Flask on AWS Serverless: A learning journey - Part 1
About 3 years ago I learnt some basic Python, which I've used almost exclusively to build back-end APIs on AWS Serverless, mostly Lambda. This includes a monitoring solution on AWS, an event-driven API integration solution, a load shedding telegram bot, a Slack bot that posts AWS News, a CDK project, and other telegram bots. None of them are front-end web apps, and thats something that has always been a gap for me. Some years back I did some Ruby on Rails, but did'nt build anything meaningful, and I've since forgotten most of it. So I've decided to learn Flask as a tool to build some web apps: primarily because its still python, and a micro-framework with a minimal learning curve. And I wanted to re-use what I've learned building python back-end apps and APIs on AWS Serverless, and see how I can build front-end apps and APIs that run on AWS Serverless. Admittedly, Flask is still server-side, which means I'm still avoiding client-side web apps (React, etc), but baby steps for now.
I'm specifically focussing on Flask web apps, that return HTML, CSS and JS. For Flask APIs on AWS Serverless, there are already awesome packages like Chalice, Zappa and Powertools for AWS Lambda (Python).
There are other AWS service that can be used to run Flask on AWS, like using EC2, Elastic Beanstalk, ECS or Lightsail. But I am specifically looking to use serverless because I don't want to manage servers or containers, I only want to to pay for what I actually use without having resource on all the time (and with the generous free tier for serverless on AWS you wont pay anything to run this tutorial), I want to fully automate the deployment process, and if I eventually have to scale, I don't want to have to re-architect anything. Serverless has a much better Developer Experience, and allows you to quickly build things with less hassle.
So in this series of posts, we will learn to build some Flask apps on AWS, and figure things out along the way. I'll probably get some stuff wrong, so Errors and omissions excepted. Onwards!
But Lambda can only be used for APIs...I hear you say!
But AWS Lambda is for APIs, as it returns JSON. And for RESTfull APIs you usually serve Lambda functions behind Amazon API Gateway or a Lambda Function URL, or behind Appsync for GraphQL APIs. Yes, you can have Lambda functions returning HTML with some customisation, but how would we run Flask on Lambda without changing anything in Flask? The answer: by using the Lambda Web Adapter, which serves as a universal adapter for Lambda Runtime API and HTTP API. It allows developers to package familiar HTTP 1.1/1.0 web applications, such as Express.js, Next.js, Flask, SpringBoot, or Laravel, and deploy them on AWS Lambda. This replaces the need to modify the web application to accommodate Lambda’s input and output formats, reducing the complexity of adapting code to meet Lambda’s requirements.
I should also call out the really good awsgi package, (and tutorial) which can also be used to run Flask on AWS serverless, with just a small handler in the flask app.
In order to demonstrate how to run Flask on AWS Serverless using the Lambda Web Adapter, I'm going to take an existing Flask app, and show you how to run it on AWS. For this, we will start using a very-well written tutorial on Digital Ocean: How to Make a Web Application Using Flask in Python 3. Using this tutorial as a vehicle, I will show you how to get this Flask app running on AWS, using AWS Lambda, Amazon API Gateway and Amazon DynamoDB, all deployed using AWS SAM. So to follow along, you may want to keep that tutorial open, as well as this blog post. I refer to the instructions in the tutorial, and advise what needs to be changed. In addition, or alternatively to following along, you can use the resources on this projects github:
- starter: use this simply as a base to start with, as you follow along and make the required changes
- completed: use this as a complete working project, that you can simply deploy without any further changes
Prerequisites
Besides a working Python 3 environment (in my case python3.12), you will also need:
- An active AWS account
- AWS Command Line Interface (AWS CLI) installed and configured
- AWS Serverless Application Model Command Line Interface (AWS SAM CLI) installed
- Optionally, you should be using an IDE like AWS Cloud9 or VS Code, with the AWS Toolkit installed
Step 1 — Installing Flask
Follow the tutorial and install Flask. In my case, the version of Flask I have locally installed is:
3.0.0Step 2 — Creating a Base Application
Follow the tutorial, and get the Hello World Flask app running locally. You can set the variables as the tutorial does, or alternatively specify it in the flask run command:
flask --app app run --debug
* Serving Flask app 'hello'
* Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
* Restarting with stat
* Debugger is active!
* Debugger PIN: 139-148-368(I realise the tutorial is using hello.py for this initial step, but to make it simpler for later on, I've started naming the file app.py from now.)
Now lets see how we can get this Hello World Flask app running on AWS. We need to create a SAM app, then build and deploy it to AWS.
We first initialise a new SAM app, using the sam-cli, based of this projects repo on github:
sam init --name flask-aws-serverless --location https://github.com/jojo786/flask-aws-serverlessthen change to the part-1 folder, and specifically the starter sub-folder:
cd flask-aws-serverless/flask-aws-serverless-part-1/flask-aws-serverless-part-1-starter/ which contains these files and folders:
.
├── README.md
├── __init__.py
├── flask
│ ├── __init__.py
│ ├── app.py
│ └── requirements.txt
├── template.yamlThe flask folder contains the python code that will run as Lambda functions - the app.py file contains the same base application from the tutorial. The template.yaml file describes the serverless application resources and properties for AWS SAM deployments.
We can now build the SAM app using sam build:
sam build
Starting Build use cache
Manifest is not changed for (HelloWorldFunction), running incremental build
Building codeuri:
.../flask-aws-serverless-part-1/flask runtime:
python3.12 metadata: {} architecture: arm64 functions: HelloWorldFunction
Running PythonPipBuilder:CopySource
Running PythonPipBuilder:CopySource
Build Succeededand deploy it to AWS using sam deploy. The first time we run it, we use the interactive guided workflow to setup the various parameters: sam deploy --guided
sam deploy --guided
Configuring SAM deploy
======================
Setting default arguments for 'sam deploy'
=========================================
Stack Name [flask-aws-serverless-part-1]:
AWS Region [af-south-1]:
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: N
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]:
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]:
HelloWorldFunction has no authentication. Is this okay? [y/N]: y
Save arguments to configuration file [Y/n]:
SAM configuration file [samconfig.toml]:
SAM configuration environment [default]:
Looking for resources needed for deployment:You can choose what to use for each argument. Please note, we haven't configured any authentication on Amazon API Gateway, so you will need to reply with y in order for the deployment to proceed.
Once the deployment has been successful, you will find the output will list the URL of the Hello World Lambda function:
CloudFormation outputs from deployed stack
------------------------------------------------------------------------------------------------------------------
Outputs
------------------------------------------------------------------------------------------------------------------
Key HelloWorldApi
Description API Gateway endpoint URL for the Hello World function
Value https://helloabc123.execute-api.af-south-1.amazonaws.com/
------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - flask-aws-serverless-part-1 in af-south-1Using your API Gateway URL, you can paste that into a browser, or call it from the command line using curl, and verify that the Flask app is working on AWS:
curl https://helloabc123.execute-api.af-south-1.amazonaws.com/
Hello, World!% You can view the logs from Amazon CloudWatch, using sam logs:
sam logs --stack-name flask-aws-serverless-part-1
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.543000 {
"time": "2023-12-18T12:59:47.543Z",
"type": "platform.initStart",
"record": {
"initializationType": "on-demand",
"phase": "init",
"runtimeVersion": "python:3.12.v16",
"runtimeVersionArn": "arn:aws:lambda:af-south-1::runtime:5eaca0ecada617668d4d59f66bf32f963e95d17ca326aad52b85465d04c429f5",
"functionName": "flask-aws-serverless-part-1-HelloWorldFunction-ovMO2mWwZDtR",
"functionVersion": "$LATEST"
}
}
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.819000 [2023-12-18 12:59:47 +0000] [12] [INFO] Starting gunicorn 21.2.0
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.820000 [2023-12-18 12:59:47 +0000] [12] [INFO] Listening at: http://0.0.0.0:8000 (12)
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.820000 [2023-12-18 12:59:47 +0000] [12] [INFO] Using worker: sync
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.822000 [2023-12-18 12:59:47 +0000] [13] [INFO] Booting worker with pid: 13
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:48.278000 {
"time": "2023-12-18T12:59:48.278Z",
"type": "platform.report",
"record": {
"requestId": "4921460f-40dc-4452-81bf-75f608101f12",
"metrics": {
"durationMs": 19.307,
"billedDurationMs": 20,
"memorySizeMB": 128,
"maxMemoryUsedMB": 76,
"initDurationMs": 713.405
},
"status": "success"
}
}Your Flask app is now live on AWS! Lets review what we have accomplished thus far. We first initialised an AWS SAM app, built it, then deployed it to AWS. What SAM actually did for us in the background was to provision the following resources on AWS:
- An AWS Lambda function to run the Flask base Hello World app. This includes a Layer for the Lambda Web Adapter
- An Amazon API Gateway HTTP API in front of the Lambda function to receive requests, which will invoke the Lambda function
- An Amazon CloudWatch group to store logs from the Lambda function
- And a few other things like IAM Roles and policies, and API Gateway stages
Step 3 — Using HTML templates
Everything in this step will be exactly the same as it is in the tutorial. After you've created all the templates in the flask folder, the file structure will now look like:
.
├── README.md
├── __init__.py
├── flask
│ ├── __init__.py
│ ├── app.py
│ ├── requirements.txt
│ ├── run.sh
│ ├── static
│ │ └── css
│ │ └── style.css
│ └── templates
│ ├── base.html
│ └── index.html
├── samconfig.toml
├── template.yamlto test it locally, change to the flask directory, and use flask run:
cd flask/
flask --app app run --debugAnd to deploy these changes to AWS, simple run:
sam build && sam deployAnd once the deploy is done, you can use the same API Gateway URL on AWS as before (e.g. https://helloabc123.execute-api.af-south-1.amazonaws.com/) in your browser.
Step 4 — Setting up the Database
AWS Lambda functions and its storage are ephemeral, meaning their execution environments only exist for a short time when the function is invoked. This means that we will eventually lose data if we setup an SQLite database as part of the Lambda function, because the contents are deleted when the Lambda service eventually terminates the execution environment. There are multiple options for managed serverless databases on AWS, including Amazon Aurora Serverless, that also supports SQL, just like SQLite as used in the tutorial. However, I prefer using Amazon DynamoDB: a fully managed, serverless, key-value NoSQL database on AWS.
So we will need to make a few changes to the tutorial to use DynamoDB, instead of SQLite is. We wont need the schema.sql or init_db.py files. Rather, we will use SAM to deploy a DynamoDB table that we referencing as PostsTable (the actual name of the table will only be known after it has been created, in the Output of sam deploy).
Add (or uncomment) the following config in template.yaml:
# PostsTable:
# Type: AWS::DynamoDB::Table
# Properties:
# AttributeDefinitions:
# - AttributeName: id
# AttributeType: N
# KeySchema:
# - AttributeName: id
# KeyType: HASH
# BillingMode: PAY_PER_REQUEST
# Tags:
# - Value: "flask-aws-serverless"
# Key: "project"
Outputs:
# DynamoDBTable:
# Description: DynamoDB Table
# Value: !Ref PostsTable DynamoDB does not have a traditional row and column data model like a relational database. Instead, DynamoDB uses a key-value format to store data.
- In DynamoDB, a table contains items rather than rows. Each item is a collection of attribute-value pairs. You can think of the attributes as being similar to columns in a relational database, but there is no predefined schema that requires all items to have the same attributes.
- DynamoDB items also do not have a fixed structure like rows in a relational database. Items can vary in content and size up to 400KB. This flexible data model allows DynamoDB tables to accommodate a wide variety of data types and workloads.
- Primary keys in DynamoDB serve a similar purpose to primary keys in a relational database by uniquely identifying each item. But DynamoDB tables have just one primary key rather than having a separate primary key for each table like in a relational DB.
So in summary, while DynamoDB shares some similarities to relational databases, it does not have the traditional row-column structure. Its flexible, non-schema model is better suited for many NoSQL and distributed applications.
Lets talk a little about the columns that are used in the tutorial, and compare it to what we using with DynamoDB:
id INTEGER PRIMARY KEY AUTOINCREMENT,
created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
content TEXT NOT NULLBut with DynamoDB, we are specifying:
- the primary key is
id, which is of TypeNumber. DynamoDB does not natively have an auto-incrementing column type. Instead of an auto-incrementing number, we will simply use generate a timestamp as theidvalue. - DynamoDB doesn’t have a dedicated datetime data type. In the flask app, we will use the python datetime class to create a timestamp.
- We dont need to specify the other columns (title, content) yet.
And to allow the Lambda function to acquire the required IAM permissions to read and write to that table, we will add a connector. Add this config to the Lambda resource in template.yaml:
PostsTable: !Ref PostsTable
Connectors:
PostsTableConnector:
Properties:
Destination:
Id: PostsTable
Permissions:
- Read
- Write The complete template.yaml is now as follows, as per github:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: flask-aws-serverless-part-1
Globals:
Function:
Tags:
project: "flask-aws-serverless"
Timeout: 3
MemorySize: 128
Runtime: python3.12
Layers:
- !Sub arn:aws:lambda:${AWS::Region}:753240598075:layer:LambdaAdapterLayerArm64:17
LoggingConfig:
LogFormat: JSON
#LogGroup: !Sub /aws/lambda/${AWS::StackName}
Architectures:
- arm64 #Graviton: cheaper and faster
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: flask/
Handler: run.sh #required for the Lambda Web Adapter
Events:
HelloWorld:
Type: HttpApi
Environment:
Variables:
AWS_LAMBDA_EXEC_WRAPPER: /opt/bootstrap
PORT: 8000
PostsTable: !Ref PostsTable
Connectors:
PostsTableConnector:
Properties:
Destination:
Id: PostsTable
Permissions:
- Read
- Write
PostsTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: id
AttributeType: N
KeySchema:
- AttributeName: id
KeyType: HASH
BillingMode: PAY_PER_REQUEST
Outputs:
HelloWorldApi:
Description: API Gateway endpoint URL for Hello World function
Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.${AWS::URLSuffix}/"
DynamoDBTable:
Description: DynamoDB Table
Value: !Ref PostsTable As usual, to deploy to AWS, run sam build && sam deploy. The Output will include the name of the DynamoDB table that SAM created.
To insert some test posts into DynamoDB, you can log into the DynamoDB console in your AWS account, or with the aws cli, using the value you got above:
aws dynamodb put-item \
--table-name flask-aws-serverless-part-1-PostsTable-abc123 \
--item '{"id": {"N": "1"}, "title": {"S": "My first post"}, "content": {"S": "Hello World"}, "created": {"S": "2023-12-18 18:05:00"}}'Step 5 — Displaying All Posts
Here we will make some changes to the Flask app, to read data from DynamoDB. We will import the boto3 package - a Python SDK for AWS. We will lookup the name of the DynamoDB table that was created by SAM, then use a scan operation to return all items.
Our app.py will now look as follows:
from flask import Flask, render_template
import os
from boto3.dynamodb.conditions import Key
from boto3 import resource
dynamodb = resource('dynamodb')
posts_table = dynamodb.Table(os.environ["PostsTable"])
app = Flask(__name__)
@app.route('/')
def index():
posts = ''
try:
response = posts_table.scan()
posts = response['Items']
except Exception as error:
print("dynamo scan failed:", error, flush=True)
return render_template('index.html', posts=posts)
You can now see the posts on the flask app. You can use the flask run command (remember to change to flask directory) to run the app locally, However, you will need to provide it with the name of the DynamoDB table. On MacOS, you can export the variable, based on output from the SAM deploy, before running flask run:
export PostsTable=flask-aws-serverless-part-1-PostsTable-abc123Instead of using the raw boto3 to interface with DynamoDB, you can look at persistence libraries that make it easier to work with DynamoDB in python and Flask.
Step 6 — Displaying a Single Post
The only change required here is to the get_post method, which will get a particular item from DynamoDB:
def get_post(post_id):
try:
response = posts_table.get_item(Key={'id': post_id})
post = response['Item']
except Exception as error:
print("dynamo get post failed:", error, flush=True)
abort(404)
return postAs usual, run sam build && sam deploy to run it on AWS, and/or flask run to test locally.
Step 7 — Modifying Posts
Creating a New Post
Our create function will work as follows. Using the DynamoDB update_item operation:
- DynamoDB does not natively have an auto-incrementing column type. Instead of an auto-incrementing, number, we will simply use a timestamp as the
idvalue, which we cast to an int - DynamoDB doesn’t have a dedicated datetime data type, so we will use the python
datetimeclass to create a timestamp, which we cast to a string
@app.route('/create', methods=('GET', 'POST'))
def create():
if request.method == 'POST':
title = request.form['title']
content = request.form['content']
created = str(datetime.now())
id = int(datetime.now().timestamp())
if not title:
flash('Title is required!')
else:
try:
#insert new post into dynamodb
posts_table.put_item(
Item={
'id': id,
'title': title,
'content': content,
'created': created
}
)
except Exception as error:
print("dynamo PUT failed:", error, flush=True)
return redirect(url_for('index'))
return render_template('create.html')Editing a Post
Our edit function will work very similar, where we lookup a particular post id, and then update that item:
@app.route('/<int:id>/edit', methods=('GET', 'POST'))
def edit(id):
post = get_post(id)
if request.method == 'POST':
title = request.form['title']
content = request.form['content']
if not title:
flash('Title is required!')
else:
try:
posts_table.update_item(
Key={
'id': id
},
UpdateExpression="set title = :title, content = :content",
ExpressionAttributeValues={
':title': title,
':content': content
}
)
except Exception as error:
print("dynamo update failed:", error, flush=True)
return redirect(url_for('index'))
return render_template('edit.html', post=post)Deleting a Post
The delete function is quite similiar again, where we lookup a particular post id, then delete it:
@app.route('/<int:id>/delete', methods=('POST',))
def delete(id):
post = get_post(id)
try:
posts_table.delete_item(
Key={
'id': id
}
)
flash('"{}" was successfully deleted!'.format(post['title']))
except Exception as error:
print("dynamo delete failed:", error, flush=True)
return redirect(url_for('index'))You can get all the final code from the completed folder in github.
As usual, you simply run sam build && sam deploy to deploy to AWS.
Conclusion
We've taken the excellent How To Make a Web Application Using Flask in Python 3 tutorial and using AWS SAM, demonstrated how you can run a Flask app on AWS Serverless. With serverless, we dont need to think of or manage servers, or worry about other mundane tasks like installing or patching the OS, database or any software packages. The beauty of SAM is that it deploys directly to AWS for us, with very little effort. We chose to use DynamoDB as the serverless database. In part 2 we use Aurora Serverless.