Flask on AWS Serverless: A learning journey - Part 1

About 3 years ago I learnt some basic Python, which I've used almost exclusively to build back-end APIs on AWS Serverless, mostly Lambda. This includes a monitoring solution on AWS,  an event-driven API integration solution, a  load shedding telegram bot, a Slack bot that posts AWS News, a CDK project, and other telegram bots. None of them are front-end web apps, and thats something that has always been a gap for me. Some years back I did some Ruby on Rails, but did'nt build anything meaningful, and I've since forgotten most of it. So I've decided to learn Flask as a tool to build some web apps: primarily because its still python, and a micro-framework with a minimal learning curve. And I wanted to re-use what I've learned building python back-end apps and APIs on AWS Serverless, and see how I can build front-end apps and APIs that run on AWS Serverless. Admittedly, Flask is still server-side, which means I'm still avoiding client-side web apps (React, etc), but baby steps for now.

I'm specifically focussing on Flask web apps, that return HTML, CSS and JS. For Flask APIs on AWS Serverless, there are already awesome packages like Chalice, Zappa and Powertools for AWS Lambda (Python).

There are other AWS service that can be used to run Flask on AWS, like using EC2, Elastic Beanstalk, ECS or Lightsail. But I am specifically looking to use serverless because I don't want to manage servers or containers, I only want to to pay for what I actually use without having resource on all the time (and with the generous free tier for serverless on AWS you wont pay anything to run this tutorial), I want to fully automate the deployment process, and if I eventually have to scale, I don't want to have to re-architect anything. Serverless has a much better Developer Experience, and allows you to quickly build things with less hassle.

So in this series of posts, we will learn to build some Flask apps on AWS, and figure things out along the way. I'll probably get some stuff wrong, so Errors and omissions excepted. Onwards!

But Lambda can only be used for APIs...I hear you say!

But AWS Lambda is for APIs, as it returns JSON. And for RESTfull APIs you usually serve Lambda functions behind Amazon API Gateway or a Lambda Function URL, or behind Appsync for GraphQL APIs. Yes, you can have Lambda functions returning HTML with some customisation, but how would we run Flask on Lambda without changing anything in Flask? The answer: by using the Lambda Web Adapter, which serves as a universal adapter for Lambda Runtime API and HTTP API. It allows developers to package familiar HTTP 1.1/1.0 web applications, such as Express.js, Next.js, Flask, SpringBoot, or Laravel, and deploy them on AWS Lambda. This replaces the need to modify the web application to accommodate Lambda’s input and output formats, reducing the complexity of adapting code to meet Lambda’s requirements.

I should also call out the really good awsgi package, (and tutorial) which can also be used to run Flask on AWS serverless, with just a small handler in the flask app.

In order to demonstrate how to run Flask on AWS Serverless using the Lambda Web Adapter, I'm going to take an existing Flask app, and show you how to run it on AWS. For this, we will start using a very-well written tutorial on Digital Ocean: How to Make a Web Application Using Flask in Python 3. Using this tutorial as a vehicle, I will show you how to get this Flask app running on AWS, using AWS Lambda, Amazon API Gateway and Amazon DynamoDB, all deployed using AWS SAM. So to follow along, you may want to keep that tutorial open, as well as this blog post. I refer to the instructions in the tutorial, and advise what needs to be changed. In addition, or alternatively to following along, you can use the resources on this projects github:

  • starter: use this simply as a base to start with, as you follow along and make the required changes
  • completed: use this as a complete working project, that you can simply deploy without any further changes

Prerequisites

Besides a working Python 3 environment (in my case python3.12), you will also need:

  • An active AWS account
  • AWS Command Line Interface (AWS CLI) installed and configured
  • AWS Serverless Application Model Command Line Interface (AWS SAM CLI) installed
  • Optionally, you should be using an IDE like AWS Cloud9 or VS Code, with the AWS Toolkit installed

Step 1 — Installing Flask

Follow the tutorial and install Flask. In my case, the version of Flask I have locally installed is:

3.0.0

Step 2 — Creating a Base Application

Follow the tutorial, and get the Hello World Flask app running locally. You can set the variables as the tutorial does, or alternatively specify it in the flask run command:

flask --app app run --debug               
 * Serving Flask app 'hello'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 139-148-368

(I realise the tutorial is using hello.py for this initial step, but to make it simpler for later on, I've started naming the file app.py from now.)

Now lets see how we can get this Hello World Flask app running on AWS. We need to create a SAM app, then build and deploy it to AWS.

We first initialise a new SAM app, using the sam-cli, based of this projects repo on github:

sam init --name flask-aws-serverless --location https://github.com/jojo786/flask-aws-serverless

then change to the part-1 folder, and specifically the starter sub-folder:

cd flask-aws-serverless/flask-aws-serverless-part-1/flask-aws-serverless-part-1-starter/ 

which contains these files and folders:

.
├── README.md
├── __init__.py
├── flask
│   ├── __init__.py
│   ├── app.py
│   └── requirements.txt
├── template.yaml

The flask folder contains the python code that will run as Lambda functions - the app.py file contains the same base application from the tutorial.  The template.yaml file describes the serverless application resources and properties for AWS SAM deployments.

We can now build the SAM app using sam build:

sam build

Starting Build use cache                                                                                 
Manifest is not changed for (HelloWorldFunction), running incremental build                      
Building codeuri:                                                                                
.../flask-aws-serverless-part-1/flask runtime:    
python3.12 metadata: {} architecture: arm64 functions: HelloWorldFunction                        
 Running PythonPipBuilder:CopySource                                                             
 Running PythonPipBuilder:CopySource                                                             

Build Succeeded

and deploy it to AWS using sam deploy. The first time we run it, we use the interactive guided workflow to setup the various parameters: sam deploy --guided

sam deploy --guided

Configuring SAM deploy
======================


        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [flask-aws-serverless-part-1]: 
        AWS Region [af-south-1]: 
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]: N
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]: 
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]:                               
        HelloWorldFunction has no authentication. Is this okay? [y/N]: y
                             
        Save arguments to configuration file [Y/n]: 
        SAM configuration file [samconfig.toml]: 
        SAM configuration environment [default]: 

        Looking for resources needed for deployment:

You can choose what to use for each argument. Please note, we haven't configured any authentication on Amazon API Gateway, so you will need to reply with y in order for the deployment to proceed.

Once the deployment has been successful, you will find the output will list the URL of the Hello World Lambda function:

CloudFormation outputs from deployed stack
------------------------------------------------------------------------------------------------------------------
Outputs                                                                                                          
------------------------------------------------------------------------------------------------------------------
Key                 HelloWorldApi                                                                                
Description         API Gateway endpoint URL for the Hello World function                             
Value               https://helloabc123.execute-api.af-south-1.amazonaws.com/                                     
------------------------------------------------------------------------------------------------------------------


Successfully created/updated stack - flask-aws-serverless-part-1 in af-south-1

Using your API Gateway URL, you can paste that into a browser, or call it from the command line using curl, and verify that the Flask app is working on AWS:

curl https://helloabc123.execute-api.af-south-1.amazonaws.com/
Hello, World!%                                                         

You can view the logs from Amazon CloudWatch, using sam logs:

sam logs --stack-name flask-aws-serverless-part-1       
                                                             
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.543000 {
  "time": "2023-12-18T12:59:47.543Z",
  "type": "platform.initStart",
  "record": {
    "initializationType": "on-demand",
    "phase": "init",
    "runtimeVersion": "python:3.12.v16",
    "runtimeVersionArn": "arn:aws:lambda:af-south-1::runtime:5eaca0ecada617668d4d59f66bf32f963e95d17ca326aad52b85465d04c429f5",
    "functionName": "flask-aws-serverless-part-1-HelloWorldFunction-ovMO2mWwZDtR",
    "functionVersion": "$LATEST"
  }
}
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.819000 [2023-12-18 12:59:47 +0000] [12] [INFO] Starting gunicorn 21.2.0
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.820000 [2023-12-18 12:59:47 +0000] [12] [INFO] Listening at: http://0.0.0.0:8000 (12)
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.820000 [2023-12-18 12:59:47 +0000] [12] [INFO] Using worker: sync
2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:47.822000 [2023-12-18 12:59:47 +0000] [13] [INFO] Booting worker with pid: 13

2023/12/18/[$LATEST]faca0569b6084bbdac895af5611c311f 2023-12-18T12:59:48.278000 {
  "time": "2023-12-18T12:59:48.278Z",
  "type": "platform.report",
  "record": {
    "requestId": "4921460f-40dc-4452-81bf-75f608101f12",
    "metrics": {
      "durationMs": 19.307,
      "billedDurationMs": 20,
      "memorySizeMB": 128,
      "maxMemoryUsedMB": 76,
      "initDurationMs": 713.405
    },
    "status": "success"
  }
}

Your Flask app is now live on AWS! Lets review what we have accomplished thus far. We first initialised an AWS SAM app, built it, then deployed it to AWS. What SAM actually did for us in the background was to provision the following resources on AWS:

  • An AWS Lambda function to run the Flask base Hello World app. This includes a Layer for the Lambda Web Adapter
  • An Amazon API Gateway HTTP API in front of the Lambda function to receive requests, which will invoke the Lambda function
  • An Amazon CloudWatch group to store logs from the Lambda function
  • And a few other things like IAM Roles and policies, and API Gateway stages

Step 3 — Using HTML templates

Everything in this step will be exactly the same as it is in the tutorial. After you've created all the templates in the flask folder, the file structure will now look like:

.
├── README.md
├── __init__.py
├── flask
│   ├── __init__.py
│   ├── app.py
│   ├── requirements.txt
│   ├── run.sh
│   ├── static
│   │   └── css
│   │       └── style.css
│   └── templates
│       ├── base.html
│       └── index.html
├── samconfig.toml
├── template.yaml

to test it locally, change to the flask directory, and use flask run:

cd flask/
flask --app app run --debug

And to deploy these changes to AWS, simple run:

sam build && sam deploy

And once the deploy is done, you can use the same API Gateway URL on AWS as before (e.g. https://helloabc123.execute-api.af-south-1.amazonaws.com/) in your browser.

Step 4 — Setting up the Database

AWS Lambda functions and its storage are ephemeral, meaning their execution environments only exist for a short time when the function is invoked. This means that we will eventually lose data if we setup an SQLite database as part of the Lambda function, because the contents are deleted when the Lambda service eventually terminates the execution environment. There are multiple options for managed serverless databases on AWS, including Amazon Aurora Serverless, that also supports SQL, just like SQLite as used in the tutorial. However, I prefer using Amazon DynamoDB: a fully managed, serverless, key-value NoSQL database on AWS.

So we will need to make a few changes to the tutorial to use DynamoDB, instead of SQLite is. We wont need the schema.sql or init_db.py files. Rather, we will use SAM to deploy a DynamoDB table that we referencing as PostsTable (the actual name of the table will only be known after it has been created, in the Output of sam deploy).  

Add (or uncomment) the following config in template.yaml:

  # PostsTable:
  #   Type: AWS::DynamoDB::Table
  #   Properties:
  #     AttributeDefinitions:
  #       - AttributeName: id
  #         AttributeType: N
  #     KeySchema:
  #       - AttributeName: id
  #         KeyType: HASH
  #     BillingMode: PAY_PER_REQUEST
  #     Tags:
  #      - Value: "flask-aws-serverless"
  #        Key: "project"
  
  Outputs:
  # DynamoDBTable:
  #   Description: DynamoDB Table
  #   Value: !Ref PostsTable  

DynamoDB does not have a traditional row and column data model like a relational database. Instead, DynamoDB uses a key-value format to store data.

  • In DynamoDB, a table contains items rather than rows. Each item is a collection of attribute-value pairs. You can think of the attributes as being similar to columns in a relational database, but there is no predefined schema that requires all items to have the same attributes.
  • DynamoDB items also do not have a fixed structure like rows in a relational database. Items can vary in content and size up to 400KB. This flexible data model allows DynamoDB tables to accommodate a wide variety of data types and workloads.
  • Primary keys in DynamoDB serve a similar purpose to primary keys in a relational database by uniquely identifying each item. But DynamoDB tables have just one primary key rather than having a separate primary key for each table like in a relational DB.

So in summary, while DynamoDB shares some similarities to relational databases, it does not have the traditional row-column structure. Its flexible, non-schema model is better suited for many NoSQL and distributed applications.

Lets talk a little about the columns that are used in the tutorial, and compare it to what we using with DynamoDB:

id INTEGER PRIMARY KEY AUTOINCREMENT,
created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
content TEXT NOT NULL

But with DynamoDB, we are specifying:

  • the primary key is id, which is of Type Number. DynamoDB does not natively have an auto-incrementing column type. Instead of an auto-incrementing number, we will simply use generate a timestamp as the id value.
  • DynamoDB doesn’t have a dedicated datetime data type. In the flask app, we will use the python datetime class to create a timestamp.
  • We dont need to specify the other columns (title, content) yet.

And to allow the Lambda function to acquire the required IAM permissions to read and write to that table, we will  add a connector. Add this config to the Lambda resource in template.yaml:

PostsTable: !Ref PostsTable
    Connectors:
      PostsTableConnector:
        Properties:
          Destination: 
            Id: PostsTable
          Permissions: 
            - Read
            - Write 

The complete template.yaml is now as follows, as per github:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: flask-aws-serverless-part-1

Globals:
  Function:
    Tags:
      project: "flask-aws-serverless"
    Timeout: 3
    MemorySize: 128
    Runtime: python3.12
    Layers:
        - !Sub arn:aws:lambda:${AWS::Region}:753240598075:layer:LambdaAdapterLayerArm64:17
    LoggingConfig:
      LogFormat: JSON
      #LogGroup: !Sub /aws/lambda/${AWS::StackName}
    Architectures:
      - arm64 #Graviton: cheaper and faster
    

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: flask/
      Handler: run.sh #required for the Lambda Web Adapter
      Events:
        HelloWorld:
          Type: HttpApi
      Environment:
        Variables:
          AWS_LAMBDA_EXEC_WRAPPER: /opt/bootstrap
          PORT: 8000
          PostsTable: !Ref PostsTable
    Connectors:
      PostsTableConnector:
        Properties:
          Destination: 
            Id: PostsTable
          Permissions: 
            - Read
            - Write 

  PostsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: N
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST

Outputs:
  HelloWorldApi:
    Description: API Gateway endpoint URL for Hello World function
    Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.${AWS::URLSuffix}/"
  DynamoDBTable:
    Description: DynamoDB Table
    Value: !Ref PostsTable  

As usual, to deploy to AWS, run sam build && sam deploy. The Output will include the name of the DynamoDB table that SAM created.

To insert some test posts into DynamoDB, you can log into the DynamoDB console in your AWS account, or with the aws cli, using the value you got above:

aws dynamodb put-item \                                                 
  --table-name flask-aws-serverless-part-1-PostsTable-abc123 \
  --item '{"id": {"N": "1"}, "title": {"S": "My first post"}, "content": {"S": "Hello World"}, "created": {"S": "2023-12-18 18:05:00"}}'

Step 5 — Displaying All Posts

Here we will make some changes to the Flask app, to read data from DynamoDB. We will import the boto3 package - a Python SDK for AWS. We will  lookup the name of the DynamoDB table that was created by SAM, then use a scan operation to return all items.

Our app.py will now look as follows:

from flask import Flask, render_template
import os
from boto3.dynamodb.conditions import Key
from boto3 import resource

dynamodb = resource('dynamodb')
posts_table = dynamodb.Table(os.environ["PostsTable"])
app = Flask(__name__)

@app.route('/')
def index():
    posts = ''
    
    try: 
        response = posts_table.scan()
        posts = response['Items']
    except Exception as error:
        print("dynamo scan failed:", error, flush=True) 
              
    return render_template('index.html', posts=posts) 

You can now see the posts on the flask app. You can use the flask run command (remember to change to flask directory) to run the app locally,  However, you will need to provide it with the name of the DynamoDB table. On MacOS, you can export the variable, based on output from the SAM deploy, before running flask run:

export PostsTable=flask-aws-serverless-part-1-PostsTable-abc123

Instead of using the raw boto3 to interface with DynamoDB, you can look at persistence libraries that make it easier to work with DynamoDB in python and Flask.

Step 6 — Displaying a Single Post

The only change required here is to the get_post method, which will get a particular item from DynamoDB:

def get_post(post_id):
    try:
        response = posts_table.get_item(Key={'id': post_id})
        post = response['Item']
    except Exception as error:
        print("dynamo get post failed:", error, flush=True) 
        abort(404)

    return post

As usual, run sam build && sam deploy to run it on AWS, and/or flask run to test locally.

Step 7 — Modifying Posts

Creating a New Post

Our create function will work as follows. Using the DynamoDB update_item operation:

@app.route('/create', methods=('GET', 'POST'))
def create():
    if request.method == 'POST':

        title = request.form['title']
        content = request.form['content']
        created = str(datetime.now())
        id = int(datetime.now().timestamp())
        
        if not title:
            flash('Title is required!')
        else:
            try: 
                #insert new post into dynamodb
                posts_table.put_item(
                    Item={
                        'id': id,
                        'title': title,
                        'content': content,
                        'created': created
                        }
                )
            except Exception as error:
                print("dynamo PUT failed:", error, flush=True) 
                  
            return redirect(url_for('index'))
    return render_template('create.html')

Editing a Post

Our edit function will work very similar, where we lookup a particular post id, and then update that item:

@app.route('/<int:id>/edit', methods=('GET', 'POST'))
def edit(id):
    post = get_post(id)

    if request.method == 'POST':
        title = request.form['title']
        content = request.form['content']

        if not title:
            flash('Title is required!')
        else:
            try:
                    posts_table.update_item(
                    Key={
                        'id': id
                        },
                        UpdateExpression="set title = :title, content = :content",
                        ExpressionAttributeValues={
                            ':title': title,
                            ':content': content
                            }
                )
            except Exception as error:
                print("dynamo update failed:", error, flush=True) 
                       
            return redirect(url_for('index'))

    return render_template('edit.html', post=post)

Deleting a Post

The delete function is quite similiar again, where we lookup a particular post id, then delete it:

@app.route('/<int:id>/delete', methods=('POST',))
def delete(id):
    post = get_post(id)

    try:
        posts_table.delete_item(
            Key={
                'id': id
                }
        )
        flash('"{}" was successfully deleted!'.format(post['title']))
    except Exception as error:
        print("dynamo delete failed:", error, flush=True)  
        
    return redirect(url_for('index'))

You can get all the final code from the completed folder in github.

As usual, you simply run sam build && sam deploy to deploy to AWS.

Conclusion

We've taken the excellent How To Make a Web Application Using Flask in Python 3 tutorial and using AWS SAM, demonstrated how you can run a Flask app on AWS Serverless. With serverless, we dont need to think of or manage servers, or worry about other mundane tasks like installing or patching the OS, database or any software packages. The beauty of SAM is that it deploys directly to AWS for us, with very little effort. We chose to use DynamoDB as the serverless database. In part 2 we use Aurora Serverless.