Creating Serverless Functions Using CloudFormation: A Step-by-Step Guide

Serverless computing has revolutionized the way we deploy applications, allowing us to run code without provisioning or managing servers. AWS CloudFormation enables us to define and provision AWS infrastructure as code, making it easier to manage and replicate environments. In this blog, we will explore how to create serverless functions using a CloudFormation template. 

We’ll break down the provided CloudFormation template and highlight the key components and their functions. For this blog we will trigger the lambda function using a cron scheduler, the lambda will trigger an ECS task on an already created docker image. The docker image is to be uploaded in the ECRRepo resource defined in the template we will discuss later.

Overview of the CloudFormation Template

The CloudFormation template defines resources for a serverless application that uses AWS Lambda and ECS Fargate. The application that we are about to discuss consists of several Lambda functions triggered by scheduled events and an ECS task for containerized workloads.

Let’s discuss the key parts of a serverless template

 

Provider Configuration:

provider:

  name: aws

  runtime: python3.9

  region: ap-northeast-1

  environment:

    CONTAINER_NAME: "service-container"

    ECS_CLUSTER: !GetAtt FargateECSCluster.Arn

    ECS_TASK_ARN: !Ref ServiceTaskDefinition

    ECS_SUBNET: !Ref FargateSubnet

    ECS_SEC_GROUP: !Ref FargateSG

This section defines the AWS provider, runtime environment for the Lambda functions, and several environment variables that will be used in the Lambda functions. It specifies the AWS region and the runtime environment (Python 3.9). It also defines references from the resources file that will be used here after defining them in the resources file

IAM Role Configuration:
iam:

  role:

    name: service-lambda-${opt:stage, 'dev'}

    permissionsBoundary: arn:aws:iam::XXXXXXXXXXXX:policy/ScopePermissions

    statements:

      - Effect: "Allow"

        Action:

          - ecs:RunTask

        Resource:

          - !Ref ServiceTaskDefinition

      - Effect: "Allow"

        Action:

          - iam:PassRole

        Resource:

          - !GetAtt FargateExecutionRole.Arn

      - Effect: Allow

        Action:

          - iam:PassRole

        Resource:

          - !GetAtt FargateTaskRole.Arn

This section configures the IAM role for the Lambda functions, including permissions to run ECS tasks and pass roles.

Execution Role: The FargateExecutionRole is required for granting permissions to the ECS tasks to pull container images from ECR and write logs to CloudWatch Logs. This role is essential for the secure execution of containers.

Task Role: The FargateTaskRole allows the ECS tasks to assume specific IAM roles to access AWS services securely. This is crucial for granting the tasks the necessary permissions to interact with other AWS resources, such as S3 buckets or Secrets Manager.

Lambda Functions

The template defines several Lambda functions, each triggered by a specific schedule:

functions:

  script_runner:

    handler: handler.run_insertion_script

    events:

      - schedule: cron(0 1,9,17 * * ? *)

  aggregation:

    handler: handler.run_aggregation

    events:

      - schedule: cron(0 7 * * ? *)

  score:

    handler: handler.run_scoring

    events:

      - schedule: cron(30 1,9,17 * * ? *)

  transfer:

    handler: handler.run_transfer

    events:

      - schedule: cron(0 2,10,18 * * ? *)

  agg_batch:

    handler: handler.run_agg_batch

Each lambda function defined here has a handler that is defined in a python file, named handler.py and the names followed by handler. are the function names defined inside the python file.

Now we will define the resources required that will support the lambda functions, as the lambda will trigger an ECS task we must define the ECS cluster and the compute boundaries of the task that will be spawned off, the ECS task will require a docker image to run which we will save in an ECR Repo.

ECS Cluster and Task Definitions

Resources:

  FargateECSCluster:

    Type: "AWS::ECS::Cluster"

    Properties:

      ClusterName: !Sub

        - "fargate-#{StackName}"

        - StackName: !Ref "AWS::StackName"

This section defines an ECS cluster where the containerized tasks will run. The cluster is named dynamically using the stack name. The ECS Cluster is essential as it provides a logical grouping of tasks or services. It enables you to manage your container instances and define the compute environment where your containers will run. In this template, the ECS Cluster is configured to host Fargate tasks, allowing you to run containers without managing the underlying EC2 instances.

 ServiceTaskDefinition:

    Type: "AWS::ECS::TaskDefinition"

    Properties:

      Cpu: 4096

      Memory: 20GB

      NetworkMode: awsvpc

      RequiresCompatibilities:

        - "FARGATE"

      ExecutionRoleArn: !GetAtt FargateExecutionRole.Arn

      TaskRoleArn: !GetAtt FargateTaskRole.Arn

      ContainerDefinitions:

        - Name: "service-container"

          Image: !Sub

            - "#{AccountId}.dkr.ecr.#{Region}.amazonaws.com/#{Repo}"

            - AccountId: !Ref AWS::AccountId

              Region: !Ref AWS::Region

              Repo: !Ref ServiceECSRepo

          LogConfiguration:

            LogDriver: awslogs

            Options:

              awslogs-group: !Ref FargateLogGroup

              awslogs-region: !Ref AWS::Region

              awslogs-stream-prefix: !Ref AWS::StackName

 

This section defines an ECS task definition with specific CPU and memory requirements, network mode, and container definitions. The container image is pulled from an ECR repository. This is fundamental for defining the configuration of the ECS tasks, including CPU, memory, networking settings, and container specifications. It specifies the Docker image to use, environment variables, and logging configurations, enabling the deployment of the application’s containers on Fargate.

Network Configuration

 FargateVPC:

    Type: "AWS::EC2::VPC"

    Properties:

      CidrBlock: 10.10.10.0/16

      EnableDnsHostnames: true

      EnableDnsSupport: true

      Tags:

        - Key: Name

          Value: !Ref "AWS::StackName"

  FargateSubnet:

    Type: "AWS::EC2::Subnet"

    Properties:

      CidrBlock: 10.10.10.0/24

      VpcId: !Ref FargateVPC

      AvailabilityZone: "ap-northeast-1a"

      Tags:

        - Key: Name

          Value: !Ref "AWS::StackName"

 

This section defines a VPC and a subnet for the ECS tasks. It ensures that the tasks run within a specific network.

Security Groups

 FargateSG:

    Type: "AWS::EC2::SecurityGroup"

    Properties:

      GroupDescription: "Generated by Serverless"

      SecurityGroupIngress:

        - IpProtocol: -1

          CidrIp: 127.0.0.1/32

      Tags:

        - Key: Name

          Value: !Ref "AWS::StackName"

      VpcId: !Ref FargateVPC

 

This section defines a security group to control inbound traffic to the ECS tasks. The security group is associated with the VPC defined earlier.

VPC: The FargateVPC defines the network boundary for the ECS tasks, providing isolation and control over network traffic. It ensures that tasks can communicate securely within the defined network.

Subnet: The FargateSG controls inbound and outbound traffic to the ECS tasks, providing security at the network level. By defining rules for allowed traffic, it helps protect the tasks from unauthorized access while permitting necessary communication with other services.

Conclusion

In this blog, we have broken down a CloudFormation template to demonstrate how to create serverless functions and set up an ECS cluster with Fargate tasks. This approach allows you to manage your infrastructure as code, making it easier to replicate and maintain your environment. By leveraging AWS CloudFormation, you can automate the deployment of complex serverless architectures, ensuring consistency and reducing the potential for human error.

By understanding the key components of this template, you can customize it to fit your specific needs and deploy your serverless applications efficiently on AWS.