In this blog, we will look into how security in the Serverless ecosystem is a shared responsibility between AWS and the developer and how one Serverless developer can follow best practices for Lambda security. This could also apply to Cloud in general.

This blog is the extended version of a talk at AWS Community Day DevSecOps Edition Pune 2024 and CorpCon 2025 at Christ University, Bengaluru.

It’s a Shared Responsibility

In the AWS whitepaper, you can learn about “Security and compliance” when building applications with AWS Lambda Functions, a shared responsibility between AWS and the developer/customer for a fully secure application.

The whitepaper showcases how the shared responsibility model is divided with the developer (customer of AWS) taking ownership in the cloud as the code of the Lambda function along with the right resource configurations and the importance of IAM roles and policies while the cloud provider (AWS) takes the responsibility of the cloud by maintaining the infrastructure and environment which Lambda is executed is secure.

A Simple Serverless API

Let’s consider building a simple Serverless API for a CRUD operation invoked from a web app UI. The API is hosted on Amazon API Gateway which triggers a Lambda function to perform CRUD operations on the data in DynamoDB.

Let’s see how the shared responsibility model could be brought into action -

• Identity and Access

• Code

• Data

• Infrastructure

Identity and Access

Identity and Access Management (IAM) plays an important role in building the right access control for the AWS Services. For the same Serverless API, let’s understand what are the different IAM permissions needed.

API Gateway endpoints require authorizers for an API used by an application with Identity Pools (IdPs). Authorized users should have an IAM policy that allows them to make API calls to Lambda Functions. The Lambda functions need an IAM execution role to perform SDK API actions with DynamoDB for CRUD operations. While this is a basic example of a simple Serverless API, the architecture can become complex, and the associated IAM policies can also become complicated.

Least Privileges Policy

One mistake that could end up being expensive is wildcard * permissions that allow all actions to a specific or a set of AWS Services, you can follow the best practice of -

• Granting only necessary permissions needed for that execution could be by granting access to specific resources along with the Resource ARN, specific API actions only that are required.

• Using managed policies for commonly used IAM policies such as DynamoDBCrudPolicy which allows the API Actions - dynamodb:GetItem, dynamodb:DeleteItem, dynamodb:PutItem, dynamodb:Scan, dynamodb:Query, dynamodb:UpdateItem, dynamodb:BatchWriteItem, dynamodb:BatchGetItem, dynamodb:DescribeTable, dynamodb:ConditionCheckItem with the specified DynamoDB table and indexes defined in the table.

Again from the simple Serverless API, the IAM execution role for the Lambda function would look something as below.

Resources:
  Users:
    Type: AWS::DynamoDB::Table
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
  .
  .
  PutItemsFunction:
    Type: AWS::Serverless::Function
    Properties:
      .
      .
      .
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref Users

When using IAM execution policies, allow only what your Lambda code needs blog talks in depth about the least privileges policy.

Code

Security with code in a Serverless application comes into play in the infrastructure layer and also in the application code level.

Enabling Request Validations with Model

When you use API Gateway, it offers Models that define the schema for your request and response. Models help validate the request body and parameters against the schema to ensure the correct parameters are passed with the right data type. This prevents data injections and invalid request body that can break the system.

One such way is defining the Model as part of your application’s IaC code as seen below.

Additionally, when adding the API trigger to the Lambda function, enable Model mapping and configure the request parameters as needed to ensure the correct parameters are passed to the Lambda function.

Using Secrets in the Codebase

One of the best practices in applications is “Not hardcoding credentials and secrets” in your application code base. So, how do you use credentials in your Lambda function code? There are options with Lambda environment variables, Secrets Manager, and Systems Manager Parameter Store.

Among the options for using credentials and secrets, AWS Secrets Manager is the top choice. It provides built-in KMS encryption and supports key rotation, which is essential for any production system. It also offers fine-grain IAM access control.

Check for vulnerability

The codebase often uses third-party libraries and dependencies that can be vulnerable to security issues. One of the easiest ways to protect against vulnerabilities is to keep your packages updated to the latest versions. To do this, you need to scan the codebase. Amazon Inspector can now scan your AWS Lambda functions, detecting affected packages and dependencies that are vulnerable to security issues and providing steps to fix them. Amazon Inspector supports codebase scans, which can be scheduled or run manually with the deployed codebase.

Data

Even in the simple Serverless API, some data is involved either stored in the database or data that is transferred across AWS Services such as Lambda function and API Gateway or through API Gateway to the API client.

Encryption for Data at Rest

AWS Services like Amazon DynamoDB, Amazon Aurora, and Amazon S3 which are data stores support encryption of data at rest with native encryption supported with Amazon Key Management Service (KMS) and also with choice of the customer’s key with KMS.

Encryption for Data in Transit

To ensure data in transit is encrypted and follows secure protocols, use services like SNS and SQS, which support encryption in transit by default. When using Lambda functions with other AWS services, Lambda automatically uses Transport Layer Security (TLS) for data transfers. As part of the shared responsibility model, AWS ensures the security of Lambda's TLS.

Additionally, the HTTPS protocol is used for endpoints exposed by API Gateway and Lambda function URLs, which are managed natively by the API Gateway and Lambda Function Service.

Infrastructure

With Serverless, the infrastructure is maintained by the cloud provider (AWS) and we as developers would also configure things to ensure the infrastructure is maintained well.

Enabling WAF

Web Application Firewall (WAF) ensures the applications are protected from common exploits and abuses such as DDoS attacks, SQL injections, bot traffic, and cross-site scripting, and ensures the underlying AWS resources are not attacked.

As a developer, you can enable WAF for the endpoints created by API services like API Gateway and AppSync to secure API traffic. You can also secure distribution endpoints from CloudFront and S3 web hosting, as well as S3 objects, by enabling WAF.

Throttling and Rate-limits

In addition to making sure web traffic comes from trusted sources, you also need to prevent traffic spikes from overloading your system. API Gateway supports throttling and rate limits for API stages, which can be configured to ensure that traffic spikes don't harm or crash the system.

Bringing It All Together!

For the same Serverless API that performs CRUD operation on DynamoDB and additionally makes API requests to third-party services, let’s apply what we discussed with the security practices.

The web app is hosted on Amazon S3 with the distribution URL enabled with CloudFront and WAF. The web app makes API requests. This ensures the users can securely access the web app.

The API layer with API Gateway has WAF enabled on the endpoint along with the defined usage plan with rate limits and throttling limits. These APIs are authenticated with IAM roles to trigger the Lambda function for execution.

The Lambda function uses VPC to ensure an elastic IP address that can be whitelisted on the third-party service. It also leverages Secrets Manager to store the API credentials of the third-party service and uses IAM to authorize Secrets Manager and DynamoDB access.

Does this sound like too much of an additional overhead for developing a simple Serverless API? Well, security is one of the pillars of the AWS Well-Architected Tool so not an overhead but a practice to follow for all kinds of workloads to ensure better security.

Hope with this blog, you have understood how you are the co-owners of security for the applications when it comes to Serverless (the cloud in general), where as a developer you can implement the recommended best practices with Security.

Parquet data in S3

Parquet is the columnar storage format that is efficient for data storage and retrieval and is widely used by different ETL and big data processing frameworks such as Apache Spark, Hive, and Amazon Athena. This makes the parquet data more accessible with the query approach by services like Athena that use SQL queries on the parquet data. Parquet-formatted data is stored as any other object in an S3 Bucket, with an additional tag specifically mentioning that it is in parquet format.

Storing this parquet data in Amazon S3 opens up the opportunity to leverage cloud storage by integrating AWS Services such as AWS Glue for ETL, Amazon Redshift for data warehouse, and Amazon SageMaker for ML workloads. Parquet stored on S3 is not only performant but also cost-efficient as the files are smaller than CSV and can be much cheaper to store on S3 with different S3 storage classes based on how frequently they are queried.

S3 Tables

Amazon S3 launched a new bucket type - tables, which is designed for structured data formats using the Apache Iceberg table storing Apache parquet format data. This offers up to 3x faster query performance with parquet and a 10x higher transaction per second when compared with parquet data stored as an object in an S3 bucket, making it ideal for data analytics workloads.

S3 tables enable structure data formatting with the structure of Table Buckets with namespace, which contains the tables that could be queried from services like Amazon Athena the catch is to set the tables in a namespace, and also loading the tables is supported with only Amazon EMR and open source Apache Spark which makes the developers go through the environment setup of EMR clusters or Spark; making the getting started experience enforced to go to EMR or self-hosted Spark.

Setting up one-time integration with AWS Analytics Services

Another region-wide, one-time setup of AWS Analytics services is an additional environment setup process one should follow.

Honestly, this could have been done natively by AWS, and given the developers to manage permissions to different AWS Analytics services.

S3 Tables with built-in management

What makes S3 Tables performance is that way S3 Tables handles the typical table management with -

• Data compaction - which combines the small table objects into larger objects which is configurable between 64MB and 512MB as a snapshot.

• Snapshot management - ensures the snapshot lifecycle has a minimum number of snapshots to retain the maximum age of the snapshot to retain, and eventually deleting the expired snapshots.

These factors weigh in to make S3 Tables performant, you can read about how S3 table uses compaction to improve query performance by up to 3 times, along with some benchmarks with uncompacted tables in general-purpose buckets v/s compacted tables in table buckets.

S3 Metadata

Along with S3 Tables, Amazon S3 also announced S3 Metadata, which is a bucket property that can be enabled to capture the metadata about each S3 object in a general-purpose S3 bucket. S3 Metadata uses S3 Tables with the power of parquet data, the metadata of the objects are now queriable making searching for S3 objects with metadata much more efficient.

Types of S3 Metadata

S3 Metadata supports each object in a bucket with metadata that are of two categories -

• System defined - the metadata that is controlled by Amazon S3 natively, such as - Date, Content-Length, Last-Modified, and ETag that are immutable values by the user along with the metadata such as - Cache-Control, Content-Disposition, and Content-Type, which allow change in value by the user.

• User defined - the metadata that could be assigned by the user when the object is being uploaded, these keys are prefixed with x-amz-meta-.

S3 Metadata in action

• Enable the S3 Metadata for the S3 Bucket from the console which requires you to set the name of the S3 Table and following the creation, it also creates the namespace aws_s3_metadata.

  

• And when you navigate to Table Buckets, you can see the S3 Table bucket with the tables created and listed.

  

• When you try to upload a new object to the S3 bucket, you can optionally define the Metadata - system-defined or user-defined.

  

• Once uploaded, you can view the metadata from the S3 Console and also query the same with Athena.

  

Why would I choose S3 Metadata?

Since the data is stored as the Apache Iceberg with S3 Tables, one question that hit me was - why would I enable S3 Metadata instead of directly implementing it with S3 Tables?

S3 Metadata is crucial for organizing and managing the data of S3 Buckets, imagine having a system that extensively uses S3 Buckets to store data (of any format), leveraging the S3 Metadata for that would mean you can have the metadata of the object (user-defined metadata) that can help with data categorization and management. Additionally, with the metadata of the objects, data discovery with additional attributes would play a pivotal role for applications.

AWS Lambda Functions is the core of our compute layer. The reasons that weigh in for Lambda as a choice are scalability, a focus on business logic rather than infrastructure management, cost efficiency with a pay-as-you-go billing model, and the ability to bring your own programming (BYOP) language for the workloads. There is also a caveat that you should be aware of, how different best practices help with better architectures at scale.

Choice of runtime

AWS Lambda makes it feasible to choose the runtime/programming language for the workload you are building. Some runtimes are managed by AWS, such as Java, Python, NodeJS, and a few more. When you want a specific runtime like PHP, you can run the custom runtime with the image, and you, as a developer, should manage the updates with the runtime.

The choice of the runtime depends on the workload and the requirement of the Lambda function. For example, a simple web application with CRUD could use NodeJS whereas for an application that is dependent on data and has extensive support with Python libraries, you may choose Python as your Lambda’s runtime. At times, it’s also considering the skillset of the team and the learning curve that could factor in for choosing a popular or performant runtime. For instance, if the team is well skilled in NodeJS and Python but knows that Rust is performant, one should not blindly start their Lambdas with Rust.

Cold starts are a thing!

Even in 2024, with Lambda being around for a decade, Cold Starts with Lambda function is still something one has to factor in. Cold Starts occur when the Lambda function is not invoked for a period of time where the Lambda function is de-provisioned for larger scalability. When the Lambda function is invoked, there is an init phase that basically initializes the Lambda function with the respective runtime and runtime dependencies and additional code dependencies that add to the latency of the first invoked Lambda execution.

Suppose you aren’t aware of how Cold Start and Lambda performance vary with different runtimes. In that case, Maxime David has built a tool, Lambda Perf, which benchmarks Cold Start across multiple runtimes supported by the Lambda function.

SnapStart is coming in hot

AWS Lambda's feature focused on reducing Cold Starts associated with Lambda functions with specific runtimes. At AWS re:Invent 2022, SnapStart made a debut with support for Java runtimes—Java 11 and extended to Java 17. After a worthwhile wait, AWS announced at AWS re:Invent 2024 that SnapStart now supports Python 3.12 or later and .NET 8 and later.

SnapStart addresses the Cold Start problem by creating a pre-initialized snapshot of the Lambda function’s environment during deployment that is used for subsequent invocations. This snapshot contains an image of all the necessary dependencies, resources, and configurations for that Lambda function to run.

Vadym Kazulkin has authored a whole series about AWS SnapStart with Java blog series that benchmarks Cold Starts, reduced Cold Starts with SnapStart and end-to-end deployment with Java runtimes.

Is Lambdalith a good approach?

Fat Lambdas or monolithic Lambdas often referred to as Lambdalith is the approach that many developers take to have all the business logic in one single Lambda function which could be invoked either by API Gateway / AppSync so that it’s one entry point for most of your executions. Unfortunately, not every invocation would need every bit of Lambda’s business logic or imported dependencies. This approach brings in cost efficiency and usage of frameworks such as Express with Node runtime (which is not the need of the hour for most cases) however, it introduces scalability issues where concurrency in production becomes a major concern with added Cold Starts due to large code and dependencies which are to be loaded during the initialization phase of the first invoked Lambda execution. Additionally, code management is also a concern given the Lambda function performs multiple tasks.

Some lessons from Dr. Werner’s keynote: we should bring in the simplexity while building our Lambda functions so that we have Lambda function for specific tasks which could give us the feasibility to have the needed dependencies alone for that specific Lambda function also configurations (reserved concurrency or timeouts or memory or runtime) based on the workload handled by the Lambda function.

Purpose fit dependencies with managing Lambda Layers

As the previous section called out, breaking down the Lambda function based on the task. A similar approach to Lambda Layers which contain additional code dependencies and packages that could be shared across multiple Lambdas.

Creating Lambda layers that are only required by the Lambda function enables the Lambda function to not only perform better but also help in code management. This approach could be achieved by understanding what’s the purpose of your Lambda layer and how are they reused by other Lambda functions. Keeping the size of Layers small avoids the bloating of Layers and Functions, especially during initialization and execution.

AWS Lambda Power Tuning for optimizing performance and cost

AWS Lambda Power Tuning tool is an AWS Step Function State Machine that runs multiple concurrent versions of the Lambda function with different memory allocations (128MB to 10GB) and later it analyses the execution logs with cost and time for that specific invocation and recommends the best possible configuration for optimized cost and best performance.

AWS Lambda Power Tuning tool helps with data-driven decisions to make the right choice of Lambda configurations with an automated approach. This tool can be integrated into your CI/CD Pipeline so that you can run the analysis based on the deployment of the Lambda function and later optimize the configuration for best performance and cost.

Choice of deployment - Canary v/s Blue-Green

The deployment strategy depends on how your Lambda function is being invoked and the traffic patterns. The factors to consider would be how are you testing the feature/change end-to-end, if something is broken, what’s your rollback plan and the traffic at the time of deployment.

Canary is a strategy where you switch the changes and roll out to a subset of users. For instance, if the rollout is expected across regions/ AWS accounts, start with one region in one AWS account, and test for completeness. In a sunny day scenario, you would continue this with a region-by-region rollout.

Blue-Green is a strategy that makes the changes instantly between two environments. This brings into account that changes have to be tested in a different environment end-to-end. The best option for applications and workloads that require extensive testing before deployment with zero downtime and isolated testing.

Choosing between a Canary or Blue-Green deployment strategy is based on how your application is architected and tolerance with impact in production.

Wrap up!

This blog talks about the specific factors we spoke about at PeerTalk meet-up, but the things to consider for best practices for a Serverless application at scale also include factors such as concurrency and approach to handling concurrency for Lambda function with reserved or provisioned when it’s in the deployed state in production. However, this blog focuses mostly on the factors and trade-offs during development and deployment.

Thanks to Arshad Zackeriya, Darshit Pandya, Jones Zachariah Noel N, Pubudu Jayawardana, and Sean Kendall for sharing their expertise and also contributing to this blog.

Application Programming Interface (APIs) are not only external facing and consumed by other systems or clients but APIs are broadly used for inter-service and layer communications.

Are you someone building APIs with AWS Serverless stack? Architecting them would mean that you primarily work with 3 specific AWS Services - Amazon API Gateway, AWS AppSync and AWS Lambda Function URLs.

API Considersations

Before looking into what these 3 API services from AWS offer, let’s understand how do we identify key features that can an architect/developer should know before making a choice of API service.

API Type

The key deciding factor is what kind of API are you trying to build.

• RESTful APIs: Stateless architecture that uses the standard HTTP protocol and methods (GET, POST, PUT, DELETE) for the CRUD operations. It is one of the widely used API type for modern APIs.

• GraphQL: Query language based API server which allows to request the data that is required by the client and support complex queries with real-time updates with subscriptions.

• Websockets: The protocol for bidirectional communication which uses full-duplex channels over a TCP connection and used in real-time systems where low latency is crucial.

API Interactions

The data exchange works between the client and backend in the real world.

• Unidirectional: Communication flows in one direction, either from the client to the server or vice versa. For example, a client requests data from a server without expecting any further communication.

• Bidirectional: Allows communication in both directions, enabling continuous interaction between the client and server. This is essential for applications needing real-time updates, such as chat apps.

API Response

The way how the API is expected to respond to the request.

• Synchronous APIs: The client waits for a response after making a request. This blocking behavior is suitable for operations requiring immediate feedback.

• Asynchronous APIs: The client can continue executing other tasks while waiting for a response. This non-blocking behavior enhances efficiency and scalability.

API Response Structure

The structure of API responses can vary:

• Standardized Formats: Many APIs return responses in standardized formats like JSON or XML.

• Customizable Structures: Some APIs allow customization of response formats based on client needs.

Supported content-type

APIs can support various content types including:

• Standard content-types like application/json, application/xml, text/html and more.

• Custom content-types defined by the API layer or application.

Authentication and Authorization

Support for various methods of authentication and authorization for the APIs, making it more secure.

• API keys: Keys which are passed to the API via headers that grants access to API actions.

• IAM roles: Roles that use polices to grant access to API actions and resources.

• Identity providers: Identity providers such as Amazon Cognito, Okta and others which authentication the users and authorizes them with tokens passed to API layers.

• Custom Lambda authorizers: Lambda functions that works as the authorizer by providing access to APIs with valid bearer tokens.

Amazon API Gateway

Amazon API Gateway is a fully managed AWS Service for the API layer with RESTful APIs and Websocket APIs. API Gateway supports various integrations with Lambda Functions, ECS and also direct integrations with other AWS Services with the Velocity Template Language (VTL).

API Gateway also facilitates creation and managing of API keys with different rate-limit and throttling capabilities that not only ensures the security but also ensures the application is not abused with excessive API invocations.

AWS AppSync

AWS AppSync is a fully managed GraphQL API server that allows clients (front-end mobile/web) and went GraphQL clients to query data from multiple data sources where data sources integrate with multiple AWS Services directly with Velocity Template Language (VTL) and also JavaScript resolvers making it easier to design queries, mutations and subscriptions from AppSync really easy!

Here is a series on AppSync that explains about various features of AppSync and also architectures leveraging AppSync.

AWS Lambda Function URLs

AWS Lambda Functions now supports URLs that can trigger the Lambda function directly without any API Gateway making is ideal for internal APIs and for public APIs it also supports authentication with IAM roles. Although the API schema is defined and structured in the core logic of Lambda function, this supports different content-types defined by the Lambda function.

The added value of Lambda function URLs is enabling streaming responses from Lambda function which improves the Time To First Byte (TTFB), performance and better user experience when used for web and multimedia content. Read more about Streaming Responses via Lambda function.

These Lambda functions URLs can use CloudFront distributions to make them available on the Edge.

When to choose

Wrap up

In a nutshell, how do you decide API Gateway v/s AppSync v/s Lambda function URLs.

This pattern for large payloads induces latency that can negatively affect the application's performance.

In this blog, we will look at how Lambda function's Response Streaming would be helpful and identify when it would be ideal to use Response Streaming for your workloads.

Lambda Response Streaming

AWS Lambda launched support for response streaming where this new pattern of API invocation allows the Lambda function to progressively send the response in chunks of data back to the client.

What does this pattern bring in?

• Improved time to first byte (TTFB) that improves the performance where the latency of the API requests is reduced and response is received as chunks of data as and when they are available.

• API response from Response Streaming supports larger payloads which have a soft limit of 20MB without the need to send it as a whole response.

• Enabling asynchronous data streaming which sends the API response when the data is available.

Gotchas of this pattern

• Runtime support: AWS Lambda supports the capability of Response Streaming with only NodeJS runtime as part of the managed runtimes but with custom runtimes, you can leverage the Runtimes API to implement this.

• Payload size: The maximum response payload size supported is 20MB which is a soft limit you can request more via AWS Support. The first 6MB of the response is streaming without bandwidth constraints but as the data exceeds 6MB, you will have a maximum throughput of 2MB/s.

• Network cost: Similar to the payload limits, the first 6MB of the response is free and there would be data transfer charges for the total data streamed from the Lambda function.

• Function timeout: This goes without saying that you need to configure the right timeout for the Lambda function, the default 3s may be short but something beyond the 60s would also result in the API client terminating with the timeout error.

• API endpoints: The Response Streaming feature is available with only Lambda Function URL and expects the Function URL to use ResponseStream as the invocation mode. APIs are best managed with AWS API Gateway or Application Load Balancer (ALB) which doesn't support the response feature yet.

Building AWS Lambda with Response Streaming

AWS Lambda uses the Node's Writable Stream API, so you can use write() from NodeJS to write into the stream, or Lambda functions introduced pipeline() which extends the capability of the stream from util.

Node's write()

Response Streaming with write() would directly write into the Streams and whenever there is data in the stream, that would be sent to the client as a response. This method would expect you to manually handle the stream end.

exports.handler = awslambda.streamifyResponse(
    async (event, responseStream, context) => {
        const httpResponseMetadata = {
            statusCode: 200,
            headers: {
                "Content-Type": "text/html",
            }
        };

        responseStream = awslambda.HttpResponseStream.from(responseStream, httpResponseMetadata);

        responseStream.write("<html>");
        responseStream.write("<p>Hello!</p>");

        responseStream.write("<h1>Let's start streaming response</h1>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<h2>Serverless</h2>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<h3>Is</h3>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<h3>Way</h3>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<h3>More</h3>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<h3>Mature</h3>");
        await new Promise(r => setTimeout(r, 1000));
        responseStream.write("<p>DONE!</p>");
        responseStream.end();
    }
);

When you publish the above Lambda function and invoke the Function URL via a web browser, you can notice how the streaming responses are received.

Lambda's pipeline()

Lambda function's pipeline() is a util extension that handles the end of the stream automatically. This is available in util package and use pipeline(requestStream, responseStream) to write data into the stream.

import util from 'util';
import stream from 'stream';
const { Readable } = stream;
const pipeline = util.promisify(stream.pipeline);

export const handler = awslambda.streamifyResponse(async (event, responseStream, _context) => {
  const httpResponseMetadata = {
    statusCode: 200,
    headers: {
      "Content-Type": "text/html",
    }
  };

  responseStream = awslambda.HttpResponseStream.from(responseStream, httpResponseMetadata);
  let requestStream = Readable.from(Buffer.from(new Array(1024 * 1024).join('🚀')));
  await new Promise(r => setTimeout(r, 1000));
  requestStream = Readable.from(Buffer.from(new Array(1024 * 1024).join('⚡️')));
  await new Promise(r => setTimeout(r, 1000));
  requestStream = Readable.from(Buffer.from(new Array(1024 * 1024).join('🚀 Serverless is not dead!')));
  await pipeline(requestStream, responseStream);
});

Let's publish the Lambda function and invoke it via the Lambda Function URL.

IaC to publish Lambda Function URL

While deploying and publishing the Lambda function, keep in mind to use the right Timeout and also FunctionUrlConfig with InvokeMode set as RESPONSE_STREAM.

Resources:
  responseStreamingLambda:
    Type: AWS::Serverless::Function
    Properties:
      Timeout: 20
      Handler: index.handler
      Runtime: nodejs20.x
      Architectures:
        - x86_64
      FunctionUrlConfig:
        AuthType: NONE
        InvokeMode: RESPONSE_STREAM

Response Streaming is the best fit in

It's important to understand when it would be ideal to use Response Streaming and some of the use cases include -

Wrap up!

Lambda functions' Response Streaming is ideal for web applications and monitoring systems where near real-time data is crucial. However, considering the limitations, you need to use Lambda Function URL with NodeJS runtime and be aware of constraints on cost and network bandwidth.

HTTP API invocation from Lambda functions

This process may require using AWS Lambda functions that utilize HTTP libraries (such as request or axios for NodeJS) to construct the request with the appropriate headers and body, ensuring that the request is sent and the response is processed successfully.

• Increased complexity: Introducing the Lambda function to invoke HTTP endpoints adds components that must be managed and maintained.

• Increased latency: Since the request response has to hop off another architectural component, it adds to the latency of the response.

• Increased cost: Running a separate Lambda function to handle the HTTP invocation will incur additional costs, as you'll be charged for the Lambda function execution in addition to the Step Functions execution.

Also, this opens up the flexibility to invoke the HTTP endpoints with whitelisted IP addresses and more security aspects of the Lambda function with Secrets Manager to maintain HTTP credentials.

HTTP API invocation with EventBridge API destinations

In an Event-Driven world, Amazon EventBridge's API destination feature allows you to configure the HTTP endpoint and credentials from Secrets Manager, collectively known as EventBridge Connection.

In a blog about How Freshworks Developer Platform integrates with AWS via EventBridge, I've explained in detail the integration but in this section of the blog, we will focus on the API destination part of the architecture.

########################   API Connection   ############################
MyConnection:
    Type: AWS::Events::Connection
    Properties:
      AuthorizationType: BASIC
      Description: 'My connection with username/password'
      AuthParameters:
        BasicAuthParameters :
          Username : !Ref APIKeyValue
          Password : !Ref Password
########################   API Destination   ############################
MyApiDestination:
    Type: AWS::Events::ApiDestination
    Properties:
      Name: 'FreshdeskAPI'
      ConnectionArn: !GetAtt MyConnection.Arn
      InvocationEndpoint: !Ref FreshdeskURL
      HttpMethod: PUT
      InvocationRateLimitPerSecond: 10
########################   Event Rule to invoke API destination   ############################
EventRule: 
    Type: AWS::Events::Rule
    Properties: 
      Description: "EventRule"
      State: "ENABLED"
      EventBusName: !Ref MyEventBus
      EventPattern: 
        source:
          - "fw-sentiment"       
      Targets: 
        - Arn: !GetAtt MyApiDestination.Arn
          RoleArn: !GetAtt EventBridgeTargetRole.Arn
          Id: "MyAPIdestination"
          InputTransformer:
            InputPathsMap:
              id: $.detail.id
              priority: $.detail.priority
            InputTemplate: >
                {
                  "priority": <priority>
                }
          HttpParameters:
            PathParameterValues:
              - $.detail.id
          DeadLetterConfig:
            Arn: !GetAtt MyDLQueue.Arn

API Destination brings to the table

• Improved Security: API connections use AWS Secrets Manager to securely store and manage authentication credentials, reducing the risk of exposing sensitive information, which is more secure than hardcoding credentials in Lambda functions.

• Reduced complexity: Using API connections with HTTP Tasks in Step Functions eliminates the need to write and maintain Lambda functions solely for making HTTP requests, simplifying the architecture and reducing the amount of code to manage.

• Dependency of EventBridge Bus: Introduces dependency on Event Buses to be able to invoke API destinations. Although it's managed, adding in additional components would mean that you have to define the Event Rules.

HTTP invoke on Step Functions

The above image from Application Composer indicates that when creating a State Machine with HTTPInvoke State, it would also require EventBridge Connection. You can also refer to the SAM template of the State Machine.

  StateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      Definition:
        StartAt: Call third-party API
        States:
          Call third-party API:
            Type: Task
            Resource: arn:aws:states:::http:invoke
            Parameters:
              Method: GET
              ApiEndpoint: ${ApiEndpoint}
              Authentication:
                ConnectionArn: ${EBConnectionARN}
            Retry:
              - ErrorEquals:
                  - States.ALL
                BackoffRate: 2
                IntervalSeconds: 1
                MaxAttempts: 3
                JitterStrategy: FULL
            ResultPath: $.ApiResponse
            End: true
      Logging:
        Level: ALL
        IncludeExecutionData: true
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt StateMachineLogGroup.Arn
      Policies:
        - AWSXrayWriteOnlyAccess
        - Statement:
            - Effect: Allow
              Action:
                - logs:CreateLogDelivery
                - logs:GetLogDelivery
                - logs:UpdateLogDelivery
                - logs:DeleteLogDelivery
                - logs:ListLogDeliveries
                - logs:PutResourcePolicy
                - logs:DescribeResourcePolicies
                - logs:DescribeLogGroups
              Resource: '*'
      Tracing:
        Enabled: true
      Type: STANDARD
      DefinitionSubstitutions:
        ApiEndpoint: !Ref APIEndpoint
        EBConnectionARN: !GetAtt EventBridgeConnection.Arn

Now that the State Machine is defined, let's also define EventBridge Connection along with the authorization credentials.

EventBridgeConnection:
    Type: AWS::Events::Connection
    Properties:
      AuthorizationType: BASIC
      AuthParameters:
        BasicAuthParameters:
          Password: !Ref EventBridgeConnectionPassword
          Username: !Ref EventBridgeConnectionUsername

The resources - StateMachine and EventBridgeConnection are defined but for the StateMachine to successfully invoke the HTTP point, you would need to authorize the State Machine IAM execution policy as below -

- Statement:
    - Effect: Allow
      Action:
          - events:RetrieveConnectionCredentials
      Resource:
          - !GetAtt EventBridgeConnection.Arn
    - Effect: Allow
      Action:
          - secretsmanager:GetSecretValue
          - secretsmanager:DescribeSecret
      Resource:
          - !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:events!connection/*
    - Effect: Allow
      Action:
          - states:InvokeHTTPEndpoint
      Resource:
          - !Sub arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*

Note that, EventBridge Connection under the hood uses AWS Secrets Manager so you would also need to allow GetSecretValue and DescribeSecret.

HTTP Invoke state with it's advantages

Since HTTP invocation on Step Functions uses EventBridge Connection, it brings in the added advantage over invoking HTTP endpoints via Lambda Functions.

• Enhanced execution insights: Step Functions provides you with built-in monitoring and logging capability of the State Machine execution which makes it easier to trace and debug the HTTP requests within the context of the workflow.

• Streamlined workflow management: Directly integrating HTTP requests within Step Functions allows for more streamlined and cohesive workflow definitions. This makes it easier to manage and update workflows without needing to modify and redeploy Lambda functions

• Error retry and handling: Step Functions' error handling techniques give you the flexibility to define the errors and their retry by handling the errors elegantly. Additionally, EventBridge Connections also retries failed requests by default.

HTTP Invoke state with it's disadvantages

• Dependency on EventBridge Connections: The HTTP invoke state depends on EventBridge connections for managing authentication credentials. This adds extra setup steps and dependencies, which can make the process more complicated. Users need to create and manage EventBridge connections and related secrets in AWS Secrets Manager, which can be confusing for some.

• No Support for Private Endpoints: The HTTP invoke state does not support private endpoints within a VPC. This limitation means it can't be used for secure, internal communication, forcing developers to use Lambda functions or other methods to interact with private APIs.

• IAM Permissions Complexity: To use the HTTP invoke state, you need to set up specific IAM permissions for the state machine's role. This includes permissions to make HTTP requests, use the EventBridge connection, and access the connection's secret. Managing these permissions can add complexity and potential security risks if not configured correctly.

• Error debugging: While error retry and handling are supported, error debugging becomes a challenge as HTTP requests could fail for various reasons - timeouts, authentication fail, missing header or parameter that can be hard to debug the error as it's an EventBridge Connection used under the hood.

Wrap up

Yes, orchestrating the HTTP invocations is made possible with Step Functions but it does have its advantages and disadvantages. While building Serverless applications or workflows, the trade-offs are important to consider.

At times, you would need the flexibility to make dynamic HTTP invocations which is possible on the Lambda function rather than via EventBridge Connection. Additionally, it would require you to be careful with the IAM execution policy to allow the needed Secrets.

Orchestration

Just as an orchestration looks, there is a concertmaster or lead who is part of the band would controls how each musician plays their instrument; in the orchestration pattern in Serverless architecture, the orchestrator controls the flow and execution of different tasks and processes based on the workflow that is defined.

Why do you need orchestration?

Orchestration as a pattern promotes Serverless applications to execute in a structured and less chaotic.

• Centralized control - With centralized control and a defined workflow for task flow and execution, orchestration assists in promptly adhering to the workflow definition. Furthermore, error-handling techniques for both the workflow and each task can be integrated into the workflow definition.

• Scalability - In the orchestration pattern, it enables scaling individual tasks within the same workflow, simplifying the scaling of the entire workflow and only the tasks that require scaling up or down. These tasks could include Lambda Functions, ECS tasks, or the implementation of asynchronous tasks.

• Monitoring - The centralized control enhances visibility into individual tasks and the workflow as a whole. This improves monitoring capabilities for each iteration of the execution, including the state and data passed between tasks. This makes debugging easier and provides a better understanding of the workflow's execution.

AWS Step Functions

In the AWS ecosystem, it is AWS Step Functions that enable orchestration of a workflow (State Machine in Step Functions terminology) with the execution of different tasks in the workflow.

Serverless orchestration of workflows

AWS Step Functions helps with different kinds of flows - parallel, map, pass, choice and more which allows the flexibility to define a workflow with a combination of flows and state.

Step Functions is Serverless by nature as there isn't any infrastructure management or provision resources for the execution. Pricing for Step functions is based on the types of workflows offered -

• Standard workflow - for longer-running workflows, which are priced with each state transition.

• Express workflow - for shorter workflow duration where the pricing is based on number of requests for the workflow and it's duration.

The State Machine or the workflow is defined with Amazon States Language (ASL) that is a JSON-based definition. The State Machine could be used in the application's Infrastructure as Code (IaC) definitions.

Step Functions supports error-handling techniques and the JitterStrategy. Additionally, Step Functions integrates with over 1000 API actions from 200+ AWS Services using SDK Integrations and intrinsic functions. These features can enhance integrations and improve how data is accessed and processed with Step Functions.

Choreography

In the world of Event-Driven Architecture (EDA), the choreography pattern works well to ensure that microservices or architectural components are coordinated based on events. It's like a dance choreography where you wait for the right music beats and cues from your dance partner to make the next move.

Why do you need choreography?

In Event-Driven Architecture (EDA), the application relies on real-time events and reactive responses, choreography allows individual microservices to reach independently to the event.

• Loosely-coupled - The microservices and independent services are decoupled to ensure they operate independently, even though they are triggered by events and share events to communicate with other services. Keeping the microservices loosely coupled also enhances fault tolerance. If one microservice goes down, the system remains operational because the other microservices are still running as intended.

• Scalability - Microservices can scale independently according to the need to scale up or down, promoting efficient resource utilization.

• Flexibility - Since microservices can scale based on the need and are loosely coupled, the architecture allows for more flexibility in making updates and rolling out upgrades while the system continues to operate as usual.

Amazon EventBridge

Amazon EventBridge enables Event-Driven Architecture (EDA) in the AWS Serverless with the EventBridge Bus offering making it easier to route events between a source and destination. EventBridge can scale up and down to support the routing of millions of events in a production-grade system.

Here is a blog about why EventBridge is a missing piece to your Serverless application -

Choreographing with EventBridge

• Asynchronous messaging - EventBridge decouples services by allowing them to post events to a shared event bus. This bus then directs the events to specific destinations, enabling the source to post the message and proceed with the rest of the execution.

• Event rules and filters - EventBridge works with event buses that use event rules to determine which events should be sent to which destination. When creating these intelligent rules, filters can be applied to the event payloads for more detailed routing of specific events.

• Archive and replay - EventBridge Bus also supports archiving and replaying events. This feature allows events to be stored for later reproduction, aiding in debugging processes.

• SaaS integrations - EventBridge supports SaaS integrations with AWS partners where the events from the AWS partner's SaaS products can be targeted to a specific SaaS Bus to process the events.

Orchestration v/s Choreography

Best of orchestration and choreography

The choice between Orchestration and Choreography is hard to choose as it depends on the use-case and architectural pros and trade-offs!

One such example is when EventBridge SaaS Event bus can integrate with AWS partners such as Freshworks events choreographed into the AWS account to route the events to trigger a State Machine execution to process the events from Freshworks product - Freshdesk.

Wrap up!

Serverless architecture on AWS includes orchestration and choreography patterns for structured execution. Orchestration offers centralized control, scalability, and monitoring using AWS Step Functions, while choreography enables loosely coupled, event-driven coordination with Amazon EventBridge. Each pattern has its advantages and trade-offs, so the decision between them depends on the specific use case.

In the "Build on Weekly" episode 12 by Darko Mesaroš and Rohini Gaonkar, Orchestration, Dancing, or big blocks of code discusses these patterns of orchestration, choreography, and monoliths, highlighting the benefits of each pattern.

Previously, I had authored why IaC should be the direction for Serverless applications.

IaC and IfC tools

When we talk about Infrastructure as Code (IaC) or Infrastructure from Code (IfC), today there is an overwhelming number of options of tools that focus on either IaC or IfC for Cloud and specifically Serverless.

Infrastructure tools have been the core of different developer workflows and how the development teams leverage these tools. And the different pillars which make your case about a tool stronger are -

Support for services

The coverage of different services the tool has for you as a developer to be able to build applications that can use different services on Cloud providers. This poses a challenge for the tool to be in sync with the Cloud providers.

Integration with CI/CD pipelines

The tool would be the ideal choice when it can integrate well into the CI/CD pipelines to automate the infrastructure provision in different environments either natively or using different plugins.

Developer experience

The Developer Experience (DX) of the tool which makes it easier for developer to adapt and help with a smooth for defining, testing, deploying the resources for your Serverless application. This also relates to how the tool can have a positive impact to the developer productivity.

Learning curve

If the tool introduces a steep learning curve about the tool and it's feature usage making it harder at one or more stages of the developer journey to build Serverless or cloud applications in general.

Readily available patterns

Today, GenAI has taken over different segments of a development process, while the tool can leverage that to enable frequent patterns or completion of different default properties for the resources either with the tool's native capability or as GenAI capability making it easier and increasing time to deploy.

Community and support

The DevTools are driven by a community that can help with unblocking a developer and also enabling with latest happenings and best practices. It makes it an easier choice for developers to try and also adapt when there is support and peer contributions to improvise the tool.

Your voice about I a/f C tools

I've been evaluating several IaC and IfC tools in the last few months, and to understand the love for these tools, I ran polls on LinkedIn, Twitter, and also my Newsletter.

Look at the poll results -

Terraform is widely adapted given that it is adapted for cloud applications in general, and the swift support with declarative IaC with Serverless services too and the fact that Terraform is already cloud agnostic makes the adoption much easier.

One thing that stood out was specifically how Serverless Framework and CDK are the popular choice for infrastructure for Serverless applications. Some of the key callouts were of declarative YAML on Serverless Framework and programming constructs with CDK.

In this blog series, I will look into different IaC and IfC tools that enable you to build Serverless applications with a much better experience than what it was a few years back, so tune in!

Learn about Serverless I a/f C DevTools

Personally, have been a big AWS SAM user so here are a few blogs about SAM -

And about how Serverless workloads also works with more configurations rather than complex application code.

In this blog, we will look at the importance of IaC and how tools like Application Composer and Workflow Studio enable it with the low-code approach of drag-and-drop of components with the comfort of VS Code.

Infrastructure as Code (IaC)

When building applications on the cloud, irrespective of Serverless, Containers, or Virtual Machines; it is recommended to use the Infrastructure as Code (IaC) approach so that provisioning and deploying resources to the cloud would be automated via workflows or with IaC tools that refers to a single source of truth where all the configurations of all the needed resources are defined.

You can read more about why IaC should be the direction for building applications.

Workflow Studio

AWS Step Functions launched Workflow Studio in 2021, where the new low code tool has been revolutionary for drag and drop for over 1000+ API Actions across multiple AWS Services.

Workflow Studio enables visual building and understanding of the workflow execution with different AWS Services API actions invocation via optimized or SDK integration. As part of the integration, you can define the API parameters and share data across different States.

Workflow Studio generates the Amazon States Language (ASL) in real-time based on the States defined visually with all the parameters. Workflow Studio also facilitates different configurations such as error handling, input, and output for different states and also defines the flow with parallel, map for loops, and choice for branching.

Application Composer

AWS Application Composer is a visual IaC builder for making drag-and-drop designs of different AWS Services on the canvas to generate an equivalent Infrastructure as Code (IaC) template with SAM (Serverless Application Model) Template using YAML. The experience of designing applications with AWS Services that generate SAM templates to synchronize when using the Google Chrome browser locally was the first step to making IaC generation and a real-time sync to the local file system.

Application Composer initially supported a few AWS Serverless services and now it supports over 1000 CloudFormation supported resources. During AWS re:Invent 2023, Application Composer announced the IDE (VS Code) experience of building IaC as part of AWS Toolkit where you can use Application Composer to design your architectures while the SAM template would be generated in your VS Code directory.

Now you are also able to use AWS Step Functions' Workflow Studio in the Application Composer VS Code experience as the integration enables using Workflow Studio that generates Amazon States Language (ASL) in real-time which synchronizes either with SAM template or as a asl.json or asl.yaml files in the local project directory.

Application Composer + Workflow Studio on VS Code

Install the latest VS Code extension of AWS Toolkit that includes Application Composer with Workflow Studio.

In an empty project directory, create a template.yaml file and click on "Application Composer" icon on the top right that launches Application Composer in VS Code.

Creating a State Machine with an external ASL file

When you drag-and-drop StateMachine resource, it generates the boilerplate State Machine with Lambda task, and the equivalent changes are made to template.yaml with the State Machine resource. Since the option of an external file for State Machine ASL was selected, statemachine.asl.json file is generated with the ASL definition of the Lambda task.

Defining State Machine using Workflow Studio

In the Application Composer, the StateMachine resource which has an option to launch Workflow Studio locally on VS Code.

Bedrock's InvokeModel task

At AWS re:Invent 2023, Step Functions launched the support for optimized integration for Amazon Bedrock that enables the state machines to invoke Bedrock APIs.

From the list of supported resources on Workflow Studio, choose Bedrock's InvokeModel API Action that requires you to define the LLM model used on Bedrock along with the input.

Below is the generated ASL from Workflow Studio.

"Bedrock InvokeModel": {
    "Type": "Task",
    "Resource": "arn:aws:states:::bedrock:invokeModel",
    "Parameters": {
        "ModelId": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-v2:1",
            "Body": {
                "prompt": "Human:Tell me a fun story about MARVEL characters using Infrastructure as Code to build Serverless Applications\nAssistant:",
                "max_tokens_to_sample": 2000
            }
        },
        "ResultPath": "$.response"
}

Adding a DynamoDB PutItem task

Now that Bedrock would generate a story, it has to be stored on DynamoDB. To do so, on Workflow Studio add PutItem API action to the state machine definition where the table name is passed with CloudFormation substitution and story from the previous task's response. Also, using STATES.UUID() intrinsic function to auto-generate UUIDs.

"DynamoDB PutItem": {
    "Type": "Task",
    "Resource": "arn:aws:states:::aws-sdk:dynamodb:putItem",
    "Parameters": {
        "TableName": "${MyDynamoDBTable}",
            "Item": {
                "id": {
                    "S": "STATES.UUID()"
                },
                "story": {
                    "S.$": "$.response.Body.completion"
                }
            }
        }
}

Defining DynamoDB resource

In the Workflow Studio, the DynamoDB's PutItem action would need the table to be created, and to do so, on Application Composer drag-and-drop DynamoDB and define the table structure.

Once the DynamoDB table is defined, update the CloudFormation references for the State Machine to use the table that is defined in the SAM template.

Application Composer can detect the different resources used in the SAM project and provides all the resource options when using !Ref.

Updating IAM execution role for State Machine

When StateMachine is created, Application Composer adds the policies AWSXrayWriteOnlyAccess and CloudWatch logs permissions by default but even though State Machine is defined; Step Function doesn't auto-generate the needed IAM execution role and we would have to update the IAM policy with access to DynamoDB PutItem and Bedrock InvokeModel APIs.

Policies:
        - AWSXrayWriteOnlyAccess
        - Statement:
            - Effect: Allow
              Action:
                - logs:CreateLogDelivery
                - logs:GetLogDelivery
                - logs:UpdateLogDelivery
                - logs:DeleteLogDelivery
                - logs:ListLogDeliveries
                - logs:PutResourcePolicy
                - logs:DescribeResourcePolicies
                - logs:DescribeLogGroups
              Resource: '*'
        - Statement:
            - Effect: Allow
              Action:
                - dynamodb:PutItem
              Resource: !GetAtt Table.Arn
        - Statement:
            - Effect: Allow
              Action:
                - bedrock:InvokeModel
              Resource: arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-v2:1

Deploy using SAM sync

Now that the SAM template is complete with Application Composer and Workflow Studio, use sam sync --stack-name "<your-stack-name>" to deploy the resources to your AWS Account.

It's deployed!

Now that the State Machine is deployed, time to start the execution to see what story about "Marvel heroes using IaC for Serverless" is generated.

Check out Arshad Zackeriya and me talking about Application Composer on The Zacs' Show Talking AWS.

Amazon PartyRock was launched a few days before re:Invent and guess what? I created an app to suggest a blog title with a few descriptions and the title of this blog is generated with PartyRock!

The theme of re:Invent was around Generative AI and Dr. Werner Vogels' keynote which emphasised cost and sustainability while being Frugal Architects. I remember how reInvent 2022 was about making everything "Serverless", just a year later we are still on track to lose the meaning of "Serverless" and making it all about "GenAI".

Everyone talking about "Generative AI" - Image generated using Amazon PartyRock and Amazon Bedrock with Stability AI.

Keynote(s)

Adam Selipsky's keynote had an announcement about a new AWS Service - Amazon Q which helps developers during development and also companion-based developer experience of using AWS Services.

Dr. Werner Vogels' keynote as I mentioned emphasized being "The Frugal Architect" with designing and architecting our applications and workloads on AWS with known pillars of The Well Architected Framework but also including cost-centric and sustainability-centric architects which go a long way!

Werner's The Frugal Architect states laws that enable architects to be cost-efficient in the lens of design, measure, and optimize. As developers, architects, and techies who are building on the cloud, we must have a sense of cost and sustainability. It's time, we take a step back and evaluate our architectures so that we are building for tomorrow not just for today.

Werner's keynote announced the new myApplications which is a new Console experience with consolidated costs, health, and performance of our applications. Application Composer gets a new makeover with an IDE extension that not only gets drag-and-drop experience but also adds on the Step Functions' workflow studio experience (hopefully, I'll get to try things around soon and more blogs around it coming up). And also, Amazon Q as part of the IDE.

As an AWS Hero, being able to get the front-row seats to witness the keynotes was one of the best experiences. A special shoutout to Heroes by Werner just before he started the keynote was a special moment!

AWS Service updates and announcements

If you are actively following updates and announcements during pre:Invent season, there were major ones concerning improving the operational excellence of Serverless applications to better monitoring and observability in the world of distributed.

ICYMI, The Serverless Terminal issue - AWS Lambda 9 years of Serverless AWSomeness #43 covers the pre:Invent announcements with Lambda updates making the troubleshooting experience better, CloudWatch Logs supporting regex filter pattern.

Dude, I've literally searched logs for specific keywords/patterns and the search on console/CLI has got on my nerves.

Troubleshooting and debugging with searching logs like a needle in a haystack- Image generated using Amazon PartyRock and Amazon Bedrock with Stability AI

I remember years ago, my first-ever interaction with Eric Johnson (Serverless DA, AWS) was about how to migrate my rookie Serverless application that I built on console to IaC, never have been a better way other than export as SAM template but now you can export that into Application Composer which is much smoother and simpler. Just that after that poor console management experience, I advocated for the IaC approach for building Serverless applications. Good to see that it's all falling into place.

While the week of re:Invent was hectic enough, I also challenged myself to publish The Serverless Terminal issue - We've always done it this way #44 that calls out the different updates from Lambda new runtime supports to Step Functions making its way with HTTPs invocation make it much easier for integration options for third party APIs, how can I forget Bedrock integration on Step Functions!? With all that integration, if the state machine fails; there is a way to restart the workflow whenever it fails (no, it's not the entire workflow rather from the point of failure).

Strong opinion about what Serverless is, because many of the Server-full services have termed some of it's offerings as "Serverless" even when you have flat pricing for non-usage. Are we losing out on the real Serverless features and services in this journey?

Connecting with Cloud/Serverless Minds

This is my first re:Invent and I'm meeting the majority of them for the first time. You ask, whom did I meet? AWS Heroes, AWS Community Builders, AWS UG Leaders, AWS folks who are listening to me whine and also patiently giving an ear to my queries and helping - AWS Developer Relations team, AWS Service teams - PMs, Engineers; the list does not end.

The conversations around what they are building with Serverless to unlocking the potential of Serverless - left me with the thought - "wow that is possible!!?" Well, I've seen so many of them building content around cloud and Serverless but in-person connection is hard to put down in words.

Imposter Syndrome is real - having such great conversations although many of them were for a few minutes and how we have known each other over the internet but everyone is so welcoming and for a first-timer, I bet most of them gave me tips on how to survive "the week".

I finally met a fellow Bangalorian - Praneeta Prakash (AWS Modern App Experience PM) her Twitter bio says Bengaluru hudugi and we conversed in my native language Kannada. Check out the tweet.

Not just meeting Serverless minds and having a conversation, did something different with Luciano Mammino (AWS Serverless Hero) who co-runs AWS Bites podcast where we took our conversation from in-person to record it with all the noise at the Expo. Check out the podcast featuring some brilliant folks!

AWS Sessions

Something that I'm not worthy to talk about 'cause I didn't attend them! Sessions of different categories and levels happened all across the Vegas Strip in multiple Hotels and conference areas although I was in Venetian; I was not able to make it to the sessions. However, I did catch up with many speakers whose sessions I had booked but didn't make it to, and spoke about mostly things around their session topics.

Guess what? All recorded sessions during re:Invent are now available on demand and I created a Twitter thread to keep a check on what sessions to watch before 2023 concludes.

Community Panel discussion

Although this being my first re:Invent, I was able to represent AWS UG Bengaluru, being AWS Hero at the community panel discussion that happened in the DevLounge area of the Expo along with Kaushik, Lena, and Shafraz.

We all shared our views of being part of the AWS Community and programs - AWS User Groups, AWS Community Builders, and AWS Heroes.

I guess I can say - "I did speak at re:Invent"

ServerlessLand Video

Eric Johnson (Serverless DA, AWS) and I had a chat on ServerlessLand Video talking about my Serverless journey and my role as a DA at Freshworks.
Check out the video.

Glorious moment for Freshworks

Freshworks was one of the sponsors for AWS re:Invent 2023 and we had our own wins be with sessions about how Freshworks is innovating with Generative AI to having Adam Selipsky (CEO, AWS) visit us at the booth and also gave a shoutout in his social media posts.

Check out Adam's LinkedIn post for more details.

What's next?

I'm still re:Covering from re:Invent but being able to connect with community folks not only as people over the internet, but now with human connection. I'd look forward to collaborating with many (I wish it's all of them) for many things with AWS UG Bengaluru, The Serverless Terminal, and The Zacs' Show Talking AWS.

To wrap up, I'd quote and agree with Werner -

There has never been a better time to be a BUILDER

With that, my first-ever in-person re:Invent is in the books!! Big cheers to Farrah Campbell (Head of Modern Compute Community, AWS), Taylor Jacobson (AWS Heroes PM, AWS), AWS Developer Relations team, AWS Community and Freshworks for making re:Invent memorable.

P.S. I planned to make this a tech blog and while I jotted down my initial thoughts, I felt that my experience of re:Invent is worthy to be shared rather than just a tech / cloud / Serverless takeaway from re:Invent.

Read more about how error handling works on Step Functions but in this blog, we will focus more on the new parameters in error handling with catch and retries on Step Functions such as MaxDelaySeconds and JitterStrategy.

Errors in Step Functions

During the execution of a State Machine, there are possibilities of execution would be interrupted by various errors such as States.Timeout when the task execution has taken more than TimeoutSeconds defined as when it failed to get a heartbeat longer than HeartbeatSeconds defined.

Some of the possible errors are listed below.

States.All wildcard is available on Step Functions to work with all the errors encountered during the execution.

Deploying a State Machine

Navigate to AWS Step Functions Create state machine and then select the Orchestrate Lambda Functions template with the new console experience.

Once the template is selected, Workflow Studio will give you more details of the template.

The Orchestrate Lambda Functions template showcases the stock buy/sell recommendation based on the stock price. Choose Run a demo option to deploy the state machine and other resources such as Lambda Functions with SNS and SQS to your AWS Account.

To test out the state machine, you could run a sample execution.

Updating the State Machine with error retries

Enabling retry

Let's update the Check Stock Price state which invokes a Lambda function with an error retry with a few retry options

{
    "Retry": [
        {
          "ErrorEquals": [
            "States.ALL"
          ],
          "BackoffRate": 2,
          "IntervalSeconds": 1,
          "MaxAttempts": 3,
          "Comment": "Check Stock Price Lambda error",
          "MaxDelaySeconds": 2
        }
}

In this error retry snippet, the wildcard States.ALL error listens to all the errors in this state to perform a retry. This retry has a few other options -

• IntervalSeconds is an integer that specifies the number of seconds before the first retry.

• BackoffRate which is a property that would multiply the IntervalSeconds property to determine the next retry would occur.

• MaxAttempts defines the maximum number of retries possible.

• MaxDelaySeconds defines the maximum time in seconds that the retry interval can increase.

When the state machine is executed with an error retry in the first state,

Enabling retry with JitterStrategy

In the second Generate Buy/Sell recommendation state, enabling retry for all States.ALL wildcard with the below retry options -

{
    "Retry": [
        {
          "ErrorEquals": [
            "States.ALL"
          ],
          "JitterStrategy": "FULL",
          "Comment": "Buy/Sell recommendation error",
          "MaxAttempts": 5
        }
      ]
}

Along with the previously enabled options, the additional property set is - JitterStrategy as FULL. When JitterStrategy is enabled with the value FULL, it randomizes delay intervals so that the retry mechanism doesn't retry excessively, this is very powerful especially when Lambda is invoked concurrently.

When the error occurs in the Generate Buy/Sell recommendation state,

Notice the retry attempts are between 1s and 5s (MaxAttempts) which are random when compared to the previous case without JitterStrategy.

Enabling Fail flow

Whenever failing the state machine, it throws an error and the cause that can customize the error states. But for this flow, enabling a custom error message defined in the state machine for the choice state.

{
    "Fail": {
      "Type": "Fail",
      "Error": "Stock prediction failed",
      "Cause": "Unable to proceed as stock prediction to buy or sell stock failed"
    }
}

When the choice state goes through a default state, the state machine fails execution.

When the error occurs, the fail flow captures the error as Stock prediction failed with the cause Unable to proceed as stock prediction to buy or sell stock failed.

Keep a watch on retries

When working with state machines that have multiple states with retries defined in all or many of them, the retry mechanism would retry until MaxAttempts.

Q: It's good to retry multiple times

Well, not if the computing brains or the state would reproduce the same error. However, definitely good when retrying would result in success.

Q: Multiple retries shouldn't cause interruptions in my state machine execution

If the retries result in a success, it won't interrupt the state machine execution. But if the same error occurs, better to fail the execution with passing events to EventBridge or DLQ.

Q: Would this be expensive?

In a Standard workflow, state machine execution is priced based on every state transition and when retry is configured there would be a state transition for each retry attempt this would be expensive.

Also, the retry attempt is invoking another AWS resource, that execution would also be billed. In this workflow, the Lambda function was invoked multiple times which also accounts for the total workflow cost.

Wrap up!

The retry mechanism would help when working with resource-based errors such as Lambda.ServiceException or any other AWS Service error based on the task. With JitterStrategy enabled, addresses the concurrent retries.

Note, this example was to showcase the features with error retry and fail flow.

Also, if you are wondering about Amazon SQS, learn about getting started with SQS and SNS and understand Standard and FIFO queues (in SQS) and topics (in SNS) with the understanding of when to use SQS and SNS in your Serverless architectures.

In this blog, we will look into Amazon EventBridge buses and Amazon SQS queues and how and where they fit right into your Serverless architectures.

Catch the bus!

Building event-driven architectures

When building EDA applications, Amazon EventBridge is best suited, as EventBridge provides you with different capabilities for service-to-service integrations across various sources in the Serverless space. Also ensuring the architecture is loosely coupled.

Event routing and filtering

EventBridge is known for its smart routing feature with event rules and also the support of filtering based on different filters and filter patterns. When building for service-to-service integrations, the routing helps with designated events routed to destined destinations.

Integrations

EventBridge supports different service integrations such as AWS Lambda, AWS Step Functions, SQS, SNS, and many more. While these are natively supported service integrations, EventBridge also supports SaaS Partner integrations and third-party HTTP end-points with API destinations.

Get in the queue!

Building message queues

When building applications, you may have to build messaging queues that need reliable and ordered delivery. SQS ensures this with FIFO queues and also enables load balancing with horizontal scaling.

Fan out patterns

Messages which have to be distributed across multiple consumers would efficiently process each message by each consumer. Fan-out patterns are best suited with Amazon SQS as it enables a balancing queue for the consumer.

Time-bound delivery

Amazon SQS has capabilities where messages could be configured to deliver the message to the consumer using the DelaySeconds parameter. Also, the messages can hold a time-to-live (TTL) attribute beyond which the message is removed from the queue. SQS supports visibility timeouts where the message is invisible and can be reversed for consumption later.

EventBridge v/s Queue

Co-existence of EventBridge and SQS

While individually EventBridge and SQS can add a lot of value to the Serverless architecture, the combination of the two in certain patterns is possible.

Messaging queues with event-driven patterns where both SQS and EventBridge can be used in combination. EventBridge in this pattern would broadcast events to multiple subscribers and SQS would be enforcing correct message ordering.

Messages in SQS could be consumed by EventBridge pipes for better event enrichment and transformation of events to the needed destination without having Lambda functions to be performing a transformational workload on messages.

Ultimately, it is about what the workload is and how each of the services - EventBridge or SQS either individually or in combination can elevate your Serverless architecture.

AI-generated image by Adobe Firefly - "Heavy traffic in Serverless cloud"

In this blog, we will understand what are the different sources for Lambda triggers and also understand the factors to ensure performance during peak traffic.

Lambda trigger sources

AWS Lambda function is well-knit with different sources triggering them either on demand or event-based. Understanding the source of Lambda function triggers would help in addressing performance during high traffic.

Invoked via HTTP API

Lambda functions would be the core compute of the backend systems which are served over API endpoints generated from Amazon API Gateway or AWS AppSync or Lambda Function URLs. These are often triggered by applications programmatically either from a front-end application or backend systems.

AWS Services invoking Lambda

In event-driven architectures, another AWS Service usually triggers the Lambda function whenever certain events occur.

A typical scenario would be, whenever there is a new object uploaded into a S3 bucket, a Lambda function could be triggered with the details of the event payload so that Lambda can further process it.

Performance with heavy traffic

When using Lambda functions, being mindful of what it offers and how best it fits into your solutions is essential.

AI-generated image by Adobe Firefly - "busy day in a traffic cop life managing traffic".

Load balancers

Load balancers, even from the traditional systems are that piece of architecture that enable smooth traffic distribution. In a serverless system also, load balances help with distributing traffic and also ensuring there isn't a burst of invocations of Lambda functions.

Enable caching

For frequently accessed data, enabling caching in your architecture at different levels would improve performance which on a high-traffic day pays off!

Allen and me (Jones) authored a blog with Momento about how caching at every layer improves app performance.

Provision and reserved concurrency

With Lambda's provision and reserved concurrency, you can allocate the resources which would be needed when you anticipate a high traffic flow. This ensures the architecture would be able to perform better on a peak traffic day.

Weighted routing

Each lambda function is assigned weights and the weighted routing would use these weights as the measure to distribute traffic amongst different Lambda functions and in the case of Lambda aliases, it helps with distributing traffic amongst different versions of the Lambda function.

Triggers with batch

When defining the event triggers for Lambda functions, different services like DynamoDB, SQS, and other messaging services support batch triggers where the service would collectively trigger the Lambda function once as per the BatchSize defined.

Options for Serverless Architecture

Application Load Balancers (ALBs)

• ALBs support targets as Lambda functions which help in distributing traffic with weights, geographical and round robin way.

• Provides endpoints available over the public internet which helps in building APIs that can be backed with ALBs and Lambda functions.

• ALBs are complex to configure when compared with other options available.

• With the management and configurations, this is not truly a serverless way!

Amazon API Gateway

• Available by design to configure API Gateway integration with Lambda functions. And also seamless to work with API Gateway when compared with ALBs.

• API Gateway enables the usage of API key based usage quotas and throttling controls.

• Supports multiple authentication and authorization options.

• Supports payload validation with models where in a peak traffic scenario, API Gateway will ensure the validated payloads are invoking the Lambda function thus eliminating junk requests.

• Supports WAF and CloudFront-based endpoints which has added security for DDoS attacks.

Lambda aliases

• Traffic distribution at the Lambda level when using it with different AWS Services as triggers.

• Configuration involves creating multiple versions of the same Lambda function and assigning an alias to one or max two versions.

• Supports weighted routing to each alias in a Lambda function.

• Not the best fit for Lambda functions which need to be invoked via the public internet as it lacks authentication and throttling.

Wrap up!

Lambda functions have proven scalability and perform well even during peak traffic. Keeping in mind different aspects of Lambda functions or other AWS Services that enrich your traffic management is the key when architecting solutions that are prone to high traffic.

With the different options of ALBs, API Gateway, or even Lambda aliases, each of them has an upside and downside depending on the use case and how they are placed in the serverless architecture.

Initial architecture of Prime Video

Few pointers from the tech blog about the initial architecture,

• Distributed and microservices approach in their initial architecture.

• A serverless architecture primarily built on AWS Lambda functions and AWS Step Functions.

• Designed with the intent of Serverless being cost-effective and scalable because it was on-demand scaling up or scaling down of Lambda functions which resulted in keeping cost intact.

Image courtesy: Prime Video blog post - https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

Reasons why Serverless was a challenge

Prime Video is one of the popular OTT platforms, and the assumption here is that they would have peak traffic and the workload in this traffic times is what the architecture to hard test in terms of -

Scalability

Yes, Serverless applications can scale well. But the main concern of scalability came in with orchestrating the Serverless workload with AWS Step Functions.

The main scaling bottleneck in the architecture was the orchestration management that was implemented using AWS Step Functions. Our service performed multiple state transitions for every second of the stream, so we quickly reached account limits.

As quoted in the blog post. Often when building Serverless applications when they hit a peak moment, there are challenges -

• Managing the distributed architecture because you have too many microservices doing various dedicated tasks even though it's a State Machine defined.

• AWS Step Functions bring in limits in payload passed across states, the number of state machine executions where there should be workarounds in place to ensure the system doesn't breach the limits.

Cost-effectiveness

Serverless applications are cost-effective because they run on demand and when certain events/requests are made.

Besides that, AWS Step Functions charges users per state transition.

AWS Step Functions have a per-state transition pricing model, for instance, in us-east-1 region that has $0.025 per 1000 state transitions, Prime Video would have a large number of state transitions happening as this state machine itself is used to detect if any defect is there in the video frames.

The second cost problem we discovered was about the way we were passing video frames (images) around different components. To reduce computationally expensive video conversion jobs, we built a microservice that splits videos into frames and temporarily uploads images to an Amazon Simple Storage Service (Amazon S3) bucket. Defect detectors (where each of them also runs as a separate microservice) then download images and processed it concurrently using AWS Lambda. However, the high number of Tier-1 calls to the S3 bucket was expensive.

The microservice which is used to split videos in frames uploads images to an S3 bucket and these images are processed to detect defects where there would be 1000s of frames in a single video which itself would have S3 bucket costs high with respect to storage, data transfer and the cost of high-performance Lambda functions which run in parallel.

Reaction to the challenges

The reasons mentioned are very valid and the Prime Video team has taken some steps to address this -

Re-think

It is important when you have scalability and cost-related concerns with your architectures that you start to take a step back and think - "what could have been the possible best solution?" or "what kind of components suit this use-case". Sometimes, it begins with the basic question of "why do we need XYZ service" when the challenges stand out.

The first approach to adapting Serverless is good but when the architecture is not meeting the need in this use-case, the approach taken to re-think says how flexible we have to be in our architecture.

Re-architect

When we re-think, it is obvious that we get more clarity with what is the use case and what architectural approach and components would be the best fit. Once we identify the best fit architectural approach or components, re-architecting the complete workload would be the way to go.

In Prime Video's case, there was a shift from Serverless to a monolith because the system would always be processing images, and data transfer between different Services would make it an expensive workload but with the new architecture, the ECS task performs the orchestration of detecting deflects using the video buffer directly from Media Converter.

Resonating my thoughts

When the internet broke out with Prime Video moving away from Serverless, me being a Serverless advocate was shocked "why"!

But reading and understanding the blog post, appreciate the architectural shift even when it means that Serverless architecture is revamped into a monolith.

Serverless first, Serverless not always

It's good to approach with the Serverless mindset and building with Serverless but if that doesn't work for your workload then "revert! revert! revert!". Sometimes, Serverless may never be the right approach, and investing in that would mean that it would turn out to be expensive in the longer run.

One size doesn't fit all

We assume that if any approach works for someone out there, it would work for me as well. As it's repeatedly mentioned, "Serverless scales and is cost-effective" it is true but also we need to understand if the Serverless approach or any other approach in general would be the best fit for the use case.

Iterative building

Oftentimes, the use-case evolves or the usage evolves which translates to the architecture should be adaptive and get there it's always with different iterations. Take software development as the instance where we go through different revisions and iterations to get to production and if we see the architecture of the software being developed, that too would have gone through revisions. These revisions are where the learning happens, to identify if the architecture is well-architected or not.

One query that stands out is that -

AppSync subscriptions work amazing when client makes mutations but I have some downstreaming patterns in my application where the backend uses subscriptions to send data from back-end to front-end, how do I achieve this with AppSync?

Ask ChatGPT

Well, this does look simple so why not ask ChatGPT?

What would be the better way to handle AppSync Subscriptions which are triggered with Lambda functions?

ChatGPT's solution

Use the appropriate trigger:

To use the appropriate trigger, you can create a trigger on the event that is relevant to your use case. For example, if you want to trigger a subscription when a record is added to a database, you can use a database trigger to invoke your Lambda function.

Here's an example using AWS Lambda and Amazon DynamoDB:

• Create a DynamoDB table with a stream enabled.

• Create a Lambda function that is triggered by the DynamoDB stream.

• In the Lambda function, write code that parses the stream event and invokes the AppSync subscription.

And also, ChatGPT generated the code snippet for this.

// Sample code for a Lambda function that is triggered by a DynamoDB stream

exports.handler = async (event) => {
  // Parse the DynamoDB stream event
  const records = event.Records.map((record) => {
    return {
      eventName: record.eventName,
      record: record.dynamodb.NewImage,
    };
  });

  // Invoke the AppSync subscription
  const appSyncClient = new AWSAppSyncClient({
    url: 'https://my-appsync-endpoint.aws.com/graphql',
    region: 'us-east-1',
    auth: {
      type: 'AWS_IAM',
      credentials: AWS.config.credentials,
    },
  });

  const mutation = gql`
    subscription OnRecordAdded {
      onRecordAdded {
        id
        name
      }
    }
  `;

  const observable = appSyncClient.subscribe({ query: mutation });
  observable.subscribe((data) => console.log('Subscription data', data));
};

And the GraphQL schema for this was also generated by AppSync.

type Record {
  id: ID!
  name: String!
}

type Subscription {
  onRecordAdded: Record!
}

type Mutation {
  addRecord(name: String!): Record!
}

type Query {
  getRecord(id: ID!): Record
}

Let's validate ChatGPT 's response

Do you notice something from ChatGPT's response? Now let's put on the Serverless architect lens and see how this would be implemented.

GraphQL schema

The schema generated by ChatGPT is a generic one that looks good at first look but when from the application's perspective, all the operations - query, mutation and subscription are accessible based on the default authorization set on AppSync.

Ensure the subscription is defined for a specific mutation, this is to be noted when you have multiple mutations. Although this looks good in this single mutation schema however it's good to define them with @aws_subscribe directive.

type Subscription {
  onRecordAdded: Record
  @aws_subscribe(mutations: ["addRecord"])
}

Irrespective of what authorization is used in your application for the client (be it Cognito User Pool, API Key, Lambda authorizers or AWS IAM role), define the mutation which would be invoked from the Lambda function as AWS IAM role.

type Mutation {
  addRecord(name: String!): Record!
  @aws_iam
}

GraphQL resolvers

The conversation with ChatGPT didn't initially give me a resolver for the mutation addRecord, but a follow-up query did generate a VTL resolver (understand JS resolver is a very new, and fair guess that model doesn't know what JS resolver is at this point).

This VTL resolver defines the PutItem operation for the Mutation type in the AppSync schema. It uses the $ctx.args variable to extract the name argument from the mutation, and then generates a new id for the item using the $util.autoId() function. The resolver constructs a DynamoDB PutItem request using these values, and then returns the request in JSON format.

#set($name = $ctx.args.name)
#set($id = $util.autoId())
{
    "version": "2018-05-29",
    "operation": "PutItem",
    "tableName": "my-table",
    "item": {
        "id": { "S": "$id" },
        "name": { "S": "$name" }
    }
}

You may notice that the VTL resolver generated is doing a DynamoDB PutItem action even though the response ChatGPT gave didn't have the schema treating the Record type as a model or even give the steps to create a DynamoDB data source.

For the simplicity of things, let's create a simple VTL resolver that generates an ID and returns the name with the data source being a None type.

Request mapping template

{
    "version": "2017-02-28",
    "payload": {
        "name": "$context.arguments.name",
        "id": "$util.autoId()"
    }
}

Response mapping template

$util.toJson($context.result)

And the mutation addRecord is ready to be tested!

Lambda function to trigger subscriptions on client-end

The solution that a Serverless architect would have come up with is the one above, where any Lambda trigger could invoke the Lambda function, and the Lambda function in turn uses an AppSync mutation and on the successful execution of the mutation, the subscription is internally invoked by AppSync.

The Lambda function generated by ChatGPT uses AppSync Client SDK. And it subscribes to an AppSync subscription rather than a mutation. 🤔 Why! 😵‍💫

const observable = appSyncClient.subscribe({ query: mutation });
observable.subscribe((data) => console.log('Subscription data', data));

Firstly, create a Lambda layer with the npm dependencies -

npm install aws-sdk aws-appsync graphql-tag isomorphic-fetch axios

Create your Lambda function with the environment variable set with the AppSync API endpoint.

require('isomorphic-fetch');
const AWS = require('aws-sdk/global');
const AUTH_TYPE = require('aws-appsync').AUTH_TYPE;
const AWSAppSyncClient = require('aws-appsync').default;
const gql = require('graphql-tag');

const config = {
    url: process.env.APPSYNC_ENDPOINT,
    region: process.env.AWS_REGION,
    auth: {
        type: AUTH_TYPE.AWS_IAM,
        credentials: AWS.config.credentials,
    },
    disableOffline: true
};

const createTodo = gql`
    mutation MyMutation($name: String!) {
      addRecord(name: $name) {
        id
        name
      }
    }
`;

const client = new AWSAppSyncClient(config);

exports.handler = async function (event) {
    console.log("event ", event);
    try {
        const result = await client.mutate({
            mutation: createTodo,
            variables: {
                "name":event.name
            }
        });
        console.log("result ", result);
        return result
    } catch (error) {
        console.log("error ", error);
        return error
    }
};

credentials: AWS.config.credentials uses the IAM credentials from Lambda's runtime and for this to work, you would have to define

Update your Lambda's execution role with the policy -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "appsync:GraphQL",
            "Resource": "<Your AppSync ARN>"
        }
    ]
}

Back to your Lambda console, create an Event JSON.

{
  "name": "This is a record from Lambda fn"
}

When testing the subscription from the AppSync console. The API by default uses API Key as authorization.

Behind the scene, the event was tested from the Lambda function console and here the mutation is authorized by AWS IAM.

Wrap up

ChatGPT gave us a high-level solution in terms of what needs to be done but the how to execute and the minor details were missed out, this is exactly where Serverless architects would come into the picture to start building on top of what AI suggests.

The TL;DR of this is that you can start to get solutions from ChatGPT but the accuracy of how well it solves is something humans would have to intervene on. And building Lambda functions that invokes an AppSync mutation to trigger subscriptions would be helpful when you are relying on real-time sync from external sources other than the support data sources from AppSync.

AWS Amplify Studio amplifies your development with front-end and back-end by provisioning the right AWS resources under the hood for the right purposes and also the ability to easily integrate into the design.

Working with data

Amplify Studio makes it really easy to define and manage your data irrespective of the environment it's used in.

Defining your data

Amplify Studio's Data allows you to define Amazon DynamoDB keeping in mind the properties with the right type and also powered with AWS AppSync where which generates a GraphQL schema under the hood.

enum TopicLevelEnum {
  INTRODUCTORY
  INTERMEDIATE
  ADVANCE
}

type InfoCard @model @auth(rules: [{allow: public}]) {
  id: ID!
  title: String!
  short_description: String!
  long_description: String!
  thumbnail_url: AWSURL!
  categories: [String!]
  services: [String!]
  topic_level: TopicLevelEnum
  aws_docs_url: String!
  aws_workshow_url: String
  community_url: String
  aws_samples_url: String
  dislikes: Int
  url: String
  s3_thumbnail: String
  likes: Int
}

type Service @model @auth(rules: [{allow: public}]) {
  id: ID!
  service_name: String!
  category_name: String!
  logo_url: AWSURL!
  url: String
  s3_logo: String
}

Now that data is defined and the AWS resources - AppSync API and DynamoDB tables for the same are created with @model directives by AppSync. Since the API is using public read, @auth(rules: [{allow: public}]) is part InfoCard and Service this will ensure that the auto-generated CRUD APIs for them will allow public access.

Generating data

The content management feature of Amplify Studio is elegant for ensuring an admin/content creator role to create or update content that the front-end application would be consuming. This uses the data definition based on your GraphQL schema and DynamoDB schema with a form-based input.

Amplify Studio also has an underrated feature that developers would love when using this during the course of development, Amplify Studio has auto-generate data which is handy for the development phase where developers need not worry about "there isn't enough data to display on the front-end" instead generate up to 100 records with the feasibility of setting some constraints so that you get valid data.

Test your data

With the data populated, you would need to ensure the GraphQL APIs with AppSync are working as expected. Amplify Studio also features a GraphQL editor to simulate and test the APIs.

The simulator provides the API docs - where you can navigate the different queries, mutations or subscription.

Amplify Storage

Amplify Storage let's you manage the media files and assets which are stored on Amazon S3. Amplify Storage will first create the Amazon S3 bucket where you can set the authorization rules for user to upload, view or delete the objects (files in this case) on S3 bucket.

Managing files on Storage

From Amplify Storage, you can navigate across different levels of folders and files and upload the files or create folders. Amplify Studio also provides your with a snippet of code that uses Storage APIs to get the file.

const file = await Storage.get("aws-amplify/auth/aws-amplify-auth.md", {
    level: "public"
});

Figma design to code

Using the getting started Figma design from the community and the components available in this design uses the design guidelines which is a subset of AWS Amplify UI Kit that has a collection of UI components for different UI frameworks both web and mobile.

With my limited knowledge of Figma designing and using the pre-defined components and modifying the components as per my requirement, I've successfully created three components - ServiceCard, FeatureCard and InfoCardDetails .

On successfully authorizing AWS Amplify Studio to access Figma designs from my Figma account, the UI Library on Amplify Studio lists all the components ready to be configured and imported into your Amplify project.

Component configuration

FeatureCard is now ready to be configured with the data that is defined. Amplify Studio provides a tree for all the elements created in this component and configuring different component properties.

Firstly, map the component to the data model that is used in the component.

Set each of the child properties with the data from the defined data model. In the above screenshot, you can see the title of the card is set with infoCard.Title to the label property of the child component.

In case of InfoCardDetails component, the data model defines URLs for documentation, samples and workshop. For those to be functional as a button onClick event, Amplify Studio allows those configurations where you can specify the target by selecting Open URL in new tab and also mapping the equivalent data from InfoCard data model.

Collection components

In the component configuration page, click on Create collection button.

The collections can be configured with properties that can make the collections UI really customizable in terms of how they would appear as a - List or Grid, ordering, spacing, alignment, enabling search functionality in the collection and also pagination with the page size.

The collections could also be configured with filters which are applied to the collections and also define the sort conditions.

Sync Amplify Studio with local IDE

With the data and UI configurations done on Amplify Studio, these changes can be synced to your local IDE with Amplify CLI.

amplify pull --appId your-amplify-project-id --envName your-env

This will create all the amplify related files in your local project folder with amplify backed and resources configurations under /amplify folder. In your src folder, the UI components and data model related files are generated in /ui-components folder and /models folder respectively.

Importing and using components

Installing the needed npm dependencies.

npm install -g @aws-amplify/cli
npm install aws-amplify @aws-amplify/ui-react

In index.js file, configure Amplify.

import { ThemeProvider } from "@aws-amplify/ui-react";
import { Amplify } from 'aws-amplify';

import awsconfig from './aws-exports';

import "@aws-amplify/ui-react/styles.css";
import { studioTheme } from "./ui-components";

Amplify.configure(awsconfig);

Using a theme from Amplify with ThemeProvider.

<ThemeProvider theme={studioTheme}>
  <App />
</ThemeProvider>

Import the UI component created and configured from Amplify Studio.

import {
 FeatureCardCollection 
} from './ui-components';

Modifying the components locally

From the above screenshots, you may have noticed the images being broken while previewing on Amplify Studio. Let's modify the InfoCardDetails component to retrieve data from Amplify Storage.

import React, { useEffect, useState } from "react";
import {
  getOverrideProps,
  useNavigateAction,
} from "@aws-amplify/ui-react/internal";
import { Button, Divider, Flex, Image, Text } from "@aws-amplify/ui-react";
import { Storage } from "aws-amplify"
import { useHistory } from "react-router-dom";

export default function InfoCardDetails(props) {
  const { infoCard, overrides, ...rest } = props;
//Set the states
  const [imageThumbnail, setImageThumbnail] = useState('')
  const [mdDoc, setMDDoc] = useState('')
  useEffect(() => {
    fetchImage()
    fetchMD()
  }, [])

//Using Amplify Storage to download images 
  async function fetchImage() {
    try {
      const image = await Storage.get(infoCard.s3_thumbnail, {
        level: "public"
      })
      setImageThumbnail(image)
    } catch (err) { console.log('error fetching images') }
  }
....
//Setting image to Image component
            <Image
              width="unset"
              height="408px"
              display="block"
              gap="unset"
              alignItems="unset"
              justifyContent="unset"
              shrink="0"
              alignSelf="stretch"
              position="relative"
              padding="0px 0px 0px 0px"
              objectFit="cover"
              src={imageThumbnail}
              {...getOverrideProps(overrides, "image")}
            ></Image>
....
}

After modifying the same in FeatureCard and ServiceCard, locally run the app with npm start.

The app renders all the images after they are downloaded from Amplify Storage.

Wrap-up

Building applications with Amplify Studio has certainly amplified developer experience with defining, provisioning, and also managing different AWS services.

Having to do it manually for a full-stack developer during development would mean that they would need in-depth knowledge of all those underlying AWS services.

Launched in 2019, it was focused on building pathways for SaaS Product events to be consumed and routing the events to the AWS Lambda function. It initially supported buses and CloudWatch Events which was previously part of AWS CloudWatch. Since then, a lot of powerful features have been added to EventBridge to build an event-driven Serverless app.

EventBridge Buses

Event Buses are the communication channel that receives events from different event sources which could be AWS Services such as Amazon S3 which is posting in events whenever there is any action happening in an S3 bucket, programmatically using EventBridge APIs to send events to a default or custom bus or events from a partner SaaS product posting events to a defined SaaS event bus. Once the events are available on the bus, event rules are used to route the event to the right designation these rules have powerful filtering capabilities which can be applied to event payloads.

Some scenarios, where you would be using it in your application -

• An S3 event that is putObject event would be routed to AWS Lambda functions for further computation or AWS Step Functions to kick-start a state machine execution.

• An S3 event that is deleteObject would result in invoking an API destination that would use the Input Transformer to restructure the event as per API's specifications.

Note: these rules are specific to event buses, and there can be multiple rules in a bus.

EventBridge Pipes

EventBridge Pipes enable direct point-to-point integrations from defined event sources like Amazon DynamoDB streams, Amazon Kinesis, Amazon SQS, and more which can be filtered with advanced EventBridge filtering on the event payloads.

{
  "eventName": [{
    "prefix": "INSERT"
  }]
}

The filtered events could be enriched to uplift the data before it is delivered to the target AWS Service. Input Transformer is available for the enrichment and target states which can transform to the expected payload structure by the AWS Service and also define the output path.

EventBridge Pipes, opening up the direct service integration with advanced filtering and enrichment

• EventBridge Pipe uses events from DynamoDB streams on any action on a DynamoDB table and would use filtering to process only eventName: INSERT for newly created items on DynamoDB and invoke a state machine execution.

• New messages from Amazon MQ could be used to start a state machine execution.

EventBridge Scheduler

EventBridge Scheduler is a way to schedule your tasks that happen from time to time and need recurring events with corn expressions, time windows, and one time with a specific date and time. These events from the scheduler could be targeted to a specific AWS Service where the event payload set in the scheduler will be delivered.

• Tasks that have to be performed every day at a specific time (like 1 am).

• Workloads that generate billing every month where the scheduler can invoke a Lambda function to generate bills.

In a nutshell

Amazon EventBridge can be a crucial component of your Serverless applications which could be solution-ed to deliver the right events effectively to the destinations. For Event-driven architectures that consume a multitude of events, EventBridge can easily scale, and events can be filtered with advanced filtering capabilities. EventBridge eliminates the glue-coded Lambda functions which might contain additional logic to validate the events against certain patterns which are necessary to the application for further processing. EventBridge Pipes enhances your Serverless workload with direct service integration that positively affects the application's performance and scalability.

In any application, the design might contain multiple features by Amazon EventBridge where specific tasks could be scheduled with an EventBridge Scheduler, all events could be sent to an Event Bus which would use Event Rules to route the events to the designated targets or have point-to-point integrations with AWS services as producer and consumer for smooth orchestration.

In this blog post, we will look into how Serverless apps can be built with just configurations and leveraging the features of AWS Services which can integrate with different services.

Why do we need direct integrations?

The AWS Services with direct integrations capabilities improve the architectures by removing the glue code needed.

Just let us think about building REST APIs with AWS API Gateway which returns a particular set of data. For this, we would have to have an API Gateway stage in which GET method invoking a Lambda function which is either using query or scan to search the items on DynamoDB. The response from DyanmoDB SDK would then be returned to API Gateway from the Lambda function.

Now, let's understand how direct integrations with this would help.

Cold Starts

Now that we are familiar with the downside of having a Lambda would be the Cold Starts that we would have to be mindful of and also have a workaround to overcome them. But when doing a direct integration with different AWS Services, since we would not have a Lambda function as the middle-man or the glue code, we would not have to worry about the problem of Cold Starts.

Latency

With the removal of an additional component from the architecture, we are reducing the latency with the requests routed with a Lambda function and returning the results back. Although this would be in milliseconds, it is very accountable in a production environment where there would be 100s and 1000s of requests at a given time.

Scalability

"Would it be highly scalable when we have concurrent requests that are coming in?" With Lambda the other question of concern is, "how can we handle concurrency and ensure the application is scalable proof?" When we do a direct integration this responsibility would be on the cloud provider and the AWS Service would ensure your application could scale better.

Direct integrations with AWS Services

AWS Step Functions

Serverless applications which would need orchestration and a dedicated workflow that would integrate with multiple AWS Services are a huge win! Starting from AWS re:Invent 2021, where Step Functions announced supporting AWS SDK-based integrations there have been over 300+ API actions that are currently been supported.

There are a lot of possibilities that would open up with SDK-based integrations with AWS Step Functions, there is a blog post series about the same.

Amazon EventBridge

Amazon EventBridge plays a vital role in event-driven, choreography-based Serverless architectures. While EventBridge supports different integrations with different event sources or destinations and also configuring HTTP endpoints as API destinations.

Not only does it support AWS Services which can post events to the event bus but also with the new EventBridge Pipes, it is possible to have configurations that can integrate service to service.

For instance, before DynamoDB Streams could only trigger a Lambda function but now with EventBridge Pipes, you can build integrations where the source of the event originating from DynamoDB Streams and then it could be targeted to invoke a Step Functions state machine execution.

Amazon API Gateway

API Gateway can integrate with Velocity Template Langauge (VTL) which could be used for request and response resolver mapping and integrate with services like DynamoDB, Step Functions, Kinesis, and more.

Like the above mentioned example, API Gateway can directly integrate with DynamoDB. Take a look at the pattern from ServerlessLand. And the snippet of the same of how you can integrate an API Gateway to query DynamoDB.

  MusicArtistMethodGet:
    Type: 'AWS::ApiGateway::Method'
    Properties:
      RestApiId: !Ref Api
      ResourceId: !Ref MusicArtistResource
      HttpMethod: GET
      ApiKeyRequired: true
      AuthorizationType: NONE
      RequestParameters:
        method.request.path.artist: true
      Integration:
        Type: AWS
        Credentials: !GetAtt APIGatewayRole.Arn
        IntegrationHttpMethod: POST
        Uri: !Sub 'arn:aws:apigateway:${AWS::Region}:dynamodb:action/Query'
        PassthroughBehavior: WHEN_NO_TEMPLATES
        RequestParameters:
          integration.request.path.artist: method.request.path.artist
        RequestTemplates:
          application/json: "{\"TableName\":\"Music\",\"IndexName\":\"Artist-Index\",\"KeyConditionExpression\":\"artist=:v1\",\"ExpressionAttributeValues\":{\":v1\":{\"S\":\"$util.urlDecode($input.params('artist'))\"}}}"
        IntegrationResponses:
          - StatusCode: '200'
            ResponseTemplates:
              application/json: "#set($inputRoot = $input.path('$'))\n{\n\t\"music\": [\n\t\t#foreach($field in $inputRoot.Items) {\n\t\t\t\"id\": \"$field.id.S\",\n\t\t\t\"artist\": \"$field.artist.S\",\n\t\t\t\"album\": \"$field.album.S\"\n\t\t}#if($foreach.hasNext),#end\n\t\t#end\n\t]\n}"
      MethodResponses:
        - StatusCode: '200'

AWS AppSync

AppSync being a Serverless GraphQL offering, the queries, mutations and subscriptions can use data sources which are DynamoDB, Aroura, Open Search, or HTTP endpoints other than the expected AWS Lambda Functions.

These direct service integrations can leverage Velocity Template Langauge (VTL) or JavaScript resolvers. Additional to the function based resolvers, you can create pipeline resolvers that can not only integrate one but multiple AWS Services.

Where to configure

These direct service configurations would make more sense when you are working with these Services from Infrastructure as Code (IaC) or Infrastructure from Code (IfC) approaches. Like the API Gateway example showcasing a YAML template, you can use AWS SAM where you can define the resources and then the integrations with resolver info.

Another simpler way to integrate services from AWS Step Functions is from the Workflow Studio on AWS Console, where you can define the task along with SDK integration by defining the parameters and results.

This generates a JSON structure for the State Machine which could be imported to your IaC project.

Similar to Workflow Studio, you can design Serverless apps that can generate SAM Template from AWS Application Composer. This currently supports EventBridge integrations with different Services.

Sometimes you may need more than a configuration

Before wrapping up, yes! Sometimes you may need more than a configuration where you need to define your Lambda function or glue codes that process the data from these Services in an efficient way. The above AWS Services support direct integration with different AWS Services, there are some of which like Amazon SNS and Amazon SQS which could use configurations to trigger a Lambda function.

Amazon Inspector supports scanning of AWS Lambda functions and Lambda layers with Java, NodeJS and Python runtimes.

Need for vulnerability checks

Often times, we have code which depends on many packages from installed via different package managers which are prone to security leaks. Although, updating to new version could resolve it, you might have dependencies which are still prone to vulnerabilities. The best way to address is a regular scanning of your codebase to ensure there aren't serious issues.

Serverless specific, until now we had to depend on a third party tool to scan but now it's possible with Amazon Inspector

Enabling Inspector

First off, you would have to enable Inspector for your AWS Account.

Your first scan

Once enabled, you will need a few minutes for Amazon Inspector to scan across your resources across Amazon EC2 instances, Amazon ECR images and now AWS Lambda functions and Lambda layers.

After Amazon Inspector has scanned you can view the report on Inspector dashboard.

[Fun Fact] As you can see, I don't have a single EC2 instance running on this AWS Account.

Scanned findings

Inspector found that 9 of my Lambda functions had a vulnerabilities with critical, high and medium levels.

If you click on one of the functions, you can find the summary for vulnerabilities in that specific AWS Lambda function or the vulnerability because of using an AWS Lambda layer.

Let's dive into the finding

One of the vulnerability is with Axios NPM package.

This also gives details about axios package and the affected with fixed version.

Inspector provides you the complete details of the vulnerability along with the link to National Vulnerability Database (NVD) report.

Along the details, you can also find how to fix it with the available remedy.

In this case, it's updating axios version.

Another way to understand the severity of the vulnerability, the score from National Vulnerability Database (NVD) and Inspector is available.

Pricing

Amazon Inspector is available as part of free trial for 15 days. For Lambda scans alone, there is monthly based on average number of Lambda functions scanned per month and price is prorated based on total Inspector coverage hours for the month.

More details on Amazon Inspector Pricing.

Action time!

Now it's time to scan your Lambda functions and layers with Amazon Inspector.