AWS Lambda

AWS Lambda is a building block for many things on AWS.

SCREENCAST: AWS Lambda as a Garage Lightbulb Hello World Lambda

SCREENCAST: Hello World Lambda Hello World Lambda

SCREENCAST: Marco Polo Lambda Hello World Lambda

gist for Marco

def lambda_handler(event, context):
  if event["name"] == "Marco":
    return "Polo"

gist for Polo

def lambda_handler(event, context):
  if event["name"] == "Polo":
    return "Marco"

Developing AWS Lambda Functions with AWS Cloud9

Cloud9 has many capabilities built in the make developing with AWS Lambda easier. These include debugging, importing remote lambda functions and a wizard.

SCREENCAST: Develop AWS Lambda functions with AWS Cloud9 Hello World Lambda

Building an API

The following code can be used as an API via API Gateway.

Python Lambda API Gateway Example

import json
import decimal


def lambda_handler(event, context):

  print(event)
  if 'body' in event:
    event = json.loads(event["body"])
  
  amount = float(event["amount"])
  res = []
  coins = [1,5,10,25]
  coin_lookup = {25: "quarters", 10: "dimes", 5: "nickels", 1: "pennies"}
  coin = coins.pop()
  num, rem  = divmod(int(amount*100), coin)
  res.append({num:coin_lookup[coin]})
  while rem > 0:
    coin = coins.pop()
    num, rem = divmod(rem, coin)
    if num:
      if coin in coin_lookup:
        res.append({num:coin_lookup[coin]})

  response = {
    "statusCode": "200",
    "headers": { "Content-type": "application/json" },
    "body": json.dumps({"res": res})
  }

  return response

AWS Step Functions

A key advantage of AWS Step functions is the ability to pipeline actions together.

Screen Shot 2020-03-24 at 2 24 27 PM

SCREENCAST: Develop AWS Lambda functions with AWS Cloud9 Hello World Lambda

Reference: Web Scraping Pipeline Github Project

Here is a Marco Polo Step Function:

{
  "Comment": "This is Marco Polo",
  "StartAt": "Marco",
  "States": {
    "Marco": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:561744971673:function:marco20",
      "Next": "Polo"
    },
    "Polo": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:561744971673:function:polo",
      "Next": "Finish"
    },
    "Finish": {
      "Type": "Pass",
      "Result": "Finished",
      "End": true
    }
  }
}

Note that the input of one Lambda goes into the input of another:

Screen Shot 2020-03-24 at 6 38 40 PM

Building a serverless data engineering pipeline

One strong use case for AWS Lambda is to build serverless data engineering pipelines.

serverless_ai_pipeline

SCREENCAST: Build Serverless Data Engineering Pipeline Build Serverless Data Engineering Pipeline

Exercise: AWS Lambda + Step Functions

  • Topic: Build a step function pipeline
  • Estimated time: 20 minutes
  • People: Individual or Final Project Team
  • Slack Channel: #noisy-exercise-chatter

  • Directions (Do one or both):

    • Basic Version: Create an AWS Lambda function that takes an input and run it inside of a Step Function
    • Advanced Version: Create an AWS Lambda function that takes an input and run it inside of a Step Function, then send the output to another AWS Lambda. See the Marco Polo Step Function above.
    • Share screenshot + gist in slack.