AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume - there is no charge when your code is not running.
COU language preference for coding a Lambda is Python, Node or Go (in this order).
https://docs.aws.amazon.com/lambda/latest/dg/getting-started-create-function.html
https://github.com/aws-samples/aws-serverless-workshops
https://github.com/awslabs/aws-sam-cli
https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html
Dealing with quotas in third-party APIs: SQS, CloudWatch Events, and Lambda functions
https://www.jeremydaly.com/throttling-third-party-api-calls-with-aws-lambda/
Limit concurrency
https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html#reserved-and-provisioned
https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html#invocation-async-errors
lambda-setuptools
https://github.com/QuiNovas/lambda-setuptools
python-lambda-setuptools
Sample using 'lambda-setuptools' with Makefile, Pipenv, Pipfile and Terraform template
https://github.com/dnvriend/python-lambda-setuptools
Sample project tree:
.
├── Makefile
├── Pipfile
├── README.md
├── setup.cfg
├── setup.py
└── src
├── app_util
│ ├── Exc.py
│ └── generic.py
└── lambda_function.py
Note: This sample lambda has the AWS Lambda default handler "lambda_function.lambda_handler".
Setuptools complains that file 'lambda_function.py' is at the root level instead of inside a package.
Although this is the default, the handler can be changed accordingly (see terraform template sample below).
Makefile:
.PHONY: pipenvini pipenvend clean package
# CONFIG
lambda_name=myapp_calendar
help:
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
pipenvini: ## creates a new python environment with all dependencies (Pipfile)
pipenv install --dev
pipenvend: ## remove the python environment
pipenv --rm
clean: ## removes all build artifacts
rm -rf build dist .eggs
package: ## create a lambda distribution
pipenv run python setup.py ldist
#zip_filename, eg: dist/myapp_calendar-0.1.2.zip
lambdaupdate:
aws lambda update-function-code --function-name $(lambda_name) \
--zip-file fileb://$(shell ls dist/*.zip) > /dev/null
deploy: ## Clean, create artifact and update function code
make clean
make package
make lambdaupdate
Pipfile:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[packages]
requests = "*"
[dev-packages]
boto3 = "*"
setuptools = "*"
pytest = "*"
mypy = "*"
yapf = "*"
pylint = "*"
[requires]
python_version = "3.8"
setup.py:
#!/usr/bin/python3.8
from setuptools import setup
setup(
#name in the form of "namespace.mypackage"
name='myapp_calendar',
version='0.1.2',
author='thatsme',
author_email='thatsme@cou.edu',
description="AWS Lambda",
license='Apache 2.0',
package_dir={"": "src"},
#package blank needed if the handler is not the Setuptools generated one
packages=['', 'app_util'],
#set of dependencies
install_requires=['pyjwt', 'pytz', 'requests'],
setup_requires=['lambda_setuptools==0.4.4'],
#lambda_function in the form of 'my_package.some_module:some_function'
lambda_function='lambda_function:lambda_handler'
)
src/lambda_function.py:
# myapp_calendar lambda using runtime Python 3.8 #
######################################################
import app_util.Exc
from app_util.generic import ag_iso_datetime
from app_util.generic import ag_log_exception
from datetime import datetime
import json
import logging
import os
from pytz import timezone
# Logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
"""Lambda entry point
:param event: -
:param context: -
:returns: event
"""
def lambda_handler(event, context):
ret = {}
try:
print('Hello')
winter = "2021-02-27"
summer = "2027-08-31"
mytime = "18:27:59"
print('B')
print("a) "+ag_iso_datetime(winter, mytime))
print("b) "+ag_iso_datetime(summer, mytime))
print('C')
raise app_exc.AppRestExc(203, '{valid: false}')
#raise app_exception.AppRestExc(201, '{valid: true}')
print('D')
except Exception as e:
ag_log_exception('EXC MAIN_TRY_CATCH', e)
finally:
# Return
return ret
Terraform sample demostrating a Lambda handler inside a package:
Note: "myapp_calendar_function.handler" is automatically generated by Setuptools and points to the desired handler.
resource "aws_lambda_function" "test_lambda" {
function_name = "myapp_calendar"
filename = "dist/myapp_calendar-0.1.2.zip"
role = "${aws_iam_role.role_for_lambda.arn}"
handler = "myapp_calendar_function.handler"
source_code_hash = "${base64sha256(file("dist/myapp_calendar-0.1.2.zip"))}"
runtime = "python3.8"
}
Given the following layout of the project:
.├── lambda_function.py├── README.md├── requirements.txt├── runZip.sh└── setup.cfgwhere lambda_function.py is the lambda function.
setup.cfg
[install]
prefix=
requirements.txt
xlrd
runZip.sh
#!/bin/bash
tech_comp=myapp_collaboration
lambda_name=collaboration-myapp-test
echo == Cleaning up
rm -rf package/
rm $tech_comp.zip
echo
echo == Installing dependencies
pip install --requirement requirements.txt --target ./package
echo
echo == Creating zip archive with dependencies
cd package
zip -r9 ../$tech_comp.zip ./ -x "bin/*"
cd ..
echo
echo == Add function code to the archive
zip -g ./$tech_comp.zip lambda_function.py
echo
echo == Update code of lambda function
aws lambda update-function-code \
--function-name $lambda_name \
--zip-file fileb://$tech_comp.zip > /dev/null
echo
Eg: COU app 'ulises'.
This Lambda using Go language example sets up 'No more index.html mess with AWS CloudFront/S3' found at:
https://github.com/artyom/cloudfront-autoindex
$ lsb_release -irs
Ubuntu
20.04
https://github.com/golang/go/wiki/Ubuntu
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt update
sudo apt install golang-go
Note that golang-go installs latest Go as default Go. If you do not want that, install golang-1.15 instead and use the binaries from /usr/lib/go-1.15/bin.
Verify installation
$ go version
go version go1.15.7 linux/amd64
git clone https://github.com/artyom/cloudfront-autoindex.git
cd cloudfront-autoindex/
Build:
GOOS=linux GOARCH=amd64 go build -o main
Compress:
zip -9 lambda.zip main
Create a new AWS Lambda:
Function name: cloudfrontautoindex-myapp-dev
Runtime: Go 1.x
(*)Use an existing role: role-lambda-s3-validaciotp-dev
Tab Code:
Upload from .zip file: lambda.zip
Handler: main (binary name built above)
Tab Configuration:
Department=manual
ci=myapp
S3 bucket: front-s3-myapp-dev
Tab Properties:
Event notifications > Create event notification
Event name: cloudfrontautoindex-event
Suffix: index.html
Event types: All object create events (except 's3:ObjectCreated:Copy')
Destination > Lambda function: cloudfrontautoindex-myapp-dev (created above)
You can configure separate destinations for events that fail processing and events that are successfully processed.
Serverless example:
resources:
DeadLetterQueue:
Type: "AWS::SQS::Queue"
Properties:
QueueName: sqs-queue-${opt:stage, self:provider.stage}-dlq
functions:
DLQFunction: # function that moves the S3 file
handler: handlers.dlq
events:
- sqs:
arn:
Fn::GetAtt:
- DeadLetterQueue
- Arn
sendEmail:
handler: handlers.email
destinations: #onfailure supports different options, among them the arn of a SQS
onFailure: ${self:custom.queue-arn} #doesn't seem to admit "Fn::GetAtt:"
events:
- s3:
bucket: myS3storage
event: s3:ObjectCreated:*
rules:
- prefix: transit/noseque
Python API for AWS infrastructure services. See https://docs.aws.amazon.com/pythonsdk/
For the S3 client, see https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html
It can be used as a client or as a resource.
Using it as a client is preferred, since as a resource makes IDEs unable to check types and autodiscover methods.
Example getting an object from a bucket using the key:
s3 = boto3.client('s3')
obj = s3.get_object(
Bucket=object_bucket_name,
Key=object_key,
)