AWS Lambda
Lambda is a compute service that can run you python / Go / Node.js /C# / Java code without a server, so a serverless service.
The runtime of your Lambda function may not have all the required libraries available, so very often you need to pack your own libraries and upload it to Lambda as a Layer. Simply consider Layer as a library to support your code in Lambda function.
Following is an example of getting realtime bitcoin price from bitfinex and save result to a S3 bucket.
Firstly, we write the python code for getting bitcoin price in local environment. This defines a simple function that calls the bitfinex website using the requests library.
import requests
def get_btc_ticker_data():
try:
req = requests.get("https://api-pub.bitfinex.com/v2/ticker/tBTCUSD")
content = eval(req.content)
field_names = ['BID', \
'BID_SIZE', \
'ASK', \
'ASK_SIZE', \
'DAILY_CHANGE', \
'DAILY_CHANGE_RELATIVE', \
'LAST_PRICE', \
'VOLUME', \
'HIGH', \
'LOW']
if 'error' in content:
return content
else:
return {name: value for name, value in zip(field_names, content)}
except Exception as e:
return str(e)
Now we pack this function into a package so that we can upload it to Lambda as a Layer.
Firstly, save the code as btc_ticker.py under a folder python.
--python
----btc_ticker.py
Note you must have the python folder, because lambda places a uploaded Layer under /opt directory so a python folder will make it /opt/python which is exactly the python path set in lambda. As your code is in the path, you can import your own python class or function.
Before we pack it, we need to download dependencies into this folder as well. So go into the python folder, and run:
pip3 install requests -t ./
The -t parameter tells it to download the dependencies into the current directory. Now the python folder has the dependencies folders as well
--python
----btc_ticker.py
----requests
----urllib3
----...
Now, zip the whole python folder (including the python folder) into a python.zip or whatevername.zip file
This zip file is self-contained and will be uploaded to lambda as a Layer.
Log onto AWS and go to Lambda. Choose "Layers" and then "Create Layer".
Put in a name and upload the previously created zip file.
Choose the Python version to run the code.
Done. You have a Layer created in your Lambda service.
Next, create a Lambda function.
Back to the Lambda dashboard and choose "Functions" and "Create Function".
use "Author from scratch", type in a name, specify the python version runtime.
Create / select an excution role. Depending on what resources you need to access, you need to attach different policies (permissions) to the role. For a demo, create/use a role attached with "AWSLambdaExecute" policy, e.g. the LambdaExecutionRole is enough. This can be done through Roles in IAM dashboard.
Now a lambda function is created.
Link a Layer to the Lambda function. On this Lambda editor panel, the bottom has a Layers section. Click 'Add a layer' to attach the previous created layer to this lambda function.
Back to the Code source section on the editor, a lambda_function.py has been created already with dummpy code:
import json
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('hello world!')
}
This does nothing but returns a hello world. Now edit the code to hook in the code to get bitcoin price.
import json
import btc_ticker
def lambda_handler(event, context):
result = btc_ticker.get_btc_ticker_data()
return {
'statusCode': 200,
'body': json.dumps(result)
}
Simple, hey. The previously created and attached layer exposes the btc_ticker.py file to this lambda function so we can just import btc_ticker to use any function defined within it. We call the get_btc_ticker_data() function to get the bitcoin price and return it as a json dump.
Hit the Deploy button to deploy it. You must deploy it before testing.
Then click Test. It will ask you to create a test event first, which puts in a dumpy json request, so leave it to the default key value pairs.
Hit the Test button, you will have something like:
Response
{
"statusCode": 200,
"body": "{\"BID\": 33293, \"BID_SIZE\": 58.48592158000002, \"ASK\": 33380, \"ASK_SIZE\": 39.754859259999996, \"DAILY_CHANGE\": -1628.89680158, \"DAILY_CHANGE_RELATIVE\": -0.0465, \"LAST_PRICE\": 33380, \"VOLUME\": 4657.08069243, \"HIGH\": 35218, \"LOW\": 33124}"
}
Hooray, successful!