AWS Step Functions is a web service that enables you to coordinate the components of distributed applications and microservices using visual workflows.
Amazon States Language (definition of state machines using JSON)
https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html
Reference > https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-cloudwatch-events-s3.html
Procedure resume:
Create event in Cloud Trail
Create rule in Cloud Watch for the event, in order to initiate Step Functions
Verify execution adding a file to the S3 bucket and then in Step Functions select the most recent execution of that state machine and expand the Input section:
This input includes information such as the bucket name and the object name. In a real-world use case, a state machine can use this input to perform actions on that object.
Sample 'input' for a file named "sample1.json":
{
"version": "0",
"id": "7a96bb1f-2914-c772-0b4a-f82148c082e8",
"detail-type": "AWS API Call via CloudTrail",
"source": "aws.s3",
"account": "434374682878",
"time": "2020-05-08T08:37:38Z",
"region": "eu-west-1",
"resources": [],
"detail": {
"eventVersion": "1.07",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDAWKIV7VD7JL6XH7PKO",
"arn": "arn:aws:iam::434374682878:user/jm.rodriguez@ibermatica.com@dam2018",
"accountId": "434374682878",
"accessKeyId": "ASIAWKIV7VD7DVTJEALR",
"userName": "jm.rodriguez@ibermatica.com@dam2018",
"sessionContext": {
"attributes": {
"creationDate": "2020-05-08T05:51:12Z",
"mfaAuthenticated": "false"
}
}
},
"eventTime": "2020-05-08T08:37:38Z",
"eventSource": "s3.amazonaws.com",
"eventName": "PutObject",
"awsRegion": "eu-west-1",
"sourceIPAddress": "93.176.187.156",
"userAgent": "[Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0]",
"requestParameters": {
"X-Amz-Date": "20200508T083737Z",
"bucketName": "dam2018-datos1-defensatf-uoc-s3-dev-test-s3",
"X-Amz-Algorithm": "AWS4-HMAC-SHA256",
"x-amz-acl": "private",
"X-Amz-SignedHeaders": "content-md5;content-type;host;x-amz-acl;x-amz-storage-class",
"Host": "dam2018-datos1-defensatf-uoc-s3-dev-test-s3.s3.eu-west-1.amazonaws.com",
"X-Amz-Expires": "300",
"key": "sample1.json",
"x-amz-storage-class": "STANDARD"
},
"responseElements": null,
"additionalEventData": {
"SignatureVersion": "SigV4",
"CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
"bytesTransferredIn": 1400,
"AuthenticationMethod": "QueryString",
"x-amz-id-2": "qAFmZxFiH4jG3irw7RdPhule06FDcLcyHoRI5BAcM7YAm/NOpEPuVT9wdoJY7J07Z9hBUkYJc9k=",
"bytesTransferredOut": 0
},
"requestID": "5892AC19E7992AA1",
"eventID": "8b323968-a92e-4f1a-9caf-3861a36f4eec",
"readOnly": false,
"resources": [
{
"type": "AWS::S3::Object",
"ARN": "arn:aws:s3:::dam2018-datos1-defensatf-uoc-s3-dev-test-s3/sample1.json"
},
{
"accountId": "434374682878",
"type": "AWS::S3::Bucket",
"ARN": "arn:aws:s3:::dam2018-datos1-defensatf-uoc-s3-dev-test-s3"
}
],
"eventType": "AwsApiCall",
"managementEvent": false,
"recipientAccountId": "434374682878",
"eventCategory": "Data"
}
}
When an step functions tasks invokes a lambda with an over-passed reserved concurrency (status code 429: Lambda.TooManyRequestsException):
reserved_concurrent_executions = 2
a retry mechanism must be configured, eg:
"calendar": {
"Type": "Task",
"Resource": "arn:aws:lambda:eu-west-1:274322772872:function:fpdefense-env-calendar",
"Next": "notifier",
"Retry": [
{
"ErrorEquals": [
"Lambda.TooManyRequestsException",
"States.TaskFailed"
],
"IntervalSeconds": 8,
"MaxAttempts": 16,
"BackoffRate": 2.0
}
]
},