Basics

Manage RESTful API. Can integrate with lambda, proxy to other AWS service. Other REST end points. Mock. VPC link. Do this by translating / mapping method/integration request/responses, authorizes, logs, etc.

Front end (API Gateway API) - Method Requests & Method Responses, can be deployed in one or more stages, can associate with a (custom) domain name.

API endpoints: host names, one for each API

• Edge-optimized - for users from all Internet, deployed to specific region while using a CloudFront distribution to facilitate access, typically improves connection time for geographically diverse clients.

• Regional API endpoint: deployed to specific region and intends to serve client (such as EC2 instances) of the same region. Requests targets directly to region-specific gateway without through CloudFront. In additional can use latency based routing to deploy API to multiple regions.

• Private: for client inside VPC. Isolated from public Internet.

App user, end user, client: client, perhaps an app.

API deployment & stage: deployment is a point-in-time snapshot of the API Gateway resources & methods. A deployment must be associated with one or more stages to be accessible for client. A stage ('dev', 'prod', 'beta', 'v2') is a logical lifecycle status.

API key: owner request API Gateway generate API key, give key to app developer (client) to id them, and owner can control access to the API by specific app.

Back end - Integration Requests & Integration Responses, integrate with & expose Lambda or other services

Gateway Response (under individual API's configuration)- Set API Gateway responses when Gateway fails to process incoming request and return to client without forwarding to backend. For example, in case insufficient permission for gateway to call backend Lambda (API Configuration Error), can specify response code, header & mapping template here.

Proxy integration: HTTP or Lambda proxy integration. HTTP - pass entire request/response. Lambda - pass entire request as input & transform output as response. Most commonly used as proxy resource - with greedy path variable ({proxy+}) combined with catch-all ANY method.

Private Integration: access resource inside a VPC through private API end points.

Mapping template: in Velocity Template Language (VTL) to transform frontend / backend data. Specified in integration request/response.

Model: schema specifying data structure of request / response payload. With model API Gateway can (1) validate request (2) transform (3) generate SDK

Developers - App developer calls API; API developer (develop API) must be an IAM user

API Resources - logical entity that an app can access through a path ("/incomes") and methods "GET,POST,PUT,PATCH,DELETE"

Usage plan: configure throttling and quota limits.

Benefits:

• Secure API with authorization & access control

• Validate request (like require parameter)

• Pre-process

  • payload transformation (template, binary/base64, compression)

  • header / query parameter etc mapping

• Traffic management

• Monitoring & analytics

• Version management

• SDK generation

API

Component: apigateway

Create API: use console or API Gateway REST API (from CLI, use SDK, REST API client - Postman, CloudFormation templates, API Gateway extension to Swagger(still not OpenAPI v3). Check availability in regions.

Lambda Integration

On Lambda Integration: API Gateway is so commonly used for Lambda integration that I first thought it's the only way to call Lambda from Internet. Not necessarily (although probably a good idea, though). See https://forum.serverless.com/t/convince-me-to-use-api-gateway-and-not-call-lambda-direct/3214/12

Integration: proxy integration / custom integration; cross account is possible.

• Custom integration is legacy, may be used if need to pre-process incoming request data, but more complex and likely to be inoperable when backend Lambda changes

• Proxy integration:

  • Lambda Input:

    • from request: headers, path, query string parameters, body

    • from API configuration settings

  • Output:

  • Node.js integration: as objects, see example

  • Java: as input/output stream, parse with JSONObject, see code example

HTTP Integration

Proxy or custom integration. Proxy is recommended whenever possible. Custom integration can transform client request or backend response.

Catch all proxy resource {proxy+} & ANY method can expose the entire set of resources.

Private Integration

AWS Integration

Integrate with AWS services. Can provide consistent protocol to access different AWS services. Examples provided for SNS, Lambda, S3, Kinesis.

Also for DynamoDB https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/

No proxy integration, hence single API method for single AWS action.

Why not direct access services instead through API-Gateway?

• Integrate with specific function without manage keys

• Apply certain restrictions

• Add a different method (create GET and proxy to POST so cache enabled), add headers (cache headers)

• All these without server code (even lambda)

Basics (my experiment & DynamoDB example)

• Follow above DynamoDB example, set up proper role for API, easy to create pass-through for the same method

• Use template to transform method request to integration request

• Consult VLT template reference, examples, and template reference, use test function

• Tips

  • Serialization error - check the request body after transformation, not valid JSON

  • Possible to pass JSON to query parameter in GET, use $util.parseJson to convert to JSON.

  • Very strict JSON, use #if($foreach.hasNext),#end

#set ($idarray = $util.parseJson($input.params('ids')))

Mock Integration

Initial Setup

• Type: Regional, edge optimized, private

API Resource (path) & Methods

Initial setup:

• Path

  • Path - Proxy resource

    • /food/{type}/{subtype}/{item} - create place holders for whatever given in request

    • /{proxy+} - "+" means which ever child resources appended to it

    • Specific resource takes precedence over generic resource when at the same level of hierarchy

• Method (ANY + GET,HEAD,OPTIONS,PATCH,POST,PUT,DELETE)

  • Method Request

    • Authorization: NONE (open), IAM, Authorizer-Cognito, Authorizer-Lambda(custom)

      • IAM - signature based on caller's acess key id & secret key

      • Cognito - ID token based on id, or access token based on custom scopes of resource

      • Lambda - token or request type

    • URL query parameters: must declare, can specify as "required"

    • Body mapping templates: for proxy integration can set up Model (JSON schema draft 4) to describe data structure, and then the Gateway can validate the request, generate an SDK (and as a class in strong typed language such as Java), initialize a mapping template, for non-proxy integration can transform body with one or more templates, based on Content-Type header.

    • Authorization:

  • Integration Request

    • Integrate with: Lambda, HTTP (Proxy or custom), Mock, AWS Service, VPC link

    • For non-proxy integration, can map headers, path parameters, query parameter, convert body with template.

    • Lambda:

      • "Invoke with caller credentials" - lambda be able to receive caller credential information

      • "Credential cache" - whether or not to add caller credential to cache key

      • Must map error from lambda to response code (which by default is 200, not intuitive)

    • HTTP

      • Content handlinig: passthrough, convert to binary (from Base64), convert to text (from Base64)

  • Integration Response

    • For non-proxy integration, can map integration response (for example Lambda error regex) to HTTP response.

  • Method Response

    • With non-proxy integration: response (code, header, body) can be mapped

    • With proxyed integration, no need to setup, pass through. Lambda must return result of required format.

    • Default status code: 500 (internal error) is reasonable

    • Headers - able to set which headers the client receives

    • Models - defines format of response body, this is for generation of a strong typed SDK. Otherwise optional.

Controlling Access

Permission to invoke:

• API resource policies - JSON policy document attach to API, control principal (typically IAM user/role) can or cannot access API

  • based on user, role, source IP range, VPC or VPC end points

• IAM permissions

  • grant IAM developer permission to create/deploy/manage API in Gateway

  • grant permission to call or refresh API caching

• Lambda Authorier

  • Token based - Uses bearer token authentication (OAuth or SAML) in certain header

  • Request based - Also use selected information in the request (header, path, query strings, etc.)

  • Authorization caching - cache the returned IAM policy that authorize the request, associated with the token (in certain header) or source request parameters

• Cognito User Pool

  • Requires ID token or Access token

  • ID token - authorize based on id of sign in user

  • Access token - based on custom scopes of the protected resources - to use, must configure Resource servers in user pool / App integration, and must match the resource server id

• Client side SSL Certificate

  • Certificate is issued by API Gateway, is self-signed.

  • To verify that call to the backend (e.g. not on AWS) are indeed from API Gateway

• Metering / tracking usage by client using API keys with Usage Plans

  • Use to extend the API as an product offering to customers

• CORS

Documenting

Can maintain structured API documentation, export a stage specific snapshot to an external Swagger file (not OpenAPI v3), distribute as a publication.

Deploying

Deployment is associated with stages. Each stage is a snapshot. When API is updated (not the backend. change to backend is depend on the backend. for example change to lambda can be immediately seen), must redeploy to existing stage or new stage. This is version control of API.

Method URL: based on host name (with restapi-id & region, or customized), stage name & resource path

Performance optimization

Account level request throttling: rate and burst

API Caching: may increase cost. select capacity (space by gb), ttl, optionally encrypt cache data...

Client certificate: select certificate that the gateway use to call integration endpoints

Stage variables: used to parametrize the integration (such as lambda authorizer), and also available in $context object of the mapping templates

Export: export as Swagger, Swagger + API Gateway Extension, or Swagger + Postman Extension - only available when payloads are application/json type, still not OpenAPI v3 as of 2018 July

SDK Generation: not necessary if call in generic REST way; do this to call in platform/language specific way, support Java, Javascript, Java for Android, Object-C or Swift for iOS, Ruby

Canary can be used to gradually phrase out old version and introduce new version with various scheme (like percentage based)

Custom Domain Name

Sell as SaaS (Software as a Service) - through AWS Marketplace

Logging & Watching

When something is wrong, test API on console, enable logging under stage configurations, need to add role that can write log to CloudWatch, give the ARN of that role to API Gateway in setting, then enable logging under stage configurations, can find detailed execution log (including received request, translation, etc.) in CloudWatch; detailed metering with cloudwatch logs, enable custom access logging; automatic log metrics on API and Stage levels if configured. can opt in Method level metrics. Can use CloudTrail to capture API calls and deliver log to S3 bucket.

In case calling Lambda and find "500 "Invalid Permission on Lambda Function", this means API Gateway doesn't have permission to call the lambda. To solve, in the function's integration request, edit lambda function name, save, ok to give permission (3) the permission is actually in Lambda, designer, event, or view permissions, function policy, or "aws lambda get-policy --function-name opt1-CreateOrUpdateContact-118YO8RD0OTMQ"

Invoking

Service Component: execute-api (not in CLI)

Manners to call:

• Console to test,

• REST API client (curl, Postman),

• REST library (Axios)

• Generated SDK

  • Not really necessary if uses Cognito user pool and not care strong type

  • If use IAM, the generated SDK has method for signature

• Amplify library

Method request & integration request:

• headers, parameters, body

Method response & integration response:

• status code, headers, payload

Postman - a convenient tool to test an API, with AWS IAM user access key, secret key & signature

Swagger Extensions

API Gateway can be created / exported to a swagger definition with AWS extensions.

Best practice:

• After creating backend resource, use console to create & test API Gateway

• Then, export swagger + extension

• Replace the hard coded resource reference Arn with (cloudformation functions can be used in swagger file):

  • { "Fn::GetAtt" : [ "logicalNameOfResource", "attributeName" ] }

Limits

See developer guide. There are certain hard limits.