cdk init app --language typescript // generates an app (in bin), a stack (lib) and test
/* IF USE LERNA: in tsconfig.json remove "typeRoots" as this prevents searching @types at parent dirs; remove package-lock.json */
cdk synth
yarn test
git commit // create initial commit
/* add "noEmit" to tsconfig, if this is not a lib */
/* IF USE STD TOOLS */ add depcheckrc.yaml and ignores - "@types/*" - "ts-node"
UPGRADE: cdk core & other packages to latest version, as CDK packages need to be in same version
cdk CLI invokes CDK Application through cdk.json "app" setting, providing inputs from certain environment variables.
CDK_OUTDIR=cdk.out
CDK_CLI_VERSION=1.109.0
CDK_DEFAULT_ACCOUNT=482175935212
CDK_DEFAULT_REGION=ap-southeast-2
CDK_CLI_ASM_VERSION=12.0.0
CDK_CONTEXT_JSON={"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId":true,"@aws-cdk/core:enableStackNameDuplicates":"true","aws-cdk:enableDiffNoFail":"true","@aws-cdk/core:stackRelativeExports":"true","@aws-cdk/aws-ecr-assets:dockerIgnoreSupport":true,"@aws-cdk/aws-secretsmanager:parseOwnedSecretName":true,"@aws-cdk/aws-kms:defaultKeyPolicies":true,"@aws-cdk/aws-s3:grantWriteWithoutAcl":true,"@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount":true,"@aws-cdk/aws-rds:lowercaseDbIdentifier":true,"@aws-cdk/aws-efs:defaultEncryptionAtRest":true,"@aws-cdk/aws-lambda:recognizeVersionProps":true,"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true,"aws:cdk:version-reporting":true,"aws:cdk:bundling-stacks":["*"]}
The CDK Application, in whatever language & however, generates the following in the output dir:
cdk.out : contains version
manifest.json: list of artifacts and meta data
AwscdkStack.template.json: Stack template
tree.json: CDK Tree
Therefore, the file structure does not have to follow the one created, as long as the CDK Application can be invoked and generates the artifacts
CDK Application bundles (compiles, copies, etc.) lambda code into cdk.out dir.
CDK Basics:
Defines cloud infrastructure (cloud formation template) in programming codes (typescript etc.)
Output (by default "cdk.out"): Cloud Assembly (package containing everything required for one or more stacks) see https://github.com/aws/aws-cdk/blob/master/design/cloud-assembly.md
Benefit:
develop infrastructure document (cloudformation template) using familiar programming language and enjoy all the benefits (tools, etc.) of the language
catch syntax, type errors and logical errors (synthesis step)
Is open source
Howto:
AWS Construct Library - provides module for AWS services
is a CLI tool
sythesize artifacts (templates)
deploy stacks
delete stacks
diff stacks (to understand the change)
AWS Toolkit for VS Code
Note: CDK project template uses directory name to name things in generated code, be careful
Apps
application written in a programming language (TS) using AWS CDK
Define one or more stacks
#!/usr/bin/env node
import 'source-map-support/register'; // for reporting source line in ts in stack trace
import * as cdk from '@aws-cdk/core';
import { HelloCdkStack } from '../lib/hello-cdk-stack';
const app = new cdk.App();
new HelloCdkStack(app, 'HelloCdkStack');
Stacks
Equivalent to Cloudformation stacks
Contains constructs
import * as cdk from '@aws-cdk/core';
import * as s3 from '@aws-cdk/aws-s3';
export class HelloCdkStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// The code that defines your stack goes here
new s3.Bucket(this, 'MyFirstBucket', {
versioned: true
});
}
}
Constructs
Define AWS resources (such as s3.CfnBucket)
Flavors:
L1: CloudFormation Only - corresponds directly to AWS resources (name always starts with "cfn", e.g. "CfnBucket")
L2: Curated - developed by CDK team for specific use cases and to simplify development, e.g. "Bucket", provides sensible defaults, best-practice security policies, may define supporting resources
L3: Pattern - multiple resources for particular use case, all plumbng is hooked up, configuration boiled down to a few important ones
Constructor of constructs: takes 3 parameters
scope: a stack or another construct (creating a tree) - almost always "this"
Id: the logical ID
this is the logical ID in CDK, e.g. MyFirstBucket, https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_core.ConstructNode.htmlhttps://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_core.ConstructNode.html
then the logical ID in synthesized template would have a suffix, e.g. MyFirstBucketB8884501
then the actual AWS resource ID may like "hellocdkstack-myfirstbucketb8884501-1isk8hbjjdpre"
props: properties
Create new class that extends Construct base class
export interface NotifyingBucketProps {
prefix?: string;
}
export class NotifyingBucket extends Construct {
constructor(scope: Construct, id: string, props: NotifyingBucketProps = {}) {
super(scope, id);
const bucket = new s3.Bucket(this, 'bucket');
const topic = new sns.Topic(this, 'topic');
bucket.addObjectCreatedNotification(new s3notify.SnsDestination(topic),
{ prefix: props.prefix });
}
}
Some values (e.g. name of a S3 bucket) only become available after deployed. Therefore CDK uses a Token (https://docs.aws.amazon.com/cdk/latest/guide/tokens.html) to represent that value. Use "cdk.isToken" to determine if a value is token.
String encoded token can be concatenated (use as a string) "const functionName = `${bucket.bucketName}Function`;". Other tokens see documentation.
When one resource refer to another resource, usually can pass that resource instance. Or if necesary can use id such as: "bucket.bucketName", "lambdaFunc.functionArn", "securityGroup.groupArn"
Can refer to resource in a different stack - as long as in same account & region. This result in automatic synthesize of "export" from the producing stack and "Fn:ImportVale" in consuming stack.
For existing resources, use the "fromXXX" static methods to obtain existing resource instance. "s3.Bucket.fromArn..."
Not recommended to set physical name - if resource needs to be replaced (some properties are fixed after creation) stack update will fail, then only way is to delete and create stack, no update.
Pass in core.PhysicalName.GENERATE_IF_NEEDED - provide but not bother to name.
import * as cdk from '@aws-cdk/core';
... ... ...
const bucket = new s3.Bucket(this, 'Bucket', {
removalPolicy: cdk.RemovalPolicy.DESTROY,
});
Guidelines for construct behaviours:
include a set of "grant" methods to grant permissions to a principal
const rawData = new s3.Bucket(this, 'raw-data');
const dataScience = new iam.Group(this, 'data-science');
rawData.grantRead(dataScience);
composition
const jobsQueue = new sqs.Queue(this, 'jobs');
const createJobLambda = new lambda.Function(this, 'create-job', {
runtime: lambda.Runtime.NODEJS_10_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('./create-job-lambda-code'),
environment: {
QUEUE_URL: jobsQueue.queueUrl
}
});
https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_core.ConstructNode.html
Is separate form the construct instance but accessed from the construct instance's "node" property
provide various information of the construct, including: scope (parent), children, root, path, id (within scope), uniqueId(within scope)
Principals (who)
IAM resources (Role - can be assumed by other principals, User, Group)
Service (new iam.ServicePrincipal('service.amazonaws.com'))
Federated principals (new iam.FederatedPrincipal('cognito-identity.amazonaws.com'))
Account (new iam.AccountPrincipal('0123456789012'))
Canonical user
Organization
ARN (new iam.ArnPrincipal(res.arn))
Composite principal (multiple principles altogether)
Grants
constructs have "grantXXX" methods to grant access to IGrantable (https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-iam.IGrantable.html)
general "grant" grant specific permissions
Roles
explicit create role, "new iamRole"
add permission "addToPolicy" method
Use "grantXXX" method to grant permissions. This returns a Grant object, which may or may not success (existing resource). Use ".success" property to check, or use "assertSuccess" to enforce success.
Use a general "grant" to grant specific priviledge, such as "table.grant(func, 'dynamodb:CreateBackup');"
Resource that requires a role to work can be passed in a role. Otherwise a role will be created automatically.
Can model Policy using PolicyDocument objects. Add statements directly to roles (or a construct's attached role) using the addToRolePolicy method, or to a resource's policy (such as a Bucket policy) using the addToResourcePolicy method.
See https://docs.aws.amazon.com/cdk/api/latest/docs/aws-iam-readme.html
See https://docs.aws.amazon.com/cdk/latest/guide/permissions.html
import * as iam from '@aws-cdk/aws-iam';
const role = new iam.Role(this, 'Role', {
assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'), // required
});
role.addToPolicy(new iam.PolicyStatement({
effect: iam.Effect.DENY,
resources: [bucket.bucketArn, otherRole.roleArn],
actions: ['ec2:SomeAction', 's3:AnotherAction'],
conditions: {StringEquals: {
'ec2:AuthorizedService': 'codebuild.amazonaws.com',
}}}));
bucket.addToResourcePolicy(new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
actions: ['s3:SomeAction'],
resources: [bucket.bucketArn],
principals: [role]
}));
Resource emit metrics, metrics can be used to define alarms.
Use "metricXXX" method of constructs to create metrics. Use "createAlarm" method to create alarm when the metric cross a threshold.
import * as s3nots from '@aws-cdk/aws-s3-notifications';
const handler = new lambda.Function(this, 'Handler', { /*…*/ });
const bucket = new s3.Bucket(this, 'Bucket');
bucket.addObjectCreatedNotification(new s3nots.LambdaDestination(handler));
App is the only construct as the root of the construct tree.
Life cycle:
Construction - tree constructed (code executed)
Preparation - all constructs with "prepare" method implemented participate this phrase for a final modification (not recommended, not used in most cases)
Validate - all constructs with "validate" method validate themselves. Recommendation - validate and throw error as early as possilble.
Synthesis - app traverses the tree and call "synthesize" methods to emit template
Deployment - deploy on cloudformation
https://docs.aws.amazon.com/cdk/latest/guide/stacks.html
Difference in parameters with cloudformation:
CF: use parameters in single template, i.e. one template multiple stack (not recommended, parameters resolve at deploy time)
CDK: create different stacks (each has own template) for "prod", "beta" etc., recommended, parameters resolve at synthesize
Environment (account & region):
without explicit specify, resolve at deploy from CLI profile
recommended for production: hard code
CDK_DEFAULT_ACCOUNT and CDK_DEFAULT_REGION process environment variable (process.env.) - resolve at syntherise time
const envEU = { account: '2383838383', region: 'eu-west-1' };
const envUSA = { account: '8373873873', region: 'us-west-2' };
new MyFirstStack(app, 'first-stack-us', { env: envUSA });
new MyFirstStack(app, 'first-stack-eu', { env: envEU });
Stack API:
Stack.of - static method to find the stack of the given construct (in the stack the construct is defined)
stackName, region, account
Stack can be nested, see document
see AWS document "bootstraping", the DefaultStackSynthesizer class can be customised using properties, or even write own synthesizer:
qualifier - to deploy two different versions of same stack in same environment (acocunt + region), so to solve name conflict
change bootstrap resource names - only when using customised bootstrap template
Usage: cdk -a <cdk-app> COMMAND
Commands:
cdk list [STACKS..] Lists all stacks in the app [aliases: ls]
cdk synthesize [STACKS..] Synthesizes and prints the CloudFormation
template for this stack [aliases: synth]
cdk bootstrap [ENVIRONMENTS..] Deploys the CDK toolkit stack into an AWS
environment
cdk deploy [STACKS..] Deploys the stack(s) named STACKS into your
AWS account
cdk destroy [STACKS..] Destroy the stack(s) named STACKS
cdk diff [STACKS..] Compares the specified stack with the deployed
stack or a local template file, and returns
with status 1 if any difference is found
cdk metadata [STACK] Returns all metadata associated with this
stack
cdk init [TEMPLATE] Create a new, empty CDK project from a
template.
cdk context Manage cached context values
cdk docs Opens the reference documentation in a browser
[aliases: doc]
cdk doctor Check your set-up for potential problems
Options:
-a, --app REQUIRED: command-line for executing your app or a
cloud assembly directory (e.g. "node bin/my-app.js")
[string]
-c, --context Add contextual string parameter (KEY=VALUE) [array]
-p, --plugin Name or path of a node package that extend the CDK
features. Can be specified multiple times [array]
--trace Print trace for stack warnings [boolean]
--strict Do not construct stacks with warnings [boolean]
--lookups Perform context lookups (synthesis fails if this is
disabled and context lookups need to be performed)
[boolean] [default: true]
--ignore-errors Ignores synthesis errors, which will likely produce
an invalid output [boolean] [default: false]
-j, --json Use JSON output instead of YAML when templates are
printed to STDOUT [boolean] [default: false]
-v, --verbose Show debug logs (specify multiple times to increase
verbosity) [count] [default: false]
--debug Enable emission of additional debugging information,
such as creation stack traces of tokens
[boolean] [default: false]
--profile Use the indicated AWS profile as the default
environment [string]
--proxy Use the indicated proxy. Will read from HTTPS_PROXY
environment variable if not specified [string]
--ca-bundle-path Path to CA certificate to use when validating HTTPS
requests. Will read from AWS_CA_BUNDLE environment
variable if not specified [string]
-i, --ec2creds Force trying to fetch EC2 instance credentials.
Default: guess EC2 instance status [boolean]
--version-reporting Include the "AWS::CDK::Metadata" resource in
synthesized templates (enabled by default) [boolean]
--path-metadata Include "aws:cdk:path" CloudFormation metadata for
each resource (enabled by default)
[boolean] [default: true]
--asset-metadata Include "aws:asset:*" CloudFormation metadata for
resources that user assets (enabled by default)
[boolean] [default: true]
-r, --role-arn ARN of Role to use when invoking CloudFormation
[string]
--toolkit-stack-name The name of the CDK toolkit stack [string]
--staging Copy assets to the output directory (use
--no-staging to disable, needed for local debugging
the source files with SAM CLI)
[boolean] [default: true]
-o, --output Emits the synthesized cloud assembly into a
directory (default: cdk.out) [string]
--no-color Removes colors and other style from console output
[boolean] [default: false]
--fail Fail with exit code 1 in case of diff
[boolean] [default: false]
--version Show version number [boolean]
-h, --help Show help [boolean]
If your app has a single stack, there is no need to specify the stack name
If one of cdk.json or ~/.cdk.json exists, options specified there will be used
as defaults. Settings in cdk.json take precedence.
https://docs.aws.amazon.com/cdk/latest/guide/assets.html
refer to assets through constructs API
the synthesized cloud assembly (output) includes:
metadata about the local asset
including source hash to check change
by default creates a copy of the asset directly under the source hash (so the cloud-assembly is self-contained)
the CloudFormation template then contains synthesized parameters
at deploy time the assets needs first be published (S3 for codes, ECR for docker images), the location is coded in template as CF parameters
use "__dirname" for current path
S3 Assets: local files and directories CDK uploads to S3
Docker Image: Docker images that CDK uploads to ECR
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as path from 'path';
... ... ... ...
new lambda.Function(this, 'myLambdaFunction', {
code: lambda.Code.fromAsset(path.join(__dirname, 'handler')),
runtime: lambda.Runtime.PYTHON_3_6,
handler: 'index.lambda_handler'
});
Deploy-time parameter of assets
Create asset, refer to local asset
Access deploy time parameter
import { Asset } from '@aws-cdk/aws-s3-assets';
import * as path from 'path';
const imageAsset = new Asset(this, "SampleAsset", {
path: path.join(__dirname, "images/my-image.png")
});
new lambda.Function(this, "myLambdaFunction", {
code: lambda.Code.asset(path.join(__dirname, "handler")),
runtime: lambda.Runtime.PYTHON_3_6,
handler: "index.lambda_handler",
environment: {
'S3_BUCKET_NAME': imageAsset.s3BucketName,
'S3_OBJECT_KEY': imageAsset.s3ObjectKey,
'S3_URL': imageAsset.s3Url
}
});
Asset permission
const asset = new Asset(this, 'MyFile', {
path: path.join(__dirname, 'my-image.png')
});
const group = new iam.Group(this, 'MyUserGroup');
asset.grantRead(group);
Docker image asset see official document
Context:
key-value pairs (both strings)
as a cache (so no need to fetch everytime)
include feature flags
can create own context values
source (recommend file under source control):
automatically from current AWS account
provide with --context option
"cdk.context.json"
"cdk.json" under "context"
~/.cdk.json
in CDK app use construct.node.setContext method.
scope
scoped to the construct that created the context
automatic / from json - scoped to the App and visible to every construct
get information - context methods can perform some lookups, see https://docs.aws.amazon.com/cdk/latest/guide/context.html#context_methods
use information
construct.node.tryGetContext method - find under current construct scope or any of its parents
viewing, managing : "cdk context"
Feature Flags:
to enable potentially breaking behaviours
full list: https://github.com/aws/aws-cdk/blob/master/packages/@aws-cdk/cx-api/lib/features.ts
Apply operation to all constructs in a given scope (like adding tags)
Node.js
aws CLI and credential (`aws configure`)
Typescript
CDK (npm install aws-cdk)
(optional) AWS Toolkit for Visual Studio Code
cdk init app --language typescript
This creates a whole opinionated project skeleton: with package.json, tsconfig.json, jest.config.js, cdk.json, tests, bin, lib..... .
What REALLY matters:
cdk.json - parameters (cannot override)
(package.json) devDependencies
"@aws-cdk/assert": "1.83.0",
"@types/jest": "^26.0.10",
"@types/node": "10.17.27",
"jest": "^26.4.2",
"ts-jest": "^26.2.0",
"aws-cdk": "1.83.0",
"ts-node": "^9.0.0",
"typescript": "~3.9.7"
(package.json) dependencies
"@aws-cdk/core": "1.83.0",
"source-map-support": "^0.5.16"
(package.json) bin
"hello-cdk": "bin/hello-cdk.js"
Add Resource
e.g. `npm install @aws-cdk/aws-s3`
then define resource in stack
import * as cdk from '@aws-cdk/core';
import * as s3 from '@aws-cdk/aws-s3';
export class HelloCdkStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
new s3.Bucket(this, 'MyFirstBucket', {
versioned: true
});
}
}
see skeletons of apps, stacks above.
`cdk synch` - synthesize a template:
by default generates CDK meta data
`cdk deploy` - deploy:
it first synthesizes
may be asked to confirm security implications
Modifying and Updating
modify app, then
`cdk diff` - diff what's already deployed with current code
then deploy again. When broadening the privileges, will be prompted.
TLDR;
use "cdk bootstrap" to deploy a standard stack to provide resources for CDK deployment (S3 etc.)
the template and other artifacts can be found here: https://github.com/aws/aws-cdk/tree/master/packages/aws-cdk/lib/api/bootstrap
What is bootstrapping:
provide resources for the deployment of a CDK bootstrap stack (S3 bucket, code pipeline, etc.) - stack name "CDKToolkit"
currently transitioning from "legacy" (still the default) to "modern" - with code pipeline
through deployment of a CLoudFormation template
resulting a "CDKToolkit" in the Environment (account + region)
When bootstrapping is needed:
Stack uses assets or very large cloudformation template - requires S3 for uploading assets
CI/CD using CDK pipelines (still in development preview)
Which template is used:
transitioning from "legacy" to "modern"
`cdk bootstrap` uses the legacy template
depends on the synthesizer:
synthesizer generates stack template, including how to use bootstrap resources (S3, pipeline)
LegacyStackSynthesizer - can use both legacy and modern template, in fact only uses a S3
DefaultStackSynthesizer - requires modern template, capacity: cross-account deployments & CDK Pipeline deployments
synthesizer can be controlled by (1) set context key `@aws-cdk/core:newStyleStackSynthesis`, or (2) pass an instance in Stack constructor
How to bootstrap:
use `cdk bootstrap` command
`cdk bootstrap` - in default account and region
`cdk bootstrap aws://ACCOUNT-NUMBER-1/REGION-1 aws://ACCOUNT-NUMBER-2/REGION-2` ...
`cdk bootstrap --profile prod` : uses aws profile
`cdk bootstrap --show-template` : get a copy of the template, or check https://github.com/aws/aws-cdk/tree/master/packages/aws-cdk/lib/api/bootstrap
deploy the template manually
Tests:
snapshot tests - test the template against a previously stored baseline template: be sure the template does not change (or accept change explicitly) during refactorying
fine-grained assertions - test specifics of the template, such as "this resource has this property with this value"...
validation tests - "fast fail" so raise error when some data is invalid - i.e. when synthesizing and when accept user input at synthesize time, validate that the inputs are valid, validate that the user input correctly translates to stack property
How to:
Use jest
Use @aws-cdk/assert
Snapsht test:
see code snippet in AWS document
the test stores snapshot in "__snapshots__" dir - better check-in in version control
`npm test -- -u` - accept change of template
Fine-grained assertions:
see code snippet, something like 'expect(stack).toHaveResource('AWS::SQS::Queue', { someProperty:somevalue});
Validation tests:
see code snippet
basically, give various values to stack, test stack has the correct resource and property, or throw if value is invalid
When constructing Lambda, needs to provide the code, see https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-lambda/lib/code.ts for CDK definition of Code. The class has a set of static methods to define code:
fromBucket - as S3 object (bucket + key + optional version)
fromInline - code as string
fromAsset - path, either a directory with code bundle or a .zip
fromDockerBuild
fromCfnParameters - use CloudFormation parameters
fromEcrImage
fromAssetImage - create an ECR image from asset
In CDK issue https://github.com/aws/aws-cdk/issues/6294 opener claims "AssetCode" is "one-file lambdas without dependencies", this is wrong. "code: lambda.Code.fromAsset("build/lambdas")," indeed package the whole directory. However if the dependencies (node_modules) is not copied to that directory, it's not packaged. So a zip or copying the dependencies to a directory is still needed.
Solution 1: https://github.com/aws/aws-cdk/issues/6294#issuecomment-639085722 just put lambda code in a dir, put a package.json, install the dependencies there, make the dir root of package; or use other scripts to copy required packages into node_modules; " You could of course zip your index.js file together with your node_modules folder and use that as your artifact. However, this approach is not recommended because it unnecessarily slows down your Lambda function due to a bigger artifact size. You’re just carrying around code which you’re not using."
Solution 2: https://github.com/aws/aws-cdk/issues/6294#issuecomment-588061918 "@aws-cdk/aws-lambda-nodejs with a NodeJsFunction construct that automatically creates bundles for Node.js functions"
Document is here: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-nodejs-readme.html (still experimental)
Testing findings:
NodejsFunction works
Docker building is slow and cannot handle the symbol linked libraries in my monorepo
local esbuild is fast, and works well with the monorepo
the local esbuild can handle Typescript directly, no need to compile Typescript into Javascript first
This API is still experimental, though. So use with care.
See:
https://github.com/bingtimren/rush-vue-sam-study/releases/tag/cdk-nodejs-function
https://github.com/aws/aws-cdk/issues/6294#issuecomment-811550747
This is actually just an out-of-the-shelf bundling solution
Solution 3: use a bundler, "esbuild" could be a good choice
Solution 4: use an external stack, possibly SAM & AWS Serverless
References:
https://dev.to/seeebiii/5-ways-to-bundle-a-lambda-function-within-an-aws-cdk-construct-1e28