AWS - CI/CD (Jenkins)
Continuous integration/continuous delivery (CI/CD) model that Jenkins uses
Continuous Delivery • A software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time •
Continuous Integration • The practice, in software engineering, of merging all developer working copies to a shared mainline several times a day
The plugin ecosystem for Jenkins offers options for integration with these AWS services:
• Amazon EC2 Elastic Computer Cloud
• Amazon ECR Elastic Container Registry
• Amazon Simple Notification Service (SNS)
• Amazon ECS Elastic Container Service
• Amazon S3
• AWS CloudFormation
• AWS CodeDeploy
• AWS CodePipeline
• AWS CodeCommit (Note that the AWS DevOps Blog offers additional insights on how to integrate AWS CodeCommit with Jenkins.)
• AWS Device Farm
• AWS Elastic Beanstalk
The solutions that follow walk you through how to use these services together to create useful patterns.
Concepts such as minimum viable product (MVP), release candidate, velocity, etc. are all derived from these new approaches. In contrast, product teams using older paradigms like waterfall development might not hear back from customers for months and, quite often, not until the product is commercialized.
Two approaches to deploying Jenkins on AWS.
Traditional deployment on top of Amazon Elastic Compute Cloud (Amazon EC2).
Containerized deployment that leverages Amazon EC2 Container Service (Amazon ECS).
Jenkins Architecture -->It’s deployed as both a server and a build agent running on the same host. You can choose to deploy Jenkins as either a server or a build agent, which allows for decoupling orchestration and build execution. This, in turn, allows for more architecture design flexibility
Jenkins Worker Node Deployments -->. A worker node contains an agent that communicates with the master server and runs a lightweight Jenkins build that allows it to receive and run offloaded jobs. Figure: Master and Worker deployment options.
Jenkins installations generally fall into one of two scenarios:
1. A single, large master server with multiple worker nodes connected to it.
2. Multiple smaller master servers with multiple worker nodes connected to each. Figure: Jenkins deployment strategies
In both cases, one or more worker nodes are present. In larger systems, this is an important practice— do not build on the master. Choosing between a single master or multiple masters depends on a few factors, but usually we see customers adopt multiple masters.
For example, Netflix runs more than 25 masters on AWS.
The following table provides some criteria that you can use when you’re choosing which strategy best fits your needs:
Components of traditional CI/CD on AWS AWS code services
Cloud software development lifecycle
Components of CI/CD with AWS Code Services DEMO
Application architecture
Resource Considerations for Right-Sizing Your Jenkins Master
As with any AWS deployment, sizing your instance in terms of CPU, memory, and storage has an impact on performance and cost profiles. It’s therefore very important to make the right choices
Operating System -->flavors of Linux or to Windows environments using the Windows installer for Jenkins
CPU and Networking --> Built for scale and high availability varies based on the number of worker nodes that connect to a master node.node launches multiple threads per connection—two for each SSH and Java Web Start (JWS) connection and three for each HTTP connection
Storage--> Have large storage volumes on the worker nodes than on the master
Instance Type When building your Jenkins environment on Amazon EC2, consider CPU, networking, and storage.
bench marked five different instance types in our evaluations: the t2.large, the m3.medium, and the m4.large, m4.xlarge, and m4.2xlarge. Each benchmark simulated traffic from 100 concurrent users loading multiple pages inside the Jenkins dashboard for a sustained period of 10 minutes.
Installation->
To install the Jenkins Master Node, do the following. Master Node
1. AWS Management Console, launch the Amazon EC2 instance from an Amazon Machine Image (AMI) that has the base operating system you want.
2. Choose a security group that will allow SSH access as well as port 80 or 8080 to access your Jenkins dashboard. You should only enable ingress from the IP addresses you wish to allow access to your server.
3. Connect to the instance via SSH. 4. Update the yum package management tool. $ sudo yum update –y
5. Download the latest Jenkins code package.
$ sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins.io/redhat/jenkins.repo
6. Import a key file from Jenkins-CI to enable installation from the package.
$ sudo rpm --import https://pkg.jenkins.io/redhat/jenkins.io.key
7. Install Jenkins.
$ sudo yum install jenkins -y
8. Start Jenkins as a service.
$ sudo service jenkins start
9. Configure Jenkins, now that it’s installed and running on your Amazon EC2 instance. Use its management interface at port 80 or 8080, or remotely access the server via SSH. In its default configuration, Jenkins versions 2.0 and later lock down access to the management interface.
The first time you use the dashboard at http://your-server-address:8080, you will be prompted to unlock Jenkins:
As noted on the user interface, you can find this password in /var/lib/jenkins/secrets/ initialAdminPassword. Paste the value into the password box, then choose Continue. 10.The installation script directs you to the Customize Jenkins page. Choose Select plugins to install and select any plugins appropriate to your particular installation. For our example, ensure that the Git plugin (under Source Code Management) and SSH Slaves plugin (under Distributed Builds) are installed
Security Considerations-->At a minimum, incoming traffic to the Jenkins master should be locked down to the specific IP address ranges from which you expect traffic
Enable SSL--> Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates can be used to secure network
CSRF Protection--> Cross-site request forgery (CSRF) is a class of attack that forces an end user to execute unwanted actions on Jenkins.
By default,Jenkins 2.x installations have the CSRF protection option enabled.
Worker Node
Access Control --> visiting “Manage Jenkins,” then “Configure Global Security,” and ensuring that “Enable Slave->Master Access Control” is enabled
Configure User
Authentication --> create a First Admin User. Create a master user that can be used to create other groups and users, and then proceed to the Jenkins dashboard.
Securing Network Access--> Port 80 or 8080, for the ability to configure Jenkins and interact with the management dashboard. By default, the Jenkins dashboard is accessible through port 8080, but you can use iptables to redirect port 80 to 8080 and allow local connections:
$ sudo iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
$ sudo iptables -t nat -I OUTPUT -p tcp -o lo --dport 80 -j REDIRECT --to-ports 8080
2. Port 22, to connect via an SSH connection to the instances and perform maintenance.
