DevOps
DevOps
2022/07/30 (新增連結)
2023/08/22 (新增連結)
2023/10/30 (新增投影片)
2023/11/12 (更新投影片)
基本概念
敏捷開發的工作從強調Integrate Often (Continuous Integration),後來更進一步強調Continuous Delivery(或Continuous Deployment),再進一步希望把開發與維運進行整合 (DevOps)。
DevOps 重視的是開發到維運的整合,包括如何從開發到維運(Dev to Ops)(如:持續性交付的管理流程)以及維運回饋給開發(Ops to Dev),除了利用工具之外,如何促成各方的合作(如:Dev & Ops)才是主要的重點(詳參: DevOps)。Davis & Daniels (2016) (Effective DevOps)強調DevOps不只是Dev及Ops兩方的合作,所以,才會有DevSecOps及DevQAOps的說法,事實上,應該是從開發一直到維運的相關活動及人員都屬於DevOps,所以,該書認為應該稱為devops。
在「鳳凰專案:看IT部門如何讓公司從谷底翻身的傳奇故事」這本書中,作者以三步工作法 (The Three Ways) 來描述如何達成上述的三件事(Dev to Ops、Ops to Dev、Dev & Ops)。書中特別強調要去找出計畫外的工作,並且控管計畫外的工作,還引用了TOC(約束理論,Theory of Constraints)去找到維運部門的瓶頸,把TOC從工廠的營運帶到資訊維運部門 (關鍵鍊把TOC應用在系統開發) ,也把Kanban從工廠的營運帶到資訊維運部門,談到風險管理,也強調要從企業的角度去看資訊系統的開發,是一本整合多種觀點的好書。
三步工作法 (參考: 鳳凰專案 The Phoenix Project & 鳳凰專案:看IT部門如何讓公司從谷底翻身的傳奇故事 & The Three Ways: The Principles Underpinning DevOps by Gene Kim)
第一步工作法 (The First Way):建立快速的工作流程
第二步工作法 (The Second Way): 縮短及增強回饋循環
第三步工作法 (The Third Way): 創造、形成公司文化,鼓勵探索進行改善活動,從錯誤中學習。
本文將本書做了一個相當精要的介紹
三步工作法
四種工作類型
也是一篇相當精要的介紹
《鳳凰專案》- 踏入 DevOps 之旅的第一本書 / 《鳳凰專案》- 踏入 DevOps 之旅的第一本書
本文提供了不少DevOps的相關資源
Work in process控制
三步工作法
四種工作類型
訂定組織目標
[讀書心得] 鳳凰專案-看IT部門如何讓公司從谷底翻身的傳奇故事
找出瓶頸
四個IT主要工作類型
三步工作法
The Phoenix project 導讀 (中央大學Ruddy Lee老師的導讀)
CIO必讀DevOps經典 (部分內容摘錄)
寶典一、DevOps:原理、方法與實踐 作者:榮國平,張賀,邵棟,何勉...等
寶典二、軟件開發本質論:追求簡約、體現價值、逐步構建 作者:羅恩·傑弗裡斯 ( Ron Jeffries )
寶典三、Continuous Delivery 中文版:利用自動化的建置、測試與部署完美創造出可信賴的軟體發佈 作者:Jez Humble, David Farley;喬梁 譯、傅育文 審校
寶典四、網站可靠性工程 - Google 的系統管理之道 作者:Betsy Beyer, Jennifer Petoff, Chris Jone 著;孫宇聰 譯
寶典五、鳳凰專案 - 看IT部門如何讓公司從谷底翻身的傳奇故事 作者:Gene Kim, Kevin Behr, George Spafford 著;楊仁和 譯
寶典六、DevOps 實踐指南 作者:吉恩·金,耶斯·亨布爾,帕特裡克·德布瓦,約翰·威爾斯
寶典七、Effective DevOps 中文版 作者:Jennifer Davis, Ryn Daniels 著;陳正瑋 譯
As a DevOps Engineer, you need to provide solutions for issues and provide the documentation/knowledge of how to use the solution itself. How to write pipelines, how to deliver similar services, how to connect monitoring and logs aggregations.
DevOps, Observability, and the need to tear down organizational boundaries
DevOps Engineer is the new SysAdmin : If you’re not doing dev… you’re just doing ops
![](https://www.google.com/images/icons/product/drive-32.png)
2021/11/26
2023/11/12 (更新)
DevOps
技術面
管理面
DevOps運作現況之個案研究
DevOps Handbook 2nd Ed.
鳳凰專案
三步工作法
第一步工作法 (The First Way):建立快速的工作流程
第二步工作法 (The Second Way): 縮短及增強回饋循環
第三步工作法 (The Third Way): 創造、形成公司文化,鼓勵探索進行改善活動,從錯誤中學習。
![](https://www.google.com/images/icons/product/drive-32.png)
2023/10/30
將存放在github的程式 (如:react或vue專案)部署到vercel
註冊
註冊過的話,直接登入
第一次,要新增一個GitHub帳號
選擇要安裝的repository
匯入repository
設定佈署的環境變數
點選「Deploy」,就完成部署了
只要push進main (或任何分支),就會自動佈署了~
持續性交付/部署 (Continuous Delivery, CD)
持續性交付/部署 (詳參: Continuous Delivery 、Continuous Delivery中文版:利用自動化的建置、測試與部署完美創造出可信賴的軟體發佈、ContinuousDelivery.com) 重視的是如何持續的交付系統(服務),包括:
建置(build)及持續整合
環境與部署
發佈管理與符合性
測試
自動化單元測試
自動化驗收測試
自動化非功能性測試
容量(Capacity)測試
資料管理
持續交付的情境下,資料的管理更顯重要,一方面資料庫結構必須有版本管理,資料內容也是,另一方面,測試資料與營運資料間的管理也是很重要的議題
設置(configuration)管理
不只是程式碼的版本管控,外部函式庫以及元件的版本管理也很重要,甚至環境 (如:資料庫、伺服器)的設置也應該要管理
Create a repeatable and reliable process for releasing software
Automate, if possible, everything
Keep everything under version control
If it hurts, do it more often — bring the pain forward
Keep the build and test process short
Make every part of the process of building, deploying, testing and releasing software visible to everybody involved
Done Means Released
Continuous Deployment: Strategies to Eliminate Roadblocks
How Should Your Team Approach DevOps?
What Are the Goals of Continuous Deployment?
How Does Continuous Deployment Work in Traditional Organizations?
How Can DevOps Better Fit in a Traditional Delivery Pipeline?
How Can You Make Hand-Offs More Effective?
Benefits of This Approach
You’re forcing your team’s changes to stay at the top of the priority queue.
You’ll have a much better sense of how long it’ll take to turn around a feature.
Reducing the overall context switching necessary for developers.
The building of relationships
How Can You Eliminate Hand-Offs Completely?
Coordinating With Security Teams
Coordinating With Compliance Teams
Coordinating With QA/QC Teams
Continuous Deployment Requires Continuous Improvement
Tools
Jenkins
One of the most demanded CI/CD tools in the world. It has become so popular because of its open-source policy.
GitHub Actions
GitLab CI
Travis CI
Do you deliver as a Tanker, Truck, or Bicycle?
Tanker Delivery Pattern - Release Windows
Family Meal Truck Delivery Pattern — Deploy after each Sprint.
Pizza Bicycle Delivery Pattern — Deploy continuously.
Continuous Delivery Metrics Will Start Being Mapped to Business Value and ROI
持續性整合 (Continuous Integration, CI)
持續性整合,基本上就是經常性的整合,過去認為至少每天整合一次,現在甚至很多專案可以一天整合很多次 (詳參: Continuous Integration)。一般而言,持續性交付涵蓋持續性整合,差別在於持續性交付涵蓋最後的上線與部屬,後來,有了DevOps的概念之後,就以DevOps來涵蓋持續性交付及持續性整合。
陳胤宏(2014)整理了過去的文獻,認為維持續性整合的主要通則為:
程式碼的版本管理與控制
頻繁的提交程式碼與建置
使用建置工具來建置程式碼
程式碼的測試
訊息的反饋
利用這些通則訪問個案公司,發現:影響持續整合落實的因素
管理階層的支持與否
公司資源是否足夠
時間、人力
價值觀上的認知
質疑為何要這麼做
不了解這些工具的效益
不會用的人會覺得不實用
為何要因此增加負擔
為何要因此多學東西、多做很多事
陳易昇(2017)整理了過去的文獻認為阻礙持續整合落實的因素,可以分為工具限制與團隊共識兩類:
在工具限制的因素大致有:
成本與管理上的考量
專案複雜度與相依性程度
建置與測試所耗費的時間
跨平台整合與測試等幾項
在團隊共識的因素大致有
團隊紀律
公司文化
高層支持與認知等幾項。
陳易昇(2017)發現
選擇持續性整合工具的考量因素除了成本之外,還會考量到與自身所開發的程式語言
導入持續整合依賴團隊共識與教育訓練
在陳胤宏(2014)中沒有看到持續性整合與敏捷式開發的相關性,然而,在本研究中看到持續性整合與敏捷式開發的相關性
與陳胤宏(2013)相較,本研究個案雖然都已經使用測試工具,但是,所使用的開發工具相對複雜許多,所以在測試方面還是很難全面自動化,例如個案B因為部分開發工作為行動平台環境,行動平台自動化測試工具尚未成熟,無法整合至持續整合平台;個案C在測試過程中需要大量外部資料,所以也無法整合至持續整合平台,這些難題尚待解決。
與陳胤宏(2013)相較,本研究發現當資深人員對持續整合接受程度不高時,管理階層需透過溝通或紀律等方法落實。
工具
陳易昇(2017)整理了個案公司所使用的工具
實作範例
Gitlab提供gitlab pages讓我們可以部署靜態網頁,所以,我們可以利用gitlab + gitlab pages部署react專案,來讓大家看一下一個非常陽春的CICD,部署後的網頁: https://benwu.gitlab.io/my-awesome-app/ (詳參: Build a React Website with full CI/CD in less than 2 minutes)
CICD設定檔
首先,我們需要一個react的專案,你需要在你的react專案裡新增一個設定檔.gitlab-ci.yml:
在設定檔的第一步是要設定一個docker,因為我們的react需要node,所以我們用的是docker hub上面的node image (https://hub.docker.com/_/node)
# Using the node image to build the React app
image: node
因為老師是利用Create-React-App (CRA)產生react專案,所以,需要一些設定:
# Announce the URL as per CRA docs
# https://create-react-app.dev/docs/advanced-configuration/
variables:
PUBLIC_URL: /my-awesome-app
# Cache node modules - speeds up future builds
cache:
paths:
- node_modules
接下來定義stage (先設定一個stage: deploy)
# Name the stages involved in the pipeline
stages:
- deploy
接下來設定要放到pages的內容
# Job name for gitlab to recognise this results in assets for Gitlab Pages
# https://docs.gitlab.com/ee/user/project/pages/introduction.html#gitlab-pages-requirements
pages:
對應的stage (deploy)
stage: deploy
放到pages之前要進行的動作,先利用npm安裝應有的packages,build,將舊檔案放到_public,將build好的檔案從build移到public
script:
- npm install # Install all dependencies
- npm run build --prod # Build for prod
- cp public/index.html public/404.html # Not necessary, but helps with https://medium.com/@pshrmn/demystifying-single-page-applications-3068d0555d46
- mv public _public # CRA and gitlab pages both use the public folder. Only do this in a build pipeline.
- mv build public # Move build files to public dir for Gitlab Pages
接下來設定路徑為public,並且設定只適用在master branch
artifacts:
paths:
- public # The built files for Gitlab Pages to serve
only:
- master # Only run on master branch
完整內容
.gitlab-ci.yml
# Using the node image to build the React app
image: node
# Announce the URL as per CRA docs
# https://create-react-app.dev/docs/advanced-configuration/
variables:
PUBLIC_URL: /my-awesome-app
# Cache node modules - speeds up future builds
cache:
paths:
- node_modules
# Name the stages involved in the pipeline
stages:
- deploy
# Job name for gitlab to recognise this results in assets for Gitlab Pages
# https://docs.gitlab.com/ee/user/project/pages/introduction.html#gitlab-pages-requirements
pages:
stage: deploy
script:
- npm install # Install all dependencies
- npm run build --prod # Build for production
- cp public/index.html public/404.html # Not necessary, but helps with https://medium.com/@pshrmn/demystifying-single-page-applications-3068d0555d46
- mv public _public # CRA and gitlab pages both use the public folder. Only do this in a build pipeline.
- mv build public # Move build files to public dir for Gitlab Pages
artifacts:
paths:
- public # The built files for Gitlab Pages to serve
only:
- master # Only run on master branch
當程式碼被push到gitlab的master branch時,會被自動佈署到Gitlab Pages (範例中只有deploy)
package.json也要改: (詳參: Using Create React App with Gitlab Pages)
{
"name": "my-awesome-app",
"version": "0.1.0",
"private": true,
"homepage": ".",
"dependencies": {
"react": "^16.8.6",
"react-dom": "^16.8.6",
"react-scripts": "2.1.8"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": "react-app"
},
"browserslist": [
">0.2%",
"not dead",
"not ie <= 11",
"not op_mini all"
]
}
參考資料
DevOps
DevOps Is Dead, Long Live NoOps
NoOps means no operation. Its philosophy is to remove all the platform management parts and reduce friction between developers and infrastructure.
How Is NoOps Possible?
A PaaS solution, like Heroku, or a cloud service hosted on Azure, AWS, and all other vendors.
Serverless computing bought from the big vendors (AWS, Azure).
Creating your replicable infrastructure (yes, this needs operations almost for the first time).
Release software in Cloud, Desktop and Enterprise Platforms: What is the production environment for Cloud, Desktop and Enterprise platforms
The Art of DevOps (從孫子兵法看DevOps)
Laying Plans (始計): Chapter 1- Skills, expertise, discipline, and process
Waging war (作戰): Chapter 2 — Size, Complexity, Uncertainty
Plan of attack (謀攻): Chapter 3 — Plan for automation, not manual processes
Importance of existing positions (軍形): Chapter 4 — configuration management, releases and versions
Use of energy (兵勢): Chapter 5 — Documentation, design and change requests
Weaknesses and Strengths (虛實): Chapter 6 — Bugs, defects, and security vulnerabilities
Maneuvering (軍爭): Chapter 7 — speed, performance, cadence, and deployment
Variations of tactics (九變): Chapter 8 — blue/green, configuration files, templates, and standards
On The March (行軍): Chapter 9 — Patching, networks, hardening and 3rd party APIs
Classification of Terrain (地形): Chapter 10 — Lack of time, cost or quality
The 9 Situations (九地): Chapter 11 — Phases of the DevOps lifecycle
Attack by Fire (火攻): Chapter 12 — direct attack and distractions
Use of Spies (用間)
DevSecOps
3 Reasons to use Containers in your DevSecOps pipeline : Improve the performance and productivity of your DevSecOps pipeline using containers.
CI
CICD
What is CICD — Concepts in Continuous Integration and Deployment
How to set up continuous deployment in your home project the easy way
Docker & Github
How To Setup Continuous Integration (CI) With React, CircleCI, and GitHub
CI vs CD vs CD — What Are The Key Differences?
CI: Continuous Integration
CD: Continuous Delivery
CD: Continuous Deployment
DevOps Tools
Must Learn DevOps Tools for 2020
Development and Build Tooling
#1 SCM + CI Tool of 2020: Gitlab and Gitlab-CI
#1 Data management tool of 2020: FlywayDB
Automated Testing Tools
#1 Integration testing tool of 2020: Cucumber
End-to-End Testing Tools
#1 End-To-End Testing Tool of 2020 — Functional: SoapUI Pro
#1 End-To-End Testing Tool of 2020 — Load Testing: LoadRunner
Deployment Tooling
#1 Artifact Management Tool of 2020: Nexus
#1 Config management tool of 2020: Ansible
#1 Deployment tool of 2020: Terraform
Runtime DevOps Tooling
#1 X-as-a-Service Tool of 2020: Amazon Web Services
#1 Orchestration tool of 2020: OpenShift
#1 Monitoring Tool of 2020: New Relic
#1 Logging Tool of 2020: Splunk
Collaboration DevOps Tooling
#1 Issue tracking tool of 2020: Jira
#1 ChatOps Tool of 2020: MatterMost
#1 Documentation Tool of 2020: Confluence
Feature Management
Feature management system requirements
Primitive feature flags vs Feature management system (part 1)
Decoupling releases from deployment (part 2)
Ring deployments (part 3)
Feature flags — best practices (part 5)
Introduction to Feature Toggling— Types, Use Cases, and Implementation
Release Toggles
Ops Toggles
Experiment Toggles
Permission Toggles
Unleash vs. LaunchDarkly: A Look at Feature Toggling Solutions
GitLab
DevOps
CI/CD
和艦長一起 30 天玩轉 GitLab 系列 (共30篇) (php)
[教學] 如何使用Gitlab CI來達到自動化測試與佈署 (node.js)
一個簡單的 GitLab CI 範例 (node.js)
Build a React Website with full CI/CD in less than 2 minutes **
Using GitLab to build, test and deploy modern front end applications
功能強大的 -- GitLab CI (php)
Gitlab-CI 入門實作教學
Continuous integration with Gitlab in Katalon
CICD on Gitlab & AWS
[Tutorial — Guide] Installing GitLab, GitLab CI on AWS EC2 from Zero (part 1)
Configuring .gitlab-ci.yml with AWS EC2 for Continuous Integration (CI) or Continuous Deplyment (CD) (part 3)
Troubleshooting GitLab and GitLab CI (part 4)
TDD & CICD
My version of the FDIC Bank Data Developer Portal is a Loopback based Node.js application generated from a Swagger, or Open API Specification, file with 2 endpoints to access FDIC Banks data.
When I submit new code to Github, the code is automatically tested using Postman and Newman using a Scripted Pipeline with Groovy in Jenkins to set up the Continuous Integration (CI).