jenkins
Some notes for now
multi-branch pipeline
points to a .git repository, with include & exclude filters so only a selected set of branches are included.
build configuration mode has only 'By Jenkinsfile' available.
specify the jenkinfile's location in your .git repository. the path is the relative path within the repository.
the jenkinsfile needs to specify the way to trigger a build, stages, etc.
click into a multi-branch pipeline will see all the branches. everytime a new branch is created it will be displayed automatically.
pipeline
select build triggers from the UI (Poll SCM, Periodically, etc.)
pipeline definition can be (1) pipeline script from SCM (2) pipeline script
with pipeline script from SCM it needs to specify the .git repository and the branch and also the relative path of the script.
with pipeline script simply type in the script in the text box
The pipeline is defined by the pipeline script. e.g. PollSCM indicates the build is triggered by polling a git branch for a commit.
The pipeline syntax page on the UI has the sample scripts (generated) for different steps.
An example pipeline that runs windows command, build sql server database, generate ssis packages by biml and deploy database and ssis packages.
my_pipeline.jenkinsfile
pipeline {
agent { label 'a jenkins server' }
options {
disableConcurrentBuilds() //dont allow concurrent builds. for multi-branch pipeline it may break stuff, e.g. updating the same table
preserveStashes() // keep stashes from the most recent build, for resumes
buildDiscarder(logRotator(numToKeepStr: '3'))
}
triggers {
pollSCM('') // trigger on repository push, needs plugin installed
}
stages {
stage('Check who commit the change') {
steps {
script {
/** run script " git show -s --pretty=%an", %%=%, %an is author name, "returnStdout: true" means returning the script's result **/
/** keep the author namein AGENT_LABEL **/
env.AGENT_LABEL = bat(returnStdout: true, script: '@git show -s --pretty=%%an')
}
echo "The author is: ${env.AGENT_LABEL}"
}
}
stage('Run a windows command line which executes a sqlcmd command to run a sql script') {
agent { label env.AGENT_LABEL }
steps {
bat "\"C:\\Program Files\\Microsoft SQL Server\\Client SDK\\ODBC\\130\\Tools\\Binn\\sqlcmd\" -S myserver -d mydatabase -b -i myfolder\\a_script.sql
}
}
stage('Build a sql server database project') {
agent { label env.AGENT_LABEL }
steps {
/** run msbuild command to build a sql project which has the definitions of tables, views, functions, etc. The dacpac file is the build output **/
/** stash command keeps a copy of the dacpac files for later use on a different agent **/
/** unstash command will be able to restore the files on a different agent **/
/** msbuild is available on jenkins as a plugin**/
echo 'Build a database project and stash the dacpac file'
bat "\"${tool name: 'SSDT', type: 'msbuild'}\\msbuild\" \"myfolder\\database_project.sqlproj\" /p:Configuration=Release /p:BuildProjectReferences=false"
stash name: 'database_project_dacpac', includes: 'myfolder\\bin\\Release\\database_project.dacpac'
}
}
stage('deploy the database to another agent and build SSIS packages on that agent') {
agent { label "another agent" }
steps {
echo " running as: ${env.USERNAME}"
/** NOTE! this is a different server / agent **/
/** run SqlPackage.exe to publish a sql project **/
/** unstash the previously created dacpac sql project (on a different agent) and use it as the source file to publish a database **/
/** The database_publish.xml file specifies the publish options and connection string pointing to the designated database instance **/
unstash 'database_project_dacpac'
bat "\"C:\\Program Files\\Microsoft Visual Studio\\2017\\Professional\\Common7\\IDE\\Extensions\\Microsoft\\SQLDB\\DAC\\150\\SqlPackage\" /Action:Publish /SourceFile:\"myfolder\\bin\\Release\\database_project_dacpac.dacpac\" /Profile:\"build\\database_publish.xml\""
/** use biml compiler (biml express, licence key required) to generate packages in .ispac format which is the entire ssis project with all packages**/
/** stash the ispac file for later deployment **/
bat script: \"C:\\Program Files\\Varigence\\BimlStudio\\5.0\\bimlc\" --source=my_generate_packages.biml --targetPath=\"..\" --version=Ssis2014 --ssisDeploymentModel=Project --licenseKey=xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-x"
stash name: 'ssis_ispac', includes: 'myfolder\\bin\\ssis_packages.ispac'
}
}
stage('Deploy SSIS Packages') {
agent { label env.AGENT_LABEL }
steps {
/**
deploy the ispac file (generated by bimlc) to database using Deployment Wizard
this goes to the integration services catalogs
**/
unstash 'ssis_ispac'
bat "\"C:\\Program Files\\Microsoft SQL Server\\120\\DTS\\Binn\\ISDeploymentWizard\" /DestinationServer:(local) /DestinationPath:\"/SSISDB/my_ssis_project\" /Silent /SourceType:File /SourcePath:\"myfolder\\bin\\ssis_packages.ispac\""
}
}
}
post {
/**
email the results
**/
success {
wrap([$class: 'BuildUser']) {
emailext (
mimeType: 'text/plain',
to: "${env.AGENT_LABEL}",
subject: "Build Succeeded: ${env.JOB_NAME}",
body: """Build Number: ${env.BUILD_NUMBER} Succeeded. :o)
Check console output at ${env.BUILD_URL} to view the results."""
)
}
}
failure {
wrap([$class: 'BuildUser']) {
emailext (
mimeType: 'text/plain',
to: "${env.AGENT_LABEL}",
subject: "BUILD FAILTURE: ${env.JOB_NAME}",
body: """Build Number: ${env.BUILD_NUMBER} FAILED! :o(
Check console output at ${env.BUILD_URL} to view the results."""
)
}
}
}
}
Add a slave agent to Jenkins.
for example, adding a local computer
1. go to manage jenkins -> manage nodes ...
create a new node, choose permanent node, and specify #exeuctors, root folder, availability, etc.
2. go to the agent's machine, and browse to jenkins instance http://yourjenkinsmaster:8080
go to manage jenkins -> manage nodes again. click on the newly created node
a connection guide for Jenkins slave pops up.
method 1. click 'launch' to run the java app slave-agent.jnlp, make sure run using admin access.
method 2 and 3. some command lines doing the same thing.
This install a local agent on the computer and connects to the jenkins master.
Refresh the jenkins nodes list, should see the new node online.
3. if want to register the agent as a service,
simply go to the 'file' option on the java app, and install as a service.
Note, make sure a proper version (latest) java runtime environment is installed.
if proxy is used, make sure http_proxy, https_proxy and no_proxy are configured in environment variables properly.