Code Manager setup

Date: may 2016

Product Version: PE 2016.1 (puppet 4.4)

## Code Manager setup to manage environment modules

Contents

Background:

Code Manager syncs environments for Puppet code. Everything is controlled by a remote "Control Repo" such as Github or Gitlab.

Prerequisites:

• Make sure youre on PE version 2016.1 and above.

• create a Github repo that will be central code repository (add your Puppet Masters SSH public key to your Github account), create 2 working branches, Development and Production

• on a Dev box (must be running same OS as Puppet Master), create a local repo with the Github repo set as the Remote (Origin). This is where you are making all changes to your code and pushing back to the Origin "control repo"

Section A - Code Manager config

1. in your basemodule path ( puppet config print | grep basemodule ), under the /code, make sure to remove all .git directories (except for .git in /environments/$environment path)

2. Code Manager doesnt like other git repos.

3. PS - make sure you DO NOT delete the .git FILE in /basemodulepath/environments/$environment, this file is used by Master when starting up.

4. Change ownership

5. chown -R pe-puppet:pe-puppet: /etc/puppetlabs/code

   1. generate SSH key for Code Manager, make sure key does not have a password, otherwise R10K wont work

   2. ssh-keygen

   3. copy the private SSH key on your Puppet master to this location /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa

   4. SSH key should be located in /root/.ssh, named 'id_rsa'

   5. set permissions of RSA file to 750, chown pe-puppet:pe-puppet

6. Open Console and click on Nodes > Classification

7. scroll down to node group PE Master, select Class puppet_enterprise::profile::master

• 1. Set code_manager_auto_configure = true

  2. r10k_private_key = his is the path to the private key that permits the pe-puppet user to access all Git repositories. This file must be owned by the pe-puppet user. Enter a string, such as '/etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa'.

  3. r10K_remote = path to Control Repo git

  4. file_sync_enabled = true

1. Run 'puppet agent -t' on Master

2. On Github, go to SSH keys page, and add your Master's SSH key as a trusted host.

Section B - Test the Control Repo

after Master is done running its catalog, run

r10k deploy display --fetch

This will attempt to connect to Control Repo (Github). If everything is setup correctly, this command will return all configured environments (branches) on your Control Repo

[root@puppetmaster3 ssh]# r10k deploy display --fetch

---

:sources:

- :name: :puppet

:basedir: "/etc/puppetlabs/code/environments"

:remote: git@github.com:perfecto25/puppet_code_manager.git

:environments:

- development

- production

- testing

Section C - Configure RBAC Role

Open up Console, click on Access Control and create a new Role "Deploy Environments", or select role called "Code Deployers" if its available

Click on the Role and click Permissions tab, select

Type="Puppet Environment"

Permissions="Deploy Code"

Object="All"

add another row,

Type="Tokens"

Permission="Override default expiry"

create a new user for Code Manager called "codemanager", give the user a password (click to generate a new password), click on "Member Users" tab, add the "codemanager" user to the "Deploy Environments" Role, commit all changes

Section D - Request Auth Token

create a new token for "codemanager" user, on Puppet Master, set lifetime to zero so its forever. (Make sure to use fully qualified hostname of PE Master, and not 'localhost')

add Puppet binaries to your path

export PATH=$PATH:/opt/puppetlabs/bin

Run Puppet Access

[root@puppetmaster3 bin]# puppet-access login --service-url https://puppetmaster3:4433/rbac-api --lifetime 0

Enter your Puppet Enterprise credentials.

Username: codemanager

Password:

Access token saved to: /root/.puppetlabs/token

your token is now ready, to see the token run

[root@puppetmaster3 bin]# puppet-access show

eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJjb2RlbWFuYWdlciIsImlhdCI6MTQ2MDc0MDQzNywiZXhwIjoxNDc2MjkyNDM3LCJzdWIiOnsiaWQiOiI0ZTNlYWQwYS1mMGU4LTQ2YTctOTI1OC1hMGM3ZGM3ZGIyYmEiLCJsb2dpbiI6ImNvZGVtYW5hZ2VyIn19.uausJ8sKvkeBYnm6J-aHNSjE9kGFep9xBLgVXcXMcfW8GmvzrXxIjjbMX1rkONjvJgn-E46wiFThFGU9K9qmvaZsgWrmDN55v52ggALK1n674XnDlDe0XGrgFw5REF8lZx31XrgICd1awOHFWLjSsg3KdYgssN-DRoflHkmQJvw7Mhy3m7G56i9zXw8CmFFLusUy6bymuilNfMfElKpF8Nx2pAG7fbeq3ULVGQHSUrfN435Zw_32cw6XiKAmO72MPl-VEPZkjwUqgxLqMQG8pJQCh0Uni8KKiUS4O683TeN1sUEb_YOfScYOJdjKID7eW4ID803JlZwh6wPQ4mrIJjYKM5EKGr3TTJX0Adkk3FXNaFl6WDv8N2lxzVO_aocIwJcbjRQ9mVjXj-PuXs0tkZqNPt1JdripogtFfZj_MOUt-gb9-1grJi0K5DpWRzxzlWqkZIaozQqgZ8W8SMhSZjNQih_QzByBbYeZdkaJP8xJWIGOBToWdb6S11_RDQ4envJIaB5mDgKelZdcFap9aQ1hEVbIicaPxP0xxL4xI7zP4wq2_dVanGkXywU8yZpRC6yBee4wca393skPGwVxiPrlZejo4a0r17YqcOOYQbKY8m5qP4UaCceIeUZKfBQwNClXNFmDaXPKj2Jps-lv6rrDtDxBiUB3KZY9Gbk8qFY

NOTE: if adding an LDAP user, see "Extras" below

Section E - Test Code Manager

puppet-code deploy my_test_environment --wait

[root@puppetmaster3 ssh]# puppet-code deploy production --wait

Deploying environment: production

[{"environment":"production","id":1,"status":"complete","file-sync":{"environment-commit":"c5a4bc11a77ac777dcb3d061c66f04ed1137d510","code-commit":"055f86dddde688ac566fa98eb9a95ed03f854c04"},"deploy-signature":"1745616811ae7c3adc48e8fa8e04947cd458aa09"}]

Code Manager will wipe out your /etc/puppetlabs/code dir of any previous modules and move modules from Puppetfile (Control Repo) to /etc/puppetlabs/code-staging

Once the code is staged, time to deploy your environments,

Section F - Deploy Code Manager using Command Line

puppet-code deploy production --wait

or to run deploy on all branches,

puppet-code deploy --all --wait

Modules will be stored in /etc/puppetlabs/code-staging dir, and then File Sync service will replicate this structure across all your Puppet Masters

IMPORTANT: If you want to manually add something to your modules or environments that is outside of your Github control repo, add it to the /etc/puppetlabs/code-staging dir, and NOT /etc/puppetlabs/code

If you add it to /etc/puppetlabs/code, it will be overwritten!

Section G - Create a Git Hook

• 1. Open up your Control Repo in Github

  2. Go to repository settings > webhooks & services > Add Webhook

• 1. for Payload URL: add your domain name for the Puppet master (Github must be able to resolve this name)

  2. https://PuppetMasterName:8170/code-manager/v1/webhook?type=github&token=<TOKEN>

  3. to get the Token, run

  4. puppet-access show

  5. copy entire block of token code into the Payload URL <TOKEN>

  6. eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJjb2RlbWFuYWdlciIsImlhdCI6MTQ2MDc0MDQzNywiZXhwIjoxNDc2MjkyNDM3LCJzdWIiOnsiaWQiOiI0ZTNlYWQwYS1mMGU4LTQ2YTctOTI1OC1hMGM3ZGM3ZGIyYmEiLCJsb2dpbiI6ImNvZGVtYW5hZ2VyIn19.uausJ8sKvkeBYnm6J-aHNSjE9kGFep9xBLgVXcXMcfW8GmvzrXxIjjbMX1rkONjvJgn-E46wiFThFGU9K9qmvaZsgWrmDN55v52ggALK1n674XnDlDe0XGrgFw5REF8lZx31XrgICd1awOHFWLjSsg3KdYgssN-DRoflHkmQJvw7Mhy3m7G56i9zXw8CmFFLusUy6bymuilNfMfElKpF8Nx2pAG7fbeq3ULVGQHSUrfN435Zw_32cw6XiKAmO72MPl-VEPZkjwUqgxLqMQG8pJQCh0Uni8KKiUS4O683TeN1sUEb_YOfScYOJdjKID7eW4ID803JlZwh6wPQ4mrIJjYKM5EKGr3TTJX0Adkk3FXNaFl6WDv8N2lxzVO_aocIwJcbjRQ9mVjXj-PuXs0tkZqNPt1JdripogtFfZj_MOUt-gb9-1grJi0K5DpWRzxzlWqkZIaozQqgZ8W8SMhSZjNQih_QzByBbYeZdkaJP8xJWIGOBToWdb6S11_RDQ4envJIaB5mDgKelZdcFap9aQ1hEVbIicaPxP0xxL4xI7zP4wq2_dVanGkXywU8yZpRC6yBee4wca393skPGwVxiPrlZejo4a0r17YqcOOYQbKY8m5qP4UaCceIeUZKfBQwNClXNFmDaXPKj2Jps-lv6rrDtDxBiUB3KZY9Gbk8qFY

  7. To check your Code Manager settings look in /etc/puppetlabs/client-tools/puppet-code.conf (do not manually edit this file on Puppet Master!)

  8. {

  9. "cacert": "/etc/puppetlabs/puppet/ssl/ca/ca_pub.pem",

  10. "token-file": "~/.puppetlabs/token",

  11. "service-url": "https://puppet:8170/code-manager"

  12. }

  13. Test the Code Deployment by making a commit to Github Control Repo, and watch Code Manager. Code Mgr should first put the changed file to /etc/puppetlabs/code-staging dir, and then File Sync service should push it across to /etc/puppetlabs/code across all Masters.

TO run code sync from another machine using CURL

run puppet-access show to get the token,

copy the token the /tmp on the machine where youre running code sync from

echo TOKEN# > /tmp/token

Run curl to get Master to sync code

curl -k -X POST -H 'Content-Type: application/json' -H "X-Authentication: `cat /tmp/token`" https://master.puppet.lan:8170/code-manager/v1/deploys -d '{"environments": ["production", "dev"]}'

Extras:

LDAP

if using LDAP for user authentication, read this first!!

https://github.com/glarizza/pe_curl_requests/blob/master/rbac/disable_nested_group_search.md

Using Puppet-Code from a client machine

To use puppet-code on an agent node or on a workstation that is not managed by PE or is not Puppet Master:

1. Install the client tools package.

2. the package is located in the Puppet Enterprise tarball, under Packages/Architecture

3. puppet-enterprise-2016.1.2-ubuntu-14.04-x86_64/packages/architecture

4. RPM based systems: pe-client-tools-1.0.2-1.el7.x86_64.rpm

5. APT systems: pe-client-tools-1.0.2-1.el7.x86_64.deb

6. Install the appropriate installer

7. Once package is installed, run 'puppet-code' and 'puppet-access'

8. If you get 'command not found', create a symlink to the binary

9. ln -s /opt/puppetlabs/client-tools/bin/puppet-code /usr/sbin/puppet-code

10. ln -s /opt/puppetlabs/client-tools/bin/puppet-access /usr/sbin/puppet-access

11. Create a config file called puppet-code.conf in the /etc/puppetlabs/client-tools folder.

12. In the puppet-code.conf file, specify your configuration settings in JSON format, like this example:

    1. {

    2. "cacert": "/etc/puppetlabs/puppet/ssl/certs/ca.pem",

    3. "token-file": "~/.puppetlabs/token",

    4. "service-url": "https://<PuppetMasterName>:8170/code-manager"

    5. }

    6. Note: if your client machine is not managed by Puppet, you will need to copy the ca.pem file from another Puppet-managed machine to your client machine and update the path to this file in the puppet-code.conf file.

    7. Open the Puppet Console, add the user's name or LDAP name to the Role you created for Code Manager (see section C), in this case "Deploy Environments" role

    8. Give the user the correct Permissions (Deploy code = All, Tokens)

• 1. Create a Puppet access-conf file that will be used to generate a login Token

  2. mkdir ~/.puppetlabs

  3. echo '{"service-url":"https://<PuppetMasterName>:4433/rbac-api"}' > ~/.puppetlabs/puppet-access.conf

  4. Generate Token:

  5. puppet-access login --lifetime 0 ## lifetime 0 = never expires, also can use --lifetime 1h, 1d, etc)

  6. [root@centos7node puppetlabs]# puppet-access login

  7. Enter your Puppet Enterprise credentials.

  8. Username: cookie_monster

  9. Password: ********

  10. Access token saved to: /root/.puppetlabs/token

  11. To see your generated token, run:

  12. puppet-access show

  13. Test the code deployment by running

  14. puppet-code deploy production --wait

Troubleshooting:

• When running code-manager, if the Puppet Server is crashing or cannot be started, delete all files in /etc/puppetlabs/code dir, and restart puppetserver

• If Puppet Server is crashing with following errors;

  • 2] [p.e.s.f.file-sync-client-service] Starting file sync client service 2016-05-16 16:51:05,585 INFO [async-dispatch-2] [p.s.l.legacy-routes-service] The legacy routing service has successfully started and is now ready to handle requests 2016-05-16 16:51:05,600 INFO [async-dispatch-2] [p.e.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests 2016-05-16 16:51:09,952 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':testing' to 2e585e239336a2254bce929f87f837f719dc4ab2 2016-05-16 16:51:12,443 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':testing' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/testing.git successful. New latest commit: 2e585e239336a2254bce929f87f837f719dc4ab2 2016-05-16 16:51:12,459 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':production' to e74f4df1f94e0bb90bd217e139f6086d1047674a 2016-05-16 16:51:13,182 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':production' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/production.git successful. New latest commit: e74f4df1f94e0bb90bd217e139f6086d1047674a 2016-05-16 16:51:13,213 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':puppet-code' to 02aeff610b9e1a6272f63a73190e2563ae51d155 2016-05-16 16:51:13,688 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':puppet-code' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git successful. New latest commit: 02aeff610b9e1a6272f63a73190e2563ae51d155 2016-05-16 16:51:13,879 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Forcefully syncing live directory at /etc/puppetlabs/code for repository :puppet-code 2016-05-16 16:51:15,946 ERROR [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fatal error during file sync, requesting shutdown. org.eclipse.jgit.api.errors.JGitInternalException: Destination path "production" already exists and is not an empty directory at org.eclipse.jgit.api.CloneCommand.init(CloneCommand.java:162) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:133) ~[puppet-server-release.jar:na] at org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:170) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.jgit_utils$eval31193$submodule_update__31198$fn__31199.invoke(jgit_utils.clj:532) ~[na:na] at puppetlabs.enterprise.jgit_utils$eval31193$submodule_update__31198.invoke(jgit_utils.clj:511) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33530$do_sync_BANG___33535$fn__33536.invoke(file_sync_client_core.clj:1086) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33530$do_sync_BANG___33535.invoke(file_sync_client_core.clj:1052) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33559$do_forced_sync_BANG___33564$fn__33565.invoke(file_sync_client_core.clj:1106) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33559$do_forced_sync_BANG___33564.invoke(file_sync_client_core.clj:1088) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667$fn__33671$fn__33674.invoke(file_sync_client_core.clj:1183) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667$fn__33671.invoke(file_sync_client_core.clj:1181) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667.invoke(file_sync_client_core.clj:1153) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__33985$service_fnk__5254__auto___positional$reify__34004.sync_live_dir_BANG_(file_sync_client_service.clj:130) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval33896$fn__33912$G__33888__33918.invoke(file_sync_client.clj:3) ~[na:na] at puppetlabs.enterprise.services.protocols.file_sync_client$eval33896$fn__33912$G__33887__33925.invoke(file_sync_client.clj:3) ~[na:na] at clojure.core$partial$fn__4527.invoke(core.clj:2494) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$sync_live_code__37978.invoke(file_sync_versioned_code_service.clj:48) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$fn__37984$fn__37995.invoke(file_sync_versioned_code_service.clj:80) ~[na:na] at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$fn__37984.invoke(file_sync_versioned_code_service.clj:72) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33011$process_callbacks_BANG___33016$fn__33017.invoke(file_sync_client_core.clj:606) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33011$process_callbacks_BANG___33016.invoke(file_sync_client_core.clj:587) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33070$process_repos_for_updates__33075$fn__33076.invoke(file_sync_client_core.clj:621) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33070$process_repos_for_updates__33075.invoke(file_sync_client_core.clj:608) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33097$sync_on_agent__33102$fn__33103.invoke(file_sync_client_core.clj:642) ~[na:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33097$sync_on_agent__33102.invoke(file_sync_client_core.clj:624) ~[na:na] at clojure.lang.AFn.applyToHelper(AFn.java:165) ~[puppet-server-release.jar:na] at clojure.lang.AFn.applyTo(AFn.java:144) ~[puppet-server-release.jar:na] at clojure.core$apply.invoke(core.clj:630) ~[puppet-server-release.jar:na] at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33372$start_periodic_sync_process_BANG___33377$fn__33378$periodic_sync__33379.doInvoke(file_sync_client_core.clj:896) ~[na:na] at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[puppet-server-release.jar:na] at clojure.core$apply.invoke(core.clj:636) ~[puppet-server-release.jar:na] at clojure.core$binding_conveyor_fn$fn__4444.doInvoke(core.clj:1928) ~[puppet-server-release.jar:na] at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[puppet-server-release.jar:na] at clojure.lang.Agent$Action.doRun(Agent.java:114) [puppet-server-release.jar:na] at clojure.lang.Agent$Action.run(Agent.java:163) [puppet-server-release.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] 2016-05-16 16:51:15,974 INFO [main] [p.t.internal] Beginning shutdown sequence

• RESOLUTION: Delete all .git repos in the following dir, /opt/puppetlabs/server/data/puppetserver/filesync/client/

• Start PuppetServer again and run code manager deploy again