Embedded Files
Understand why docker is useful
Understand why docker is useful
This video (https://www.youtube.com/watch?v=Q5POuMHxW-0) explains what problem is solved.
Play with commands
Play with commands
This video (https://www.youtube.com/watch?v=UV3cw4QLJLs) explains how to create docker instance and also, more useful basic commands.
Understanding Docker volume
Understanding Docker volume
Create docker volume and attach to the container
root@ubuntu:~/personal/foss# docker volume create dktest
dktest
root@ubuntu:~/personal/foss# docker volume ls
DRIVER VOLUME NAME
local c4da78acbfbc0953e3f517a9a52fa60e90bceb84436a3813dafc329e8c060476
local dktest
local f0d1ee365eaf654225c40bfb49a56a511ff16122c2a36445f98cc6ed3a3a6311
root@ubuntu:~/personal/foss# docker run -dt -v dktest:/test pytest1:custom
51669488a92051d94a2ad5d004bd00adac63b8d06367fd63392c45f85f288b92
root@ubuntu:/test# docker exec -it 5166 bash
root@51669488a920:/# cd /test/
root@51669488a920:/test# touch abcd
root@51669488a920:/test# ls
abcd
root@ubuntu:/test# ls /var/lib/docker/volumes/dktest/_data/abcd
How networking work in Docker
How networking work in Docker
https://www.youtube.com/watch?v=vb7U_9AO7Ww
Difference between docker-engine and docker.io
Difference between docker-engine and docker.io
docker.io is maintained by Ubuntu
docker-engine is maintained by Docker
useful link: https://www.quora.com/What-is-the-difference-between-docker-engine-and-docker-io-packages
Understanding Dockerfiile
Understanding Dockerfiile
Useful link: https://stackoverflow.com/questions/24958140/what-is-the-difference-between-the-copy-and-add-commands-in-a-dockerfile
Docker for Fedora OS
Docker for Fedora OS
Guideline to install Docker
Guideline to install Docker
Refer each step in https://docs.docker.com/engine/installation/linux/docker-ce/fedora/#install-using-the-convenience-script . I needed to perform 'dnf update' to resolve the docker start issue by systemctl.
Programmatic way to manage Docker container
Programmatic way to manage Docker container
Python way
Python way
Use docker-python library. A Python library for the Docker Engine API. It lets you do anything the docker command does, but from within Python apps – run containers, manage containers, manage Swarms, etc
pip install docker-py
Useful link: https://docker-py.readthedocs.io/en/stable/
Troubleshooting
Troubleshooting
Method to override Entrypoint while creating container
Method to override Entrypoint while creating container
Use --enrypoint option in docker run (before the image name)
Method to override entry point in docker run
Useful link: http://www.johnzaccone.io/entrypoint-vs-cmd-back-to-basics/
Method to know storage used by Docker
Method to know storage used by Docker
Use docker system df
root@ubuntu:~/personal/# docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 15 5 2.92GB 2.139GB (73%)
Containers 7 1 547.6MB 447.8MB (81%)
Local Volumes 1 0 0B 0B
Useful link: https://stackoverflow.com/questions/26753087/docker-how-to-analyze-a-containers-disk-usage
Method to install specific docker version
Method to install specific docker version
Below example is to install 1.7.1 version
wget -O docker.deb https://apt.dockerproject.org/repo/pool/main/d/docker-engine/docker-engine_1.7.1-0~trusty_amd64.deb sudo dpkg -i docker.deb
Useful link: https://forums.docker.com/t/how-can-i-install-a-specific-version-of-the-docker-engine/1993/5
Method to clean all exited containers in one go
use grep, awk andn xargs
Cleanup exited containers
deepak@deepak-VirtualBox:~/promo/prometheus-exporter/test$ sudo docker ps -a | grep Exit
27e3a745a242 test_exporter "python /exporter.py…" 4 seconds ago Exited (0) 2 seconds ago test_exporter_run_3
6deb8969b879 test_exporter "python /exporter.py…" 4 minutes ago Exited (0) 4 minutes ago test_exporter_run_2
285def7c4510 test_exporter "python /exporter.py…" 17 minutes ago Exited (0) 17 minutes ago test_exporter_run_1
e7c06427b8d6 test_exporter "python /exporter.py…" 8 hours ago Exited (1) 24 minutes ago
deepak@deepak-VirtualBox:~/promo/prometheus-exporter/test$ sudo docker ps -a | grep Exit | awk '{print $1}' | xargs sudo docker rm
27e3a745a242
6deb8969b879
285def7c4510
e7c06427b8d6
deepak@deepak-VirtualBox:~/promo/prometheus-exporter/test$
Method to enable Dockerfile to run command in bash shell
Method to enable Dockerfile to run command in bash shell
Use /bin/bash -c option
RUN /bin/bash -c "source /usr/local/bin/virtualenvwrapper.sh"
Useful link: http://stackoverflow.com/questions/20635472/using-the-run-instruction-in-a-dockerfile-with-source-does-not-work
Command to install latest Docker in Ubuntu
Command to install latest Docker in Ubuntu
Install Docker
Approach 1-
Below command will install docker using apt-get tool
apt-get install docker-engine
Approach 2 -
Below command will install latest docker by running script
wget -qO- https://get.docker.com/ | sh
Check the version after any of above two approaches
root@ubuntu:~/personal/# docker --version
Docker version 17.03.0-ce, build 60ccb22
Useful link: https://forums.docker.com/t/how-can-i-install-a-specific-version-of-the-docker-engine/1993/5
Method to get memory usage and other resource usage for a docker container
Method to get memory usage and other resource usage for a docker container
'docker stats' command is useful for this purpose.
Example to get resource usage stats
[root@ubuntu ~]# docker ps | grep myubuntu
0e2dc4440e57 ubuntu:12.04 "/bin/bash" 44 seconds ago Up 41 seconds myubuntu
[root@ubuntu ~]# docker stats myubuntu --no-stream
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
myubuntu 0.00% 4.121 MiB / 5.868 GiB 0.07% 648 B / 648 B 3.781 MB / 0 B 0
Method to create docker image from running container
Method to create docker image from running container
'docker commit' command is useful for this purpose.
Method to create docker image from scratch
Method to create docker image from scratch
'docker build' command is useful for this purpose.
Example docker image creation from Dockerfile
[root@ubuntu /personal/learning_docker]# cat Dockerfile
#my docker
FROM ubuntu:14.04
RUN apt-get update && apt-get install -y vim curl
[root@ubuntu /personal/learning_docker]# docker build -t dk/myimage:11.1 .
.....
[root@ubuntu /personal/learning_docker]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
dk/myimage 11.1 578c9bbef73b 46 seconds ago 264.9 MB
Method to see bridge network in Linux
Method to see bridge network in Linux
brctl show
refer https://docs.docker.com/engine/userguide/networking/default_network/custom-docker0/
Method to assign static ip to the docker container in bridge mode
Method to assign static ip to the docker container in bridge mode
Example to configure static IP
docker network create --subnet=172.18.0.0/16 mynet123 -> Create a custom network
docker run --net mynet123 --ip 172.18.0.22 -it ubuntu bash --> instruct to use static IP
docker inspect 4d0bb | grep IP -> to confirm if 172.18.0.22 is assgined to the container
ping 172.18.0.22 -> to check if IP is pingable via docker host
Refer http://stackoverflow.com/questions/27937185/assign-static-ip-to-docker-container
Method to create multiple interfaces in a docker container using bridge networking
Method to create multiple interfaces in a docker container using bridge networking
Docker provides option to add custom bridge network with a docker container which is using docker default bridge network
docker network command to associate multiple interfaces to a docker container
Below example shows how an ubuntu container has been configured another interface (eth1 in this case).
[root@ubuntu /personal]# docker ps | grep ubuntu
5810abdd3b55 ubuntu:latest "/bin/bash" 34 hours ago Up 34 hours myubuntu
[root@ubuntu /personal]# docker exec -it myubuntu bash
root@5810abdd3b55:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:00:03
inet addr:172.17.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:30140 errors:0 dropped:0 overruns:0 frame:0
TX packets:25135 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:188053576 (188.0 MB) TX bytes:1695670 (1.6 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
[root@ubuntu /personal]# exit
[root@ubuntu /personal]# docker network ls
NETWORK ID NAME DRIVER SCOPE
d1bf12886f4f bridge bridge local
6b8724210568 host host local
e548f243e4ec none null local
[root@ubuntu /personal]# docker network create -d bridge --subnet 172.25.0.0/16 isolated_nw
6e0788bf5d2dc01f6d16118b8d25269be1e2f88254713e77c159fef335d99090
[root@ubuntu /personal]# docker network ls
NETWORK ID NAME DRIVER SCOPE
d1bf12886f4f bridge bridge local
6b8724210568 host host local
6e0788bf5d2d isolated_nw bridge local
e548f243e4ec none null local
[root@ubuntu /personal]# docker network connect isolated_nw myubuntu
[root@ubuntu /personal]# docker exec -it myubuntu bash
root@5810abdd3b55:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:00:03
inet addr:172.17.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:30140 errors:0 dropped:0 overruns:0 frame:0
TX packets:25135 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:188053576 (188.0 MB) TX bytes:1695670 (1.6 MB)
eth1 Link encap:Ethernet HWaddr 02:42:ac:19:00:02
inet addr:172.25.0.2 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe19:2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1296 (1.2 KB) TX bytes:648 (648.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Useful link:- https://docs.docker.com/engine/userguide/networking/work-with-networks/#basic-container-networking-example
Method to change shared memory size
Method to change shared memory size
There is an option in docker run command to modify shared memory size (--shm-size). However, documentation says that it works for posix shared memory management.
Increase shared memory max size using option in docker run command
root@ubuntu:~/personal/multi_intf# docker run -it -P --cap-add=ALL --shm-size=256m ubuntu:14.04 cat /proc/mounts | grep shm
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=262144k 0 0
Useful link: https://github.com/moby/moby/issues/24040
http://man7.org/linux/man-pages/man7/shm_overview.7.html
Dockerfile restructuring to improve installation time
Dockerfile restructuring to improve installation time
RUN commands clubbing
RUN commands clubbing
We can improve Docker installation time by clubbing multiple RUN commands. Its due to fact that docker executes commit command(docker commit) for each RUN command.
Example of RUN command optimization
Before:
RUN apt-get update && apt-get install -y openssh-server
RUN apt-get install -y openssh-client
RUN apt-get install -y tcpdump
After:
RUN apt-get update && apt-get install -y openssh-server openssh-client tcpdump
Unable to delete 211323c72f0d (must be forced) - image is referenced in one or more repositories
Unable to delete 211323c72f0d (must be forced) - image is referenced in one or more repositories
Check if the same image ID is shared among different images.
Handling case where same ID is shared among docker imags
[root@ubuntu /]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nslx latest 3ed03f312aa6 6 days ago 2.599 GB
dpkumar/testexample 1.0 211323c72f0d 7 days ago 264.9 MB
hello2 latest 211323c72f0d 7 days ago 264.9 MB
dk/myimage 11.1 578c9bbef73b 7 days ago 264.9 MB
hello latest 578c9bbef73b 7 days ago 264.9 MB
=============
Note that 211323c72f0d is shared among 2 images. To delete, we should use docker rmi <tag>. For example,
docker rmi hello2:latest
Example error
[root@ubuntu ~]# docker push dpkumar/testexample:latest
The push refers to a repository [docker.io/dpkumar/testexample]
7bf04ad1d006: Preparing
023b7696bd58: Preparing
ad37eb38337a: Preparing
350bf4dddc59: Preparing
554f627f09ee: Preparing
toomanyrequests: too many failed login attempts for username or IP address
You need to retry after 10 minutes. It is docker security feature.
Useful link:https://forums.docker.com/t/too-many-failed-login-attempts/7502/8
Method to enable pull from insecure registry
Method to enable pull from insecure registry
Create an entry for exception in /etc/docker/daemon.json . You need to restart docker service (service docker restart)
adding entry to /etc/docker/daemon.json
root@ubuntu:~# cat /etc/docker/daemon.json
{ "insecure-registries":["10.106.76.229:5000"] }
Useful link: https://stackoverflow.com/questions/42211380/add-insecure-registry-to-docker
How to create docker bridge mode container with assigned IP in the range of Host network IP
How to create docker bridge mode container with assigned IP in the range of Host network IP
Use macvlan feature of docker container as shown in below example.
Text Box
root@ubuntu:~/personal/auto# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether d2:7b:ed:29:3c:92 brd ff:ff:ff:ff:ff:ff
inet 10.102.169.37/24 brd 10.102.169.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::d07b:edff:fe29:3c92/64 scope link
valid_lft forever preferred_lft forever
Create a docker network:
#docker network create -d macvlan --subnet=10.102.169.0/24 --ip-range=10.102.169.192/26 --gateway=10.102.169.1 -o macvlan_mode=bridge -o parent=eth0 macvlantest
root@ubuntu:~/personal/auto# docker network inspect macvlantest
[
{
"Name": "macvlantest",
"Id": "e49d8bf11ba4a7be622a591799987f7af289c07c8e7c0d68f4aa526bf522b525",
"Created": "2017-11-23T16:12:18.461084242+05:30",
"Scope": "local",
"Driver": "macvlan",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "10.102.169.0/24",
"IPRange": "10.102.169.192/26",
"Gateway": "10.102.169.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"5555b439b553591df89f0bb6dbc0fb5b8243c9f2c75ef0606e327d27383cca5e": {
"Name": "jovial_jepsen",
"EndpointID": "8baf3935a047ca1e0ecee1612e01d90a8e5e6575f44c5f4b285280edcdd01de3",
"MacAddress": "02:42:0a:66:a9:c0",
"IPv4Address": "10.102.169.192/24",
"IPv6Address": ""
},
"a0cdd1fccdd0973b6d52899eda68341a8daae473a1df8670d293ae86c9841638": {
"Name": "dktrest",
"EndpointID": "4a616fb8bd713d3c32215f9e1be1522998dffc28be4abbd0ec5e8df8ad167bc7",
"MacAddress": "02:42:0a:66:a9:c1",
"IPv4Address": "10.102.169.193/24",
"IPv6Address": ""
}
},
"Options": {
"macvlan_mode": "bridge",
"parent": "eth0"
},
"Labels": {}
}
]
root@ubuntu:~/personal/auto# docker run -dt --net=macvlantest --name myub ubuntu:16.04
59cd3fdab98a3d63470731e327929521958a8afdecb3e451274315849dc5a862
root@ubuntu:~/personal/auto#
root@ubuntu:~/personal/auto# docker inspect myub | grep -i ipaddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "10.102.169.194",
Here 10.102.169.194 will be accessible from outside
Useful link: https://www.youtube.com/watch?v=3vI2DVb8qWk
https://docs.docker.com/engine/userguide/networking/get-started-macvlan/#pre-requisites
How to create docker host mode container with assigned IP in the range of Host network IP
How to create docker host mode container with assigned IP in the range of Host network IP
Use Linux macvlan feature as shown in below example.
configure docker container with macvlan
root@device-31822:~# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether fa:16:3e:58:57:29 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether fa:16:3e:7a:fa:83 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether fa:16:3e:cd:52:30 brd ff:ff:ff:ff:ff:ff
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
link/ether 02:42:4d:47:c3:64 brd ff:ff:ff:ff:ff:ff
root@device-31822:~# ip link add mac0 link eth0 type macvlan
root@device-31822:~#
root@device-31822:~# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether fa:16:3e:58:57:29 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether fa:16:3e:7a:fa:83 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether fa:16:3e:cd:52:30 brd ff:ff:ff:ff:ff:ff
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
link/ether 02:42:4d:47:c3:64 brd ff:ff:ff:ff:ff:ff
6: mac0@eth0: <BROADCAST,MULTICAST> mtu 8950 qdisc noop state DOWN mode DEFAULT group default
link/ether 4e:ed:8f:3c:2a:4e brd ff:ff:ff:ff:ff:ff
root@ubuntu:~# docker run -dt --privileged=true -e EULA=yes --ulimit core=-1 --net=host -e NS_NETMODE=HOST -P -e CPX_NW_DEV='mac0 eth1' -e NS_IP=10.102.169.42 -e NS_GATEWAY=255.255.255.0 --name dktest cpx:latest
root@ubuntu:~# docker exec -it dktest bash
root@ubuntu:/# cli_script.sh 'sh ip'
exec: sh ip
Ipaddress Traffic Domain Type Mode Arp Icmp Vserver State
--------- -------------- ---- ---- --- ---- ------- ------
1) 10.102.169.42 0 NetScaler IP Active Enabled Enabled NA Enabled
2) 192.0.0.1 0 SNIP Active Enabled Enabled NA Enabled
3) 1.1.1.100 0 SNIP Active Enabled Enabled NA Enabled
Done
Method to view namespace assigned to container
Method to view namespace assigned to container
use /proc data with the help of readlink
reading namespaces of all processes
root@ubuntu:~/personal/# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
167765dbef7c hyperledger/composer-playground "pm2-docker compos..." 7 days ago Up 7 days 0.0.0.0:8080->8080/tcp composer
85e03aa0b32b hyperledger/fabric-peer:x86_64-1.0.4 "peer node start -..." 7 days ago Up 7 days 0.0.0.0:7051->7051/tcp, 0.0.0.0:7053->7053/tcp peer0.org1.example.com
b95503df7ffb hyperledger/fabric-couchdb:x86_64-1.0.4 "tini -- /docker-e..." 7 days ago Up 7 days 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp couchdb
42adc8aefa7d hyperledger/fabric-ca:x86_64-1.0.4 "sh -c 'fabric-ca-..." 7 days ago Up 7 days 0.0.0.0:7054->7054/tcp ca.org1.example.com
60e8735ec29d hyperledger/fabric-orderer:x86_64-1.0.4 "orderer" 7 days ago Up 7 days 0.0.0.0:7050->7050/tcp orderer.example.com
root@ubuntu:~/personal/# readlink /proc/*/task/*/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026531839]
ipc:[4026532362]
ipc:[4026532421]
ipc:[4026532480]
ipc:[4026532541]
ipc:[4026532602]
mnt:[4026531840]
mnt:[4026531857]
mnt:[4026532338]
mnt:[4026532339]
mnt:[4026532360]
mnt:[4026532419]
mnt:[4026532478]
mnt:[4026532539]
mnt:[4026532600]
net:[4026532101]
net:[4026532365]
net:[4026532424]
net:[4026532483]
net:[4026532544]
net:[4026532605]
pid:[4026531836]
pid:[4026532363]
pid:[4026532422]
pid:[4026532481]
pid:[4026532542]
pid:[4026532603]
user:[4026531837]
uts:[4026531838]
uts:[4026532361]
uts:[4026532420]
uts:[4026532479]
uts:[4026532540]
uts:[4026532601]
Display namespaces for a particular process in a container
root@ubuntu:~/personal/# docker exec -it composer ps
PID USER TIME COMMAND
1 composer 1:53 node /home/composer/.npm-global/bin/pm2-docker composer-p
24 composer 6:03 PM2 v2.7.2: God Daemon (/home/composer/.pm2)
30 composer 3:18 node /home/composer/.npm-global/bin/composer-playground
54 composer 0:00 ps
root@ubuntu:~/personal/# ps ax | grep composer-playground
25549 ? Ssl 1:53 node /home/composer/.npm-global/bin/pm2-docker composer-playground
25675 ? Ssl 3:18 node /home/composer/.npm-global/bin/composer-playground
32161 pts/6 S+ 0:00 grep --color=auto composer-playground
root@ubuntu:~/personal/# readlink /proc/25675/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026532602]
mnt:[4026532600]
net:[4026532605]
pid:[4026532603]
user:[4026531837]
uts:[4026532601]
Method to compare namespaces in different containers of same image source
root@ubuntu:~# docker run -dt --cap-add=NET_ADMIN -e EULA=yes --ulimit core=-1 store/citrix/netscalercpx:12.0-53.16
7722eb15115850ada0545ad2b8847693ae5cf29f76f0117bd984c3d9dffd142b
root@ubuntu:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAM
ES
7722eb151158 store/citrix/netscalercpx:12.0-53.16 "/var/netscaler/bi..." 4 seconds ago Up 3 seconds 22/tcp, 80/tcp, 443/tcp, 161/udp gif
ted_brahmagupta
root@ubuntu:~# ps ax | grep nsppe
19664 ? Ss 0:01 /var/netscaler/bins/nsppe 1
root@ubuntu:~# readlink /proc/19664/task/*/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026532358]
mnt:[4026532356]
net:[4026532361]
pid:[4026532359]
user:[4026531837]
uts:[4026532357]
root@ubuntu:~# ps ax | grep nsconfigd
19777 ? Ss 0:00 /var/netscaler/bins/nsconfigd -S
19865 pts/11 S+ 0:00 grep --color=auto nsconfigd
root@ubuntu:~# readlink /proc/19777/task/*/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026532358]
mnt:[4026532356]
net:[4026532361]
pid:[4026532359]
user:[4026531837]
uts:[4026532357]
root@ubuntu:~# ps ax | grep nsnetsvc
6360 pts/2 S+ 0:00 gdb dbgbins/bin/nsnetsvc
19761 ? Ss 0:00 /var/netscaler/bins/nsnetsvc -S
19877 pts/11 S+ 0:00 grep --color=auto nsnetsvc
root@ubuntu:~# readlink /proc/19761/task/*/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026532358]
mnt:[4026532356]
net:[4026532361]
pid:[4026532359]
user:[4026531837]
uts:[4026532357]
root@ubuntu:~# docker run -dt --cap-add=NET_ADMIN -e EULA=yes --ulimit core=-1 store/citrix/netscalercpx:12.0-53.16
9d16297a998652599457ef8c955c4768db66a869521795421a4d51eec671f349
root@ubuntu:~# ps ax | grep nsnetsvc
6360 pts/2 S+ 0:00 gdb dbgbins/bin/nsnetsvc
19761 ? Ss 0:00 /var/netscaler/bins/nsnetsvc -S
20133 ? Ss 0:00 /var/netscaler/bins/nsnetsvc -S
20221 pts/11 S+ 0:00 grep --color=auto nsnetsvc
root@ubuntu:~# readlink /proc/20133/task/*/ns/* | sort -u
cgroup:[4026531835]
ipc:[4026532421]
mnt:[4026532419]
net:[4026532424]
pid:[4026532422]
user:[4026531837]
uts:[4026532420]
useful link: https://unix.stackexchange.com/questions/113530/how-to-find-out-namespace-of-a-particular-process
Method to install latest docker version in ubuntu
Method to install latest docker version in ubuntu
Use docker CE which is maintained by community
Example approach to install latest docker version in ubuntu
deepak@deepak-VirtualBox:~$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
OK
deepak@deepak-VirtualBox:~$ sudo apt-key fingerprint 0EBFCD88
pub 4096R/0EBFCD88 2017-02-22
Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid Docker Release (CE deb) <docker@docker.com>
sub 4096R/F273FCD8 2017-02-22
deepak@deepak-VirtualBox:~/personal/docker$ uname -a
Linux deepak-VirtualBox 4.10.0-40-generic #44~16.04.1-Ubuntu SMP Thu Nov 9 15:37:44 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
deepak@deepak-VirtualBox:~/personal/docker$ sudo add-apt-repository \
> "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
> $(lsb_release -cs) \
> stable"
deepak@deepak-VirtualBox:~/personal/docker$ sudo apt-get update
Get:1 https://download.docker.com/linux/ubuntu xenial InRelease [49.8 kB]
Hit:2 http://in.archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages [2,756 B]
Get:5 http://in.archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:6 http://in.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 359 kB in 1s (199 kB/s)
Reading package lists... Done
deepak@deepak-VirtualBox:~/personal/docker$ apt-cache search docker-ce
docker-ce - Docker: the open-source application container engine
deepak@deepak-VirtualBox:~/personal/docker$ apt-get install docker-ce
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
deepak@deepak-VirtualBox:~/personal/docker$ sudo apt-get install docker-ce
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
linux-headers-4.10.0-28 linux-headers-4.10.0-28-generic
linux-image-4.10.0-28-generic linux-image-extra-4.10.0-28-generic
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
aufs-tools cgroupfs-mount git git-man liberror-perl
Suggested packages:
git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk
gitweb git-arch git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
aufs-tools cgroupfs-mount docker-ce git git-man liberror-perl
0 upgraded, 6 newly installed, 0 to remove and 13 not upgraded.
Need to get 25.0 MB of archives.
After this operation, 126 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://download.docker.com/linux/ubuntu xenial/stable amd64 docker-ce amd64 17.09.1~ce-0~ubuntu [21.0 MB]
Get:2 http://in.archive.ubuntu.com/ubuntu xenial/universe amd64 aufs-tools amd64 1:3.2+20130722-1.1ubuntu1 [92.9 kB]
Get:3 http://in.archive.ubuntu.com/ubuntu xenial/universe amd64 cgroupfs-mount all 1.2 [4,970 B]
Get:4 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 liberror-perl all 0.17-1.2 [19.6 kB]
Get:5 http://in.archive.ubuntu.com/ubuntu xenial-updates/main amd64 git-man all 1:2.7.4-0ubuntu1.3 [736 kB]
Get:6 http://in.archive.ubuntu.com/ubuntu xenial-updates/main amd64 git amd64 1:2.7.4-0ubuntu1.3 [3,102 kB]
Fetched 25.0 MB in 8s (2,859 kB/s)
Selecting previously unselected package aufs-tools.
(Reading database ... 224641 files and directories currently installed.)
Preparing to unpack .../aufs-tools_1%3a3.2+20130722-1.1ubuntu1_amd64.deb ...
Unpacking aufs-tools (1:3.2+20130722-1.1ubuntu1) ...
Selecting previously unselected package cgroupfs-mount.
Preparing to unpack .../cgroupfs-mount_1.2_all.deb ...
Unpacking cgroupfs-mount (1.2) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../docker-ce_17.09.1~ce-0~ubuntu_amd64.deb ...
Unpacking docker-ce (17.09.1~ce-0~ubuntu) ...
Selecting previously unselected package liberror-perl.
Preparing to unpack .../liberror-perl_0.17-1.2_all.deb ...
Unpacking liberror-perl (0.17-1.2) ...
Selecting previously unselected package git-man.
Preparing to unpack .../git-man_1%3a2.7.4-0ubuntu1.3_all.deb ...
Unpacking git-man (1:2.7.4-0ubuntu1.3) ...
Selecting previously unselected package git.
Preparing to unpack .../git_1%3a2.7.4-0ubuntu1.3_amd64.deb ...
Unpacking git (1:2.7.4-0ubuntu1.3) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for systemd (229-4ubuntu21) ...
Setting up aufs-tools (1:3.2+20130722-1.1ubuntu1) ...
Setting up cgroupfs-mount (1.2) ...
Setting up docker-ce (17.09.1~ce-0~ubuntu) ...
Setting up liberror-perl (0.17-1.2) ...
Setting up git-man (1:2.7.4-0ubuntu1.3) ...
Setting up git (1:2.7.4-0ubuntu1.3) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for systemd (229-4ubuntu21) ...
Processing triggers for ureadahead (0.100.0-19) ...
deepak@deepak-VirtualBox:~/personal/docker$ docker version
Client:
Version: 17.09.1-ce
API version: 1.32
Go version: go1.8.3
Git commit: 19e2cf6
Built: Thu Dec 7 22:24:23 2017
OS/Arch: linux/amd64
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.32/version: dial unix /var/run/docker.sock: connect: permission denied
Useful link: https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository
Method to check which line of Dockerfile contributed to docker image size bloating
Method to check which line of Dockerfile contributed to docker image size bloating
Use docker history command
checking image size increase by each line in Dockerfile
deepak@deepak-VirtualBox:~/personal/docker$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 20c44cd7596f 3 weeks ago 123MB
store/citrix/netscalercpx 12.0-53.16 7293642af318 8 weeks ago 371MB
deepak@deepak-VirtualBox:~/personal/docker$ sudo docker history 729
IMAGE CREATED CREATED BY SIZE COMMENT
7293642af318 8 weeks ago /bin/sh -c #(nop) EXPOSE 161/udp 22/tcp 4... 0B
<missing> 8 weeks ago /bin/sh -c #(nop) CMD ["/var/netscaler/bi... 0B
<missing> 8 weeks ago /bin/sh -c easy_install /var/netscaler/egg... 11.4kB
<missing> 8 weeks ago /bin/sh -c (cd /tmp && mv /var/netscaler/... 35.5MB
<missing> 8 weeks ago /bin/sh -c mkdir -p /tmp 0B
<missing> 8 weeks ago /bin/sh -c echo "export VISIBLE=now" >> /e... 594B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV PLATFORM=CP1000 0B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV LS_PORT=27000 0B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV CPX_CONFIG={"YIELD"... 0B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV CPX_CORES=1 0B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV CPX_MAX_MEM=1024 0B
<missing> 8 weeks ago /bin/sh -c #(nop) ENV NOTVISIBLE=in users... 0B
<missing> 8 weeks ago /bin/sh -c sed 's@session\s*required\s*pam... 2.13kB
<missing> 8 weeks ago /bin/sh -c /usr/sbin/sshd 0B
<missing> 8 weeks ago /bin/sh -c sed -i 's/^PermitRootLogin.*/Pe... 2.53kB
<missing> 8 weeks ago /bin/sh -c echo 'nsroot:nsroot' | chpasswd 874B
<missing> 8 weeks ago /bin/sh -c useradd -ou 0 -g 0 -ms /bin/bas... 8.74kB
<missing> 8 weeks ago /bin/sh -c echo 'root:linux' | chpasswd 749B
<missing> 8 weeks ago /bin/sh -c "ldconfig" 28.1kB
<missing> 8 weeks ago /bin/sh -c "cp" "/var/netscaler/conf/nsld... 113B
<missing> 8 weeks ago /bin/sh -c "cp" "/var/netscaler/conf/nsss... 8.84kB
<missing> 8 weeks ago /bin/sh -c #(nop) ADD file:56a5727e4e14467... 121MB
<missing> 8 weeks ago /bin/sh -c mkdir /var/run/sshd 0B
<missing> 8 weeks ago /bin/sh -c #(nop) MAINTAINER NSCPX-Dev <N... 0B
<missing> 4 months ago 214MB Imported from -
Method to change / customise the docker storage directory (/var/lib/docker)
Method to change / customise the docker storage directory (/var/lib/docker)
Step guide for changing storage
root@ubuntu-14:/var/lib/docker# docker info
Containers: 0
Images: 0
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 0
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.19.0-25-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 7.788 GiB
Name: ubuntu-14
ID: 2ARA:MGPR:23DT:J7SN:E5VS:RD5N:NS3M:F2TT:XPC7:IYNR:ATCM:VF33
WARNING: No swap limit support
root@ubuntu-14:/var/lib/docker# mkdir /drive/docker
root@ubuntu-14:/var/lib/docker# vim /etc/default/docker
# Docker Upstart and SysVinit configuration file
# Customize location of Docker binary (especially for development testing).
#DOCKER="/usr/local/bin/docker"
# Use DOCKER_OPTS to modify the daemon startup options.
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"
DOCKER_OPTS="-g /drive/docker"
# If you need Docker to use an HTTP proxy, it can also be specified here.
#export http_proxy="http://127.0.0.1:3128/"
# This is also a handy place to tweak where Docker's temporary files go.
#export TMPDIR="/mnt/bigdrive/docker-tmp"
root@ubuntu-14:/var/lib/docker# service docker stop
docker stop/waiting
root@ubuntu-14:/var/lib/docker# service docker start
docker start/running, process 17901
root@ubuntu-14:/var/lib/docker# ps ax | grep docker
17901 ? Ssl 0:00 /usr/bin/docker -d -g /drive/docker
17944 pts/5 S+ 0:00 grep --color=auto docker
root@ubuntu-14:/var/lib/docker# docker info
Containers: 0
Images: 0
Storage Driver: aufs
Root Dir: /drive/docker/aufs
Backing Filesystem: extfs
Dirs: 0
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.19.0-25-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 7.788 GiB
Name: ubuntu-14
ID: 2ARA:MGPR:23DT:J7SN:E5VS:RD5N:NS3M:F2TT:XPC7:IYNR:ATCM:VF33
WARNING: No swap limit support
root@ubuntu-14:/var/lib/docker# du -hs /drive/docker/
9.9M /drive/docker/
root@ubuntu-14:/var/lib/docker# ls /drive/docker/
aufs containers graph init linkgraph.db repositories-aufs tmp trust volumes
root@ubuntu-14:/var/lib/docker# docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.2.1
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.2.1
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64
Useful link: https://forums.docker.com/t/how-do-i-change-the-docker-image-installation-directory/1169
Method to restrict docker log size (log rotation in docker)
Method to restrict docker log size (log rotation in docker)
Refer '--log-opt max-size=50m '
Create docker container with 50m max size
root@minikube:~/personal/cic/cpx# docker run -dt --log-opt max-size=50m --privileged=true -e EULA=yes --ulimit core=-1 --name dktest in-docker-reg.eng.citrite.net/cpx-dev/cpx:label2_703989
cb212e71877b296b044b35e0ec7418434173bfdfd1ef2406bb62d83850dfa431
Useful link: https://stackoverflow.com/questions/31829587/docker-container-logs-taking-all-my-disk-space
Method to know storage used by a container
Method to know storage used by a container
Use 'docker ps -a -s'
usage of docker ps -s to know storage of container
root@ubuntu:~# docker ps -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
fda243dc08c5 10.106.76.229:5000/nsoslx-appbuild-1.1-201705100:latest "bash" 8 minutes ago Up 7 minutes 0.0.0.0:32770->22/tcp nslx_cpx 1.511 GB
191a7cb69799 nslx:latest "bash" 13 months ago Up 2 days nslx_cpx_tot_devbox 6.059 GB
root@ubuntu:~# docker ps -a -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
fda243dc08c5 10.106.76.229:5000/nsoslx-appbuild-1.1-201705100:latest "bash" 8 minutes ago Up 8 minutes 0.0.0.0:32770->22/tcp nslx_cpx 1.661 GB
191a7cb69799 nslx:latest "bash" 13 months ago Up 2 days nslx_cpx_tot_devbox 6.059 GB
064ff3301d36 nslx:latest "bash" 14 months ago Exited (0) 18 minutes ago nslx_cpx_devbox_kopis 161.3 MB
Useful link: https://stackoverflow.com/questions/26753087/docker-how-to-analyze-a-containers-disk-usage
Sample Dockerfiles
Sample Dockerfiles
Dockerfile for creating python based docker container
Dockerfile for creating python based docker container
Dockerfile
FROM library/python:3.6.0-alpine
RUN pip install tornado’
ADD web-server.py /web-server.py
CMD ["python", "/web-server.py"]
Webserver.py
root@ubuntu:~# cat web-server.py
import tornado.ioloop
import tornado.web
import socket
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hostname: " + socket.gethostname())
def make_app():
return tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
app = make_app()
app.listen(8888)
tornado.ioloop.IOLoop.current().start()
root@ubuntu:~#
Reference
Reference
Personal experiment
https://www.youtube.com/watch?v=Q5POuMHxW-0
https://www.youtube.com/watch?v=UV3cw4QLJLs
http://stackoverflow.com/questions/32944391/how-to-remove-multiple-docker-images-with-the-same-imageid
Page updated
Google Sites
Report abuse