Deployment Automation using Docker Compose

Goal - we will explore a use case where we have a two tier application - web tier and db tier. We will use docker to build deploy both together.

Directory tree

$ tree ~/docker-app/

/home/training/docker-app/

├── cassandra

│   ├── Dockerfile

│   └── init.cql

├── docker-compose.yml

└── flask-app

    ├── app.py

    ├── Dockerfile

    └── requirements.txt

Create docker-compose file. It defines two services - web-service and db-service. DB-service depends on web-service. DB_service hosts a cassandra database. Data of cassandra is stored on /var/lib/cassandra/data. This data directory is mapped to a volume on the docker host machine - with a purpose that if the container is recreated, we want to persist the data outside the container environment.

$ cat ~/docker-app/docker-compose.yml

version: "3"

services:

  web-service:

    build: ./web-service

    ports:

      - "5000:5000"

    volumes:

      - "./web-service:/app"

    depends_on:

      - db-service

  db-service:

    build: ./db-service

    volumes:

      - "/data:/var/lib/cassandra/data"

Define a dockerfile for DB-service.

$ cat ~/docker-app/db-service/Dockerfile

FROM cassandra:3.11.3

VOLUME /var/lib/cassandra/data

USER cassandra

RUN cassandra

Define a docker file web service. We are using a customized ubuntu instance that consists of python and python libraries - flask web framework and cassandra-driver - as the name indicates it is a python driver for Cassandra database.

$ cat ~/docker-app/web-service/Dockerfile

FROM  abasar/web_tier:1.0

COPY . /app

WORKDIR /app

CMD ["python", "app.py"]

Below is code of a web application - it exposes three path

/ - show hello world
/ init - initializes the database with requires schema
/counter - increment view count and show the current count

$ cat ~/docker-app/web-service/app.py

from cassandra.cluster import Cluster, NoHostAvailable

import socket

from flask import Flask

app = Flask(__name__)

session_ = None

db_host_ = "db-service"

def get_session():

  global session_

 if session_:

  return session_

 try:

  socket.gethostbyname(db_host_)

  cluster = Cluster([db_host_])

  session_ = cluster.connect()

 except socket.error:

  print("DB host is not found")

 except NoHostAvailable:

  print("DB host is up but the DB service is not available")

 return session_

@app.route("/", methods = ["get", "post"])

def hello():

    return "Hello world"

@app.route("/init")

def init():

    session = get_session()

    if session:

        session.execute("CREATE KEYSPACE IF NOT EXISTS demo WITH replication = {'class': 'SimpleStrategy','replication_factor': 1}")

        session.execute("CREATE TABLE IF NOT EXISTS demo.counters(name text primary key, count counter)")

        return "DB has been initialized"

    return "DB Session is not active"

@app.route("/counter")

def counter():

    session = get_session()

    session.execute("update demo.counters  set count += 1 where name = 'home'")

    count = session.execute("select count from demo.counters where name = 'home'").one()[0]

    return "Page view count: " + str(count)

if __name__ == "__main__":

    app.run(debug = True, host = "0.0.0.0", port = 5000)

Validate the docker-compose.yml file

$ sudo docker-compose config

services:

  db-service:

    build:

      context: /home/training/docker-app/db-service

    volumes:

    - /data:/var/lib/cassandra/data:rw

  web-service:

    build:

      context: /home/training/docker-app/web-service

    depends_on:

    - db-service

    ports:

    - 5000:5000/tcp

    volumes:

    - /home/training/docker-app/web-service:/app:rw

version: '3.0'

$ sudo docker-compose build

Building db-service

Step 1/4 : FROM cassandra:3.11.3

 ---> d05bb5bbfe7d

Step 2/4 : VOLUME /var/lib/cassandra/data

 ---> Using cache

 ---> c80e5ea58ce9

Step 3/4 : USER cassandra

 ---> Using cache

 ---> 29e1c9c5b13b

Step 4/4 : CMD ["cassandra"]

 ---> Using cache

 ---> de307c093a33

Successfully built de307c093a33

Successfully tagged docker-app_db-service:latest

Building web-service

Step 1/4 : FROM  abasar/web_tier:1.0

 ---> d6a39aa114c8

Step 2/4 : COPY . /app

 ---> Using cache

 ---> f70fccac3702

Step 3/4 : WORKDIR /app

 ---> Using cache

 ---> 117171fb6177

Step 4/4 : CMD ["python", "app.py"]

 ---> Using cache

 ---> 1ea7ab33e4bf

Successfully built 1ea7ab33e4bf

Successfully tagged docker-app_web-service:latest

You can find the files in the github link below.

https://github.com/abulbasar/docker-examples/tree/master/docker-app

Since we are using volume mapping from the DB service to /data, create the directory and give necessary permission - in this case we are giving full permission.

$ sudo mkdir /data

$ sudo chmod 777 /data

Start the services

$ sudo docker-compose up

Starting docker-app_db-service_1 ... done

docker-app_web-service_1 is up-to-date

Attaching to docker-app_db-service_1, docker-app_web-service_1

web-service_1  |  * Serving Flask app "app" (lazy loading)

web-service_1  |  * Environment: production

web-service_1  |    WARNING: Do not use the development server in a production environment.

web-service_1  |    Use a production WSGI server instead.

web-service_1  |  * Debug mode: on

web-service_1  |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

web-service_1  |  * Restarting with stat

web-service_1  |  * Debugger is active!

...

Initialize the data base

$ curl localhost:5000/init

DB has been initialized

$ curl localhost:5000/counter

Page view count: 1

Simple stress testing

On host machine, install apache-utils

$ sudo apt install apache-utils

Run the stress test

$ ab -n 1000 -c 10 -k http://localhost:5000/counter

This is ApacheBench, Version 2.3 <$Revision: 1706008 $>

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Completed 100 requests

Completed 200 requests

Completed 300 requests

Completed 400 requests

Completed 500 requests

Completed 600 requests

Completed 700 requests

Completed 800 requests

Completed 900 requests

Completed 1000 requests

Finished 1000 requests

Server Software:        Werkzeug/0.14.1

Server Hostname:        localhost

Server Port:            5000

Document Path:          /counter

Document Length:        19 bytes

Concurrency Level:      10

Time taken for tests:   8.359 seconds

Complete requests:      1000

Failed requests:        984

   (Connect: 0, Receive: 0, Length: 984, Exceptions: 0)

Keep-Alive requests:    0

Total transferred:      178067 bytes

HTML transferred:       20067 bytes

Requests per second:    119.64 [#/sec] (mean)

Time per request:       83.586 [ms] (mean)

Time per request:       8.359 [ms] (mean, across all concurrent requests)

Transfer rate:          20.80 [Kbytes/sec] received

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0    0   0.4      0       6

Processing:    35   83 100.4     69    1174

Waiting:       35   82 100.5     68    1173

Total:         35   83 100.9     69    1179

Percentage of the requests served within a certain time (ms)

  50%     69

  66%     75

  75%     81

  80%     85

  90%    101

  95%    115

  98%    151

  99%    861

 100%   1179 (longest request)