Cloud Computing‎ > ‎microservice‎ > ‎

CRUD micro-service





This post covers the CRUD microservice creation using Golang\Cassandra and packaging micro-service as docker container and pushing to Git\Docker repository.

The CRUD micro-service is deployed over AWS and integrated with AWS ELB and Region53 as http://deepagar.net:8080/api/v1/policies/common 


Create a micro-service using Golang and Cassandra


Follow the steps at rest microservice with gin 

This page also covers pushing the micro-service to github repository at crud-service

Create the Dockerfile container using above micro-service


Create the Dockerfile project




Push the Dockerfile project to Git


Create a repo

https://github.com/deepagargit/crud-service

Clone the repo

git clone https://github.com/deepagargit/crud-service

List the current branch

$ git branch -a
* develop
  remotes/origin/HEAD -> origin/develop
  remotes/origin/develop

Switch to develop branch

$ git checkout develop
Already on 'develop'

Create docker folder 

$ mkdir docker
$ cd docker

Copy dockerfile and other project files


$ ls -R
.:
cassandra  crud  crud-cassandra

./cassandra:

./crud:

./crud-cassandra:
build.sh  Dockerfile  ReadMe  src

./crud-cassandra/src:
cassandra.yaml  datastax.repo  epel7.repo  main.go  start.sh  supervisord.conf



Add files to git


Touch the gitkeep file in empty folder.
touch .gitkeep

git add .

 git status
# On branch develop
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   cassandra/.gitkeep
#       new file:   crud-cassandra/Dockerfile
#       new file:   crud-cassandra/ReadMe
#       new file:   crud-cassandra/build.sh
#       new file:   crud-cassandra/src/cassandra.yaml
#       new file:   crud-cassandra/src/datastax.repo
#       new file:   crud-cassandra/src/epel7.repo
#       new file:   crud-cassandra/src/main.go
#       new file:   crud-cassandra/src/start.sh
#       new file:   crud-cassandra/src/supervisord.conf
#       new file:   crud/.gitkeep
#


Add User Config

git config --global user.email "deepak.aagarwal@gmail.com"

git config --global user.name "Deepak Agarwal"       


Commit the files


$ git commit -m "Added Docker project for CRUD service with Golang"
[develop 9aaaedb] Added Docker project for CRUD service with Golang
 11 files changed, 1421 insertions(+)
 create mode 100644 docker/cassandra/.gitkeep
 create mode 100644 docker/crud-cassandra/Dockerfile
 create mode 100644 docker/crud-cassandra/ReadMe
 create mode 100755 docker/crud-cassandra/build.sh
 create mode 100755 docker/crud-cassandra/src/cassandra.yaml
 create mode 100755 docker/crud-cassandra/src/datastax.repo
 create mode 100755 docker/crud-cassandra/src/epel7.repo
 create mode 100755 docker/crud-cassandra/src/main.go
 create mode 100755 docker/crud-cassandra/src/start.sh
 create mode 100755 docker/crud-cassandra/src/supervisord.conf
 create mode 100644 docker/crud/.gitkeep


Push to the develop branch


git push origin develop
Username for 'https://github.com': deepagargit
Password for 'https://deepagargit@github.com':
Counting objects: 14, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (13/13), 16.14 KiB | 0 bytes/s, done.
Total 13 (delta 1), reused 0 (delta 0)
To https://github.com/deepagargit/crud-service
   87c3096..9aaaedb  develop -> develop


Build the container image


./crud-cassandra/build.sh

Spawn the container


docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra

Login to container bash


sudo docker exec -i -t p4 bash

Validate cqlsh in container


$ cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.2.4 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> use demo;
cqlsh:demo> select * from policy;

 class | name | blob
-------+------+------

$ exit


Push the container image to docker repository

docker login
docker push deepagar/crud-cassandra



Deploy (Manually) the micro-service over AWS VM


Create the VM


Create the VM in EC2

Install docker


sudo curl -sSL https://get.docker.com/ | sh

sudo usermod -aG docker centos

List docker container


$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Start the docker contain using docker image from repository


docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra

Unable to find image 'deepagar/crud-cassandra:latest' locally
latest: Pulling from deepagar/crud-cassandra
fa5be2806d4c: Pull complete
98783644cfab: Pull complete
4609e0ebb5d4: Pull complete
c812b6264db1: Pull complete
ea1036165fa4: Pull complete
7a21760c41c7: Pull complete
542acddd307e: Pull complete
4ade23629f70: Pull complete
e94a69157b27: Pull complete
7e6c33318a29: Pull complete
96a5ae6fbdba: Pull complete
6500fd2abf12: Pull complete
c318cbc3e970: Pull complete
7a375ec1e6fb: Pull complete
b6495ac2f1b9: Pull complete
474f92c6f273: Pull complete
1f9b54b17fa3: Pull complete
dcec5e78b34d: Pull complete
39792fa55bc7: Pull complete
e5deca7a2da3: Pull complete
f19b231d0906: Pull complete
4003b883c0eb: Pull complete
4664ac8d5be6: Pull complete
cfe4c7737d0d: Pull complete
f8f5bc5925a5: Pull complete
1c8afedc0bc9: Pull complete
a6efcfa258d0: Pull complete
77fdb57d5964: Pull complete
ad9e0a7dcbb2: Pull complete
97e7e6dc8cdb: Pull complete
8e83abb7513b: Pull complete
Digest: sha256:0a76c40c82d269c717ecf8361a3077a8c398d9b3626efe1f9dd478ab1d30b0d8
Status: Downloaded newer image for deepagar/crud-cassandra:latest
Starting Cassandra
/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2015-12-20 11:29:38,777 CRIT Supervisor running as root (no user in config file)
2015-12-20 11:29:38,781 INFO supervisord started with pid 6
2015-12-20 11:29:39,787 INFO spawned: 'sshd' with pid 9
2015-12-20 11:29:39,789 INFO spawned: 'crud-service' with pid 10
2015-12-20 11:29:39,790 INFO spawned: 'cassandra-crud' with pid 11
2015-12-20 11:29:39,791 INFO spawned: 'cassandra' with pid 12
2015-12-20 11:29:40,798 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: crud-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: cassandra-crud entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: cassandra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)                                                                                



Validate accessing the crud interface


READ

curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:31:57 GMT
Content-Length: 40

{"error":"no policy(s) into the table"}



Create

curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-1\", \"blob\": \"rule1\" }" http://52.35.62.110:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:34:24 GMT
Content-Length: 51

{"class":"common","name":"email-1","blob":"rule1"}
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$ curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-2\", \"blob\": \"rule2\" }" http://52.35.62.110:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:34:32 GMT
Content-Length: 51

{"class":"common","name":"email-2","blob":"rule2"}

curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:01 GMT
Content-Length: 104

[{"class":"common","name":"email-1","blob":"rule1"},{"class":"common","name":"email-2","blob":"rule2"}]


Update

curl -i -X PUT -H "Content-Type: application/json" -d "{ \"blob\": \"{rule-x}\" }" http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:21 GMT
Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$ curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:24 GMT
Content-Length: 107

[{"class":"common","name":"email-1","blob":"{rule-x}"},{"class":"common","name":"email-2","blob":"rule2"}]


curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:15 GMT
Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}


Delete

curl -i -X DELETE http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:35 GMT
Content-Length: 54

{"class":"common","name":"email-1","blob":"{rule-x}"}

curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:41 GMT
Content-Length: 53

[{"class":"common","name":"email-2","blob":"rule2"}]

curl -i http://52.35.62.110:8080/api/v1/policies/common/email-2
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:50 GMT
Content-Length: 51

{"class":"common","name":"email-2","blob":"rule2"}

curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:53 GMT
Content-Length: 31

{"error":"no policy(s) found"}


Integrate AWS Deployment with a domain using ELB and Region53 


Create EC2 ELB and associate instance


The AWS EC2 ELB got created following the Getting Started with ELB.
The ELB could be accessed from EC2 -> EC2 Dashboards -> Load Balancers from AWS console.


EC2 ELB with DNS deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com gets created

Access the CRUD service using AWS ELB

curl -i http://deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:02:09 GMT
Content-Length: 53
Connection: keep-alive

[{"class":"common","name":"email-2","blob":"rule2"}]

Integrating with AWS Route 53


The first step is to register a new domain or transfer an existing domain. I did with new domain over AWS.

Once domain is successfully registered (takes ~60 minutes), it is visible at route53 home

Create resource record sets to tell Amazon Route 53 how you want to route traffic for the domain. For more information, see Adding Resource Record Sets for a New Domain.

If you already have a hosted zone for your domain, go to the hosted zones page from  route53 home

On the Hosted Zones page, choose the name of the hosted zone in which you want to create resource record sets. Click Create Record Set.

Follow Using domain-names with elb to create record-set.
Click Create Record Set. Under Create Record Set, do the following: 
Leave the default name, which is the name of your domain. 
From Type, select A — IPv4 address. 
For Alias, click Yes. An alias enables Amazon Route 53 to associate your domain name with an AWS resource, such as a load balancer. 
Click Alias Target. Select your load balancer from the list. The console adds the dualstack prefix. Note that the value of Alias Hosted Zone ID is based on the load balancer that you selected. 
From Routing Policy, select Simple. 
Leave Evaluate Target Health set to No. 
Click Create.


Validate CRUD service using AWS Route53 domain name


curl -i http://deepagar.net:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:41:34 GMT
Content-Length: 53
Connection: keep-alive

[{"class":"common","name":"email-2","blob":"rule2"}]


curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-3\", \"blob\": \"rule3\" }" http://deepagar.net:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:51:41 GMT
Content-Length: 51
Connection: keep-alive

{"class":"common","name":"email-3","blob":"rule3"}

curl -i http://deepagar.net:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:51:47 GMT
Content-Length: 104
Connection: keep-alive

[{"class":"common","name":"email-2","blob":"rule2"},{"class":"common","name":"email-3","blob":"rule3"}]

Conclusion


The above demonstrates the AWS end-to-end service deployment.

I have intentionally left an anti-pattern in micro-service design and would be addressed in a separate post.

Disclaimer : This is for demo purpose and there is much more for production...


References














Comments