Embedded Files
This post covers the CRUD microservice creation using Golang\Cassandra and packaging micro-service as docker container and pushing to Git\Docker repository.
The CRUD micro-service is deployed over AWS and integrated with AWS ELB and Region53 as http://deepagar.net:8080/api/v1/policies/common
Create a micro-service using Golang and Cassandra
Create a micro-service using Golang and Cassandra
Follow the steps at rest microservice with gin
This page also covers pushing the micro-service to github repository at crud-service
Create the Dockerfile container using above micro-service
Create the Dockerfile container using above micro-service
Create the Dockerfile project
Create the Dockerfile project
Push the Dockerfile project to Git
Push the Dockerfile project to Git
Create a repo
https://github.com/deepagargit/crud-service
Clone the repo
git clone https://github.com/deepagargit/crud-service
List the current branch
$ git branch -a
* develop
remotes/origin/HEAD -> origin/develop
remotes/origin/develop
Switch to develop branch
$ git checkout develop
Already on 'develop'
Create docker folder
$ mkdir docker
$ cd docker
Copy dockerfile and other project files
$ ls -R
.:
cassandra crud crud-cassandra
./cassandra:
./crud:
./crud-cassandra:
build.sh Dockerfile ReadMe src
./crud-cassandra/src:
cassandra.yaml datastax.repo epel7.repo main.go start.sh supervisord.conf
Add files to git
Touch the gitkeep file in empty folder.
touch .gitkeep
git add .
git status
# On branch develop
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# new file: cassandra/.gitkeep
# new file: crud-cassandra/Dockerfile
# new file: crud-cassandra/ReadMe
# new file: crud-cassandra/build.sh
# new file: crud-cassandra/src/cassandra.yaml
# new file: crud-cassandra/src/datastax.repo
# new file: crud-cassandra/src/epel7.repo
# new file: crud-cassandra/src/main.go
# new file: crud-cassandra/src/start.sh
# new file: crud-cassandra/src/supervisord.conf
# new file: crud/.gitkeep
#
Add User Config
git config --global user.email "deepak.aagarwal@gmail.com"
git config --global user.name "Deepak Agarwal"
Commit the files
$ git commit -m "Added Docker project for CRUD service with Golang"
[develop 9aaaedb] Added Docker project for CRUD service with Golang
11 files changed, 1421 insertions(+)
create mode 100644 docker/cassandra/.gitkeep
create mode 100644 docker/crud-cassandra/Dockerfile
create mode 100644 docker/crud-cassandra/ReadMe
create mode 100755 docker/crud-cassandra/build.sh
create mode 100755 docker/crud-cassandra/src/cassandra.yaml
create mode 100755 docker/crud-cassandra/src/datastax.repo
create mode 100755 docker/crud-cassandra/src/epel7.repo
create mode 100755 docker/crud-cassandra/src/main.go
create mode 100755 docker/crud-cassandra/src/start.sh
create mode 100755 docker/crud-cassandra/src/supervisord.conf
create mode 100644 docker/crud/.gitkeep
Push to the develop branch
git push origin develop
Username for 'https://github.com': deepagargit
Password for 'https://deepagargit@github.com':
Counting objects: 14, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (13/13), 16.14 KiB | 0 bytes/s, done.
Total 13 (delta 1), reused 0 (delta 0)
To https://github.com/deepagargit/crud-service
87c3096..9aaaedb develop -> develop
Build the container image
./crud-cassandra/build.sh
Spawn the container
docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra
Login to container bash
sudo docker exec -i -t p4 bash
Validate cqlsh in container
$ cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.2.4 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> use demo;
cqlsh:demo> select * from policy;
class | name | blob
-------+------+------
$ exit
Push the container image to docker repository
docker login
docker push deepagar/crud-cassandra
Deploy (Manually) the micro-service over AWS VM
Deploy (Manually) the micro-service over AWS VM
Create the VM
Create the VM in EC2
Install docker
sudo curl -sSL https://get.docker.com/ | sh
sudo usermod -aG docker centos
List docker container
$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Start the docker contain using docker image from repository
docker run -it -p 8080:8080 --name p4 deepagar/crud-cassandra
Unable to find image 'deepagar/crud-cassandra:latest' locally
latest: Pulling from deepagar/crud-cassandra
fa5be2806d4c: Pull complete
98783644cfab: Pull complete
4609e0ebb5d4: Pull complete
c812b6264db1: Pull complete
ea1036165fa4: Pull complete
7a21760c41c7: Pull complete
542acddd307e: Pull complete
4ade23629f70: Pull complete
e94a69157b27: Pull complete
7e6c33318a29: Pull complete
96a5ae6fbdba: Pull complete
6500fd2abf12: Pull complete
c318cbc3e970: Pull complete
7a375ec1e6fb: Pull complete
b6495ac2f1b9: Pull complete
474f92c6f273: Pull complete
1f9b54b17fa3: Pull complete
dcec5e78b34d: Pull complete
39792fa55bc7: Pull complete
e5deca7a2da3: Pull complete
f19b231d0906: Pull complete
4003b883c0eb: Pull complete
4664ac8d5be6: Pull complete
cfe4c7737d0d: Pull complete
f8f5bc5925a5: Pull complete
1c8afedc0bc9: Pull complete
a6efcfa258d0: Pull complete
77fdb57d5964: Pull complete
ad9e0a7dcbb2: Pull complete
97e7e6dc8cdb: Pull complete
8e83abb7513b: Pull complete
Digest: sha256:0a76c40c82d269c717ecf8361a3077a8c398d9b3626efe1f9dd478ab1d30b0d8
Status: Downloaded newer image for deepagar/crud-cassandra:latest
Starting Cassandra
/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
'Supervisord is running as root and it is searching '
2015-12-20 11:29:38,777 CRIT Supervisor running as root (no user in config file)
2015-12-20 11:29:38,781 INFO supervisord started with pid 6
2015-12-20 11:29:39,787 INFO spawned: 'sshd' with pid 9
2015-12-20 11:29:39,789 INFO spawned: 'crud-service' with pid 10
2015-12-20 11:29:39,790 INFO spawned: 'cassandra-crud' with pid 11
2015-12-20 11:29:39,791 INFO spawned: 'cassandra' with pid 12
2015-12-20 11:29:40,798 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: crud-service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: cassandra-crud entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-12-20 11:29:40,798 INFO success: cassandra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Validate accessing the crud interface
READ
curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:31:57 GMT
Content-Length: 40
{"error":"no policy(s) into the table"}
Create
curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-1\", \"blob\": \"rule1\" }" http://52.35.62.110:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:34:24 GMT
Content-Length: 51
{"class":"common","name":"email-1","blob":"rule1"}
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$ curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-2\", \"blob\": \"rule2\" }" http://52.35.62.110:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:34:32 GMT
Content-Length: 51
{"class":"common","name":"email-2","blob":"rule2"}
curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:01 GMT
Content-Length: 104
[{"class":"common","name":"email-1","blob":"rule1"},{"class":"common","name":"email-2","blob":"rule2"}]
Update
curl -i -X PUT -H "Content-Type: application/json" -d "{ \"blob\": \"{rule-x}\" }" http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:21 GMT
Content-Length: 54
{"class":"common","name":"email-1","blob":"{rule-x}"}
[centos@ip-172-31-14-230 ~]$
[centos@ip-172-31-14-230 ~]$ curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:35:24 GMT
Content-Length: 107
[{"class":"common","name":"email-1","blob":"{rule-x}"},{"class":"common","name":"email-2","blob":"rule2"}]
curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:15 GMT
Content-Length: 54
{"class":"common","name":"email-1","blob":"{rule-x}"}
Delete
curl -i -X DELETE http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:35 GMT
Content-Length: 54
{"class":"common","name":"email-1","blob":"{rule-x}"}
curl -i http://52.35.62.110:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:41 GMT
Content-Length: 53
[{"class":"common","name":"email-2","blob":"rule2"}]
curl -i http://52.35.62.110:8080/api/v1/policies/common/email-2
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:50 GMT
Content-Length: 51
{"class":"common","name":"email-2","blob":"rule2"}
curl -i http://52.35.62.110:8080/api/v1/policies/common/email-1
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 11:36:53 GMT
Content-Length: 31
{"error":"no policy(s) found"}
Integrate AWS Deployment with a domain using ELB and Region53
Integrate AWS Deployment with a domain using ELB and Region53
Create EC2 ELB and associate instance
Create EC2 ELB and associate instance
The AWS EC2 ELB got created following the Getting Started with ELB.
The ELB could be accessed from EC2 -> EC2 Dashboards -> Load Balancers from AWS console.
EC2 ELB with DNS deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com gets created
Access the CRUD service using AWS ELB
curl -i http://deepagar-ELB-1344047162.us-west-2.elb.amazonaws.com:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:02:09 GMT
Content-Length: 53
Connection: keep-alive
[{"class":"common","name":"email-2","blob":"rule2"}]
Integrating with AWS Route 53
Integrating with AWS Route 53
The first step is to register a new domain or transfer an existing domain. I did with new domain over AWS.
Once domain is successfully registered (takes ~60 minutes), it is visible at route53 home
Create resource record sets to tell Amazon Route 53 how you want to route traffic for the domain. For more information, see Adding Resource Record Sets for a New Domain.
If you already have a hosted zone for your domain, go to the hosted zones page from route53 home
On the Hosted Zones page, choose the name of the hosted zone in which you want to create resource record sets. Click Create Record Set.
Follow Using domain-names with elb to create record-set.
Click Create Record Set. Under Create Record Set, do the following:
Leave the default name, which is the name of your domain.
From Type, select A — IPv4 address.
For Alias, click Yes. An alias enables Amazon Route 53 to associate your domain name with an AWS resource, such as a load balancer.
Click Alias Target. Select your load balancer from the list. The console adds the dualstack prefix. Note that the value of Alias Hosted Zone ID is based on the load balancer that you selected.
From Routing Policy, select Simple.
Leave Evaluate Target Health set to No.
Click Create.
Validate CRUD service using AWS Route53 domain name
Validate CRUD service using AWS Route53 domain name
curl -i http://deepagar.net:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:41:34 GMT
Content-Length: 53
Connection: keep-alive
[{"class":"common","name":"email-2","blob":"rule2"}]
curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-3\", \"blob\": \"rule3\" }" http://deepagar.net:8080/api/v1/policies
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:51:41 GMT
Content-Length: 51
Connection: keep-alive
{"class":"common","name":"email-3","blob":"rule3"}
curl -i http://deepagar.net:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sun, 20 Dec 2015 15:51:47 GMT
Content-Length: 104
Connection: keep-alive
[{"class":"common","name":"email-2","blob":"rule2"},{"class":"common","name":"email-3","blob":"rule3"}]
Conclusion
Conclusion
The above demonstrates the AWS end-to-end service deployment.
I have intentionally left an anti-pattern in micro-service design and would be addressed in a separate post.
Disclaimer : This is for demo purpose and there is much more for production...
References
Designing a RESTful API with Python and Flask
Building a RESTful API with Pyramid - Setup
Page updated
Google Sites
Report abuse