Programming‎ > ‎Golang‎ > ‎

REST Microservices with Gin



Contents

  1. 1 Introduction
  2. 2 Services in Go
    1. 2.1 Schema
    2. 2.2 Routes
    3. 2.3 Table
    4. 2.4 Gin for Routes
    5. 2.5 Code
  3. 3 Deployment 
    1. 3.1 Cassandra Node Setup
      1. 3.1.1 Install Java
      2. 3.1.2 Install Cassandra
      3. 3.1.3 Configure Cassandra
      4. 3.1.4 Disable firewall
      5. 3.1.5 Start Cassandra
      6. 3.1.6 Check cassandra
      7. 3.1.7 Cassandra Tools
      8. 3.1.8 Cassandra Create Table
      9. 3.1.9 Start Consul Server [Optional for Consul Service Discovery]
      10. 3.1.10 Register Cassandra Service in Consul [Optional for Consul Service Discovery]
    2. 3.2 CRUD-Server Node Setup
      1. 3.2.1 Disable firewall
      2. 3.2.2 Start Consul Client [Optional for Consul Service Discovery]
      3. 3.2.3 Start the CRUD service
    3. 3.3 CRUD-Client Node Setup
      1. 3.3.1 Read Operation (DB empty)
      2. 3.3.2 Create Operation
      3. 3.3.3 Update Operation
      4. 3.3.4 Delete Operation
  4. 4 Push the CRUD service into Git Repository
    1. 4.1 Create a repo
    2. 4.2 Clone the repo
    3. 4.3 Create new git branch
    4. 4.4 Copy the file in repo
    5. 4.5 Add the files to Git
    6. 4.6 Add User Config
    7. 4.7 Commit the files
    8. 4.8 Push to the develop branch








Introduction


This post gives an overview with example to write RESTful API services in Golang.


It used Gin, Cassandra.


Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster.


The setup contains 1 node hosting both Cassandra and Golang service 




Services in Go


Schema


Routes


Let us first define the routes (REST APIs)

[Create]  POST : http://127.0.0.1:8080/api/v1/policies

[Read]    GET : http://127.0.0.1:8080/api/v1/policies

[Read]    GET : http://127.0.0.1:8080/api/v1/policies/1

[Update] PUT : http://127.0.0.1:8080/api/v1/policies/1

[Delete]  DELETE : http://127.0.0.1:8080/api/v1/policies/1


Table


Let's also define the table structure to store the policies blob

Table 'policies'

id — big_int(20) & auto-increment

policy — varchar(255)


Gin for Routes


We’re gonna use Gin who is a micro framework routing. It’s friendly easy and fast.

go get github.com/gin-gonic/gin


Code


Ok, let’s start coding ! Firstly in a new “main.go” file, we call our libraries.


package main
import
(
   "github.com/gin-gonic/gin"
   "strconv"
   "github.com/hashicorp/consul/api"
)


Secondly, we declare the structure “Policy” :


type User struct {
    Id int64 `db:"id" json:"id"`
    Policy string `db:"policy" json:"policy"`
}

The “db” param will be used later, with the database connection…


Thirdly, in the main() function, we regoup our routes in a single group :


func main() {
  r := gin.Default()
  v1 := r.Group("api/v1")
  {
  v1.GET("/policies", GetPolicies)
  v1.GET("/policies/:id", GetPolicies)
  v1.POST("/policies", PostPolicy)
  v1.PUT("/policies/:id", UpdatePolicy)
  v1.DELETE("/policies/:id", DeletePolicy)
  }
  r.Run(":8080")
}


Then we declare the five functions calling in the routes





Deployment 


Create a VMa with Redhat flavor CentOS 7.1


Cassandra Node Setup



Install Java


$ sudo yum -y install java-1.8.0-openjdk


$ java -version

openjdk version "1.8.0_65"

OpenJDK Runtime Environment (build 1.8.0_65-b17)

OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)


Install Cassandra


$ wget http://www.carfab.com/apachesoftware/cassandra/2.2.4/apache-cassandra-2.2.4-bin.tar.gz 

$ tar -xzvf apache-cassandra-2.2.4-bin.tar.gz

$ cd apache-cassandra-2.2.4


Configure Cassandra


$ sudo mkdir /opt/lib

$ sudo mkdir /opt/lib/cassandra/

$ sudo mkdir /opt/lib/cassandra/data

$ sudo chmod 777 /opt/lib/cassandra/ -R


$ cd conf/

$ cp cassandra.yaml cassandra.yaml.orig


$vi cassandra.yaml


Old

# The name of the cluster. This is mainly used to prevent machines in

# one logical cluster from joining another.

cluster_name: 'Test Cluster'


New

cluster_name: 'CRUD-Cluster'


Old\New - Leave this as it is

# If you already have a cluster with 1 token per node, and wish to migrate to

# multiple tokens per node, see http://wiki.apache.org/cassandra/Operations

num_tokens: 256


Old

# - AllowAllAuthenticator performs no checks - set it to disable authentication.

# - PasswordAuthenticator relies on username/password pairs to authenticate

#   users. It keeps usernames and hashed passwords in system_auth.credentials table.

#   Please increase system_auth keyspace replication factor if you use this authenticator.

#   If using PasswordAuthenticator, CassandraRoleManager must also be used (see below)

authenticator: AllowAllAuthenticator


New

authenticator: PasswordAuthenticator (Recommended)

authenticator: AllowAllAuthenticator



Old

# - AllowAllAuthorizer allows any action to any user - set it to disable authorization.

# - CassandraAuthorizer stores permissions in system_auth.permissions table. Please

#   increase system_auth keyspace replication factor if you use this authorizer.

authorizer: AllowAllAuthorizer


New

authorizer: CassandraAuthorizer (Recommended)

authorizer: AllowAllAuthorizer


Old\New - Use existing only

# Besides Murmur3Partitioner, partitioners included for backwards

# compatibility include RandomPartitioner, ByteOrderedPartitioner, and

# OrderPreservingPartitioner.

#

partitioner: org.apache.cassandra.dht.Murmur3Partitioner


Old

# Directories where Cassandra should store data on disk.  Cassandra

# will spread data evenly across them, subject to the granularity of

# the configured compaction strategy.

# If not set, the default directory is $CASSANDRA_HOME/data/data.

# data_file_directories:

#     - /var/lib/cassandra/data


New - Ensure no-space before data_file_directories

# If not set, the default directory is $CASSANDRA_HOME/data/data.

data_file_directories:

      - /opt/lib/cassandra/data


Old

# commit log.  when running on magnetic HDD, this should be a

# separate spindle than the data directories.

# If not set, the default directory is $CASSANDRA_HOME/data/commitlog.

# commitlog_directory: /var/lib/cassandra/commitlog


New - Ensure no-space before commlilog

# If not set, the default directory is $CASSANDRA_HOME/data/commitlog.

# commitlog_directory: /var/lib/cassandra/commitlog

commitlog_directory: /opt/lib/cassandra/commitlog


Old

saved caches

# If not set, the default directory is $CASSANDRA_HOME/data/saved_caches.

# saved_caches_directory: /var/lib/cassandra/saved_caches


New

# saved_caches_directory: /var/lib/cassandra/saved_caches

saved_caches_directory: /opt/lib/cassandra/saved_caches


Old\New - No change for now, need to change for cluster

# any class that implements the SeedProvider interface and has a

# constructor that takes a Map<String, String> of parameters will do.

seed_provider:

    # Addresses of hosts that are deemed contact points.

    # Cassandra nodes use this list of hosts to find each other and learn

    # the topology of the ring.  You must change this if you are running

    # multiple nodes!

    - class_name: org.apache.cassandra.locator.SimpleSeedProvider

      parameters:

          # seeds is actually a comma-delimited list of addresses.

          # Ex: "<ip1>,<ip2>,<ip3>"

          - seeds: "127.0.0.1"


# If IP based cqlsh access           - seeds: "127.0.0.1,192.168.0.164"


Old

# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6.

listen_address: localhost


New

# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6.

listen_address: 192.168.0.164


#If IP based cqlsh access  listen_address: 192.168.0.164



Old

# If you choose to specify the interface by name and the interface has an ipv4 and an ipv6 address

# you can specify which should be chosen using rpc_interface_prefer_ipv6. If false the first ipv4

# address will be used. If true the first ipv6 address will be used. Defaults to false preferring

# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6.

rpc_address: localhost


New

# ipv4. If there is only one address it will be selected regardless of ipv4/ipv6.

rpc_address: 192.168.0.164


# If IP based cqlsh access rpc_address: 192.168.0.164



Note that unlike listen_address, you can specify 0.0.0.0, but you must also

# set broadcast_rpc_address to a value other than 0.0.0.0.

# Also set broadcast_rpc_address: 1.2.3.4, if rpc_address is set to 0.0.0.0


Old\New - Same, In production use GossipingPropertyFileSnitch, single node use simplesnitch

# You can use a custom Snitch by setting this to the full class name

# of the snitch, which will be assumed to be on your classpath.

endpoint_snitch: SimpleSnitch


if use GossipingPropertyFileSnitch, chance as per your topology

# vi cassandra-rackdc.properties

# These properties are used with GossipingPropertyFileSnitch and will

# indicate the rack and dc for this node

dc=DC1

rack=RAC1


Disable firewall

sudo systemctl disable firewalld

sudo systemctl stop firewalld

sudo systemctl status firewalld


Start Cassandra

$ cd /home/centos/bin/cassandra/apache-cassandra-2.2.3/bin/

$ ./cassandra &


Check cassandra

ps auwx | grep cassandra



Cassandra Tools

$ cd /home/centos/bin/cassandra/apache-cassandra-2.2.3/bin/

$ ./nodetool help

$ ./cqlsh --help


$ ./nodetool version

ReleaseVersion: 2.2.3


$ ./nodetool status

Datacenter: datacenter1

=======================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address        Load       Tokens       Owns    Host ID                               Rack

UN  127.0.0.1  131.31 KB  256          ?       39fc1d89-ea3d-45aa-ab1c-68139c76945d  rack1


Cassandra Create Table


$ ./cqlsh [192.168.0.164 -u cassandra -p cassandra] # Default user\password not required if authenticator and authorizor is Allow All


$ ./cqlsh

Connected to CRUD-Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 2.2.4 | CQL spec 3.3.1 | Native protocol v4]

Use HELP for help.

cqlsh> 



Before you execute the program, Launch `cqlsh` and execute:

> create keyspace demo with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };

> use demo;

> create table demo.policy(class text, name text, blob text, PRIMARY KEY(class, name));


This is not required as class, name is already part of primary key.

> create index on demo.policy(class);

> create index on demo.policy(name);


cqlsh> use demo;

cqlsh:demo> desc policy;


CREATE TABLE demo.policy (

    class text,

    name text,

    blob text,

    PRIMARY KEY (class, name)

) WITH CLUSTERING ORDER BY (name ASC)

    AND bloom_filter_fp_chance = 0.01

    AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'

    AND comment = ''

    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}

    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}

    AND dclocal_read_repair_chance = 0.1

    AND default_time_to_live = 0

    AND gc_grace_seconds = 864000

    AND max_index_interval = 2048

    AND memtable_flush_period_in_ms = 0

    AND min_index_interval = 128

    AND read_repair_chance = 0.0

    AND speculative_retry = '99.0PERCENTILE';


cqlsh:demo> select * from policy;


 class  | name       | blob

--------+------------+-------

 common | email-sign | rule5

 common |  email-web | rule6


(2 rows)



Start Consul Server [Optional for Consul Service Discovery]


The consul is used for service discovery of cassandra by CRUD service.


Download consul

$ cd /home/centos

$ wget https://dl.bintray.com/mitchellh/consul/0.5.2_linux_amd64.zip

$ sudo yum -y install unzip

$ unzip 0.5.2_linux_amd64.zip


$ mkdir /tmp/consul

$ sudo mkdir /etc/consul.d

$ ./consul agent -server -bootstrap-expect=1 -data-dir /tmp/consul -config-dir /etc/consul.d -node=consul-server-db --bind=192.168.0.158 &    


$ ./consul members

    2015/12/18 19:16:52 [INFO] agent.rpc: Accepted client: 127.0.0.1:58388

Node              Address             Status  Type    Build  Protocol  DC

consul-server-db  192.168.0.158:8301  alive   server  0.5.2  2         dc1



Register Cassandra Service in Consul [Optional for Consul Service Discovery]


curl -X PUT http://localhost:8500/v1/agent/service/register -d '{"ID":"cassandra", "name":"cassandra","tags": ["rails"], "port": 9160, "Address":"192.168.0.158"}'

$ curl http://localhost:8500/v1/catalog/service/cassandra | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   181  100   181    0     0   1180      0 --:--:-- --:--:-- --:--:--  1183
[
    {
        "Address": "192.168.0.158",
        "Node": "consul-server-db",
        "ServiceAddress": "192.168.0.158",
        "ServiceID": "cassandra",
        "ServiceName": "cassandra",
        "ServicePort": 9160,
        "ServiceTags": [
            "rails"
        ]
    }
]



CRUD-Server Node Setup


Install Golang

$ sudo yum -y update

$ sudo yum -y install golang


$ go version

go version go1.4.2 linux/amd64



Install gin\gocql\api for Routes

$ go help gopath

$ mkdir /home/centos/repos

$ export GOPATH=/home/centos/repos/

$ go get github.com/gin-gonic/gin

$ go get github.com/gocql/gocql

$ go get github.com/hashicorp/consul/api



Create directory structure for Go service


$ cd /home/centos/repos

$ mkdir crud-service

$ mkdir crud-service/crud-server

$ mkdir crud-service/crud-client

$ mkdir crud-service/crud-test

$ mkdir crud-service/crud-server/bin

$ mkdir crud-service/crud-server/pkg

$ mkdir crud-service/crud-server/src

$ mkdir crud-service/crud-client/bin

$ mkdir crud-service/crud-client/pkg

$ mkdir crud-service/crud-client/src

$ mkdir crud-service/crud-test/bin

$ mkdir crud-service/crud-test/pkg

$ mkdir crud-service/crud-test/src


Create the service main.go


/home/centos/repos/crud-service/crud-server/src/main.go


main.go



 cat src/main.go


package main


import

(

   "fmt"

   "log"

   "github.com/gin-gonic/gin"

//   "strconv"

   "github.com/gocql/gocql"

//   "github.com/hashicorp/consul/api"

)



type Policy struct {

    Class string `db:"class" json:"class"`

    Name string `db:"name" json:"name"`

    Blob string `db:"blob" json:"blob"`

}


/* Before you execute the program, Launch `cqlsh` and execute:

create keyspace demo with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };

create table demo.policy(class text, name text, blob text, PRIMARY KEY(class, name));


// Being primary key, below won;t work

create index on demo.policy(class);

create index on demo.policy(name);


curl -X PUT http://localhost:8500/v1/agent/service/register -d '{"ID":"cassandra", "name":"cassandra","tags": ["rails"], "port": 9160, "Address":"192.168.0.168"}'

curl http://localhost:8500/v1/catalog/service/cassandra | python -m json.tool

*/



func checkErr(err error, msg string) {

    if err != nil {

        log.Fatalln(msg, err)

    }

}



func main() {

  r := gin.Default()

  v1 := r.Group("api/v1")


  {

  v1.GET("/policies/:class", GetPolicies)

  v1.GET("/policies/:class/:name", GetPolicy)

  v1.POST("/policies", PostPolicy)

  v1.PUT("/policies/:class/:name", UpdatePolicy)

  v1.DELETE("/policies/:class/:name", DeletePolicy)

  }


  r.Run(":8080")

}



func GetCassandraIP() (IP string) {


   //var IP string

/*

   //Get a new client

   client, err := api.NewClient(api.DefaultConfig())

   if err != nil {

      panic(err)

   }


    agent := client.Agent()

    services, err := agent.Services()


    fmt.Println("services : ", services)

        if err != nil {

                log.Fatal("err: %v", err)

        }


        if _, ok := services["cassandra"]; !ok {

                log.Fatal("missing service: %v", services)

        }


    IP = services["cassandra"].Address

    fmt.Println(" Service cassandra IP : ", services["cassandra"].Address)

*/

    IP = "127.0.0.1"

    fmt.Println(" Service cassandra IP : ", IP)

    return


}


// CRUD Operations





// READ all policies


func GetPolicies(c *gin.Context) {


    // connect to the cluster

    cluster := gocql.NewCluster(GetCassandraIP())


    // A keyspace in Cassandra is a namespace that defines data replication on nodes. A cluster contains one keyspace per node.

    cluster.Keyspace = "demo"


    // May use gocql.Quorum

    cluster.Consistency = gocql.LocalOne

    session, _ := cluster.CreateSession()


    // Make sure that the connection can close once you are done.

    defer session.Close()


//    ####################################### Query Logic ##########################################


    class := c.Params.ByName("class")


    fmt.Println("class : ", class)


    var policies []Policy

    var policy Policy


    iter := session.Query("SELECT class, name, blob FROM policy WHERE class=?", class).Iter()

    for iter.Scan(&policy.Class, &policy.Name, &policy.Blob) {


        // fmt.Println("1 : ", policy.Class, policy.Name, policy.Blob)

        policies = append(policies, policy)

    }


    if len(policies) > 0 {

        c.JSON(200, policies)

    } else {

        c.JSON(404, gin.H{"error": "no policy(s) into the table"})

    }


    if err := iter.Close(); err != nil {

        log.Fatal(err)

    }



    // curl -i http://localhost:8080/api/v1/policies/common

}




// Read a user



func GetPolicy(c *gin.Context) {


    // connect to the cluster

    cluster := gocql.NewCluster(GetCassandraIP())


    // A keyspace in Cassandra is a namespace that defines data replication on nodes. A cluster contains one keyspace per node.

    cluster.Keyspace = "demo"


    // May use gocql.Quorum

    cluster.Consistency = gocql.LocalOne

    session, _ := cluster.CreateSession()


    // Make sure that the connection can close once you are done.

    defer session.Close()


//    ####################################### Query Logic ##########################################


    class := c.Params.ByName("class")

    name := c.Params.ByName("name")

    var policy Policy


    err := session.Query("SELECT class, name, blob FROM policy WHERE class=? AND name=?", class, name).Scan(&policy.Class, &policy.Name, &policy.Blob);



    if err == nil {

        c.JSON(200, policy)

    } else {

        c.JSON(404, gin.H{"error": "no policy(s) found"})

    }


    // curl -i http://localhost:8080/api/v1/policies/common/email

}





// Create a user


func PostPolicy(c *gin.Context) {


    // connect to the cluster

    cluster := gocql.NewCluster(GetCassandraIP())


    // A keyspace in Cassandra is a namespace that defines data replication on nodes. A cluster contains one keyspace per node.

    cluster.Keyspace = "demo"


    // May use gocql.Quorum

    cluster.Consistency = gocql.LocalOne

    session, _ := cluster.CreateSession()


    // Make sure that the connection can close once you are done.

    defer session.Close()


//    ####################################### Query Logic ##########################################


    var policy Policy

    if c.BindJSON(&policy) == nil {

       fmt.Println("Bind success", c.Params.ByName("Class"))

    } else {

       fmt.Println("Bind failure")

    }



    fmt.Println("Policy: ", policy.Class, policy.Name, policy.Blob)


    if policy.Class != "" && policy.Name != "" && policy.Blob != "" {


    err := session.Query("INSERT INTO policy (class, name, blob) VALUES (?, ?, ?)", policy.Class, policy.Name, policy.Blob).Exec()



            if err == nil {

              c.JSON(201, policy)


/* // Commenting consul KV store

              // Get a new client

                client, err := api.NewClient(api.DefaultConfig())

                if err != nil {

                    panic(err)

                }


                // Get a handle to the KV API

                kv := client.KV()


                key := "notify" + "/" + "policy" + "/" + policy.Class + "/" + policy.Name

                value := []byte("POST")


                fmt.Println(key, " : ", value)




                // PUT a new KV pair

                p := &api.KVPair{Key: key, Value: value}

                _, err = kv.Put(p, nil)

                if err != nil {

                    panic(err)

                } else {

                        fmt.Println("kv updated")

                }

*/


            } else {

                checkErr(err, "Insert failed")

            }

    } else {

        c.JSON(422, gin.H{"error": "fields are empty"})

    }



    // curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email\", \"blob\": \"{rule-2}\" }" http://localhost:8080/api/v1/policies

}



// Update a user


func UpdatePolicy(c *gin.Context) {



    // connect to the cluster

    cluster := gocql.NewCluster(GetCassandraIP())


    // A keyspace in Cassandra is a namespace that defines data replication on nodes. A cluster contains one keyspace per node.

    cluster.Keyspace = "demo"


    // May use gocql.Quorum

    cluster.Consistency = gocql.LocalOne

    session, _ := cluster.CreateSession()


    // Make sure that the connection can close once you are done.

    defer session.Close()


//    ####################################### Query Logic ##########################################


    fmt.Println("In Update")



    var json Policy

    c.BindJSON(&json)


    json.Class = c.Params.ByName("class")

    json.Name = c.Params.ByName("name")



    fmt.Println("json : ", json.Class, json.Name, json.Blob)

    var policy Policy


    err := session.Query("SELECT class, name, blob FROM policy WHERE class=? AND name=?", json.Class, json.Name).Scan(&policy.Class, &policy.Name, &policy.Blob);



    if err == nil {

        fmt.Println("select query success ")

        if json.Blob != "" {


        err := session.Query("UPDATE policy SET blob=? WHERE class=? AND name=?", json.Blob, json.Class, json.Name).Exec();


            if err == nil {

              c.JSON(200, json)


/* // Commenting consul KV


             // Get a new client

                client, err := api.NewClient(api.DefaultConfig())

                if err != nil {

                    panic(err)

                }


                // Get a handle to the KV API

                kv := client.KV()


                key := "notify" + "/" + "policy" + "/" + json.Class + "/" + json.Name

                value := []byte("PUT")


                fmt.Println(key, " : ", value)


                // PUT a new KV pair

                p := &api.KVPair{Key: key, Value: value}

                _, err = kv.Put(p, nil)

                if err != nil {

                    panic(err)

                } else {

                        fmt.Println("kv updated")

                }

*/


            } else {

                checkErr(err, "Update failed")

            }

        } else {

            c.JSON(422, gin.H{"error": "fields are empty"})

        }


    } else {

        fmt.Println("select query failed")

        c.JSON(404, gin.H{"error": "policy not found"})

    }




    // curl -i -X PUT -H "Content-Type: application/json" -d "{ \"blob\": \"{rule-2}\" }" http://localhost:8080/api/v1/policies/common/email

}



// Delete a user


func DeletePolicy(c *gin.Context) {



    // connect to the cluster

    cluster := gocql.NewCluster(GetCassandraIP())


    // A keyspace in Cassandra is a namespace that defines data replication on nodes. A cluster contains one keyspace per node.

    cluster.Keyspace = "demo"


    // May use gocql.Quorum

    cluster.Consistency = gocql.LocalOne

    session, _ := cluster.CreateSession()


    // Make sure that the connection can close once you are done.

    defer session.Close()


//    ####################################### Query Logic ##########################################



    class := c.Params.ByName("class")

    name := c.Params.ByName("name")

    var policy Policy


    err := session.Query("SELECT class, name, blob FROM policy WHERE class=? AND name=?", class, name).Scan(&policy.Class, &policy.Name, &policy.Blob);


    if err == nil {

        if err := session.Query("DELETE FROM policy WHERE class=? AND name=?", class, name).Exec(); err != nil {

                        log.Fatal(err)

            }


        if err == nil {

          c.JSON(200, policy)


/* // Commented consul KV


             // Get a new client

                client, err := api.NewClient(api.DefaultConfig())

                if err != nil {

                    panic(err)

                }


                // Get a handle to the KV API

                kv := client.KV()


                key := "notify" + "/" + "policy" + "/" + policy.Class + "/" + policy.Name

                value := []byte("DELETE")


                fmt.Println(key, " : ", value)


                // PUT a new KV pair

                p := &api.KVPair{Key: key, Value: value}

                _, err = kv.Put(p, nil)

                if err != nil {

                    panic(err)

                } else {

                        fmt.Println("kv updated")

                }

*/



        } else {

            checkErr(err, "Delete failed")

        }


    } else {

        c.JSON(404, gin.H{"error": "policy not found"})

    }


    // curl -i -X DELETE http://localhost:8080/api/v1/policies/common/email

}






Disable firewall

sudo systemctl disable firewalld

sudo systemctl stop firewalld

sudo systemctl status firewalld



Start Consul Client [Optional for Consul Service Discovery]


The consul is used for service discovery of cassandra by CRUD service.


Download consul

$ cd /home/centos

$ wget https://dl.bintray.com/mitchellh/consul/0.5.2_linux_amd64.zip

$ sudo yum -y install unzip

$ unzip 0.5.2_linux_amd64.zip


$ mkdir /tmp/consul

$ sudo mkdir /etc/consul.d

$ ./consul agent -data-dir /tmp/consul -config-dir /etc/consul.d -node=consul-server-db --bind=192.168.0.164 &    


$ ./consul join 192.168.0.164 (IP of Cassandra Node)

$ ./consul members

    2015/12/18 19:22:27 [INFO] agent.rpc: Accepted client: 127.0.0.1:53227

Node                Address             Status  Type    Build  Protocol  DC

consul-server-crud  192.168.0.164:8301  alive   client  0.5.2  2         dc1

consul-server-db    192.168.0.158:8301  alive   server  0.5.2  2         dc1


Start the CRUD service


$ cd /home/centos/repos/crud-service/crud-server/src/

$ go run src/main.go
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET   /api/v1/policies/:class   --> main.GetPolicies (3 handlers)
[GIN-debug] GET   /api/v1/policies/:class/:name --> main.GetPolicy (3 handlers)
[GIN-debug] POST  /api/v1/policies          --> main.PostPolicy (3 handlers)
[GIN-debug] PUT   /api/v1/policies/:class/:name --> main.UpdatePolicy (3 handlers)
[GIN-debug] DELETE /api/v1/policies/:class/:name --> main.DeletePolicy (3 handlers)
[GIN-debug] Listening and serving HTTP on :8080
 


CRUD-Client Node Setup


Create a VM say 192.168.0.168


Access the CRUD service REST interface and validate.


Read Operation (DB empty)


$ curl -i http://192.168.0.164:8080/api/v1/policies/common

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:30:00 GMT

Content-Length: 40


{"error":"no policy(s) into the table"}


Server Log:
[GIN] 2015/12/18 - 21:00:00 | 404 |  126.750701ms | 192.168.0.168:50952 |   GET     /api/v1/policies/common
                                                                                                           


$ curl -i http://192.168.0.164:8080/api/v1/policies/common/email

HTTP/1.1 404 Not Found

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:30:48 GMT

Content-Length: 31


{"error":"no policy(s) found"}


Server Log:

[GIN] 2015/12/18 - 21:00:48 | 404 |   91.628026ms | 192.168.0.168:51389 |   GET     /api/v1/policies/common/email

                                                                                                                 


Create Operation


$ curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-sign\", \"blob\": \"rule5\" }" http://192.168.0.164:8080/api/v1/policies

HTTP/1.1 201 Created

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:33:03 GMT

Content-Length: 54


{"class":"common","name":"email-sign","blob":"rule5"}


Server Log: 

[GIN] 2015/12/18 - 21:08:51 | 201 |   72.241961ms | 192.168.0.168:55797 |   POST    /api/v1/policies


$ curl -i http://192.168.0.164:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:33:11 GMT

Content-Length: 56


[{"class":"common","name":"email-sign","blob":"rule5"}]



$ curl -i -X POST -H "Content-Type: application/json" -d "{ \"class\": \"common\", \"name\": \"email-web\", \"blob\": \"rule6\" }" http://192.168.0.164:8080/api/v1/policies

$ curl -i http://192.168.0.164:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 18 Dec 2015 15:39:45 GMT
Content-Length: 109

[{"class":"common","name":"email-sign","blob":"rule5"},{"class":"common","name":"email-web","blob":"rule6"}]


Update Operation


$ curl -i -X PUT -H "Content-Type: application/json" -d "{ \"blob\": \"{rule-2}\" }" http://192.168.0.164:8080/api/v1/policies/common/email-sign
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 18 Dec 2015 15:41:19 GMT
Content-Length: 57

{"class":"common","name":"email-sign","blob":"{rule-2}"}

Server Log:
[GIN] 2015/12/18 - 21:11:19 | 200 |   63.486158ms | 192.168.0.168:57144 |   PUT     /api/v1/policies/common/email-sign

$ curl -i http://192.168.0.164:8080/api/v1/policies/common
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 18 Dec 2015 15:42:21 GMT
Content-Length: 112

[{"class":"common","name":"email-sign","blob":"{rule-2}"},{"class":"common","name":"email-web","blob":"rule6"}]

$ curl -i http://192.168.0.164:8080/api/v1/policies/common/email-sign
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 18 Dec 2015 15:42:34 GMT
Content-Length: 57

{"class":"common","name":"email-sign","blob":"{rule-2}"}


Delete Operation


$ curl -i -X DELETE http://192.168.0.164:8080/api/v1/policies/common/email-sign

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:43:38 GMT

Content-Length: 57


Server Log:

[GIN] 2015/12/18 - 21:13:38 | 200 |   81.548881ms | 192.168.0.168:58409 |   DELETE  /api/v1/policies/common/email-sign


{"class":"common","name":"email-sign","blob":"{rule-2}"}


$ curl -i http://192.168.0.164:8080/api/v1/policies/common

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Date: Fri, 18 Dec 2015 15:43:44 GMT

Content-Length: 55


[{"class":"common","name":"email-web","blob":"rule6"}]




Push the CRUD service into Git Repository


Create a repo

https://github.com/deepagargit/crud-service

Clone the repo

git clone https://github.com/deepagargit/crud-service

Create new git branch

$ git checkout -b develop
Switched to a new branch 'develop'

Copy the file in repo


$ ls -R
.:
commands.sh  common  crud-client  crud-server  crud-test  run.sh

./common:
commands.cql

./crud-client:
bin  pkg  src

./crud-client/bin:

./crud-client/pkg:

./crud-client/src:

./crud-server:
bin  pkg  src

./crud-server/bin:

./crud-server/pkg:

./crud-server/src:
main.go

./crud-test:
bin  pkg  src

./crud-test/bin:

./crud-test/pkg:

./crud-test/src:


$ cat run.sh
#!/usr/bin/env bash
cd $(dirname $0)
go run crud-server/src/main.go

$ cat commands.sh
#!/usr/bin/env bash
cd $(dirname $0)
sleep 10s
cqlsh -f common/commands.cql

$ cat common/commands.cql
create keyspace demo with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
create table demo.policy(class text, name text, blob text, PRIMARY KEY(class, name));


Add the files to Git


Touch the gitkeep file in empty folder.
touch .gitkeep

git add .

$ git status
# On branch develop
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
#       new file:   commands.sh
#       new file:   common/commands.cql
#       new file:   crud-client/.gitkeep
#       new file:   crud-client/bin/.gitkeep
#       new file:   crud-client/pkg/.gitkeep
#       new file:   crud-client/src/.gitkeep
#       new file:   crud-server/bin/.gitkeep
#       new file:   crud-server/pkg/.gitkeep
#       new file:   crud-server/src/main.go
#       new file:   crud-test/.gitkeep
#       new file:   crud-test/bin/.gitkeep
#       new file:   crud-test/pkg/.gitkeep
#       new file:   crud-test/src/.gitkeep
#       new file:   run.sh
#

Add User Config

git config --global user.email "deepak.aagarwal@gmail.com"

git config --global user.name "Deepak Agarwal"

Commit the files

$ git commit -m "Added CRUD service with Golang"

[develop (root-commit) 7611268] Added CRUD service with Golang

 14 files changed, 432 insertions(+)

 create mode 100755 commands.sh

 create mode 100644 common/commands.cql

 create mode 100644 crud-client/.gitkeep

 create mode 100644 crud-client/bin/.gitkeep

 create mode 100644 crud-client/pkg/.gitkeep

 create mode 100644 crud-client/src/.gitkeep

 create mode 100644 crud-server/bin/.gitkeep

 create mode 100644 crud-server/pkg/.gitkeep

 create mode 100644 crud-server/src/main.go

 create mode 100644 crud-test/.gitkeep

 create mode 100644 crud-test/bin/.gitkeep

 create mode 100644 crud-test/pkg/.gitkeep

 create mode 100644 crud-test/src/.gitkeep

 create mode 100755 run.sh


Push to the develop branch

$ git push origin develop

Username for 'https://github.com': deepagargit

Password for 'https://deepagargit@github.com':

Counting objects: 12, done.

Delta compression using up to 16 threads.

Compressing objects: 100% (8/8), done.

Writing objects: 100% (12/12), 3.21 KiB | 0 bytes/s, done.

Total 12 (delta 0), reused 0 (delta 0)

To https://github.com/deepagargit/crud-service

 * [new branch]      develop -> develop






References





Comments