Using the Rexter 2.7 Graph Server with Couchbase Server

One of my side projects at Couchbase is a Graph-API on top of it. This article explains how you can use the Rexter Graph Server with this implementation of the Blueprints 2.x API.

There is a forked version of Rexter available here: https://github.com/dmaier-couchbase/rexster .

However, the modifications are quite simple.

I disabled the enforcer plug-in in the main pom.xml file.

The 'rexter-server/pom.xml' file contains additional dependencies.

<dependency>
  <groupId>com.couchbase.graph</groupId>
  <artifactId>couchbase-blueprints</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>

<dependency>
  <groupId>com.couchbase.client</groupId>
  <artifactId>java-client</artifactId>
  <version>2.1.3</version>
</dependency>

Also the file 'rexter-server/config/rexter.xml' was edited in order to use the Couchbase instance.

<graph>
    <graph-enabled>true</graph-enabled>
    <graph-name>cbsample</graph-name>
    <graph-type>com.couchbase.graph.rexter.CouchbaseGraphConfiguration</graph-type>
    <graph-location>/tmp/cb-graph</graph-location>
</graph>

To have this working it is expected that you have previously built CBGraph and that it sits in your local Maven cache. It can be found here: https://github.com/dmaier-couchbase/cb-graph

Your Couchbase connection is currently configured via the file 'couchbase.properties' which is part of the source code of CBGraph.

## Connection to Couchbase
#cb.con.hosts=192.168.56.104,192.168.56.105,192.168.56.106
cb.con.hosts=192.168.7.160
cb.con.port=8091
cb.con.bucket.name=graph
cb.con.bucket.pwd=test
cb.timeout.op=30000
cb.admin.user=couchbase
cb.admin.pwd=couchbase

## Views on side of Couchbase
cb.view.autocreate=true
cb.view.designdoc=graph_views
cb.view.alledges=all_edges
cb.view.allvertices=all_vertices
cb.view.alledgelabels=all_edge_labels
cb.view.allvertexprops=all_vertex_props
cb.view.alledgeprops=all_edge_props

So the steps are:

• Run 'mvn install' in the CBGraph project
• Run 'mvn install' in the Rexter project

Rexter's target folder now should contain a folder which is called 'rexster-server-2.7.0-SNAPSHOT-standalone'.

• Execute 'bin/rexter.sh -s'

You should see a message like

[INFO] Slf4JLogger - Connected to Node 192.168.7.160

[INFO] Slf4JLogger - Opened bucket graph

[INFO] Slf4JLogger - Connected to Node 192.168.7.155

[INFO] Slf4JLogger - Connected to Node 192.168.7.159

Now let's play a bit with the Rexter REST interface:

• All vertices: http://localhost:8182/graphs/cbsample/vertices
• All edges: http://localhost:8182/graphs/cbsample/edges
• A specific vertex called 'Bart': http://localhost:8182/graphs/cbsample/vertices/tae_bart
• Get all outgoing from Bart: http://localhost:8182/graphs/cbsample/vertices/tae_bart/outE
• ...

You can also browse the Graph via Rexter's UI:  http://localhost:8182/doghouse 

Here an example screen shot:

Using Couchbase Server with Docker

There was a very good talk at Couchbase Connect about Docker some days ago (Deploying Couchbase Server Using Docker: Couchbase Connect 2015). So I just played a bit around with it and want to share my impressions.

I am running Docker on MacOS which means that 'boot2docker' is required. Because Docker is using Linux kernel features,  a Linux-VM with a Docker daemon is running in the background. So this VM is used to host our containers.

• Install 'boot2docker'
• Start it by using the application menu, which opens you a shell which has all required environment variables set (boot2docker init --> boot2docker start --> boot2docker shellinit)

Couchbase has now a fully supported Docker image. So the installation of a node is quite simple:

• docker run -d -P --name couchbase-1 couchbase

This starts a container in a daemon mode with the id 'couchbase-1' by using the Couchbase image. Ports are exposed/mapped to the outside.

In order to double check which containers are running use the following command:

docker ps

To see the port mappings, the following command can be used

docker port couchbase-1

This gives me the following output for my first Couchbase node:

11207/tcp -> 0.0.0.0:32772
11210/tcp -> 0.0.0.0:32773
11211/tcp -> 0.0.0.0:32774
18091/tcp -> 0.0.0.0:32768
18092/tcp -> 0.0.0.0:32769
8091/tcp -> 0.0.0.0:32770
8092/tcp -> 0.0.0.0:32771

The Docker host VM is accessible via the following command which returns 192.168.59.103 in my case.

boot2docker ip

So we can access the Admin-UI of our node by using http://192.168.59.103:32770 (instead of using port 8091).

The internal IP can be determined by using Docker's exec command:

docker exec -it couchbase-1 ifconfig

So far so good. But how to build a cluster?

• Setup Couchbase within the container 'couchbase-1' via the Admin-UI by creating a new cluster. I used the internal IP address as the name. Productive environments should care about the name resolution.
• Just run a second Couchbase container and name it 'couchbase-2'
• Retrieve the port mapping for 'couchbase-2' and access the Admin-UI via the port which is mapped to 8091.
• Join the existing cluster by pointing to the internal IP of the container which was called 'couchbase-1'. The default ports are accessible internally (between containers).
• Repeat the steps for a third node 'couchbase-3'

Additional containers could host applications. By default my containers could reach each other via the internal IP addresses. I think it will be more challenging to access the Cluster from the outside even if the ports are mapped, but the idea here is anyway to deploy especially your applications via Docker. Access from the outside would need to configure the used Couchbase client library specifically regarding these port mappings. Also, here some further details about Docker's networking: https://docs.docker.com/articles/networking/ .

!Important! The Docker trend causes that containers are used within VM-s (those are deployed via IaaS cloud systems like Amazon EC2). I even saw it in a VMWare environment whereby the intention was to use the VM-s more efficiently and to unify the application deployment process. So what you see there are multiple containers per VM. Unlike Application Servers, Couchbase is designed to use dedicated resources, so for instance a specific per node RAM quota. The cluster is sized in a way that it fits your requirements, so it's not expected to have a real waste of resources in this case. Especially for performance critical use cases it would NOT follow Couchbase's best practices to run Couchbase Server in a container (Docker) which is again running in a container (VM).

• Docker in general is useful for setting up Test and Development environments (even for containers those are hosted in VM-s)
• As soon as your use case is more performance critical then my opinion is that you should use Docker at least on bare-metal for your productively used Couchbase Server deployments

This is especially underlined by the documentation of the Docker image: https://registry.hub.docker.com/_/couchbase/ . There you can see that the only productively recommended deployment is the one which is named 'Multiple hosts, single container on each host'.

Feedback is highly appreciated! :-)