The return of RPC - or how REST is no longer the only respectable solution for APIs
Post date: Mar 04, 2018 7:55:30 PM
With all the debate over REST APIs and the many difficulties awaiting on the path to REST nirvana (at least for the purists - for example, did you know that you're doing it wrong if you version your REST API and don't use HATEOAS), it's interesting to observe a certain resurgence of one of the oldest methods of writing distributed applications: RPC (Remote Procedure Call).
If you want to get stuff done and are not interested in this philosophical discussion about what REST is and what it's not, RPC can be a time-saver. And it can even be the right tool for the job!
But for some time, RPC was considered a bad word (not without reason, things like Java RMI and Corba made it look ugly, complicated and unreliable). Not anymore! There's a lot options available in the "RPC world" (if we can say there's such thing).
In this blog post, I investigate in detail some of those options with the objective of measuring them up against the standard REST approach that has become the default.
I've used Gradle to build the code and Java 8 and Groovy to implement the servers/clients, but whatever stack you use, the procedure to create RPC-based communications shouldn't be very different from this.
All code shown in this article can be found on GitHub.
gRPC
gRPC is Google's own take on the old RPC mechanism. With support for over 10 different languages, you can benefit from using the shiniest languages and frameworks just like with your REST-based code, but with the additional benefit of extra efficiency and an actual generated interface you can use directly in your code, in your programming language of choice (rather than having to manufacture HTTP requests yourself or hope that a good SDK is provided in the language you want to use).
It does require a lot of boilerplate to setup. Below, I describe the exact steps you would need to take to get a RPC server and client communicating via gRPC using Gradle (build system) and Java (notice that the procedure is nearly the same for any other JVM language).
1. Create a Gradle build file like this:
plugins {
id "com.google.protobuf" version "0.8.4"
id "java"
}
group 'com.athaydes.tutorials'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile 'io.grpc:grpc-netty:1.10.0'
compile 'io.grpc:grpc-protobuf:1.10.0'
compile 'io.grpc:grpc-stub:1.10.0'
testCompile 'junit:junit:4.12'
}
sourceSets.main.java {
srcDirs "$buildDir/generated/source/proto/main/grpc"
srcDirs "$buildDir/generated/source/proto/main/java"
}
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.5.1-1"
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.10.0'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
build.gradle
2. create a proto file (in a protobuffer-specific declaration language) that declares your RPC service(s) and data:
syntax = "proto3";
option java_package = "com.athaydes.tutorials.rpc.grpc.api";
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
// Sends another greeting
rpc SayHelloAgain (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
src/main/proto/helloworld.proto
3. Run gradle build to generate the Java code (you can configure it to generate other languages as well).
This will generate the source code you need to make RPC calls (both the service and the data you exchange).
4. Now, implement the server-side of the service:
package com.athaydes.tutorials.rpc.grpc;
import com.athaydes.tutorials.rpc.grpc.api.GreeterGrpc;
import com.athaydes.tutorials.rpc.grpc.api.Helloworld.HelloReply;
import com.athaydes.tutorials.rpc.grpc.api.Helloworld.HelloRequest;
import io.grpc.stub.StreamObserver;
public class SimpleGreeter extends GreeterGrpc.GreeterImplBase {
@Override
public void sayHello( HelloRequest req, StreamObserver<HelloReply> responseObserver ) {
HelloReply reply = HelloReply.newBuilder().setMessage( "Hello " + req.getName() ).build();
responseObserver.onNext( reply );
responseObserver.onCompleted();
}
@Override
public void sayHelloAgain( HelloRequest req, StreamObserver<HelloReply> responseObserver ) {
HelloReply reply = HelloReply.newBuilder().setMessage( "Hello again " + req.getName() ).build();
responseObserver.onNext( reply );
responseObserver.onCompleted();
}
}
src/main/java/com/athaydes/tutorials/rpc/grpc/SimpleGreeter.java
5. Add the service to a server and run it in the server's main class:
package com.athaydes.tutorials.rpc.grpc;
import io.grpc.ServerBuilder;
import java.io.IOException;
public class Server {
public static void main( String[] args )
throws IOException, InterruptedException {
io.grpc.Server server = ServerBuilder.forPort( 8081 )
.addService( new SimpleGreeter() )
.build();
server.start().awaitTermination();
}
}
src/main/java/com/athaydes/tutorials/rpc/grpc/Server.java
6. Now that you have a server, you can call it from your client code:
package com.athaydes.tutorials.rpc.grpc;
import com.athaydes.tutorials.rpc.grpc.api.GreeterGrpc;
import com.athaydes.tutorials.rpc.grpc.api.GreeterGrpc.GreeterBlockingStub;
import com.athaydes.tutorials.rpc.grpc.api.Helloworld.HelloReply;
import com.athaydes.tutorials.rpc.grpc.api.Helloworld.HelloRequest;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
public class Client {
public static void main( String[] args ) {
ManagedChannel channel = ManagedChannelBuilder.forAddress( "127.0.0.1", 8081 )
.usePlaintext( true ).build();
GreeterBlockingStub greeter = GreeterGrpc.newBlockingStub( channel );
HelloRequest joe = HelloRequest.newBuilder().setName( "Joe" ).build();
HelloRequest mary = HelloRequest.newBuilder().setName( "Mary" ).build();
HelloReply joesReply = greeter.sayHello( joe );
HelloReply marysReply = greeter.sayHello( mary );
System.out.println( joesReply.getMessage() );
System.out.println( marysReply.getMessage() );
}
}
src/main/java/com/athaydes/tutorials/rpc/grpc/Client.java
Notice the call usePlainText( true ) when creating the client's channel. That's just to avoid using TLS! The server in this example does not enable TLS, so if you copy this code, remember to do that... and don't blame me if you forget, this is just a hello-world example!
And that's it... run the Server class first, then the Client in a separate shell. You should see the expected hello messages:
Hello Joe
Hello Mary
You've probably never worked so hard to get a couple of hello world messages printed in the terminal, but hey, this is a distributed application now! And you didn't even once have to build a HTTP call!!!
Hm... maybe something else could do this but a little bit more easily?!
SOAP is definitely out of the question as using it could ruin our reputation (even though it didn't look too different from the above, to be honest... except when all the specs that developed around SOAP are taken into account, then yeah, we don't want to go back to that)... but how about its simpler, nearly forgotten predecessor, XML-RPC?! Well, let's have a look (if you hate XML so much you wouldn't even think, skip to the next section!).
XML-RPC
XML-RPC is an old technology. It was designed in 1998, according to Wikipedia, and later developed into SOAP. But despite that, it is really simple! So simple you can read the full specification in around 5 minutes.
Because of its simplicity and age, it has been implemented in every language imaginable (here are the libraries for JavaScript, PHP, Python, C and C++, Haskell, Go, .NET, OCaml, Common Lisp, Clojure, Rust, Swift and, of course, Java - or this one for Android).
It's amazing how XML-RPC already in 1998 brought us a way of making remote calls based on open internet protocols people were already familiar with, something that SOAP managed to totally mess up just a few years later...
Anyway, even really ancient technologies might sometimes turn out to still be useful, so let's have a look at what exactly XML-RPC looks like using Gradle and Java.
1. Create a Gradle build file:
plugins {
id "java"
}
group 'com.athaydes.tutorials'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile 'org.apache.xmlrpc:xmlrpc-server:3.1.3'
compile 'org.apache.xmlrpc:xmlrpc-client:3.1.3'
}
build.gradle
I've added dependencies on both xmlrpc-server and xmlrpc-client to that we don't need separate modules for the server and the client in this example.
2. Create a Handler class that you want to be called remotely.
package com.athaydes.tutorials.xmlrpc;
public class Handler {
public String sayHello( String name ) {
return "Hello " + name;
}
public String sayHelloAgain( String name ) {
return "Hello again " + name;
}
}
src/main/java/com/athaydes/tutorials/xmlrpc/Handler.java
3. Create a Server.
package com.athaydes.tutorials.xmlrpc;
import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.server.PropertyHandlerMapping;
import org.apache.xmlrpc.webserver.WebServer;
import java.io.IOException;
public class Server {
public static void main( String[] args ) throws XmlRpcException, IOException {
PropertyHandlerMapping mapping = new PropertyHandlerMapping();
mapping.addHandler( "Handler", Handler.class );
WebServer server = new WebServer( 8081 );
server.getXmlRpcServer().setHandlerMapping( mapping );
server.start();
}
}
src/main/java/com/athaydes/tutorials/xmlrpc/Server.java
4. Create a Client.
package com.athaydes.tutorials.xmlrpc;
import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;
import java.net.MalformedURLException;
import java.net.URL;
public class Client {
public static void main( String[] args ) throws MalformedURLException, XmlRpcException {
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();
config.setServerURL( new URL( "http://127.0.0.1:8081" ) );
XmlRpcClient client = new XmlRpcClient();
client.setConfig( config );
Object[] params = { "Joe" };
String result = ( String ) client.execute( "Handler.sayHelloAgain", params );
System.out.println( result );
}
}
src/main/java/com/athaydes/tutorials/xmlrpc/Client.java
This Client call implementation is quite ugly, I am not sure why the Java library does not provide a Proxy wrapper around the XmlRpcClient so that calls to the remote method look like a normal call, but that's how things are currently.
I've implemented a Proxy wrapper anyway in a few lines of code, if you think it's useful, you can use it in your own projects:
package com.athaydes.tutorials.xmlrpc.proxy;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;
import java.lang.reflect.Proxy;
import java.net.URL;
public class ClientProxy {
@SuppressWarnings( "unchecked" )
public static <T> T wrap( Class<T> handlerType, URL serverUrl, String handlerName ) {
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();
config.setServerURL( serverUrl );
XmlRpcClient client = new XmlRpcClient();
client.setConfig( config );
return ( T ) Proxy.newProxyInstance( handlerType.getClassLoader(),
new Class[]{ handlerType },
( proxy, method, args ) -> client.execute( handlerName + "." + method.getName(), args ) );
}
}
src/main/java/com/athaydes/tutorials/xmlrpc/proxy/ClientProxy.java
This proxy is very simple to use and makes the client code much simpler:
package com.athaydes.tutorials.xmlrpc.proxy;
import java.net.MalformedURLException;
import java.net.URL;
public class SimplerClient {
public interface HandlerApi {
String sayHello( String name );
String sayHelloAgain( String name );
}
public static void main( String[] args ) throws MalformedURLException {
HandlerApi handler = ClientProxy.wrap( HandlerApi.class,
new URL( "http://127.0.0.1:8081" ),
"Handler" );
System.out.println( handler.sayHelloAgain( "Joe" ) );
}
}
src/main/java/com/athaydes/tutorials/xmlrpc/proxy/SimplerClient.java
Notice that we can only proxy via interfaces, so we needed to create a interface representing the remote service to be able to use the proxy... but you should use interfaces anyway when exposing remote services to avoid the client depending on the actual server implementation, so I don't see really this as a downside.
Build everything with gradle build.
Now we're ready to run the Server class in one shell, and the Client class in another. You should see the Hello again Joe message in the client, as expected.
It's difficult for me to see why XML-RPC is not so popular nowadays, or why people thought that it was not enough and some crazy complexity was necessary around it, so they created SOAP to solve whatever problem they think they had. As SOAP ultimately ended up being almost completely replaced with REST APIs, and that XML-RPC is much closer to REST than SOAP (in fact, you could architecture your XML-RPC API around REST principles), it seems that doing that was a serious mistake by the industry of the time.
Sure, XML is quite inefficient and verbose if performance is a requirement, but to solve that all they needed was a more efficient transport mechanism and serialization format (which is what gRPC did).
As you're probably aware, there's also a JSON-RPC specification, which is basically the same as XML-RPC except that JSON is used instead of XML, so I will not include it in this article.
GraphQL
Let's get this out of the way first: is GraphQL a RPC implementation? Well, some influential people certainly think so... and as I will show below, it definitely can be used like an RPC, so let's go with that!
GraphQL was developed by Facebook to make it easier for them to develop applications that required data from its servers in a variety of forms. It is described as "a query language for your API" in the home page and has been gaining substantial momentum recently.
As gRPC, GraphQL supports a dozen different languages. But how hard is it to use, both on the server and on the client side?
Only one way to find out! Let's implement a GraphQL Java HTTP server based on spark-java, and a Groovy client:
1. as usual, create a Gradle build file declaring the project's dependencies:
plugins {
id "java"
}
group 'com.athaydes.tutorials'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile 'com.graphql-java:graphql-java:6.0'
compile 'com.graphql-java:graphql-java-tools:4.3.0'
compile 'com.sparkjava:spark-core:2.7.1'
compile 'com.google.code.gson:gson:2.8.2'
runtime 'org.slf4j:slf4j-simple:1.7.25'
}
build.gradle
We'll use spark-java to wrap the GraphQL engine into a HTTP endpoint as that's probably the easiest way to do it in Java.
To convert the data objects to and from JSON, I chose Gson.
2. define your GraphQL schema. For example, here's mine:
schema {
query: QueryType
}
type QueryType {
sayHello(name: String!): Hello!
sayHelloAgain(name: String!) : Hello!
}
type Hello {
name: String!
message: String!
}
src/main/resources/hello_world.graphqls
GraphQL schemas can get pretty advanced. Its type system includes union types, enumerations, lists, interfaces, and non-null types (indicated by a ! at the end of the type name, as I used in the schema above for everything).
3. Write a Java definitions for the GraphQL types:
In this example, we make use of graphl-java-tools, which makes it considerably easier to use GraphQL, even though you could only use plain graphql-java and use a much lower-level API to feed data into the GraphQL engine.
package com.athaydes.tutorials.graphql;
import com.coxautodev.graphql.tools.GraphQLQueryResolver;
public class Query implements GraphQLQueryResolver {
public Hello sayHello( String name ) {
return new Hello( name, "Hello " + name );
}
public Hello sayHelloAgain( String name ) {
return new Hello( name, "Hello again " + name );
}
}
src/main/java/com/athaydes/tutorials/graphql/Query.java
package com.athaydes.tutorials.graphql;
public class Hello {
private final String name;
private final String message;
public Hello( String name, String message ) {
this.name = name;
this.message = message;
}
public String getName() {
return name;
}
public String getMessage() {
return message;
}
}
src/main/java/com/athaydes/tutorials/graphql/Hello.java
package com.athaydes.tutorials.graphql;
import com.coxautodev.graphql.tools.GraphQLResolver;
public class HelloResolver implements GraphQLResolver<Hello> {
public String name( Hello hello ) {
return hello.getName();
}
public String message( Hello hello ) {
return hello.getMessage();
}
}
src/main/java/com/athaydes/tutorials/graphql/HelloResolver.java
4. Expose the GraphQL engine via HTTP:
package com.athaydes.tutorials.graphql;
import com.coxautodev.graphql.tools.SchemaParser;
import com.google.common.reflect.TypeToken;
import com.google.gson.Gson;
import graphql.ExecutionInput;
import graphql.ExecutionResult;
import graphql.GraphQL;
import graphql.schema.GraphQLSchema;
import spark.Spark;
import java.util.Map;
public class Main {
public static void main( String[] args ) {
SchemaParser schemaParser = SchemaParser.newParser()
.file( "hello_world.graphqls" )
.resolvers( new Query(), new HelloResolver() )
.build();
GraphQLSchema executableSchema = schemaParser.makeExecutableSchema();
GraphQL graphQL = GraphQL.newGraphQL( executableSchema ).build();
Gson gson = new Gson();
// expose the GraphQL engine using a HTTP server
Spark.get( "/graphql-api", "application/json", ( req, res ) -> {
Map<String, Object> variables = gson.fromJson(
req.queryParamOrDefault( "variables", "{}" ),
new TypeToken<Map<String, Object>>() {
}.getType() );
ExecutionInput executionInput = ExecutionInput.newExecutionInput()
.query( req.queryParams( "query" ) )
.variables( variables )
.build();
ExecutionResult result = graphQL.execute( executionInput );
res.header( "Content-Type", "application/json" );
return gson.toJson( result );
} );
}
}
src/main/java/com/athaydes/tutorials/graphql/Main.java
The above implementation exposes a single /graphql-api endpoint which accepts GET requests with a query and variables query parameters. You could use POST as well, of course. Check the GraphQL docs for details on how GraphQL HTTP APIs are supposed to work.
At this point, you can build the server with gradle build.
If you're not into wiring everything together in your applications explicitly, you could use a framework with GraphQL support to do it for you.
5. Write a Groovy client (or use your favourite language):
@Grab( "com.athaydes.rawhttp:rawhttp-core:1.1.0" )
import com.athaydes.rawhttp.core.*
import com.athaydes.rawhttp.core.client.TcpRawHttpClient
query = URLEncoder.encode '''query($n:String!) {
sayHelloAgain(name: $n) {
name
message
}
}''', 'UTF-8'
variables = URLEncoder.encode '{"n": "Michael"}', 'UTF-8'
client = new TcpRawHttpClient()
req = new RawHttp().parseRequest( """\
GET localhost:4567/graphql-api?query=$query&variables=$variables HTTP/1.1
Accept: application/json
User-Agent: rawhttp
""" )
println client.send( req ).eagerly()
client.close()
src/graphql-client.groovy
I used raw-http (a Java library with 0 dependencies that I wrote myself to make these kind of HTTP-based prototypes easy!) to create and send a HTTP request.
Notice how a GraphQL "query" can look just like a RPC call, with the difference that it describes what the response it wants back should look like. query is a static String, but it is accompanied by a variables object containing values to be substituted in the query (allowing for more dynamic behaviour and efficiency).
Run the server Main class, then run the client with groovy src/graphql-client.
It will print the full HTTP response provided by our GraphQL server:
HTTP/1.1 200 OK
Date: Fri, 02 Mar 2018 20:07:24 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(9.4.6.v20170531)
{"data":{"sayHelloAgain":{"name":"Michael","message":"Hello again Michael"}},"errors":[]}
Unfortunately, on the Java server, we had to write the data classes by hand, and on the client, I didn't bother using types for the model at all (we would have to convert the JSON to a Java/Groovy type, but that's left as an exercise to the interested reader). That's because, unlike with gPRC, generating the types does not seem to be the standard approach in GraphQL, even though it is supported via graphql-java-type-generator or the more polyglot graphql-code-generator (if you're into TypeScript, Swift or Scala, there's also apollo-code-gen). But all these type generators seemed clunky and under-documented to me, so I decided not to use them in this example.
So, in summary, GraphQL is very powerful but not very simple. Matching the server code to your data storage may be challenging (but there are some helper libraries and frameworks that may make this easier, and even databases that support GraphQL natively). But losing any kind of type-safety and a language-idiomatic way to make calls on the client is a major bummer.
Anyway, GraphQL may be appropriate for clients that deal with complex data that changes frequently or back-ends that need to use several data sources to serve a single request, I guess.
Thrift
Thrift is Facebook's answer to Google's gRPC. It has been open-source since a whitepaper describing it was published in 2007 (when gRPC's predecessor, Stubby, was closed-source) and has now support for nearly 20 different languages. It is an Apache project since 2010.
To use Thrift with a Java, Gradle build, follow these steps:
1. Download and install the Thrift compiler:
Unlike gRPC, it seems that the Gradle plugin is not capable of automatically downloading the compiler, so we need to go to http://thrift.apache.org/download and install it manually. This can compilicate the build in CI servers.
The tutorial tells us to extract the tar ball, enter the thrift directory and run the following command:
./configure && make
What it doesn't say is that it will run for several minutes, as it builds and tests the Thrift implementation for several languages!
Well, nevermind... once you're done waiting, make sure you can run thrift from anywhere by adding a link to the executable:
ln -s $(pwd)/compiler/cpp/thrift /usr/local/bin/thrift
Apparently, if you're on Ubuntu, you can just run apt install thrift-compiler.
2. Create a Gradle build file:
plugins {
id "java"
id "org.jruyi.thrift" version "0.4.0"
}
group 'com.athaydes.tutorials'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile 'org.apache.thrift:libthrift:0.11.0'
}
sourceSets.main.java {
srcDirs "$buildDir/generated-sources/thrift/gen-java"
}
build.gradle
3. Create a Thrift file defining the data and service(s):
namespace java com.athaydes.tutorials.thrift.api
service HelloService {
string sayHello(1:string name)
string sayHelloAgain(1:string name)
}
src/main/thrift/hello_world.thrift
4. Run gradle build to generate the Java files.
This will generate a single file at build/generated-sources/thrift/gen-java/com/athaydes/tutorials/thrift/api/HelloService.java containing all the definitions required for the implementation of the RPC server and client.
5. Implement the server-side service:
package com.athaydes.tutorials.thrift;
import com.athaydes.tutorials.thrift.api.HelloService;
import org.apache.thrift.TException;
public class SimpleHelloService implements HelloService.Iface {
@Override
public String sayHello( String name ) throws TException {
return "Hello " + name;
}
@Override
public String sayHelloAgain( String name ) throws TException {
return "Hello again " + name;
}
}
src/main/java/com/athaydes/tutorials/thrift/SimpleHelloService.java
6. Implement the Server exposing the service:
package com.athaydes.tutorials.thrift;
import com.athaydes.tutorials.thrift.api.HelloService;
import org.apache.thrift.server.TServer;
import org.apache.thrift.server.TThreadPoolServer;
import org.apache.thrift.transport.TServerSocket;
import org.apache.thrift.transport.TTransportException;
public class Server {
public static void main( String[] args ) {
try {
TServerSocket serverTransport = new TServerSocket( 7911 );
HelloService.Processor processor = new HelloService.Processor<>( new SimpleHelloService() );
TServer server = new TThreadPoolServer( new TThreadPoolServer.Args( serverTransport ).
processor( processor ) );
server.serve();
} catch ( TTransportException e ) {
e.printStackTrace();
}
}
}
src/main/java/com/athaydes/tutorials/thrift/Server.java
7. Implement the Client:
package com.athaydes.tutorials.thrift;
import com.athaydes.tutorials.thrift.api.HelloService;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
public class Client {
public static void main( String[] args ) throws Exception {
TSocket tSocket = new TSocket( "localhost", 7911 );
tSocket.open();
TProtocol tProtocol = new TBinaryProtocol( tSocket );
HelloService.Client client = new HelloService.Client( tProtocol );
System.out.println( client.sayHello( "Mary" ) );
System.out.println( client.sayHelloAgain( "Mary" ) );
tSocket.close();
}
}
src/main/java/com/athaydes/tutorials/thrift/Client.java
As with the other examples, now you can run the Server in a shell and the Client in another, which should print the hello messages as expected:
Hello Mary
Hello again Mary
Thrift is really, really similar to gRPC. It's amazing that Google and Facebook both figured that the existing (at the time) RPC solutions were not enough and came up with something so similar, more or less independently (the Thrift whitepaper does mention Protobuffers but as it was closed-souce at the time, it's impossible to tell if they had access to the RPC design Google was using).
However, I do think that both of their solutions are sub-optimal in that they require more boilerplate than I think is justifiable for simple projects, including a custom IDL and its compiler, and that all code written to integrate with their frameworks cannot be re-purposed to use another RPC implementation without a lot of work.
Protobuf-TCP-RPC
Protobuf-TCP-RPC is a Java library I wrote myself based on Google's Protobuffer serialization format and TCP (the original idea was to implement an efficient Apache Aries' DistributionProvider for OSGi remote services). You might think that with gRPC, which is also a RPC based on Protobuffers, there should be no need for something like this, but I disagree.
First of all, gRPC only works with Protobuffer-generated data types, not with JVM types, which is quite annoying.
Secondly, having not only data types be generated by protoc but also the service base classes is quite a big limitation as all the code implementing the services needs to be specifically written for gRPC.
Hence, I thought that having a simpler, more JVM-friendly (but still usable in other platforms) RPC mechanism based on Protobuffers (which are great for serialization) was really needed. The disadvantage is that it's not as easy to use with non-JVM languages... but adding support for any language is simple enough using just protoc, given that the RPC implementation used by Protobuf-TCP-RPC is just a proto file describing a generic method call, and some kind of mapper from/to the language's data types, if desired.
Anyway, even though this is not nearly as mature and flexible as the other alternatives, it's simpler... here's how it works.
1. The Gradle build file:
plugins {
id "java"
}
group 'com.athaydes.tutorials'
version '1.0-SNAPSHOT'
sourceCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile 'com.athaydes.protobuf:protobuf-tcp-rpc:0.2.1'
}
build.gradle
2. Create a service interface representing the remote service:
package com.athaydes.tutorials.protobuftcprpc;
public interface HelloService {
String sayHello( String name );
String sayHelloAgain( String name );
}
src/main/java/com/athaydes/tutorials/protobuftcprpc/HelloService.java
3. Create a server-side implementation of the service:
package com.athaydes.tutorials.protobuftcprpc;
public class SimpleHelloService implements HelloService {
@Override
public String sayHello( String name ) {
return "Hello " + name;
}
@Override
public String sayHelloAgain( String name ) {
return "Hello again " + name;
}
}
src/main/java/com/athaydes/tutorials/protobuftcprpc/SimpleHelloService.java
4. Create a server exposing the service:
package com.athaydes.tutorials.protobuftcprpc;
import com.athaydes.protobuf.tcp.api.RemoteServices;
import java.io.Closeable;
public class Server {
public static void main( String[] args ) throws Exception {
Closeable server = RemoteServices.provideService(
new SimpleHelloService(), 8081, HelloService.class );
System.out.println( "Type something to stop the server" );
System.in.read();
server.close();
}
}
src/main/java/com/athaydes/tutorials/protobuftcprpc/Server.java
5. Create a client that uses the service interface:
package com.athaydes.tutorials.protobuftcprpc;
import com.athaydes.protobuf.tcp.api.RemoteServices;
public class Client {
public static void main( String[] args ) {
HelloService helloService = RemoteServices.createClient(
HelloService.class, "localhost", 8081 );
System.out.println( helloService.sayHello( "Joe" ) );
System.out.println( helloService.sayHelloAgain( "Joe" ) );
}
}
src/main/java/com/athaydes/tutorials/protobuftcprpc/Client.java
Build everything with gradle build. Run the Server class on a shell, and the Client on another. You should see the expected messages:
Hello Joe
Hello again Joe
This is, as far as I know, the simplest way to run RPC calls in the JVM, at least. If you need complex data types (i.e. custom data classes or structs), you need to create a proto file and generate the Java type for it, but if all you need is Java's basic types like int, float, double, char, boolean, String (coming soon, List, Set or Map of those), it will work without custom serialization logic.
I am even thinking of adding support for Kotlin data classes in a Kotlin-specific module, which would make this library much more powerful (and it's not difficult at all - if you would like to see this implemented, vote for it on GitHub).
Notable alternatives
This article is already a little bit longer than I think it should be (and I've already spent more time than is reasonable on it!), so without giving more details, here are some other alternatives you might want to look at:
Apache Avro
Apache Avro is a data serialization system that also supports RPC. It seems to be focused on dynamic languages and smaller payloads. As most alternatives, has its own custom IDL (defined with JSON), but defining one is optional.
It seems to have been developed for use in Apache Hadoop, and is also utilized in Apache Spark SQL.
JSON-RPC
As metioned in the XML-RPC section, JSON-RPC is quite similar to XML-RPC and for that reason, I decided to not describe it in any detail in this article.
If you're interested in learning more about it, have a look at simple-is-better.org's article about it. This website maintains a list of Java tools for working with JSON-RPC, including even a shell for experimentation.
Messaging systems
An alternative to REST and RPC which can be used to solve similar problems in distributed applications is a messaging system like JMS, ZeroMQ (highly recommended read even if you don't intend to use it!) and Apache Kafka (a really powerful system that can do a lot more than just implement the publisher/subscriber pattern).
Conclusion
REST is a great solution to distributed applications in many cases. As this article from 2013 shows, it was a welcome back-to-basics wake-up call to the industry at a time when complexity, much of it incidental, was starting to win over simplicity, causing applications to become difficult to write, maintain and evolve.
With REST, calling a remote service became as easy as sending a HTTP request.
But using REST for everything, including things like internal microservices used only by one or two other internal applications seems like a over-reaction. RPC is, and probably has always been, ideal for situations like this.
With the help of network-aware libraries like Netflix's Hystrix, RPC-based applications can be easier to develop and at the same time more efficient and reliable than a REST alternative.
If you ever need to break up a service into two simpler services, but you don't want to completely change how the different parts interact, do not hesitate to reach out for a RPC solution.
If your clients must be really flexible and customizable, and your data does look a little bit like a graph, GraphQL seems to be the way to go.
If performance is paramount and you want the option to use many different languages, go for either gRPC or Thrift.
You might even want to go with XML-RPC (or JSON-RPC) if you need to use some obscure languages or performance is not more important than simplicity for you.
To optimize for the absolute minimum network bandwidth, a good choice seems to be Apache Avro, specially if you are using dynamic languages.
If you're happy to stay in the JVM and use one of the many languages it supports, but don't want to rely on code-generation tools just to be able to call inter-process/remote services, then help me continue the development of protobuf-tcp-rpc (I think the library needs a better name, suggestions welcome!).
Thanks for reading.