[Miwi-middleware] KIARA/DFKI Java Design Document

[Dmitri Rubinstein]

Jaime requested me to send this separately. Please note this is a first 
draft, and I would like to see comments, issues, etc.

FYI, in the directory Benchmarks of the kiara-java repository:

https://github.com/dmrub/kiara-java/tree/master/Benchmarks

you will find user scenario based on 
http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201004.html implemented 
with different middlewares:

ApacheAxis2
ApacheThrift
RMI
RabbitMQ
Zero ICE
Plain Java TCP sockets

They are also used for benchmarking.

Best,

Dmitri
-------------- next part --------------
This documents describes design ideas behind the KIARA Java API
that was developed by DFKI: https://github.com/dmrub/kiara-java

Our starting point was the design of the KIARA/DFKI C/C++ API,
but we also took into account features specific to Java.

API examples are in KIARA/src/main/java/de/dfki/kiara/example

Notes: 

This API is the result of our internal discussions. We are always open
to other ideas and would be happy to consider alternative designs
(given suitable suggestions).

This API design is driven by the original KIARA ideas about a unified
middleware API that 
-- operates using the native data types of the application,
-- supports several different transport mechanisms and protocols,
-- supports to negotiate and select a suitable protocol/mechanims at run-time,
-- can transfers data directly between the memories of the two application (e.g. by RDMA),
-- allows for optimized transformation between the native data and the used protocol formats.

This current Java API is designed to support Remote Procedure Call
(RPC) pattern, so far. Changes to support PubSub should be minimal and
are under consideration now. RDMA-like support might require a small
additional API to register instances of data types with the middleware
as send/receive buffers.

This API is compatible with Java SE version 7 and higher.

-- Client --

1. Opening connection

   Like in the C++ version we start first with creating context and using it
   for opening the connection. A URI is used as the universal server address:

     Context context = Kiara.createContext();
     Connection connection = context.openConnection("http://localhost:8080/service");

   We assume following IDL in our example:

    namespace * calc "
    service calc {
      i32 add(i32 a, i32 b)
      float addf(float a, float b) "
    }

   After the connection has been opened:
    1) An IDL provided by the server under the URL is loaded and parsed
    2) The best protocol and transport available by both client and server are
       selected.

2. Binding Java methods to the IDL

   As the interface to KIARA will be in terms of the native types of
   the application, the next step is to bind suitable Java methods
   (specified/defined through Java interfaces) to the remote service
   methods provided by the server (and described in the IDL). Later we
   will invoke the bound Java methods, which will trigger the
   communication and execute the remote service methods on the server.

   In the C/C++ API we bind the interface methods to individual
   functions. Because Java does not support single functions (like
   .NET delegates or C function pointers), here we bind to methods of
   a Java interface:

    1) User provides Java interface that corresponds to the service that
       he would like to call:

         public interface Calc {
             public int add(int a, int b);

             public float addFloat(float a, float b);
         }

    2) User uses the generic KIARA class MethodBinding<T> to describe
       which methods of the interface are bound to which service
       methods in the IDL provided by the server.

         MethodBinding<Calc> binder
           = new MethodBinding<>(Calc.class)
           .bind("calc.add", "add")
           .bind("calc.addf", "addFloat");

       The first argument to the bind method is the full name of the
       service, the second argument is the name of the Java interface
       method.  The third optional variable length argument allows to
       pass classes in order to identify overloaded interface methods.

       Parameterizing MethodBinding with the interface type enables
       KIARA to check whether the passed interface method names are
       correctly typed.

3. Now the user can generate instance of the interface Calc with
   bindings registered with MethodBinding class:

     Calc calc = connection.generateClientFunctions(binder);

   This operation might throw an exception when the service methods
   registered through binder are not available in the IDL or when
   argument or result types are not compatible.
     
   All of the above code can easily be hiden in a setup
   function. Internally our implementation uses
   java.lang.reflect.Proxy for implementing this.

4. Once the setup it has been performed, the user can now execute
   remote service methods by calling the generated interface methods:

     {
       int result = calc.add(21, 32);
       System.out.printf("calc.add: result = %d\n", result);
     }

5. Finally Context and Connection can be closed with the close method:

     connection.close();
     context.close();

A1. Note: Besides the Calc interface the object returned by
   generateClientFunctions() also implements KIARA's RemoteInterface
   interface. It can provide additional information about the remote
   service (e.g. the used connection):

     RemoteInterface ri = (RemoteInterface) calc;
     Connection c = ri.getConnection();

-- Server --

1. Creating the service
   
   As in the client we use the C/C++ API as the basis:

     Context context = Kiara.createContext();
     Service service = context.newService();

   The service class represents the remote service that is provided by
   the server.  A single server instance can provide multiple
   services.

2. Specifying the IDL

   At first user need to define what the service will provide by loading IDL:

     service.loadServiceIDLFromString("KIARA",
         "namespace * calc "
       + "service calc { "
       + "    i32 add(i32 a, i32 b) "
       + "    float addf(float a, float b) "
       + "} ");

3. Binding the Java class to the IDL

   Let's assume we are given an implementation of the service as follows:

     public class CalcImpl {
        public int add(int a, int b) {
            return a + b;
        }

        public float add(float a, float b) {
            return a + b;
        }
     }

  Given this implementation, the user can now create an instance of
  CalcImpl and bind it to the IDL:

     CalcImpl impl = new CalcImpl();

     service.registerServiceFunction("calc.add", impl, "add", Integer.TYPE, Integer.TYPE);
     service.registerServiceFunction("calc.addf", impl, "addf", Float.TYPE, Float.TYPE);

  The first argument is the service name, the second argument is the
  implementation instance, the third argument is the name of the
  implementing method. The finally arguments provide the types of the
  arguments.

4. Creating the server.

   Again this step is similar to the C/C++ API:

     // Create new server on specified port and host
     // negotiation document with path: /service
     Server server = context.newServer("0.0.0.0", port, "/service");

     // Register service under the relative URL "/rpc/calc"
     // and available via "jsonrpc" protocol.
     // Other options are possible as well.
     server.addService("/rpc/calc", "jsonrpc", service);

     // Run server.
     server.run();

5. Finally the Context and Server objects can be closed:

    server.close()
    context.close()


-- Asynchronous API --

What we described before allow us to perform synchronous API calls.
However, our API is designed in such a way that no extension is required in order to
call methods asynchronously. The user only needs to change signature of the interface
by using java.util.concurrent.Future<V> class. For example:

  public interface Calc {
    public Future<Integer> add(int a, int b);

    public Future<Float> addFloat(float a, float b);
  }

Now when we call Calc.add() or Calc.addFloat() methods we will get no immediate
results but a future handle. Program execution will continue since we are
not waiting for a result. A call to Future<V>.get() will enforce
immediate execution and block until result or an error arrives.

The only limitation of this approach is that Future<V> type does not support
notification of the computation end. We can either block and wait for the result
or check if the result is available. As an extension we also support  
ListenableFuture<T> which provides support for notification callback:

https://code.google.com/p/guava-libraries/wiki/ListenableFutureExplained

Future-s can be used as argument and result types in both client interfaces
and server implementation classes.