Hi,
I just finished the first version of our internal API description that I
developed. Please note that we don't want this to be an official public
KIARA API. Possibly it can be used as a KIARA plugin API, but I expect
more modifications in future. All of this is work in progress and change
from day to day when new features are added.
It is possible that my description is not fully clear, so please comment
and ask about details.
Best,
Dmitri
-------------- next part --------------
KIARA DFKI Internal API Design
------------------------------
In this document we explain internal design and implementation of the current KIARA/DFKI Java version
available on https://github.com/dmrub/kiara-java.
1. Overall design
The basic idea behind the internal API is that given a communication channel (e.g. TCP socket) both
endpoints send and receive messages. The message can be seen as an universal data unit that can represent
an RPC request, response, notification, etc.
2. Transport
The transport layer works on top of the communication channel and takes care of splitting input data stream(s)
into transport messages, and also to send transport messages via output data stream(s).
Design was based on following decisions:
* Transport layer is independent of other layers,
we have no dependencies on serialization and dispatching layers of the framework.
* We keeping all information provided by the implementation of the transport layer, since it can be required by
the serialization and dispatching layers.
* API is fully asynchronous
* Convenience for the end-user (developer) is not a goal, since API is internal
Transport layer API consists of following classes:
TransportMessage - represents message and all the information that is provided by the transport layer.
(source: http://goo.gl/bk0ohV)
Transport - factory that produces TransportConnection and TransportAddress instances.
(source: http://goo.gl/p7SF2m)
TransportConnection - represents single communication channel between client and server (e.g. opened TCP socket),
is used to send and receive transport messages.
(source: http://goo.gl/Y0NlWK)
TransportAddress - represents abstract address identifying TransportConnection, Transport constructs TransportAddress instances from URI.
(source: http://goo.gl/jcRvTR)
TransportRegistry - allow to register Transport instances with the transport name. (e.g. HttpTransport with "http" and "https").
(source: http://goo.gl/jFIfL7)
Transport layer can also be seen as a higher level abstraction of a (not necessarily network-based) communication library,
that allow to exchange messages containing payload (ByteBuffer) and related metadata. The current design is actually
dictated by the features of the HTTP protocol, not TCP. In HTTP a single request message contains following information:
Request line, such as GET /logo.gif HTTP/1.1 or Status line, such as HTTP/1.1 200 OK
Headers
Optional HTTP message body data
For further processing and dispatch we need to keep and make available all this information.
We map HTTP body data to the payload of the TransportMessage and represent it by the ByteBuffer.
Everything else is stored as the metadata in a String to Object mapping. The mapping has a number
of predefined keys with the fixed interpretation:
session-id - ID of the current communication session, required for security
content-type - Mime Type of the payload data encoding
request-uri - URI used in HTTP request, or similar transport protocol
http-method - HTTP method used in HTTP request, or similar transport protocol
More keys can be easily defined later without changing the API.
Additionally to the abstract representation of the transport connection and message we also provide abstract representation of the
transport address. We need this for the advanced message dispatch, because in protocols like HTTP dispatch of the request is not based
only on the combination of host and port. For example assume we provide KIARA services via HTTP server.
The same host and port combination can be shared between multiple services (e.g. static HTML pages, files, etc.).
In order to figure out if we should handle the request we need to check additionally the request path.
Since transport layer is not taking care of the dispatch we need to provide an abstract API that allow to match requests.
The idea is that URI's can have patterns (e.g. glob patterns with '*' and '?' letters) so we can register services not just for
a specific path but also for a pattern. For example:
String patternUri = "http://localhost:8080/*/bar";
String addrUri1 = "http://localhost:8080/foo/bar";
String addrUri2 = "http://localhost:8080/moo/tar";
Kiara.init();
TransportAddress patternAddr = TransportRegistry.createTransportAddressFromURI(patternUri);
TransportAddress addr = TransportRegistry.createTransportAddressFromURI(addrUri1);
boolean result = patternAddr.acceptConnection(addr);
System.out.println(result); // Will output true
addr = TransportRegistry.createTransportAddressFromURI(addrUri2);
result = patternAddr.acceptConnection(addr);
System.out.println(result); // Will output false
The transport API is fully self-contained, so it can be used directly without the rest of the KIARA API.
Here are examples of the client and server: http://goo.gl/1cNbo1, http://goo.gl/Z7sRcC.
It is built on top of the Netty library (http://netty.io/), but it is 100% wrapper in order to be able to change internal implementation
easily. Currently TCP and HTTP transport protocols are supported.
The only limitation of this API is that without modification it can only support a transport protocol where data stream can be
split into single messages.
For example our TCP protocol sends a 32-bit unsigned integer specifying payload length at the start of every message.
3. Protocol
Protocol layer is responsible for deserializing TransportMessage-s into request and response messages and vice versa.
Protocol layer API consists of following classes:
Message - represents a deserialized message that can be either serialized into TransportMessage or dispatched.
Message contains its kind (REQUEST, RESPONSE or EXCEPTION), RPC method name, RPC call arguments or RPC call result.
(source: http://goo.gl/1WUeQW)
Protocol - provides construction of Message instances for requests and responses from TransportMessage payload (ByteBuffer).
(source: http://goo.gl/ViC5ek)
ProtocolRegistry - factory that provides registration of Protocol classes, and construction of Protocol instances.
(source: http://goo.gl/YMx2rv)
Important here is a difference between TransportMessage and Message. They represent usually the same message but on different
abstraction levels. TransportMessage instance contains just data, but no interpretation (request, exception or response).
Message instance represents an RPC message and allows to get or set arguments and a result.
Additionally Message implementations (e.g. JsonRpcMessage, source: http://goo.gl/mh2fQP) store data that are specific to the protocol.
In the case of JSON-RPC we need to store message ID required for the dispatch.
Note that in our current implementation there is no serialization API, because we use external libraries
(jackson for JSON-RPC: https://github.com/FasterXML/jackson, Java Serialization for JavaObjectSerialization format).
Also depending on the protocol serialization encoding can be too different, so even multiple APIs are possible.
Instead Message class currently require that user pass type information about Java class that should be deserialized,
and serializer implementation takes care of correct conversion.
4. Dispatching
Most of the server-side dispatch mechanism is implemented in ServerConnectionHandler.java (source: http://goo.gl/vMwqCr) and
ServiceHandler.java (source: http://goo.gl/HTOIXp).
1) ServerConnectionHandler.onRequest is called each time when new incoming TransportMessage arrives.
We prepare TransportMessage instance for a response.
2) First we check if the request is a part of negotiation protocol, and then we return JSON-encoded configuration document.
3) When request is not processed yet we search for the service (ServiceHandler instance) accepting requests on the transport address
of the transport message.
(Note: this part of the code is work in progress, ServerConnectionHandler will store later all ServiceHandlers that accept
requests on the opened connection).
4) When ServiceHandler.performCall() is executed it converts TransportMessage to a Message by using internal Protocol instance.
Then bound Java method is invoked, result or exception of its execution is serialized into a response Message and then into
a TransportMessage.
The client-side dispatch mechanism is implemented in the DefaultInvocationHandler class (source: http://goo.gl/bOzLzB).
Client side dispatch is required because we can send and receive messages asynchronously.
Because of this the order in which responses arrive is not specified. We solve this problem by having a list of handlers
(that we call Pipeline) for pending responses. When we perform a call we add a handler to the pipeline and invoke asynchronous
function that waits for the handler and then deserializes the result.
When response arrives we search for a corresponding handler and pass a response (TransportMessage) to it.
Our asynchronous function deserializes the result and returns it to the caller.
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy