[Fiware-middleware] DFKI Thrift-based IDL Proposal

[Thomas Michael Bohnert]

Jaime, you need to speed up a bit. We had agreed that you provide 
feedback asap.

Thanks, Thomas

On 10/22/2014 11:35 AM, Jaime Martin Losa wrote:
> 	Hi Dimitri,
>
> 	We will send you soon an updated document about this. Please be patient.
>
> Regards,
> Jaime.
>
> -----Original Message-----
> From: fiware-middleware-bounces at lists.fi-ware.org [mailto:fiware-middleware-bounces at lists.fi-ware.org] On Behalf Of Dmitri Rubinstein
> Sent: miércoles, 22 de octubre de 2014 11:29
> To: Thomas Michael Bohnert; Bohnert Thomas Michael (bohe); fiware-middleware at lists.fi-ware.org
> Subject: Re: [Fiware-middleware] DFKI Thrift-based IDL Proposal
>
> Hi Thomas,
>
> On 10/21/14 15:11, Thomas Michael Bohnert wrote:
>> HI Dimitry,
>>
>> The starting point will be OMG IDL 3.5. You will sooner or later have
>> to get familiar with it.
>>
>> Please adhere to that decision and use the time to get into the topic.
>> If you need some more time to study it please let me know.
>
> Of course I need more time. I received no link to the OMG IDL spec and found with google this one: http://www.omg.org/spec/IDL35/3.5/PDF/. It has 100 pages. As far as I understood we don't plan to support everything, just a subset. But which parts of the spec ?
>
> I already found a feature that we can't easily support: preprocessing (Section 5.3). Since DFKI part of the proposal requires IDL processing at the runtime of the program it also requires downloading and parsing of the IDL at the runtime. IDL file with #include directives can only be correctly preprocessed when all included files are downloaded in advance. One option is to support only already preprocessed IDL files without preprocessor directives.
>
> Best,
>
> Dmitri
>
>>
>> I will have a look into what you have provided but we will not start
>> from a Thrift foundation.
>>
>> Thanks for your cooperation,
>> Thomas
>>
>> On 10/21/2014 03:05 PM, Dmitri Rubinstein wrote:
>>> Hi Thomas,
>>>
>>> On 10/21/14 14:40, Thomas Michael Bohnert wrote:
>>>> Dimitri,
>>>>
>>>> Sorry, but then this is beyond was what requested yesterday.
>>>>
>>>> Thanks for your contribution but we wont consider it.HI
>>>>
>>>> Please read the minutes again and start from the OMG-IDL 3.5.
>>>
>>> This is just impossible, since first of all we are not experts in
>>> OMG-IDL. In order to fulfill your task I need not just to see a OMG-IDL
>>> grammar but usage examples for any feature. Thrift provides them as a
>>> part of tutorial and test suite, and I refer to them in my text. It is
>>> hard to me to find similar OMG IDL examples online.
>>>
>>> Also because of this Philipp requested yesterday a link to the OMG IDL
>>> 3.5 PDF, and I requested answers to my questions regarding annotation
>>> examples. What I mean is when somebody would like to convince us to use
>>> OMG IDL then they should show how our features described in Thrift IDL
>>> will be expressed in OMG IDL.
>>>
>>> IMHO it does not really matter which spec I use as a foundation since  I
>>> describe differences together with examples. Also I documented features
>>> that AFAIK are missing in OMG IDL (2.5 Unused Thrift features) but are
>>> available in Thrift.
>>>
>>> Best,
>>>
>>> Dmitri
>>>
>>>>
>>>> Best - Thomas
>>>>
>>>> On 10/21/2014 02:26 PM, Dmitri Rubinstein wrote:
>>>>> No, our starting point is Thrift IDL, and I describe all differences
>>>>> including a justification. Task with OMG IDL as foundation is
>>>>> eProsima's
>>>>> task, since they are experts in OMG IDL.
>>>>>
>>>>> https://docs.google.com/document/d/1a0pttnIaHX6aGvCqvostEm8iB1n1hHD7vg7JxbfNe5Y/edit:
>>>>>
>>>>>
>>>>>
>>>>> Deadline Thu EOB
>>>>>       Joined document that describes the new IDL
>>>>>       Features that have been dropped out of OMG (ePros)
>>>>>         Including a justification
>>>>>       Features that are proposed by Thrift (DFKI)
>>>>>         Including a justification
>>>>>         In particular related to features that are required by KIARA
>>>>> innovations
>>>>>
>>>>> Best,
>>>>>
>>>>> Dmitri
>>>>>
>>>>> On 10/21/14 14:16, Thomas Michael Bohnert wrote:
>>>>>> Dimitry,
>>>>>>
>>>>>> Thanks - just one question. Is this starting from the OMG IDL as
>>>>>> foundation?
>>>>>>
>>>>>> I assume so since this was the task to complete.
>>>>>>
>>>>>> Best - Thomas
>>>>>>
>>>>>> On 10/21/2014 02:13 PM, Dmitri Rubinstein wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> as discussed in the last meeting I put here information about our IDL
>>>>>>> proposal. When final document appears online I will copy this text
>>>>>>> to it.
>>>>>>>
>>>>>>> 1. KIARA/DFKI IDL is based on Apache Thrift IDL, with the grammar
>>>>>>> available here:
>>>>>>>
>>>>>>> https://thrift.apache.org/docs/idl
>>>>>>>
>>>>>>> Our grammar is available here:
>>>>>>>
>>>>>>> https://forge.fi-ware.org/plugins/mediawiki/wiki/fi-ware-private/index.php/Middleware_-_KIARA_-_User_and_Programmers_Guide#KIARA_Interface_definition_language
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Note: our goal was not to keep a full backwards compatibility with
>>>>>>> Thrift, but to modify a grammar only when necessary. Still
>>>>>>> technically
>>>>>>> we can be backwards compatible (we can parse already *existing*
>>>>>>> Thrift
>>>>>>> IDL files), since we avoided to remove any functionality from Thrift
>>>>>>> (only 'oneway' keyword was removed, see below). However, this was not
>>>>>>> tested yet.
>>>>>>>
>>>>>>> 2. We did following modifications:
>>>>>>>
>>>>>>> 2.1 Base types
>>>>>>>
>>>>>>> 2.1.1 Added unsigned types missing in Thrift: u8, u16, u32, u64
>>>>>>>
>>>>>>> 2.1.2 Renamed bool to boolean, byte to i8 (signed byte).
>>>>>>>
>>>>>>> 'boolean' is used more frequently than 'bool' in other IDLs. We can
>>>>>>> switch back to 'bool' if desired.
>>>>>>>
>>>>>>> 'byte' as a signed integer is not consistent with names of other
>>>>>>> signed
>>>>>>> integers (i16, i32, i64). We can switch back to 'byte' as signed
>>>>>>> 8-bit
>>>>>>> integer and use 'ubyte' as unsigned 8-bit integer if desired.
>>>>>>>
>>>>>>> A comparison table for basic types for DFKI/KIARA, CORBA, Thrift,
>>>>>>> etc.
>>>>>>> is here:
>>>>>>>
>>>>>>> https://forge.fi-ware.org/plugins/mediawiki/wiki/fi-ware-private/index.php/Middleware_-_KIARA_-_User_and_Programmers_Guide#Primitive_types
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2.2 Annotations
>>>>>>>
>>>>>>> The official Thirft grammar available on the web (see link above)
>>>>>>> *does
>>>>>>> not* contain flexible annotation syntax. There is an annotation
>>>>>>> support
>>>>>>> available in the latest Thrift version:
>>>>>>>
>>>>>>> https://github.com/apache/thrift/blob/master/test/AnnotationTest.thrift
>>>>>>>
>>>>>>>
>>>>>>> However, annotations are limited to key-value pairs and even more
>>>>>>> important: seems that function arguments and return types can't be
>>>>>>> annotated, there are at least no examples.
>>>>>>>
>>>>>>> Instead of using Thrift annotations we decided to introduce a
>>>>>>> different
>>>>>>> syntax inspired by WebIDL:
>>>>>>> http://www.w3.org/TR/WebIDL/#idl-extended-attributes.
>>>>>>>
>>>>>>> 2.2.1 Added syntax for defining custom annotations, similar to
>>>>>>> structs
>>>>>>> and exceptions:
>>>>>>>
>>>>>>> annotation HTTPPort {
>>>>>>>        i32 port = 80
>>>>>>> }
>>>>>>>
>>>>>>> 2.2.2 Added ability to annotate following entities: services, service
>>>>>>> functions, return types of service functions, arguments of service
>>>>>>> functions. We need this especially for security-related features. For
>>>>>>> example:
>>>>>>>
>>>>>>> namespace * enc
>>>>>>>
>>>>>>> // HTTPPort annotates the 'enc' service with argument 8080
>>>>>>> [HTTPPort(8080)]
>>>>>>> service enc {
>>>>>>>
>>>>>>>          // Function ping is annotated with Oneway and Encrypted
>>>>>>> annotations
>>>>>>>          // without parameters
>>>>>>>          [Oneway, Encrypted]
>>>>>>>          void ping()
>>>>>>>
>>>>>>>          // In function saveString Argument s of type string
>>>>>>>          // is annotated with annotation OptionallyEncrypted without
>>>>>>>          // parameters
>>>>>>>          void saveString(string fileName, [OptionallyEncrypted]
>>>>>>> string s)
>>>>>>>
>>>>>>>          // Return type of function loadString is annotated with
>>>>>>>          // annotation OptionallyEncrypted without parameters
>>>>>>>          string [OptionallyEncrypted] loadString(string fileName)
>>>>>>> }
>>>>>>>
>>>>>>> Also see our documentation:
>>>>>>> https://forge.fi-ware.org/plugins/mediawiki/wiki/fi-ware-private/index.php/Middleware_-_KIARA_-_User_and_Programmers_Guide#Functions_2
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2.3 Generic types
>>>>>>>
>>>>>>> Original Thrift IDL supports following predefined container types:
>>>>>>> list<T>, map<T1,T2>, and set<T>. We decided to extend this syntax to
>>>>>>> arbitrary generic types: GenericType<T1, T2, ..., Tn> where Ti can be
>>>>>>> either type or constant. In this way we can support types like
>>>>>>> bounded
>>>>>>> arrays but still keep backwards compatibility with the original
>>>>>>> syntax:
>>>>>>>
>>>>>>> array< i32, 2> // bounded array of size 2
>>>>>>> array< i32, array< i32, 4> > // two-dimensional array
>>>>>>>
>>>>>>> 2.4 Removed Thrift features
>>>>>>>
>>>>>>> We removed 'oneway' keyword from Thrift because we can use annotation
>>>>>>> Oneway:
>>>>>>>
>>>>>>> Original Thrift:
>>>>>>>
>>>>>>> service Test {
>>>>>>>        oneway void zip()
>>>>>>> }
>>>>>>>
>>>>>>> Our grammar:
>>>>>>>
>>>>>>> service Test {
>>>>>>>        [Oneway] void ping()
>>>>>>> }
>>>>>>>
>>>>>>> If backwards-compatibility to Thrift is desired we can try to support
>>>>>>> both styles.
>>>>>>>
>>>>>>> 2.5 Unused Thrift features
>>>>>>>
>>>>>>> We currently parse but ignore following Thrift features that are
>>>>>>> missing
>>>>>>> in OMG IDL:
>>>>>>>
>>>>>>> Struct fields can have optional integer identifier.
>>>>>>> Struct fields can be 'required' or 'optional'.
>>>>>>>
>>>>>>> 2.6 Planned extensions
>>>>>>>
>>>>>>> Thrift does not provide entity like CORBA's 'module'. The namespace
>>>>>>> keyword used by Thrift only allow to specify namespace in the
>>>>>>> generated
>>>>>>> code. We plan to add module NAME { ... } entity in order to provide
>>>>>>> support for nested IDL namespaces as well as annotations per module.
>>>>>>>
>>>>>>> Best,
>>>>>>>
>>>>>>> Dmitri
>>>>>>> _______________________________________________
>>>>>>> Fiware-middleware mailing list
>>>>>>> Fiware-middleware at lists.fi-ware.org
>>>>>>> https://lists.fi-ware.org/listinfo/fiware-middleware
>>>>>>>
>>>>>
>>>
>>
>
> _______________________________________________
> Fiware-middleware mailing list
> Fiware-middleware at lists.fi-ware.org
> https://lists.fi-ware.org/listinfo/fiware-middleware
>

-- 
Thomas Michael Bohnert (TMB)
Head of Service Engineering (ICCLab, SPLab)
Zurich University of Applied Sciences
eMail: thomas.bohnert at zhaw.ch
mobile: +41791758136
web: www.cloudcomp.ch, tmb.nginet.de
twitter: tmbohnert
skype: tbohnert