[Fiware-data] On the relation of the Metadata Processing GE (Siemens) and the CEP GE (IBM)

[Guy Sharon]

Hi,

I will comment in the order Andreas listed his understanding
- "manage incoming connections" - in my understanding this is the role for 
the gathering data GE. CEP GE is configured with the events it should 
process and not too focused on the connections and where the events come 
from. Although IBM's asset includes an adapter for introducing new types 
of connections, it is not the focus of the asset and the thoughts were to 
support adapters only for internal purposes of the Data\Context 
management.
- "prepare for further consumption" - CEP GEs should support simple 
(better term is stateless) operations as well (as IBM's asset does) such 
as filter-in\out and transformations. As for XML to Java Classes - we want 
to abstract this part in the entire chapter, meaning there will be no 
constraint on the structure and method of representing data, events and 
context (a part from what is of basic as in the discussions between 
Juanjo, Boris and myself). So - if you get metadata in other format than 
XML we should be able to support that as well.
-"provide to consuming GE" - any processing GE should support this

Therefore it initially looks very similar but before concluding I would 
ask the following questions as well:
1. What does it mean the metadata of streamed data? Are you analyzing the 
metadata separately? How is this then combined with the data? Or is your 
assumption that you 'propagate' the same stream with additional 
information or a slightly different stream?
2. How different is the processing on the metadata from the data? If you 
had the data in XML and need to convert to Java Classes and attach 
detection time - whats the difference? Is there something at the level of 
the filter or transformation that is semantical and you need someone to 
set this up in such a way that could never be done on data?
3. Not all data is events, CEP deals with events, what would be the issues 
of handling the metadata as events in order to process with CEP?
4. How do you configure such instructions\processing on metadata? Who does 
this and how? Could they be happy with a CEP like tooling\management?
Perhaps there are additional questions to help us understanding what are 
the differences and breaking points in processing for metadata

Guy Sharon
Manager
Event-based Middleware & Solutions Group


Event-based Middleware & Solutions

phone : 
+972 4 8296587
mobile : 
+972 54 6976417
address : 
IBM R&D Labs in Israel, Haifa University Campus, Mount Carmel, Haifa, 
31905, Israel
email : 
guysh at il.ibm.com




From:   "Hutter, Andreas" <andreas.hutter at siemens.com>
To:     "fiware-data at lists.fi-ware.eu" <fiware-data at lists.fi-ware.eu>
Date:   17/06/2011 12:59
Subject:        [Fiware-data] On the relation of the Metadata Processing 
GE (Siemens) and the CEP GE (IBM)
Sent by:        fiware-data-bounces at lists.fi-ware.eu



Dear Guy, all,
 
as agreed during the telco, I am taking up the discussion on the relation 
between the Metadata Processing GE (Siemens) and the CEP GE (IBM). 
The question to be discussed is whether the CEP GE would already cover the 
functionalities of the Metadata Processing GE and, therefore, the latter 
one could be discarded.
 
My current understanding is the following: 
 
a) The functionality of the (Meta)data Processing GE is to:
- manage the incoming streaming / data connections
- prepare the incoming data for further consumption in the analytics 
platform (e.g. transforming an XML document into JavaClasses, performing 
simple filtering operations, attaching reception time stamp, ...). There 
are no complex processing steps involved (e.g. correlations)
- provide the prepared data to the target consuming GE
 
b) These functions have corresponding functions that are provided inside 
the CEP GE. 
 
Hence we need to answer two questions:
1. Do we provide an stand-alone GE to provide the functions currently 
proposed for the (Meta)data Processing GE ? 
--> I believe that this would be beneficial because the tasks addressed by 
these functions are required all consuming GEs in the analytics platform. 
At the same time GE's like the CEP GE could provide their own internal 
"preprocessing components" to provide these functions, i.e. there is 
always the option to bypass the (Meta)data Processing GE.
 
2. Should we use the (Meta)data Processing GE proposed by Siemens or 
should we carve out the corresponding functions from the CEP GE by 
defining the appropriate interfaces and asking IBM to provide their 
implementation with these APIs  attiached ?
--> I do not have a strong opinion here. Either way is fine.
 
Best regards,
Andreas_______________________________________________
Fiware-data mailing list
Fiware-data at lists.fi-ware.eu
http://lists.fi-ware.eu/listinfo/fiware-data

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/old-fiware-data/attachments/20110619/d7b44fc7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 2558 bytes
Desc: not available
URL: <https://lists.fiware.org/private/old-fiware-data/attachments/20110619/d7b44fc7/attachment.gif>