[Fiware-ngsi] GSMA contribution: New "Historic" API suggestions

[Ernoe Kovacs]

Correction:

[B.1] Graph Features
For examples, NGSI was meant for distributed stored entities.
So every link/relationship between NGSI entities are contained in attributes and are
usually URLs. Following those URLs means another REST query.  Of course, in a central
database this would not be done differently. You would directly link to the other
object and have query languages following that link.



______________________________________________________________________________________
Dr. Ernö Kovacs        エルノー・コヴァーチェ
NEC Europe Ltd.        NEC Laboratories Europe     Senior Manager Cloud Service and Smart Things
Kurfürsten-Anlage 36 | D-69115 Heidelberg          E-Mail: ernoe.kovacs at neclab.eu<mailto:ernoe.kovacs at neclab.eu>
Tel.  +49 6221 4342 – 131 | Fax. +49 6221 4342 – 115  Mobile: +49 (163) 2086046
NEC Europe Limited | Registered Office: Athene, Odyssey Business Park, West End Road, London, HA4 6QE, GB |
Registered in England 2832014
______________________________________________________________________________________

[NEC_Email_Footer_leafenginev2]<http://www.leafengine.com/>



From: fiware-ngsi-bounces at lists.fi-ware.org [mailto:fiware-ngsi-bounces at lists.fi-ware.org] On Behalf Of Ernoe Kovacs
Sent: Dienstag, 11. August 2015 10:04
To: Craig Aldridge; fiware-ngsi at lists.fi-ware.eu
Subject: Re: [Fiware-ngsi] GSMA contribution: New "Historic" API suggestions

Hi Craig, all
thanks for sharing the evaluation of the different query languages.
I always like weighted ranking matrix because it makes decision criteria
so much more clear.

[A] Clarification
In this specific case, it would be good to have more details on the selected
criteria, for example, for us, the geographic queries need to have circle
and polygones, but should also include 3D coordinates,.

P.8 of your slide set seem to indicate that circles are possible. Are you considering polygones
as well?
Too my understating you propose “geographic queries” as extension to  MQL?
The title of the powerpoint seem to suggest so. Please clarify, apologies if this has been explained somewhere
[B] NGSI Aspects
For me it is important that NGSI design philosophies are adequately reflected in
the query language.
[B.1] Graph Features
For examples, NGSI was meant for distributed stored entities.
So every link/relationship between NGSI entities are contained in attributes and are
usually URLs. Following those URLs means another REST query.  Of course, in a central
database this would not be done differently. You would directly link to the other
object and have query languages following that link.

To illustrate the problem…
Context_source A will report a UPDATE saying

{
  entity_id: “my_name”,
  entity_type: “moving_robot”,
  location : “http://<entity_id__of_room><http://%3centity_id__of_room%3e>”
}

His location will be an URL reference that later need to resolved in subsequent queries
to the target entity (not necessary in the same database but somewhere in the IoT network).
It is not guaranteed that the other entity <entity_id_of_room> is sending its own updates
to the ddatabase. So you need to QUERY.

If the historical query only looks for the entities, okay, if you want to specify operators that
follow relationships(Graph Features), you see the problem. Not everything is in the database
and not everything is using database-references.

[B.2] For example, we like to have “type“ names (for entities, attributes,
meta-data) as Ontology concepts including type hierarchy relationships.
/* using syntax from here: http://wiki.freebase.com/wiki/MQL */
(With simple words:
Our ontology says “Porsche” isA “car”
Then the entity
{
  "type": "porsche",
  "name": "My beloved rocket providing me a stream of endless speed tickets",
  "color": “red”
}
should match the query
{
  "type": "car",
  "name": [],
  "album": []
}
Is this feasible? I have the feeling that the type concept of MQL has some domain specific
aspects, but it is not really considering ontology-driven type matching.

I am finding documentation with freebase (example: here
http://www.freebase.com/base/fbontology/metaschema/relationship_id/superclass_subclass)
Which seem to support subclass/superclass type hierarchies.

… mony other NGSI-based aspects …

[C] I fear also that the type concept introduced here http://mql.freebaseapps.com/ch02.html#mqltypes
need to be carefully checked against NGSI usage. Not sure you want to support all
of those concepts and related queries in the NGSI historical API.
[D] IMPLEMENTATIONS
Most important, is there an existing implementation that checks how
the suggested queries are working against NGSI based objects?

During the standardization of NGSI in OMA, the core player had implementations  of similar systems
available and could check how features would work in their system. This helped. A lot.
I think we can learn from IETF standardization that requires running implementations before making
something into a standard.


After so many questions, let me state one thing.
I think having a separate API for historical queries is good. And we need a good/expressive query language.
I fear implementation effort and feature creep into specification-only work.
In the FIWARE context we have two choices:

-          Trying to get an agreement on the language and the supported concepts (might be MQLv2 or subset or…)
If we go down this route, IMHO we should follow some pragmatic approach saying
that only features implemented (and tested) by at least two NGSI_based enablers
should be included into the spec.

-          Being pragmatic and simple define a NGSI storage enable following the freebase schema and
the MQLv2 ideas. Get an implementation going. When storing into the NGSI storage enable,
make transformations optimizing for a central storage system, e.g.
map an URL to an internal database reference where feasible.
And allowing all NGSI-based enablers to support this API as an extra API.
But do not make it mandatory. Maybe add an mandatory “feature supported” API that can
be queried for which feature is implemented..

My personal opinion (which I might change ASAP when I see better arguments ;-) is to be pragmatic
and combine both.
Define a NGSI storage enabler based on Freebase/MQLv2. This should be powerful and
provide your features and storage extensions.

Pragmatically include historic query features one after the other into the NGSI spec following
the “at least two implementations” approach.

Recommend to the FIWARE uses…
(a) connect the FIWARE NGSI brokers to the NGSI storage enabler.
     Use MQLv2 for querying the NGSI storage enable
(b) If you need to query IoT agents with their own “memory” == historical database
     Directly on the agent or in the sensor, use NGSIv2, maybe with some limited
     historical queries.


-          Ernoe


______________________________________________________________________________________
Dr. Ernö Kovacs        エルノー・コヴァーチェ
NEC Europe Ltd.        NEC Laboratories Europe     Senior Manager Cloud Service and Smart Things
Kurfürsten-Anlage 36 | D-69115 Heidelberg          E-Mail: ernoe.kovacs at neclab.eu<mailto:ernoe.kovacs at neclab.eu>
Tel.  +49 6221 4342 – 131 | Fax. +49 6221 4342 – 115  Mobile: +49 (163) 2086046
NEC Europe Limited | Registered Office: Athene, Odyssey Business Park, West End Road, London, HA4 6QE, GB |
Registered in England 2832014
______________________________________________________________________________________

[NEC_Email_Footer_leafenginev2]<http://www.leafengine.com/>



From: fiware-ngsi-bounces at lists.fi-ware.org<mailto:fiware-ngsi-bounces at lists.fi-ware.org> [mailto:fiware-ngsi-bounces at lists.fi-ware.org] On Behalf Of Craig Aldridge
Sent: Mittwoch, 5. August 2015 17:22
To: fiware-ngsi at lists.fi-ware.eu<mailto:fiware-ngsi at lists.fi-ware.eu>
Cc: Ben Howes
Subject: [Fiware-ngsi] GSMA contribution: New "Historic" API suggestions

Dear all,

Following on from the GSMA’s previous communication on suggested contributions to the NGSIv2 specification, we have been discussing in detail and have agreed with Telefonica a proposed way forward. It is felt that the best way forward is to develop a new API that will deal with complex and historical queries rather than make breaking changes to the existing draft of the NGSIv2 specification. As such, please find detailed below (and attached) our thoughts on the new ‘Historic’ API.

GSMA have evaluated several choices of language for the historic API query language by means of a weighted ranking matrix. The criteria which each query DSL was ranked by was based on requirements defined in collaboration with the FIWARE team at Telefonica. MQL and elastic search are the two main contenders as a result of this, however neither is perfectly suited.

GSMA had already been working to create a specification (which you will see as MQLv2 in the ranking matrix) prior to involvement with FIWARE, which we feel offers a great choice of DSL to meet all the current requirements for querying, as well as offering scope to expand features in the future without breaking existing queries.

The attached slides provide an overview of the 3 key areas GSMA have sought to improve in MQL, whilst the document shows in more detail how these improvements look from a more technical perspective, as well as some examples of these features in use.

Can I please ask that this group review our suggestions and provide any feedback by no later than close of business on the 11th August; our target is to provide a full specification of the MQLv2 specification to this group by 14th August for final review.

If you have any questions please feel free to be contact me,

Kind regards,

Craig Aldridge | Project Manager | IoT Big Data Ecosystem|GSMA | +44 7725 221 299
GSMA, Floor 2, The Walbrook Building, 25 Walbrook, London, EC4N 8AF
P Please consider the environment – don’t print this mail unless you really need to...



This email and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this email or call +44 207 356 0600 and highlight the error.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-ngsi/attachments/20150811/ee3c0948/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 68508 bytes
Desc: image001.jpg
URL: <https://lists.fiware.org/private/fiware-ngsi/attachments/20150811/ee3c0948/attachment.jpg>