IN ORDER TO CONTRIBUTE TO THE DISCUSSION, PLEASE ANSWER TO THIS EMAIL IN THE FOLLOWING FORMAT:
- ANSWER INLINE TO THE DISCUSSION POINTS
- DO NOT REMOVE ANY DISCUSSIONS (AND DO NOT REMOVE THIS TEXT EITHER)
- HIGHLIGHT YOUR ANWERS SO THAT THEY CAN BE RECOGNIZED AS NEW. ALSO REMOVE THE HIGHLIGHTING FROM YOUR PREVIOUS ANSWERS.
- SEPARATE NEW DISCUSSION POINTS BY A LINE OF “---------------------------------------“. ADD NEW DISCUSSION AT THE BOTTOM, GIVING IT A CONSECUTIVE NUMBER.
1: ----------------------------------:1
[TJ] I was wondering how I can contribute to the API description on git (for example, the introduction is missing a description the overall NGSI concepts – what is an entity, what is an attribute, what is a registration, etc.)
Theoretically I understand that the way to do it is to create a branch, do the changes on “doc\apiary\v2\fiware-ngsiv2-reference.apib”, commit the new branch back to the server and create a pull request to merge my branch with the “develop” branch.
The only thing I do not understand is how I can preview my edits using the apiary viewer. Is there a way I can get a preview?
[JMCF] You can use the Apiary Client
https://github.com/apiaryio/apiary-client
For contributing please create a Pull Request. Then it will be reviewed, once it is r+ed it will be merged.
[FGM] You can use the apiary client (https://github.com/apiaryio/apiary-client) to preview the render the .apib into an HTML file locally, eg:
apiary preview --path doc/apiary/v2/fiware-ngsiv2-reference.apib --output /tmp/tmp.html
2: ---------------------------------- :2
[TJ] could you list the most important points in terms of functionality that are added / removed by your proposal? E.g. I think that the attribute domains have been thrown out; at least I do not see them currently.
[FGM] Added:
* Geolocation and filtering capabilities (actually, I'd say that they aren't addition to NGSI itself -as their implemention in the standard operations family relies on existing Scope/Restriction structures-, but an addition in the way of using them for filtering).
* Pagination (already in Orion API for v1)
* Get entity types
* Get subscription list
* Get registration list
* Get subscription availability list
* Removed:
* Special semantics for metadata ID, i.e. ID is now a regular metadata (nobody was using these special semantcis, see my email to this list on June 2nd, 2015)
* Attribute domains (you can get the same using attributes which value is a key-map).
[TJ] One important point would be the restriction language. We used to have XPATH expressions for the XML message bodies, but they do not seem to have been used so much, and we talk about json messages now.
In the current ngsi2 proposal it seems restrictions on attribute values are supported as URL parameters, but filtering based on metadata (e.g. for retrieving only attribute values received from a certain provider) is not possible by this, right?
Have you thought about the language for restrictions? Something as powerful/complex as the previous xpath?
[FGM] Yes, we have considered the possibility of allowing a more powerfull query language (in addition to the simple one used by the "q" argument, that may suffice most of use cases) and even considering some alternatives, e.g. MQL. A complex language doesn't seems to be a good idea to be included in a URL param, so maybe makes more sense using query.
Please have a look to https://github.com/telefonicaid/fiware-orion/issues/1051. Jose Manuel is leading this part, so probably he may add additional comments about the status as it evolves.
[TJ] Have you also considered JSONPATH (see e.g. https://github.com/jayway/JsonPath) ?
Using JSONPATH would result in about the same expressiveness as the previous XPATH, and its syntax is potentially compact enough to add it to URL parameters.
[FGM] I have added a comment about JsonPath as alternative for the advance/full query language at https://github.com/telefonicaid/fiware-orion/issues/1051.
3: -------------:3
[TJ] Do you intend to keep the distinction between NGSI 9 and NGSI 10? In my view it would be useful to at least sharply distinguish between operations for exchanging information and operations for exchanging “availability information”.
[FGM] That difference could be explained in the introductory sections, before depicting each operation. What we have found is that people find confusion "NGIS9" and "NGSI10" (they tend to confuse with version 9 and version 10 of a thing called "NGSI"), so probably we should discuss about "context" and "context availability" but without mentioning NGSI9 and NGSI10.
[JH] I rather believe that distinguishing between NGSI9 and NGSI10 doesn't make so much sense from a developer's perspective.
[TJ] NEC agrees that we could get rid of the “API version numbers” NGSI-9 and NGSI-10.
But from the IoT Chapter point of view there should be a distinction between context-data and context-availability. In the IoT Backend the functionality is split between a component maintaining the availability information (“IoT Discovery”) and data (“IoT Broker”).
This sharp split e.g. also better enables distributed setups where a central IoT Discovery component provides several instances of IoT Broker with availability information.
Concretely, what we propose is to have context data (former NGSI-10) and context availability (former NGSI-9) on different resource trees like we had it previously; e.g. {serverRoot}/contextData and {serverRoot}/contextAvailability .
[TJ] Any further comment on this? This is a major design principle question, so we need to clarify it before being able to start working on the context availability specs.
4: -------------:4
- [TJ] I think it would be good to keep the approach of having a set of basic operations (queryContextRequest etc.) with which in principle one can do everything, so that convenience operations only add convenience but not add functionality as compared to the basic operations?
[FGM] In general, I agree. In fact, we plan to include standards operations also as part of the reference documentation. As I mention in my previous email: "The current draft at the above URL is focused on not-standard operations (sometime referred as "convenience operations") but at the end it will also include a specification for the standard operations (e.g. queryContext, updateContext, registerContext, etc.) based in the one we already have for the JSON encoding of that operations in the v1 API". However, there are some exceptions, eg: the convenience operation to get all the entity types ("GET /v2/types") in the sytesm doesn't have any counterpart in the standards operation family. But I understand that this is not a mayor problem.
[JH] We definitively should keep the non-convenience operations (I wouldn't call them "standards" by the way, maybe we need to call them in another way). However, I understand that we will implement some of the simplifications in the returned JSON structures, aren't we? Leaving the developer the option to choose between self-contained (verbose :-) JSON structures and more simplified structures also would apply for these kind of operations.
[MC] As newcomers to the NGSI / Fiware ecosystem, me and some colleagues of mine have found very confusing that NGSI v1 exposes multiple ways of accessing the exact same information thought the standard and convenience operations. After a quite heated discussion with Tobias at the F2F meeting in Heidelberg, I now have a better understanding of the historic reasons that pushed the decision to have a subset of the NGSI standard operations duplicated as "convenience" REST endpoints... but I still think it was an error in the first place. Accepting that some developers need a more "convenient" way to access data is accepting that standard operation are not convenient (by not being RESTfull) and that design of the NGSI v1 APIs is flawed in some way.
The first thing to decide (and there was no consensus at the F2F meeting) is whether NGSI v2 is about :
- breaking compatibility with v1 allowing to change all the APIs,
- keeping compatibility with v1 and just adding missing functionalities (more a v1.1),
We need to decide on that first.
Then, *if* the NGSI v2 specification is about breaking compatibility with v1 and simplifying it using REST endpoints, it *may* be the right time to discuss if the standard operations need to be included in v2 and if we can move all the NGSI functionalities to "REST-like" endpoints. I say only "REST-like" as some NGSI concepts cannot be mapped fully to REST properly.
On another side, I don't have anything against keeping the standard operations for v2. But then, we should accept that the NGSI model is not adapted to the REST conventions and don't try to apply REST partially by duplicating endpoints.
[FGM] NGSIv2 is a major rework of the FIWARE NGSI binding, so we can assume breaking compatibility with v1. From a client perspective, there should be no problem as long as the endpoint for v1 and v2 are separated, e.g. in Orion v1 uses <host>:<port>/v1 and v2 uses <host>:<port>/v2. In this context, keeping or not keeping standard operations is an open point. I think that they could be kept as long as they provide functionality not covered by non-standard operations, e.g. standard queryContext can query by a list of entities thus is a kind of bulk/batch query operation that make sense to be included. Probably we need to examine standard operation by standard operation to decide which ones are useful and which ones aren't.
By the moment, in seems that queryContext make sense (not yet included in http://telefonicaid.github.io/fiware-orion/api/v2/, PR pending https://github.com/telefonicaid/fiware-orion/pull/1060)
[TJ] I think one thing we should avoid in any case is to have the “standard operations” cover a subset of the functionality and the “convenience operations” cover a different subset of functionalities, such that users are forced to use “convenience operations” for certain things the standard operations do not support.
If we keep both standard and convenience operations we should design it such that with standard operations one can do everything, and we should make this really clear in the documentation.
One thing we almost surely cannot avoid is request message bodies. The NGSI standard allows custom operations scopes and we cannot assume that these scopes would always fit into the request URL. (E.g. geographic scopes can be arbitrarily complex polygons.).
5: ------------- :5
[NEC]: For NEC it is important to preserve the possibility to have multiple instances of attribute values. This might not be so important for most of the cases where NGSI is used these days, where entities represent sensors and attributes represent their readings. However, in we expect that in IoT scenarios entities will represent arbitrary objects like rooms, houses, etc. and then there can indeed be several providers of the same attribute values who should be distinguishable.
However, we also see that this possibility should not make the life of users difficult who do not care about multiple attribute value instances because in their scenarios this does not play a role. Therefore our proposal is that NGSI systems treat attribute values with no given ID as having a specific “null” ID.
By this, the value ID possibility can be completely ignored by scenarios where it is not needed, while the NGSI standard still offers the possibility to have multiple values.
[FGM] In order to fully understand you idea about IDs, could you elaborate an example aligned with the current NGSIv2 in order to see how it would looks like, please?
[TJ] Assume you have a context entity representing a street, and the co2-level of the street is measures by two different devices. In the NGSI model, the attribute “co2level” would then have two relevant values, each delivered by a distinct device, and these two values would be distinguished by distinct value IDs.
In FIWARE NGSI 1, applications who wish to retrieve only a specific attribute value could
- make a queryContextRequest and add to it an XPATH restriction which would limit the response to the attribute value instance having the desired value ID as metadata
- or use the convenience function making a GET on {ngsi10path}/contextEntities/myStreet/attributes/co2level/id1
A query not specifying the ID would, in contrast, return two attribute values.
In FIWARE NGSI 2 we can discuss whether the possibility to query a specific value needs to be included in the convenience function as a sub-resource of resources representing attributes – maybe this not necessary, and specific value instances can still be queried by using the corresponding restriction [restrictions are another question to discuss].
However, conceptually the NGSI system should distinguish between several value instances, so if two attribute values with different ID metadata exist, then e.g. an update on one instance should not overwrite the other.
[JMCF] I’m sympathetic to what you are proposing but I was wondering if what you are pursuing might result in API over engineering just to support a use case that can be seamlessly handled by the application itself. For instance, you can have a CO2 level value which can be obtained from the contribution of several measurements coming from different sensors . You might even automatically update the resulting value when a new measurement arrives by using the CEP. For instance.
{
id: “Friedrichstrasse”,
type: “Street”,
co2Level: 0.1
}
{
id: “Sensor-789”,
type: “CO2Sensor”,
location: “ Friedrichstrasse”,
measurement: 0.15
}
{
id: “Sensor-900”,
type: “CO2Sensor”,
location: “ Friedrichstrasse”,
measurement: 0..09
}
[TJ]: In your example you refer to entities representing the different sensors, and of course such a representation is always possible.
In the IoT chapter we typically speak of “higher-level” entities representing more abstract Things like streets, and we expect that the IoT user is not aware of which are the names of the sensors who deliver the information., so the user would query for “street” instead of “Sensor-900”. And it is in such scenarios where the same attribute is potentially delivered by several providers.
[FGM] In an scenario in which you combine low level and high level entities, you can always ask to low level entities for the particular values is you need. Moreover, the advantage of having the high level entity is to hide the details of the low level entities (i.e. abstraction), thus is you consider the posibility to ask for low level entities through the high level entity (using IDs or other means) you are breaking that abstraction.
[TJ] Hmm…my point was that when someone asks for an attribute of a more abstract entity, there could be several providers of that attribute, so there could be several valid instances of that value, and all of them should be returned, not only one of them. And the way to distinguish between the value instances would then be through the ID.
[FGM] The problem of using metadata for that is that with the "flat" JSON rendering you cannot have duplicated key, e.g. the following example is not valid JSON:
{
"id": "Friedrichstrasse",
"type": "Street",
...
"temp" {
"value": 22.2,
"id": "sensor4"
},
"temp" {
"value": 23.3,
"id": "sensor5"
}
...
}
An alternative would be to use compound attribute values for this, in this way (at the same time, note that is more compact than the previous one):
{
"id": "Friedrichstrasse",
"type": "Street",
...
"temp" {
"sensor4": 22.2,
"sensor5": 23,3
...
}
However, I'm not sure if this behaviours should be standarized in NGSIv2, as it seems to be specific of a given use case / application domain.
[TJ] OK, I see where the problem with „flat“ JSON rendering is. So it might be better to make the “canonical representation” the standard one, and declare the “flat” one as syntactic sugar.
[FGM] The canonical representation doesn't solve the problem, given that
{
"id": "Friedrichstrasse",
"type": "Street",
"attrs": {
"temp": {
"value": 22.2,
"type": null,
"metadata": {
"id": "sensor4"
}
},
"temp": {
"value": 23.3,
"type": null,
"metadata": {
"id": "sensor5"
}
}
...
}
}
also have the problem of duplicated keys (this time in the "attrs" object).
6: --------------------------------:6
[TJ] The NGSI v2 proposal is very rich and there will be many different things to discuss or clarify. I am not sure what is the best “format” for the discussion, but let me start in this email with a few points (most of them questions). We are not done yet analyzing the complete proposal and more discussions will follow.
[FGM] Email discussions are perfectly ok. Other alternative ways to provide feedback would be issues in github.com and even pull request suggesting modification to the specification or cookbook .apib files (.apib files at https://github.com/telefonicaid/fiware-orion/tree/develop/doc/apiary/v2 are the "source code" for Apiary web at http://telefonicaid.github.io/fiware-orion/api/v2/ and http://telefonicaid.github.io/fiware-orion/api/v2/cookbook)
(If you are not familiar with the pull request way of working in github.com, please tell me and we try to provide an introduction).
[TJ] OK; I think I prefer at the moment the email format for the basic discussions, and when it comes to the subtle details we can consider moving to pull requests. [Personally I am also not so well aware yet on how this works, but I have colleagues who can help me :)]
7: -----------------------------------: 7
1) “flat” JSON message:
In general NEC agrees to reserve “id”, “type”, “value” to have special semantics. Even if this means that entities can have no attributes called “id”, “type” or “value”
Alternatively (and for maybe making the descriptions easier to understand) we might think about calling “id”, “type”, “value” special attributes with special semantics; some being mandatory (e.g. “id” is mandatory in entities).
8:-----------------------------------: 8
2) Predefined value types: in general NEC supports to have predefined types in order to improve interoperability
9: -------------------------------------------------------------------: 9
3) JSON types as predefined value types: We understand that a JSON property whose value is not a JSON object (and thus has no explicitly stated type) is implicitly either of type “number”, “string” or “boolean”, depending on the type of the JSON property. So for example "temperature": 32.5 would be implicitly of type “number”.
[FGM] Yes, that's the idea. I would recommend you to have a new look to the specification, as we have merged some changes this morning that (we hope) make this idea clearer.
(The diff of the newest changes is at https://github.com/telefonicaid/fiware-orion/pull/1042/files in the case you prefer to see only the modifications)
[TJ] It seems in the newest version that you removed the “implicit” types number, boolean and string, so that every attribute value now needs to be a JSON object. Is this correct?
[FGM] Not exactly... Every attribute needs to be a JSON type, which can be one of the following ones: string, number, boolean, object or array (i.e. not necessarily an "object", which is only one among 5 possibilities).
What I meant was that now an entity looking like
{
id: myStreet
co2Level: “aLot”
}
is not allowed, but needs to be something like
{
id: myStreet
co2Level: {
value: ”aLot”
type: “fun-string”
}
}
So the actual value is of JSON type string, but needs to be nested inside a JSON object where also the value type is declared.
At least this is how I interpret the “JSON entity representation” description in the ngsi2 draft, but maybe I still get it wrong?
[JMCF]
{
id: myStreet
co2Level: “aLot”
}
In this variant there is no NGSI type informed. In the variant which include type you are informing a NGSI type but from the point of view of the implementation they are both strings.
[TJ] OK, then my question would probably be whether an NGSI value type is mandated for attribute values. From reading the current specs my guess would be “yes”, but I would like to have it confirmed…
[FGM] NGSI type for attributes is not mandatory in NGSIv2 (in fact, it is not mandatory in OMA NGSI, see Section 5.5.3), for example the fragment above from JMCF.
[TJ] I think we can close this point because the next one is equivalent…
10: ----------------------------------- :10
4) Further predefined value types: We also understand that there will be further predefined types like for example “geo:point”. But we believe that attribute values having this kind of type will still need to be a JSON object with explicitly stated “type” property, because otherwise e.g. one cannot distinguish a “date” value from a “string” value. This would mean that the example "timestamp": "2017-06-17T07:21:24.238Z" is incorrect and would instead have to look like
“timestamp”:{
“value”: "2017-06-17T07:21:24.238Z"
“type”: date
}
[FGM] In the new version of the spec, there are only two special NGSI types ("date" and "geo:point"), both associated to the JSON string type. I think it works in the way you suggest here, but a double checking the new version of the spec to get your feedback would be great.
[TJ] Yes, in the new versions the values are objects with explicitly given types.
[FGM] Actually, values are JSON valid types, among string, number, boolean, object and array. For JSON strings, the attribute type (from a NGSI point of view) "date" and "geo:point" would have the special meaning described in the specification.
[TJ] see the example previous discussion point. What I meant is that now each attribute value is nested inside a JSON object…
[FGM] Not exaclty... actually, there are two possibilities. Citing from current NGSIv2 spec draft: "Properties representing attributes with no metadata or type are rendered as regular JSON values [i.e. not nested] Properties representing attributes which specify a type or include metadata are rendered as JSON objects [i.e. netsted]".
[TJ] OK, I understand the intention now, thanks.
A potential problem pointed out by Marc in the F2F meeting last week was how to distinguish between an
- attribute without NGSI type, whose value is a JSON object, and
- attribute with NGSI type, which is represented in a JSON object
For example, in
{
id: myStreet
co2Level: {
value: ”aLot”
type: “fun-string”
}
}
The attribute “co2Level” can be interpreted as
- an attribute with NGSI type “fun-string” and value “a lot”
- an attribute without NGSI type and value
{value: ”aLot”, type: “fun-string” }.
--> OK, I just see that there is a discussion point about this below…
11: -----------------------------------: 11
5) Further predefined value types: We assume that some predefined value types can refer to specific JSON object formats, not only to specific string formats.
[FGM] It could be so from a theoretical point of view. However, by the moment, the only two special types we have found make sense are "date" and "geo:point", both associated to JSON string. Maybe you have some other case in mind?
[TJ] I do not have a specific example in my mind at the moment, so it is indeed more a theoretical consideration. I imagine that for the sake of interoperability specific domains could publish something like a catalogue of their entity types and the format of the values they are using.
[FGM] To be precise, I understand you mean "a catalogue of their entity types and the format of the values of the entity attributes they are using"
[TJ] Yes
[TJ] Probably we should first discuss what other predefined value types we want to have (if any), and only then discuss whether some of them should rather be JSON objects than specific string formats.
[FGM] By the moment we have found "date" and "geo:point" and both fit into string JSON type, i.e. we haven't found the need to use a JSON obejct for them. Maybe some more examples arise as we progress in the NGSIv2 specification, but we don't foresee any new one at the current point of time.
12: -----------------------------------:12
6) User-defined types: It is currently written that “The values of user-defined informative types are represented by JSON string”, but we believe that user-defined types should be representable by numbers, Booleans and JSON objects as well.
[FGM] This has changed in the new version of the spec and I guess now works in the way you suggest. Now, NGSI type and JSON type are "orthogonal" aspects on the attribute.
[TJ] OK, I read that even arrays would be allowed. Makes sense to me.
13: ----------------------------------- :13
7) Error format: Currently an error code is not specified. Is this because it is intended to use the status code in the HTTP header instead? Can this lead to conflicts between standard HTTP error codes (e.g. 404 resource not found) and NGSI error codes (eg. 404 context element not found)?
[FGM] Exactly. The idea is to use HTTP status code for the numeric error code, so no conflict should occur. Note that one of the criticized aspects in NGSI v1 is not using HTTP code to convey the error code, as this is not the common way of working in RESTful APIs. E.g. this would be a "not found" error response in NGSIv2:
HTTP/1.1 404 Not Found
Content-Length: 95
Content-Type: application/json
Date: ...
{
"error": "NotFound",
"description": "The requested entity has not been found. Check type and id"
}
[TJ] I need to further discuss with my colleagues about this before agreeing or proposing an alternative.
One concern is that not having the status code in the response message forces applications to also look into the response header, which is not always convenient.
[FGM] (I understand that your refer to the status line, i.e. "HTTP/1.1 404 Not Found", not to response headers, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html)
On the other side, one of the concerns (that I have personally listened several times when presenting Orion in FIWARE events) is exactly the opposite :), i.e. that with the current way of working in NGSI you always need to check the payload to know the error. Even worse, depending on the request type (queryContext, updateContext, etc.) the error may be in different places on the payload, which is a little bit crazy from the point of view of the client programmer.
Anyway, the above approach should make both people happy, I mean:
• For people not wanting to check the status code, they can access to the payload and check the "error" message.
• For people not wanting to check the payload, they can get the status code in the status line.
[JMCF] Could you give us a real programming example on which getting access to response headers is not convenient? I think you must always check for response status if you are aiming at creating a robust application that deals with any kind of unexpected error which might happen (500, etc.)
14: ------------------------------------------------------------ :14
8) API entry point
In general NEC agrees to make this resource somehow an overview of the main API urls. Would you make the format if this link list predefined, or would you leave the specific format to the API developer?
[FGM] I think Jose Manuel could answer this better than me.
[JMCF] The API entry point must list at least all the URLs that allow to get access to the core functionalities offered by the API i.e.
Entities, types, subscriptions and registrations . In theory the only mandatory part would be the keys of the returned JSON object i.e.
entities_url, types_url, etc. As values yo could return your own URLs, but in practice it is very likely that you will be returning the same as in the example provided. I think this should be clarified in the spec.
15: ----------------------------------- :15
10) Pagination: In principle NEC supports the concept of pagination. However we are not sure how it can be realized in practice, as query results are dynamic (especially when getting the queried data live from a distributed system like IoT systems do). What the user probably expects is that the query results are “frozen” for him and then delivered page by page. But then something like a query ID is needed to collect the query results.
Alternatively the query result could be represented by a specific http resource (which is similar to a query id).
In both cases things become slightly more complex…
Do you have a proposal how to deal with that issue?
[FGM] Frozen the query at NGSI server implementation could have an important performance impact (as the frozen results would need to be maintained in memory from an undefined amount of time while the client gets all the result) and would make NGSI sever implementation stateful, which is very bad from the point of view of HA and horizontal scaling.
Some ordering criteria may alleviate the problem In particular, entity creation date, which ensures that any new entity have to be put at the end. However, even in this case, there could be some problems if an entity already seen page is deleted (as it would introduce a shift in the next pages).
Thus, the solution based on limit/offset is not perfect, but may suffice most of the cases and have a good simplicity tradeoff. So, it is probably "the best of the worse" solution. But this point is still opened.
[TJ] I understand the scalability aspect.
On the other hand, NGSI systems who do not have the information ready in a database but need to assemble it from various sources anyway somehow have to cache the query result when it is retrieved page by page, as otherwise they would have to run the same queries on the subsystem again and again.
>From this point of view the question is whether clients should have separate control on data generation and on data retrieval, or if this difference should be transparent to applications. To be discussed.
[FGM] I agree that pagination in a distributed context (as the architecture based on CB + CPrs is) is hard. We have been discussing about it at https://github.com/telefonicaid/fiware-orion/issues/847 long time, but without having found a perfect solution.
16: -----------------------:16
[MC] Query expressions of the "q" parameter only applies on attributes and not metadata. We may need a syntax to tell the matching should occur on metadata of a given attribute like "temperature/unit" to match the unit metadata field of a temperature attribute.
[TJ] See the discussion point above, where I have proposed to use JSONPATH. [Maybe it is better to answer there.]
17: ------------------------:17
[MC] There would be 3 formats for attributes, as simple JSON values, as JSON object (for type and metadata) and canonical. Maybe only one could be standardized, others would be implementation specific (only supported in Orion CB for third party clients) ?
[FGM] I think the different possibilities should be standardized to ensure interoperability. However, they could correspond to different "compliance levels" in the standard, e.g. one implementation may support only the basic format (whatever we define as "basic") and other implementation (more advanced) may support all them. Although the drawbacks of having different compliance levels should also be evaluated (it is easier to achieve compatibility when only one level exists).
[TJ] Is there a good reason for the canonical representation? (If there is I will probably support to define this as the basic one; if not then I suggest to drop it).
[FGM] The point of having a canonical represenstation is that it makes easier to implement generic NGSIv2 clients. The point of having a "compact" (i.e. not canonical representation) is that is makes easier to implement specific clients.
For example, for an application that deals with attribute temperature as a number without any extra information (a case of specific client) it is easier:
{
"id": "myEnt1",
"temperature": 32
}
than:
{
"id": "myEnt1",
"type": null,
"temperature": {
"value": 32,
"type": null,
"metadata": [ ]
}
}
but a generic client (e.g. a context browser based implemented as a web portal) needs to take into account all the posisibilities (e.g. that the attribute have a NGSI type, metadata, etc.). Thus, representing the information in a regular syntax (i.e. the canonical mode) is easier for it.
[TJ] The discussion above about multiple values of the same attribute makes me now think that it is actually very useful to have a canonical representation. I would even recommend to make this the “standard” one, so that NGSI-compliant enablers have at least to implement the canonical representation.
18: --------------- :18
[MC] For registrations/subscriptions, how do we define if we want to match a complete id or an idpattern in the subject/entity field ?
[FGM] We would use "id" for exact ID and "idPattern" for patterns.
[TJ] Naughty question: what happens when both id and idPattern are present?
[FGM] In that case a bad syntax error would be returned.
[Unfortunately, as far as I know, at JSON Schema Language level one cannot express that exactly one of the two have to be present].
[MC] Fermin, understood, spec could have a small correction like "* `entity` : Id or pattern ididPattern which identifies the affected entities (optional)"
Tobias, JSON Schema (draft v4) provides a "oneOf" keyword to handle such case.
[FGM] Correction is on the way... we are a bit delayed merging pull requests into the main spec.
[TJ] OK, good to know there is such a keyword, that would have been useful earlier in the XML specs. Fine from my side.
19: ------------------ :19
[MC] Orion CB supports Service/ServicePath HTTP headers. Should they be defined in the spec ? Is it a requirement for other GE to implement for interoperability ?
[FGM] Fiware-Service HTTP header is a way of implementing multitenancy, i.e. having completely isolated instances of the API at the same API endpoint (i.e. without needing extra ports/URLs). In this sense, Fiware-Service is quite "orthogonal" to the API. Do you really think we should include it in the spec?
Regarding ServicePath, we think that the idea of grouping entities is a functional aspect that should be part of the specification. It will be added to the apiary document in the future, although we would try to find a better name than "ServicePath", such as "group", "domain", "collection", etc. Any suggestion regarding the name is welcome.
[TJ] Actually the NGSI-way to define things like that would be the definition of a specific scope type. Standardizing such a “domain” scope might be a good idea, as many NGSI based systems could use such a feature. At least it makes more sense to me than the previous “attribute domain” concept.
20: ------------------ :20
[MC] Mixing of structured attributes values could be a problematic to handle (already discussed on the issue tracker: https://github.com/telefonicaid/fiware-orion/issues/1040).
[FGM] At attribute creation/update time, an heuristic can be used to solve the ambiguity, e.g. if the value of the JSON property representing the attribute is a JSON object having the key "value", then the value of the attribute is the value of the "value" property, otherwise the attribute value is the value of the JSON property representing the attribute. At attribute retrieving time, there are some situations which cannot be distinguished but we think they are corner cases and shouldn't be a problem as long as the way it works would be explained in the API specification document.
[TJ] OK, I think for the API user this is indeed only a corner case; only for the implementer of the API it is some limited extra effort to check what kind of attribute representation is used.
21: --------------------- :21
[MC] Shoud registration/subscription support the DELETE verb, or are we still relying on duration set to 0 ?
[FGM] Setting duration to 0 to cancel a registration is a very ugly workaround, so I think we should use DELETE in this case. Regarding subscriptions, there are the"unsubscribeContext" and "unsubscribeContextAvailability" operations in the OMA NGSI standard operations set but probably they are examples of standard operations that we don't need as they are not adding any significant functionality compared with the non-standard counterparts.
[TJ] Something that was not there in OMA NGSI was the possibility to cancel a registration. I would agree that such an operation is useful, and I agree that registrations are something that can well be represented by the REST model and thus from functionality viewpoint do not need a “standard operation”.
[Unless we decide that everything should be covered by standard operations for consistency]
22: ----------------------- :22
[MC] Does registration (like subscription) support multiple callbacks (providingApplication) for the same subject ? Would it allow fanout for query/updates of (IoT Agent) lazy/commands attributes ?
[FGM] The providingApplication is a kind of "primary key" for the registration, so multiple callbacks in the same registration are not supported. However, you could create two registrations using exactly the same entities/attributes with different providing applications to achieve the same case.
Could you elaborate on what you mean by fanout for query/updates of (IoT Agent) lazy/commands attributes, please?
[MC] When a third app makes update or query request on the Orion CB, it can forward the request to the providing application (like an IoT Agent) that previously registered the entities/attributes.
If multiple IoT Agents can register the same Entities/Attributes (with each a diffrent providingApplication url) then Orion CB has to update/query each of these IoT Agents. For one incoming request, multiple outgoing requests would be done by the Orion CB (this is what is was calling fanout).
Lazy/commands are just are special type of attributes IoT Agents expect the third party apps to either update (commands) or query (lazy attributes). nicely explained here : https://github.com/telefonicaid/iotagent-node-lib/blob/master/README.md
[FGM] Thanks for clarification. Orion support this kind of fanout, e.g. if there are two providers for the Ent1-Attr1, a query on Ent1-Attr1 is forwarded to both. However, current implementation only takes one of the values, making sense for backup scenarios when both providers are acting one the backup of the other. In the future this may evolve (e.g. to get the newer value, to aggregate them, etc.) but I think this is an implementation aspect not fully related with NGSIv2 specification.
23: ----------------------------- :23
[MC] Support for JSON-LD was discussed during the F2F meeting : JSON-LD @id, @type could simply be stored as normal attributes. But an interesting solution is proposed in the tracker (https://github.com/telefonicaid/fiware-orion/issues/1053)
24: -----------------------------: 24
[MC] The specification is missing some NGSI definitions like Service Description and Information Model part of the v1 specification.
[FGM] Not sure to which definitions do you refer... Could you elaborate or provide a link to them, please?
[MC] The NGSI v1 introduced abstract concepts like Entities, Attributes and Metadata, but also application roles like Context Providers, Context Managers in the "Service Description" and "Information Model" sections of the OMA document<http://technical.openmobilealliance.org/Technical/release_program/docs/NGSI/V1_0-20120529-A/OMA-TS-NGSI_Context_Management-V1_0-20120529-A.pdf>.
[TJ] We started to write at least some background information on the data model of entities/attributes, see the pull request at https://github.com/telefonicaid/fiware-orion/pull/1078
But much stuff is still missing.
25:------------------------------: 25
[TJ] Now this is the discussion about whether we need to separate between context elements and context entities. See the discussion at https://github.com/telefonicaid/fiware-orion/pull/1078
The separation would be that the context entity represents a real-world object, and so each context entity is unique. A context element is a container for information about a context entity, and so there can be multiple such context elements for one entity.
For achieving a greater simplicity one could alternatively remove the distinction and call both just context entity.
While this simplifies terminology (one term less), it makes it more difficult to understand that the data object called “context entity” in general only contains partial information about the entity.
[FGM] Maybe we can leave this as it is now in http://telefonicaid.github.io/fiware-orion/api/v2/ (with the disclaimer in "Context Elements" section) and take the decision at the end, once the spec gets mature and we can see if "context element" term is needed or not.
26:------------------------------: 26
[TJ] NEC proposes to make in the “contextRegistration” structure the “providingApplication” URL optional.
The reason is that in some cases it makes sense to register entities without providing NGSI servers where they can be retrieved:
- for announcing that in principle a system can provide information about that entity, even if at the moment no provider is available.
- for registering associations which describe how to compute entity information from another other entities (sensor2things concept). Up to now we have used a dummy URL in that case as a workaround…
27:------------------------------: 27
[TJ] NEC proposes a new optional field in the contextRegistration structure. This field is called “sourceInformation”. Its value describes how the attributes of the entities registered can be obtained by other means than retrieving them from an NGSI host.
More specifically, the value of “sourceInformation” is a data structure called “SourceInformation”. This structure has a field “sourceType” and a field “sourceValue”.
The idea is that, similarly to NGSI scopes, source types can be user-defined, but the NGSI specs would already contain one or more basic types of sources, one of them being the associations (entity-to-entity mappings).
By this, we achieve that
- Associations and more advanced entity computation rules can be registered
- Also non-NGSI data sources can be registered, together with specific information on how the data should be translated into NGSI
- the IoT Discovery (aka ConfMan) component does not have to understand the specific source types, as it still simply returns the associations whose entities and attributes comply with the request
- Brokers who do not understand the registered source types can simply ignore registrations that do not mention a providingApplication. Or maybe the understand a subset of the source types but ignore all others.
- IoT Agents can also read this data from IoT Discovery and get information from it on how they should translate the incoming protocol to NGSI. In other words, the translation rules can be dynamically changed instead of having to statically configure them.
[FGM] Could you provide an usage example of that "sourceInformation" element based in the current registration JSON (see "Context Provider Registration" - "List registraitons" in http://telefonicaid.github.io/fiware-orion/api/v2/) in order to fiigure out how would it be, please?
________________________________
Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.
The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.
Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-ngsi/attachments/20150722/5177ed2d/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy