Hi Aitor,
I have few question on this extension. How is it different from the
"standard" DataStore /{CKAN-URL}/datastore/dump/{RESOURCE-ID}/ API,
which is described as/"A DataStore resource can be downloaded in the CSV
file format"/? I see you implemented different format but, expect this
aspect, does your extension relies on this to produce the CSV resource
representation?
Lastly, I would ask you how it is possible to know if a resource in
available as CSV download or not. Currently, we use the standard CKAN
API to list all the available dataset, filtering the related resources
depending on their format (CSV, XLS, etc). This help us to show to the
user only the type of resources that we can actually handle in SpagoBI.
Having to care only about the CSV format it's a big plus for us, but I
don't think you can convert any resource as CSV, so how to know which
one is actually available through your extension?
Thanks,
A.
Il 07/07/2015 11:14, Aitor Magán ha scritto:
> No... The extension we are using to generate the CSVs is not running
> on FILAB... I'm sorry. But you can use the example that is running on
> http://130.206.126.145/. As stated in the documentation, you can
> access the resources in CSV by accessing this URL:
>
> GET http://130.206.126.145/resource/[RESOURCE_ID]/entry <about:blank>
>
> Accept: text/csv
>
>
> For example, you can access resource "Load 07/07/2015" in this URL:
> http://130.206.126.145/resource/827fc9f3-098c-4a13-89cd-ef68f2ffbc56/entry.
>
> I'll update the entities with the location info today.
>
> BR
> Aitor
>
> 2015-07-07 11:05 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it
> <mailto:davide.zerbetto at eng.it>>:
>
> Hi all
> we updated the entities with actual location info.
> Regarding the DataStore, is there any example in the data portal
> on FILab we can use in order to make some tests?
> Thanks and best regards
> Davide
>
>
> Il 03/07/2015 09:15, Aitor Magán ha scritto:
>> Ok! I'll try to implement your approach! Can you update the
>> documentation? The documentation to retrieve data from CKAN
>> DataStore in CSV is here:
>> https://github.com/conwetlab/ckanext-datastore_restful/wiki/API.
>> If you have any doubt, please, don't hesitate to contact me :)
>>
>> Have a nice day!
>>
>> BR
>> Aitor
>>
>> 2015-07-02 18:12 GMT+02:00 Davide Zerbetto
>> <davide.zerbetto at eng.it <mailto:davide.zerbetto at eng.it>>:
>>
>> Hi Aitor
>> I'm sorry for my late reply, but I was very busy these days.
>> We changed the names of the entities, removing namespaces,
>> that's way we removed the entities, sorry for the inconvenience.
>> regarding how to store the data into CKAN, see my comments below.
>> Thanks and best regards
>> Davide
>>
>>
>> Il 02/07/2015 17:39, Aitor Magán ha scritto:
>>> Hi Davide, Hi Monica, Hi All,
>>>
>>> @Davide: I've created the component to retrieve the data
>>> from the Context Broker and push it to CKAN. I've used the
>>> approach that I told you yesterday since I've no news from
>>> you. However, I cannot test the service since all the
>>> entities have been removed from the CB instance. Why?
>>>
>>> @Monica: Have you asked Manuel how demo tickets should be
>>> created? (pattern,...)
>>>
>>> BR
>>> Aitor
>>>
>>> 2015-07-01 15:53 GMT+02:00 Aitor Magán <amagan at conwet.com
>>> <mailto:amagan at conwet.com>>:
>>>
>>> Hi Davide, Hi all,
>>>
>>> I'm not pretty sure, but I think that the information
>>> about the prosumers is static (it does not changes) so I
>>> don't understand why this information has to be stored
>>> in the Context Broker. I think that the best place to
>>> store static information is CKAN.
>>>
>>
>> I think it's better to mantain those entities in the CB: they
>> also may change in some attributes. Is it a big issue to
>> leave them in the CB?
>>
>>>
>>> I don't know if I've understood you, but you want two
>>> different datastes a day: one for meters and another one
>>> for loads.
>>>
>>
>> yes
>>
>>> Should every row of both dataset contains prosumer
>>> information? I think this is a waste of space since the
>>> information about the prosumers will be replicated a lot
>>> of times.
>>>
>>
>> The datasets will be more self-explaining when containing
>> prosumers data. With ad-hoc reporting capabilities (within
>> SpagoBI but also with CKAN I think), users will be able to
>> display values grouping by prosumers attributes (city, type,
>> ...) in a very simple way if the dataset contains that
>> information; otherwise, the user has to join the dataset to
>> another one (that's possible but more complicated).
>> We have to clarify a difference between "dataset" term in
>> SpagoBI and CKAN:
>> 1. in SpagoBI a dataset is 1 file or 1 SQL statement or 1
>> script.....
>> 2. in CKAN a dataset is a container for 1 or more resources.
>> When CKAN datasets are imported into SpagoBI, a SpagoBI
>> dataset is defined for each resource available within the
>> CKAN dataset.
>>
>>>
>>> In my opinion, the best approach is the following one:
>>> one dataset for each prosumer. Everyday, the system will
>>> attach two different resources to these datasets: one
>>> for meters and one for loads. The prosumer information
>>> can be stored in the dataset metadata. What do you think?
>>>
>>
>> I don't like this approach very much: according to me, it's
>> better to put information about all prosumers in the same
>> file: in this way, final users (with ad-hoc reporting) will
>> be able both to aggregate on all prosumers and to filter data
>> for a single prosumer quite easily. If the users wants to
>> aggregate values for several prosumers, it is quite difficult
>> if values are stored in different datasets.
>> What do you think?
>> According to me we don't have to worry about space occupation
>> (at least not in this moment) or to think about optimization,
>> we have to show how components work and interoperate.
>>
>>>
>>> P.S. I've installed the extension that allows you to
>>> download information stored in the datastore as CSV. :)
>>>
>>
>> Good! Can you please provide us some documentation in order
>> to understand how to use it?
>>
>>
>>>
>>> BR
>>> Aitor
>>>
>>> 2015-07-01 13:46 GMT+02:00 Davide Zerbetto
>>> <davide.zerbetto at eng.it <mailto:davide.zerbetto at eng.it>>:
>>>
>>> Hi all
>>> we updated the CB entities as we discussed yesterday
>>> and their documentation on the technical document.
>>> @Aitor: your component should subscribe for Meter
>>> and Load entities creation and update, then data
>>> should be collected in flat datasets: I propose 1
>>> dataset for meters and 1 dataset for loads,
>>> collecting data for 1 day. The best would be CSV
>>> files containing information about the prosumer and
>>> the meter/load: each line of the csv should contain
>>> prosumer info plus meter/load.
>>> Regarding names of the columns: we used namespaces,
>>> but we are thinking about removing them. Is it
>>> possible for you to define more simple CSV columns
>>> names? for example:
>>> BusinessDemo:Prosumer:address:name --> Prosumer Name
>>> BusinessDemo:Prosumer:address:city --> City
>>> BusinessDemo:Prosumer:address:country --> Country
>>> BusinessDemo:Prosumer:address:number--> Address number
>>> and so on.
>>> Thanks and best regards
>>> Davide
>>>
>>>
>>> Il 29/06/2015 10:47, Aitor Magán ha scritto:
>>>> I need to know the entities that you are using in
>>>> the Context Broker and the structure of the CKAN
>>>> datasets that you expect.
>>>>
>>>> BR
>>>> Aitor
>>>>
>>>> 2015-06-29 10:38 GMT+02:00 Davide Zerbetto
>>>> <davide.zerbetto at eng.it
>>>> <mailto:davide.zerbetto at eng.it>>:
>>>>
>>>> Hi Aitor
>>>> referring to the work plan
>>>> <https://docs.google.com/document/d/1I9FoZ8KkzXOXNPvrWOklzU5ajboQ1hSEZevfGomEWas/edit>,
>>>> do you have enough information to start with
>>>> task T.2.2? I don't know if the information we
>>>> provided on the technical document if enough
>>>> for you...
>>>> Thanks and best regards
>>>> Davide
>>>>
>>>> Il 29/06/2015 10:26, Aitor Magán ha scritto:
>>>>> Sorry Davide, I do not understand the
>>>>> question. What information do I need to start
>>>>> collecting data?
>>>>>
>>>>> BR
>>>>> Aitor
>>>>>
>>>>> 2015-06-29 10:23 GMT+02:00 Davide Zerbetto
>>>>> <davide.zerbetto at eng.it
>>>>> <mailto:davide.zerbetto at eng.it>>:
>>>>>
>>>>> Hi all
>>>>> we installed the Prosumers Simulator (the
>>>>> mock component that generates anonymous
>>>>> data) on the same VM hosting the CB.
>>>>> Now it is running. I updated the technical
>>>>> document also.
>>>>> Of course if you have comments, don't
>>>>> hesitate to ask.
>>>>> @Aitor: do you have enough information to
>>>>> start collecting data into CKAN datasets?
>>>>> Thanks and best regards
>>>>> Davide
>>>>> --
>>>>>
>>>>> --
>>>>>
>>>>> *Davide Zerbetto*
>>>>> SpagoBI Consultant
>>>>>
>>>>> *SpagoBI Labs
>>>>> Engineering Group*
>>>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>>>> Phone: +39 049 8283411
>>>>> <tel:%2B39%20049%208283411>
>>>>> Fax: +39-049.8700272 <tel:%2B39-049.8700272>
>>>>> www.spagobi.org <http://www.spagobi.org> -
>>>>> www.eng.it
>>>>> <http://www.eng.it/web/eng_en/home>
>>>>>
>>>>>
>>>>> Respect the environment. Please don't
>>>>> print this e-mail unless you really need to.
>>>>>
>>>>> The information transmitted is intended
>>>>> only for the person or entity to which it
>>>>> is addressed and may contain confidential
>>>>> and/or privileged material. Any review,
>>>>> retransmission, dissemination or other use
>>>>> of, or taking of any action in reliance
>>>>> upon, this information by persons or
>>>>> entities other than the intended recipient
>>>>> is prohibited. If you received this in
>>>>> error, please contact the sender and
>>>>> delete the material from any computer.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Fiware-apps mailing list
>>>>> Fiware-apps at lists.fi-ware.org
>>>>> <mailto:Fiware-apps at lists.fi-ware.org>
>>>>> https://lists.fi-ware.org/listinfo/fiware-apps
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>> Aitor Magán
>>>>>
>>>>> Twiter <https://twitter.com/AitorMagan>
>>>>> <https://es.linkedin.com/in/aitormagan>
>>>>> <https://github.com/AitorMagan>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Fiware-apps mailing list
>>>>> Fiware-apps at lists.fi-ware.org <mailto:Fiware-apps at lists.fi-ware.org>
>>>>> https://lists.fi-ware.org/listinfo/fiware-apps
>>>>
>>>> --
>>>>
>>>> --
>>>>
>>>> *Davide Zerbetto*
>>>> SpagoBI Consultant
>>>>
>>>> *SpagoBI Labs
>>>> Engineering Group*
>>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>>> Phone: +39 049 8283411 <tel:%2B39%20049%208283411>
>>>> Fax: +39-049.8700272 <tel:%2B39-049.8700272>
>>>> www.spagobi.org <http://www.spagobi.org> -
>>>> www.eng.it <http://www.eng.it/web/eng_en/home>
>>>>
>>>>
>>>> Respect the environment. Please don't print
>>>> this e-mail unless you really need to.
>>>>
>>>> The information transmitted is intended only
>>>> for the person or entity to which it is
>>>> addressed and may contain confidential and/or
>>>> privileged material. Any review,
>>>> retransmission, dissemination or other use of,
>>>> or taking of any action in reliance upon, this
>>>> information by persons or entities other than
>>>> the intended recipient is prohibited. If you
>>>> received this in error, please contact the
>>>> sender and delete the material from any computer.
>>>>
>>>>
>>>> _______________________________________________
>>>> Fiware-apps mailing list
>>>> Fiware-apps at lists.fi-ware.org
>>>> <mailto:Fiware-apps at lists.fi-ware.org>
>>>> https://lists.fi-ware.org/listinfo/fiware-apps
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>> Aitor Magán
>>>>
>>>> Twiter <https://twitter.com/AitorMagan>
>>>> <https://es.linkedin.com/in/aitormagan>
>>>> <https://github.com/AitorMagan>
>>>
>>> --
>>>
>>> --
>>>
>>> *Davide Zerbetto*
>>> SpagoBI Consultant
>>>
>>> *SpagoBI Labs
>>> Engineering Group*
>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>> Phone: +39 049 8283411 <tel:%2B39%20049%208283411>
>>> Fax: +39-049.8700272 <tel:%2B39-049.8700272>
>>> www.spagobi.org <http://www.spagobi.org> -
>>> www.eng.it <http://www.eng.it/web/eng_en/home>
>>>
>>>
>>> Respect the environment. Please don't print this
>>> e-mail unless you really need to.
>>>
>>> The information transmitted is intended only for the
>>> person or entity to which it is addressed and may
>>> contain confidential and/or privileged material. Any
>>> review, retransmission, dissemination or other use
>>> of, or taking of any action in reliance upon, this
>>> information by persons or entities other than the
>>> intended recipient is prohibited. If you received
>>> this in error, please contact the sender and delete
>>> the material from any computer.
>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Aitor Magán
>>>
>>> Twiter <https://twitter.com/AitorMagan>
>>> <https://es.linkedin.com/in/aitormagan>
>>> <https://github.com/AitorMagan>
>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Aitor Magán
>>>
>>> Twiter <https://twitter.com/AitorMagan>
>>> <https://es.linkedin.com/in/aitormagan>
>>> <https://github.com/AitorMagan>
>>
>> --
>>
>> --
>>
>> *Davide Zerbetto*
>> SpagoBI Consultant
>>
>> *SpagoBI Labs
>> Engineering Group*
>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>> Phone: +39 049 8283411 <tel:%2B39%20049%208283411>
>> Fax: +39-049.8700272 <tel:%2B39-049.8700272>
>> www.spagobi.org <http://www.spagobi.org> - www.eng.it
>> <http://www.eng.it/web/eng_en/home>
>>
>>
>> Respect the environment. Please don't print this e-mail
>> unless you really need to.
>>
>> The information transmitted is intended only for the person
>> or entity to which it is addressed and may contain
>> confidential and/or privileged material. Any review,
>> retransmission, dissemination or other use of, or taking of
>> any action in reliance upon, this information by persons or
>> entities other than the intended recipient is prohibited. If
>> you received this in error, please contact the sender and
>> delete the material from any computer.
>>
>>
>>
>>
>> --
>> Best Regards
>> Aitor Magán
>>
>> Twiter <https://twitter.com/AitorMagan>
>> <https://es.linkedin.com/in/aitormagan>
>> <https://github.com/AitorMagan>
>>
>>
>> _______________________________________________
>> Fiware-apps mailing list
>> Fiware-apps at lists.fi-ware.org <mailto:Fiware-apps at lists.fi-ware.org>
>> https://lists.fi-ware.org/listinfo/fiware-apps
>
>
> _______________________________________________
> Fiware-apps mailing list
> Fiware-apps at lists.fi-ware.org <mailto:Fiware-apps at lists.fi-ware.org>
> https://lists.fi-ware.org/listinfo/fiware-apps
>
>
>
>
> --
> Best Regards
> Aitor Magán
>
> Twiter <https://twitter.com/AitorMagan>
> <https://es.linkedin.com/in/aitormagan> <https://github.com/AitorMagan>
>
>
> _______________________________________________
> Fiware-apps mailing list
> Fiware-apps at lists.fi-ware.org
> https://lists.fi-ware.org/listinfo/fiware-apps
--
--
*Alessandro Portosa*
SpagoBI Consultant
*SpagoBI Labs
Engineering Group*
Via Marconi, 10 - 40125 Bologna - Italy
Tel. + 39 051 0435090
Skype: alessandro.portosa
www.spagobi.org <http://www.spagobi.org> - www.eng.it
<http://www.eng.it/web/eng_en/home>
Respect the environment. Please don't print this e-mail unless you
really need to.
The information transmitted is intended only for the person or entity to
which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited. If you
received this in error, please contact the sender and delete the
material from any computer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0002.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0003.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: spagobi.png
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/b40e8736/attachment.jpg>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy