Hi Alessandro,
I didn't know that the DataStore has an option to dump the stored data in
CSV. I think it's better to get the CSV in this way instead of using my
extension :). However, my extension is more powerful that the standard way,
since the standard way gives you the entire dataset while my extension
allows you to set some filters (the fields that you want to retrieve, the
number of rows to return,...)
Related to your last question: I'm not pretty sure. Maybe the best solution
is to access http://{CKAN-URL}/api/action/datastore_search?resource_id={RESOURCE_ID}
and check the answer. If the server returns a 404 response, you can assume
that the DataStore is not active for this resource so users won't be able
to access the CSV.
BR
Aitor
2015-07-07 12:18 GMT+02:00 Alessandro Portosa <alessandro.portosa at eng.it>:
> Hi Aitor,
> I have few question on this extension. How is it different from the
> "standard" DataStore *{CKAN-URL}/datastore/dump/{RESOURCE-ID}* API,
> which is described as* "A DataStore resource can be downloaded in the CSV
> file format"*? I see you implemented different format but, expect this
> aspect, does your extension relies on this to produce the CSV resource
> representation?
>
> Lastly, I would ask you how it is possible to know if a resource in
> available as CSV download or not. Currently, we use the standard CKAN API
> to list all the available dataset, filtering the related resources
> depending on their format (CSV, XLS, etc). This help us to show to the user
> only the type of resources that we can actually handle in SpagoBI. Having
> to care only about the CSV format it's a big plus for us, but I don't think
> you can convert any resource as CSV, so how to know which one is actually
> available through your extension?
>
> Thanks,
> A.
>
>
> Il 07/07/2015 11:14, Aitor Magán ha scritto:
>
> No... The extension we are using to generate the CSVs is not running on
> FILAB... I'm sorry. But you can use the example that is running on
> http://130.206.126.145/. As stated in the documentation, you can access
> the resources in CSV by accessing this URL:
>
> GET http://130.206.126.145/resource/[RESOURCE_ID]/entry
>
> Accept: text/csv
>
>
> For example, you can access resource "Load 07/07/2015" in this URL:
> http://130.206.126.145/resource/827fc9f3-098c-4a13-89cd-ef68f2ffbc56/entry
> .
>
> I'll update the entities with the location info today.
>
> BR
> Aitor
>
> 2015-07-07 11:05 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it>:
>
>> Hi all
>> we updated the entities with actual location info.
>> Regarding the DataStore, is there any example in the data portal on FILab
>> we can use in order to make some tests?
>> Thanks and best regards
>> Davide
>>
>>
>> Il 03/07/2015 09:15, Aitor Magán ha scritto:
>>
>> Ok! I'll try to implement your approach! Can you update the
>> documentation? The documentation to retrieve data from CKAN DataStore in
>> CSV is here:
>> https://github.com/conwetlab/ckanext-datastore_restful/wiki/API. If you
>> have any doubt, please, don't hesitate to contact me :)
>>
>> Have a nice day!
>>
>> BR
>> Aitor
>>
>> 2015-07-02 18:12 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it>:
>>
>>> Hi Aitor
>>> I'm sorry for my late reply, but I was very busy these days.
>>> We changed the names of the entities, removing namespaces, that's way we
>>> removed the entities, sorry for the inconvenience.
>>> regarding how to store the data into CKAN, see my comments below.
>>> Thanks and best regards
>>> Davide
>>>
>>>
>>> Il 02/07/2015 17:39, Aitor Magán ha scritto:
>>>
>>> Hi Davide, Hi Monica, Hi All,
>>>
>>> @Davide: I've created the component to retrieve the data from the
>>> Context Broker and push it to CKAN. I've used the approach that I told you
>>> yesterday since I've no news from you. However, I cannot test the service
>>> since all the entities have been removed from the CB instance. Why?
>>>
>>> @Monica: Have you asked Manuel how demo tickets should be created?
>>> (pattern,...)
>>>
>>> BR
>>> Aitor
>>>
>>> 2015-07-01 15:53 GMT+02:00 Aitor Magán <amagan at conwet.com>:
>>>
>>>> Hi Davide, Hi all,
>>>>
>>>> I'm not pretty sure, but I think that the information about the
>>>> prosumers is static (it does not changes) so I don't understand why this
>>>> information has to be stored in the Context Broker. I think that the best
>>>> place to store static information is CKAN.
>>>>
>>>
>>> I think it's better to mantain those entities in the CB: they also may
>>> change in some attributes. Is it a big issue to leave them in the CB?
>>>
>>>
>>>> I don't know if I've understood you, but you want two different
>>>> datastes a day: one for meters and another one for loads.
>>>>
>>>
>>> yes
>>>
>>> Should every row of both dataset contains prosumer information? I
>>>> think this is a waste of space since the information about the prosumers
>>>> will be replicated a lot of times.
>>>>
>>>
>>> The datasets will be more self-explaining when containing prosumers
>>> data. With ad-hoc reporting capabilities (within SpagoBI but also with CKAN
>>> I think), users will be able to display values grouping by prosumers
>>> attributes (city, type, ...) in a very simple way if the dataset contains
>>> that information; otherwise, the user has to join the dataset to another
>>> one (that's possible but more complicated).
>>> We have to clarify a difference between "dataset" term in SpagoBI and
>>> CKAN:
>>> 1. in SpagoBI a dataset is 1 file or 1 SQL statement or 1 script.....
>>> 2. in CKAN a dataset is a container for 1 or more resources.
>>> When CKAN datasets are imported into SpagoBI, a SpagoBI dataset is
>>> defined for each resource available within the CKAN dataset.
>>>
>>>
>>>> In my opinion, the best approach is the following one: one dataset
>>>> for each prosumer. Everyday, the system will attach two different resources
>>>> to these datasets: one for meters and one for loads. The prosumer
>>>> information can be stored in the dataset metadata. What do you think?
>>>>
>>>
>>> I don't like this approach very much: according to me, it's better to
>>> put information about all prosumers in the same file: in this way, final
>>> users (with ad-hoc reporting) will be able both to aggregate on all
>>> prosumers and to filter data for a single prosumer quite easily. If the
>>> users wants to aggregate values for several prosumers, it is quite
>>> difficult if values are stored in different datasets.
>>> What do you think?
>>> According to me we don't have to worry about space occupation (at least
>>> not in this moment) or to think about optimization, we have to show how
>>> components work and interoperate.
>>>
>>>
>>>> P.S. I've installed the extension that allows you to download
>>>> information stored in the datastore as CSV. :)
>>>>
>>>
>>> Good! Can you please provide us some documentation in order to
>>> understand how to use it?
>>>
>>>
>>>
>>>> BR
>>>> Aitor
>>>>
>>>> 2015-07-01 13:46 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it>:
>>>>
>>>>> Hi all
>>>>> we updated the CB entities as we discussed yesterday and their
>>>>> documentation on the technical document.
>>>>> @Aitor: your component should subscribe for Meter and Load entities
>>>>> creation and update, then data should be collected in flat datasets: I
>>>>> propose 1 dataset for meters and 1 dataset for loads, collecting data for 1
>>>>> day. The best would be CSV files containing information about the prosumer
>>>>> and the meter/load: each line of the csv should contain prosumer info plus
>>>>> meter/load.
>>>>> Regarding names of the columns: we used namespaces, but we are
>>>>> thinking about removing them. Is it possible for you to define more simple
>>>>> CSV columns names? for example:
>>>>> BusinessDemo:Prosumer:address:name --> Prosumer Name
>>>>> BusinessDemo:Prosumer:address:city --> City
>>>>> BusinessDemo:Prosumer:address:country --> Country
>>>>> BusinessDemo:Prosumer:address:number--> Address number
>>>>> and so on.
>>>>> Thanks and best regards
>>>>> Davide
>>>>>
>>>>>
>>>>> Il 29/06/2015 10:47, Aitor Magán ha scritto:
>>>>>
>>>>> I need to know the entities that you are using in the Context Broker
>>>>> and the structure of the CKAN datasets that you expect.
>>>>>
>>>>> BR
>>>>> Aitor
>>>>>
>>>>> 2015-06-29 10:38 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it>:
>>>>>
>>>>>> Hi Aitor
>>>>>> referring to the work plan
>>>>>> <https://docs.google.com/document/d/1I9FoZ8KkzXOXNPvrWOklzU5ajboQ1hSEZevfGomEWas/edit>,
>>>>>> do you have enough information to start with task T.2.2? I don't know if
>>>>>> the information we provided on the technical document if enough for you...
>>>>>> Thanks and best regards
>>>>>> Davide
>>>>>>
>>>>>> Il 29/06/2015 10:26, Aitor Magán ha scritto:
>>>>>>
>>>>>> Sorry Davide, I do not understand the question. What information do I
>>>>>> need to start collecting data?
>>>>>>
>>>>>> BR
>>>>>> Aitor
>>>>>>
>>>>>> 2015-06-29 10:23 GMT+02:00 Davide Zerbetto <davide.zerbetto at eng.it>:
>>>>>>
>>>>>>> Hi all
>>>>>>> we installed the Prosumers Simulator (the mock component that
>>>>>>> generates anonymous data) on the same VM hosting the CB.
>>>>>>> Now it is running. I updated the technical document also.
>>>>>>> Of course if you have comments, don't hesitate to ask.
>>>>>>> @Aitor: do you have enough information to start collecting data into
>>>>>>> CKAN datasets?
>>>>>>> Thanks and best regards
>>>>>>> Davide
>>>>>>> --
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Davide Zerbetto*
>>>>>>> SpagoBI Consultant
>>>>>>>
>>>>>>> *SpagoBI Labs Engineering Group*
>>>>>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>>>>>> Phone: +39 049 8283411 <%2B39%20049%208283411>
>>>>>>> Fax: +39-049.8700272
>>>>>>> www.spagobi.org - www.eng.it <http://www.eng.it/web/eng_en/home>
>>>>>>> Respect the environment. Please don't print this e-mail
>>>>>>> unless you really need to.
>>>>>>>
>>>>>>> The information transmitted is intended only for the person or
>>>>>>> entity to which it is addressed and may contain confidential and/or
>>>>>>> privileged material. Any review, retransmission, dissemination or other use
>>>>>>> of, or taking of any action in reliance upon, this information by persons
>>>>>>> or entities other than the intended recipient is prohibited. If you
>>>>>>> received this in error, please contact the sender and delete the material
>>>>>>> from any computer.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Fiware-apps mailing list
>>>>>>> Fiware-apps at lists.fi-ware.org
>>>>>>> https://lists.fi-ware.org/listinfo/fiware-apps
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards
>>>>>> Aitor Magán
>>>>>>
>>>>>> [image: Twiter] <https://twitter.com/AitorMagan>
>>>>>> <https://es.linkedin.com/in/aitormagan>
>>>>>> <https://github.com/AitorMagan>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Fiware-apps mailing listFiware-apps at lists.fi-ware.orghttps://lists.fi-ware.org/listinfo/fiware-apps
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Davide Zerbetto*
>>>>>> SpagoBI Consultant
>>>>>>
>>>>>> *SpagoBI Labs Engineering Group*
>>>>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>>>>> Phone: +39 049 8283411 <%2B39%20049%208283411>
>>>>>> Fax: +39-049.8700272
>>>>>> www.spagobi.org - www.eng.it <http://www.eng.it/web/eng_en/home>
>>>>>> Respect the environment. Please don't print this e-mail unless
>>>>>> you really need to.
>>>>>>
>>>>>> The information transmitted is intended only for the person or entity
>>>>>> to which it is addressed and may contain confidential and/or privileged
>>>>>> material. Any review, retransmission, dissemination or other use of, or
>>>>>> taking of any action in reliance upon, this information by persons or
>>>>>> entities other than the intended recipient is prohibited. If you received
>>>>>> this in error, please contact the sender and delete the material from any
>>>>>> computer.
>>>>>>
>>>>>> _______________________________________________
>>>>>> Fiware-apps mailing list
>>>>>> Fiware-apps at lists.fi-ware.org
>>>>>> https://lists.fi-ware.org/listinfo/fiware-apps
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>> Aitor Magán
>>>>>
>>>>> [image: Twiter] <https://twitter.com/AitorMagan>
>>>>> <https://es.linkedin.com/in/aitormagan>
>>>>> <https://github.com/AitorMagan>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> --
>>>>>
>>>>> *Davide Zerbetto*
>>>>> SpagoBI Consultant
>>>>>
>>>>> *SpagoBI Labs Engineering Group*
>>>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>>>> Phone: +39 049 8283411 <%2B39%20049%208283411>
>>>>> Fax: +39-049.8700272
>>>>> www.spagobi.org - www.eng.it <http://www.eng.it/web/eng_en/home>
>>>>> Respect the environment. Please don't print this e-mail unless
>>>>> you really need to.
>>>>>
>>>>> The information transmitted is intended only for the person or entity
>>>>> to which it is addressed and may contain confidential and/or privileged
>>>>> material. Any review, retransmission, dissemination or other use of, or
>>>>> taking of any action in reliance upon, this information by persons or
>>>>> entities other than the intended recipient is prohibited. If you received
>>>>> this in error, please contact the sender and delete the material from any
>>>>> computer.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>> Aitor Magán
>>>>
>>>> [image: Twiter] <https://twitter.com/AitorMagan>
>>>> <https://es.linkedin.com/in/aitormagan>
>>>> <https://github.com/AitorMagan>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Aitor Magán
>>>
>>> [image: Twiter] <https://twitter.com/AitorMagan>
>>> <https://es.linkedin.com/in/aitormagan> <https://github.com/AitorMagan>
>>>
>>>
>>> --
>>>
>>> --
>>>
>>> *Davide Zerbetto*
>>> SpagoBI Consultant
>>>
>>> *SpagoBI Labs Engineering Group*
>>> Corso Stati Uniti, 23/C - 35127 Padua - Italy
>>> Phone: +39 049 8283411 <%2B39%20049%208283411>
>>> Fax: +39-049.8700272
>>> www.spagobi.org - www.eng.it <http://www.eng.it/web/eng_en/home>
>>> Respect the environment. Please don't print this e-mail unless
>>> you really need to.
>>>
>>> The information transmitted is intended only for the person or entity to
>>> which it is addressed and may contain confidential and/or privileged
>>> material. Any review, retransmission, dissemination or other use of, or
>>> taking of any action in reliance upon, this information by persons or
>>> entities other than the intended recipient is prohibited. If you received
>>> this in error, please contact the sender and delete the material from any
>>> computer.
>>>
>>
>>
>>
>> --
>> Best Regards
>> Aitor Magán
>>
>> [image: Twiter] <https://twitter.com/AitorMagan>
>> <https://es.linkedin.com/in/aitormagan> <https://github.com/AitorMagan>
>>
>>
>> _______________________________________________
>> Fiware-apps mailing listFiware-apps at lists.fi-ware.orghttps://lists.fi-ware.org/listinfo/fiware-apps
>>
>>
>>
>> _______________________________________________
>> Fiware-apps mailing list
>> Fiware-apps at lists.fi-ware.org
>> https://lists.fi-ware.org/listinfo/fiware-apps
>>
>
>
>
> --
> Best Regards
> Aitor Magán
>
> [image: Twiter] <https://twitter.com/AitorMagan>
> <https://es.linkedin.com/in/aitormagan> <https://github.com/AitorMagan>
>
>
> _______________________________________________
> Fiware-apps mailing listFiware-apps at lists.fi-ware.orghttps://lists.fi-ware.org/listinfo/fiware-apps
>
>
> --
>
> --
>
> *Alessandro Portosa*
> SpagoBI Consultant
>
> *SpagoBI Labs Engineering Group*
> Via Marconi, 10 - 40125 Bologna - Italy
> Tel. + 39 051 0435090
> Skype: alessandro.portosa
> www.spagobi.org - www.eng.it <http://www.eng.it/web/eng_en/home>
> Respect the environment. Please don't print this e-mail unless you
> really need to.
>
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you received
> this in error, please contact the sender and delete the material from any
> computer.
>
--
Best Regards
Aitor Magán
[image: Twiter] <https://twitter.com/AitorMagan>
<https://es.linkedin.com/in/aitormagan> <https://github.com/AitorMagan>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0001.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0002.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: spagobi.png
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 1379 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0003.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3639 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-apps/attachments/20150707/91f7fdda/attachment-0004.png>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy