[Fiware-tech-help] [FI-WARE-JIRA] (HELP-5452) Problema al intentar grabar datos en CKAN desde Cygnus - Cygnus side

[FRANCISCO ROMERO BUENO]

Hola Alfonso,

El fallo se debe a que el modo fila de CKAN crea automáticamente el
recurso con todos los campos de tipo ³text². Es la desventaja de usar el
modo row, ya que en la versión actual Cygnus no inspecciona el campo type
del atributo, más que nada porque ese campo ahora mismo en Orion no
significa nada, no tiene semántica, es solo una anotación de cara al
usuario. De suerte que si en tu valor de atributo no mandas un string
(rodeado de ³ ³) no va a funcionar el modo fila.

La solución pasa por usar el modo columna, en el que tú provisionas el
recurso, y por tanto, tienes total control sobre los tipos CKAN de todos
los campos, incluido el de valor de atributo, que tendría tipo ³json².

Saludos,
Paco

El 3/12/15 8:25, "Manuel Escriche (JIRA)" <jira-help-desk at fi-ware.org>
escribió:

>
>     [
>https://jira.fiware.org/browse/HELP-5452?page=com.atlassian.jira.plugin.sy
>stem.issuetabpanels:all-tabpanel ]
>
>Manuel Escriche reassigned HELP-5452:
>-------------------------------------
>
>    Assignee: Francisco Romero  (was: OKFN CKAN team)
>
>> [Fiware-tech-help] Problema al intentar grabar datos en CKAN desde
>>Cygnus - Cygnus side
>>
>>-------------------------------------------------------------------------
>>--------------
>>
>>                 Key: HELP-5452
>>                 URL: https://jira.fiware.org/browse/HELP-5452
>>             Project: Help-Desk
>>          Issue Type: extRequest
>>          Components: FIWARE-TECH-HELP
>>            Reporter: FW External User
>>            Assignee: Francisco Romero
>>
>> Buenas tardes.
>> Estoy intentando grabar datos en CKAN a través de Cygnus, pero no
>>consigo
>> que funcione correctamente cuando el tipo del atributo es JSON. Lo
>>primero
>> que hago es la llamada a ContextBroker:
>> Accept: application/json
>> X-AUTH-TOKEN: <mytoken>
>> Fiware-Service: PapelClubDemo
>> Fiware-ServicePath: /events/leonliterario
>> {
>>     "contextElements": [
>>         {
>>             "type": "Events",
>>             "isPattern": "false",
>>             "id": "thisweek",
>>             "attributes": [
>>                 {
>>                     "name": "schedule",
>>                     "type": "json",
>>                     "value": [{"title": "Presentación Viva Mi
>> Gente","date": "2015-11-30","location": "Salón de actos del
>> ICAL","url": "http://www.papel.club"}]
>>                 }
>>             ]
>>         }
>>     ],
>>     "updateAction": "APPEND"
>> }
>> Con la correspondiente suscripción de cygnus para esta entidad, obtengo
>> esta entrada en el log de cygnus:
>> 01 Dec 2015 19:05:13,701 INFO [891993589 at qtp-1988714671-0]
>> (com.telefonica.iot.cygnus.handlers.OrionRestHandler.getEvents:232) -
>> Received data ({"subscriptionId" : "565dd3497b72b7c7092d5a29",
>> "originator" : "localhost", "contextResponses" : [ { "contextElement"
>> : { "type" : "Events","isPattern" : "false", "id" : "thisweek",
>> "attributes" : [ { "name" : "schedule", "type" : "json", "value" : [ {
>> "title" : "Presentación Viva Mi Gente", "date" : "2015-11-30",
>> "location" : "Salón de actos del ICAL", "url" :
>> "http://www.papel.club" }, { "title" : "Presentación Viva Mi
>> Gente2","date" : "2015-11-30", "location" : "Salón de actos del ICAL",
>> "url" : "http://www.papel.club" } ] } ] }, "statusCode" : { "code" :
>> "200","reasonPhrase" : "OK" } } ]})
>> 01 Dec 2015 19:05:13,702 INFO
>>
>>[891993589 at qtp-1988714671-0](com.telefonica.iot.cygnus.handlers.OrionRest
>>Handler.getEvents:255)
>> - Event put in the channel (id=2134043204, ttl=10) 01 Dec 2015
>> 19:05:16,842 INFO[SinkRunner-PollingRunner-DefaultSinkProcessor]
>> (com.telefonica.iot.cygnus.sinks.OrionCKANSink.persistOne:207) -
>> [ckan-sink] Persisting data atOrionCKANSink (orgName=papelclubdemo,
>> pkgName=papelclubdemo_events_leonliterario,
>>
>>resName=thisweek004_events,data=1448989513702,2015-12-01T17:05:13.702Z,th
>>isweek,Events,schedule,json,[{"title":"Presentación
>> Viva MiGente","date":"2015-11-30","location":"Salón de actos del
>> ICAL","url":"http://www.papel.club"},{"title":"Presentación Viva
>> MiGente2","date":"2015-11-30","location":"Salón de actos del
>> ICAL","url":"http://www.papel.club"}],[])
>> 01 Dec 2015 19:05:19,479 ERROR
>> [SinkRunner-PollingRunner-DefaultSinkProcessor]
>> (com.telefonica.iot.cygnus.sinks.OrionSink.process:224) - Runtime
>> error (Don't know how to treat response code 503)
>> 01 Dec 2015 19:05:19,480 INFO
>> [SinkRunner-PollingRunner-DefaultSinkProcessor]
>> (com.telefonica.iot.cygnus.sinks.OrionSink.process:233) - Finishing
>> transaction (1448984542-601-0000000018)
>> Esta es la configuración de mi fichero agent en cygnus:
>> # Flume handler that will parse the notifications, must not be changed
>> cygnusagent.sources.http-source.handler =
>> com.telefonica.iot.cygnus.handlers.OrionRestHandler
>> # URL target
>> cygnusagent.sources.http-source.handler.notification_target = /notify
>> # Default service (service semantic depends on the persistence sink)
>> cygnusagent.sources.http-source.handler.default_service = def_serv
>> # Default service path (service path semantic depends on the
>>persistence sink)
>> cygnusagent.sources.http-source.handler.default_service_path =
>>def_servpath
>> # Number of channel re-injection retries before a Flume event is
>> definitely discarded (-1 means infinite retries)
>> cygnusagent.sources.http-source.handler.events_ttl = 10
>> # Source interceptors, do not change
>> cygnusagent.sources.http-source.interceptors = ts gi
>> # TimestampInterceptor, do not change
>> cygnusagent.sources.http-source.interceptors.ts.type = timestamp
>> # GroupinInterceptor, do not change
>> cygnusagent.sources.http-source.interceptors.gi.type =
>> com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
>> # Grouping rules for the GroupingInterceptor, put the right absolute
>> path to the file if necessary
>> # See the doc/design/interceptors document for more details
>> cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file
>> = /usr/cygnus/conf/grouping_rules.conf
>> # ============================================
>> # OrionCKANSink configuration
>> # channel name from where to read notification events
>> cygnusagent.sinks.ckan-sink.channel = ckan-channel
>> # sink class, must not be changed
>> cygnusagent.sinks.ckan-sink.type =
>>com.telefonica.iot.cygnus.sinks.OrionCKANSink
>> # the CKAN API key to use
>> #cygnusagent.sinks.ckan-sink.api_key = <mykey>
>> # the FQDN/IP address for the CKAN API endpoint
>> cygnusagent.sinks.ckan-sink.ckan_host = demo.ckan.org
>> # the port for the CKAN API endpoint
>> cygnusagent.sinks.ckan-sink.ckan_port = 80
>> # Orion URL used to compose the resource URL with the convenience
>> operation URL to query it
>> cygnusagent.sinks.ckan-sink.orion_url = http://127.0.0.1:1026
>> # how the attributes are stored, either per row either per column (row,
>>column)
>> cygnusagent.sinks.ckan-sink.attr_persistence = row
>> # enable SSL for secure Http transportation; 'true' or 'false'
>> cygnusagent.sinks.ckan-sink.ssl = false
>> Cuando cygnus reciba la información la envía a ckan y este la recibe,
>> creándome la organización (si no existe), el dataset y el recurso, pero
>> cuando accedo a ver el contenido del recurso este está vacío. Estoy
>> utilizando demo.ckan.org.
>> Muchas gracias,
>> Alfonso García Fernández
>> Since January 1st, old domains won't be supported and messages sent to
>>any domain different to @lists.fiware.org will be lost.
>> Please, send your messages using the new domain
>>(Fiware-tech-help at lists.fiware.org) instead of the old one.
>> _______________________________________________
>> Fiware-tech-help mailing list
>> Fiware-tech-help at lists.fiware.org
>> https://lists.fiware.org/listinfo/fiware-tech-help
>> [Created via e-mail received from:
>>=?UTF-8?Q?Alfonso_Garc=C3=ADa_Fern=C3=A1ndez?= <alfongf at gmail.com>]
>
>
>
>--
>This message was sent by Atlassian JIRA
>(v6.4.1#64016)


________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição