[ https://jira.fiware.org/browse/HELP-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=100073#comment-100073 ]
Andres Muñoz commented on HELP-12977:
-------------------------------------
Hello Peter,
I am still checking your issue but I need that you perform some other tests, please once again run the script that I provide you but this time with this 2 more test values :
./notification.sh http://localhost:5050/notify myservice /traffic 40.361 -3.4099
remember that you must execute this script pointing to your cygnus host and port
then tell me if it was successful persisted in the cartodb otherwise give me the output log.
If and only if it fails please try to execute the next command for testing if the cartodb entities are correctly created and available
for inserting data to cartodb
curl -G "https://<my_user>.cartodb.com/api/v2/sql?api_key=<api_key>" --data-urlencode "q=INSERT INTO <table_name> (recvTime, fiwareServicePath, entityId, entityType, <attr_1>, <attr_1>_md, ..., <attr_n>, <attr_n>_md, the_geom) VALUES ('2016-04-19T07:09:53.116Z', '<service_path>', '<entity_id>', '<entity_type>', '<attr_1_value>', '<attr_1_metadata>', ..., '<attr_n_value>', '<attr_n_metadata>', 'ST_SetSRID(ST_MakePoint(<lat>, <lon>), 4326))"
for query this data
curl -G "https://<my_user>.cartodb.com/api/v2/sql?api_key=<api_key>" --data-urlencode "q=SELECT * FROM <table_name>"
I will be waiting for your comments.
Best Regards
> [Fiware-tech-help] IMPACT: Delivering CartoDb events through Cygnus
> -------------------------------------------------------------------
>
> Key: HELP-12977
> URL: https://jira.fiware.org/browse/HELP-12977
> Project: Help-Desk
> Issue Type: extRequest
> Components: FIWARE-TECH-HELP
> Reporter: FW External User
> Assignee: Andres Muñoz
> Attachments: carto-cygnus-log.txt, cygnus-log.txt, notification-json-geom.sh
>
>
> I've setup Orion and Cygnus so whenever a new entity is created in Orion
> it's transferring its data to Carto using the Cygnus CartoDbSink.
> However I'm having some issues with this implementation. I'm getting the
> data correctly from Orion to Cygnus but when the sink tries to forward
> those data to Carto I'm getting this error message (also attached for
> better formatting) when trying to persist it in Carto
> cygnus_1 | time=2017-11-23T13:40:11.175Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[294] :
> [NGSIRestHandler] Starting internal transaction
> (efab5d17-12c3-4716-8b1b-15352ec6ae18)
> cygnus_1 | time=2017-11-23T13:40:11.176Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[312] :
> [NGSIRestHandler] Received data
> ({"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"type":"Event","isPattern":"false","id":"getif","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808"}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[320] :
> [NGSIRestHandler] Parsed NotifyContextRequest:
> {"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[345] :
> [NGSIRestHandler] NGSI event created for ContextElementResponse:
> {"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[350] :
> [NGSIRestHandler] Header added to NGSI event (fiware-service: default)
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[353] :
> [NGSIRestHandler] Header added to NGSI event (fiware-servicepath: /)
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[356] :
> [NGSIRestHandler] Header added to NGSI event (fiware-correlator:
> d3be117c-d053-11e7-ad11-0242ac120009)
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[359] :
> [NGSIRestHandler] Header added to NGSI event (transaction-id:
> efab5d17-12c3-4716-8b1b-15352ec6ae18)
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[385] :
> [NGSIRestHandler] NGSI events put in the channel, ids=1516398619
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[84] :
> [gi] Event intercepted, id=1516398619
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[117] :
> [gi] Adding flume event header (notified-entity: getif_Event)
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[120] :
> [gi] Adding flume event header (grouped-entity: getif_Event)
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[123] :
> [gi] Adding flume event header (grouped-servicepath: /)
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[127] :
> [gi] Event put in the channel, id=1516398619
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
> corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
> subsrv=pending | comp=Orion |
> op=httpRequestSend.cpp[620]:httpRequestSendWithCurl | msg=Notification
> Successfully Sent to http://staging.familiohq.com:5050/notify
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
> corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
> subsrv=pending | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd |
> msg=Transaction ended
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=processNewBatches |
> msg=com.telefonica.iot.cygnus.sinks.NGSISink[554] : Batch completed
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistBatch |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[381] : [cartodb-sink]
> Processing sub-batch regarding the default_/_getif_Event destination
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[570]
> : [cartodb-sink] Processing context element (id=getif, type=Event)
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[592]
> : [cartodb-sink] Processing context attribute (name=location,
> type=geo:point)
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistRawAggregation |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[616] : [cartodb-sink]
> Persisting data at NGSICartoDBSink. Schema (default), Table
> (x002fxffffgetifxffffx0045vent), Data
> (('2017-11-23T13:40:11.241Z','/','getif','Event',ST_SetSRID(ST_MakePoint(79.00555,-148.99808),
> 4326)))
> cygnus_1 | time=2017-11-23T13:40:11.255Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-*15352ec6ae18
> | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.*
> *cygnus_1 | java.lang.NullPointerException*
> *cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistRawAggregation(NGSICartoDBSink.java:625)*
> *cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistBatch(NGSICartoDBSink.java:417)*
> *cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:558)*
> *cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)*
> *cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)*
> *cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)*
> *cygnus_1 | at java.lang.Thread.run(Thread.java:748)*
> cygnus_1 | time=2017-11-23T13:40:16.256Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> cygnus_1 | time=2017-11-23T13:40:18.648Z | lvl=DEBUG | corr=N/A
> | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[126]
> : Checking file:/opt/apache-flume/conf/agent.conf for changes
> cygnus_1 | time=2017-11-23T13:40:18.654Z | lvl=WARN | corr=N/A
> | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=<init> |
> msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[49] : No
> grouping rules have been read. Details:
> /usr/cygnus/conf/grouping_rules.conf (No such file or directory)
> cygnus_1 | time=2017-11-23T13:40:21.257Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> cygnus_1 | time=2017-11-23T13:40:26.258Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> Are you able to help me figure out why I can't send this information to
> carto?
> --
> *Peter Theill*
> CTO
> +45 6171 5096 <61%2071%2050%2096>
> peter at familiohq.com
> *Familio*
> Artillerivej 86
> DK-2300 Copenhagen S
> https://familiohq.com/
> __________________________________________________________________________________________
> You can get more information about our cookies and privacy policies on the following links:
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Privacy_Policy
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Cookies_Policy_FIWARE
> Fiware-tech-help mailing list
> Fiware-tech-help at lists.fiware.org
> https://lists.fiware.org/listinfo/fiware-tech-help
> [Created via e-mail received from: Peter Theill <peter at familiohq.com>]
--
This message was sent by Atlassian JIRA
(v6.4.1#64016)
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy