>From FIWARE JIRA - Main Help Desk ----
-------------------------------------------------------------------------------
Comments:
FW External User - Today 3:40 PM
------------------
Comment by peter at familiohq.com :
Hi Andres,
I haven't received the script file .. it's not attached to the emails I'm
getting. Are you able to send it directly to my email at peter at familiohq.com --
thanks
On Wed, Dec 20, 2017 at 3:27 PM Help-Desk <jira-help-desk at jira.fiware.org>
wrote:
>
>
Andres Muñoz - Today 3:26 PM
------------------
Hello,
Did you make the test that I wrote you before?
FW External User - Today 1:49 PM
------------------
Comment by peter at familiohq.com :
Hi Andres,
Anything news on this? Also are you able to provide me with your
"agent.conf" and "cartodb_keys.conf" file (removing your actual key is
fine) so I'm able to compare it with mine to see if there is anything
different? Maybe also your "docker-compose.yml" file if you're using Docker
to setup orion/cygnus.
Thanks!
On Mon, Dec 11, 2017 at 4:07 PM Help-Desk <jira-help-desk at jira.fiware.org>
wrote:
>
>
Andres Muñoz - 12/Dec/17 12:05 PM
------------------
Hello,
I have been checking your agent.conf file and It looks good configurated, the log error (null pointer exception) that you sent us give me some clues about the creation of the object to insert to de cartodb (I am not sure, I am speculating), so I want to be sure that cygnus can handle and convert the event to the object, to this comment I am attaching a script that generates a test event and sen this to cygnus in order to persist in the cartoDB the event contains geolocation data so is pretty similar to the events that you generate. please execute this script as follows:
./notification-json-geom.sh http://your-cygnus-ip:5050/notify myservice /myservicepath
and let me know if it event is persisted in cartodb or send me again the cygnus log
Andres Muñoz - 11/Dec/17 4:06 PM
------------------
Hello,
I am trying to replicate the error with the settings of your agent.conf file, please give a little more of time.
FW External User - 11/Dec/17 10:51 AM
------------------
Comment by peter at familiohq.com :
Hi guys,
Have you been able to look at my issue? I would appreciate your help with
this issue.
On Mon, Dec 4, 2017 at 1:53 PM Peter Theill <peter at familiohq.com> wrote:
> Sure, this is my file agent.conf
>
> cygnus-ngsi.sources = http-source
>
> cygnus-ngsi.sinks = cartodb-sink
>
> cygnus-ngsi.channels = cartodb-channel
>
>
> cygnus-ngsi.sources.http-source.channels = cartodb-channel
>
> cygnus-ngsi.sources.http-source.type =
> org.apache.flume.source.http.HTTPSource
>
> cygnus-ngsi.sources.http-source.port = 5050
>
> cygnus-ngsi.sources.http-source.handler =
> com.telefonica.iot.cygnus.handlers.NGSIRestHandler
>
> cygnus-ngsi.sources.http-source.handler.notification_target = /notify
>
> cygnus-ngsi.sources.http-source.handler.default_service = default
>
> cygnus-ngsi.sources.http-source.handler.default_service_path = /
>
> cygnus-ngsi.sources.http-source.interceptors = ts gi
>
> cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
>
> cygnus-ngsi.sources.http-source.interceptors.gi.type =
> com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
>
> cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file =
> /usr/cygnus/conf/grouping_rules.conf
>
>
> cygnus-ngsi.sinks.cartodb-sink.type =
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink
>
> cygnus-ngsi.sinks.cartodb-sink.channel = cartodb-channel
>
> cygnus-ngsi.sinks.cartodb-sink.keys_conf_file =
> /opt/apache-flume/conf/cartodb_keys.conf
>
>
> cygnus-ngsi.channels.cartodb-channel.type = memory
>
> cygnus-ngsi.channels.cartodb-channel.capacity = 1000
>
> cygnus-ngsi.channels.cartodb-channel.transactionCapacity = 100
>
>
> Let me know if you want more info.
>
> On Mon, Dec 4, 2017 at 1:13 PM Help-Desk <jira-help-desk at jira.fiware.org>
> wrote:
>
>>
>>
>
FW External User - 04/Dec/17 1:54 PM
------------------
Comment by peter at familiohq.com :
Sure, this is my file agent.conf
cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = cartodb-sink
cygnus-ngsi.channels = cartodb-channel
cygnus-ngsi.sources.http-source.channels = cartodb-channel
cygnus-ngsi.sources.http-source.type =
org.apache.flume.source.http.HTTPSource
cygnus-ngsi.sources.http-source.port = 5050
cygnus-ngsi.sources.http-source.handler =
com.telefonica.iot.cygnus.handlers.NGSIRestHandler
cygnus-ngsi.sources.http-source.handler.notification_target = /notify
cygnus-ngsi.sources.http-source.handler.default_service = default
cygnus-ngsi.sources.http-source.handler.default_service_path = /
cygnus-ngsi.sources.http-source.interceptors = ts gi
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
cygnus-ngsi.sources.http-source.interceptors.gi.type =
com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file =
/usr/cygnus/conf/grouping_rules.conf
cygnus-ngsi.sinks.cartodb-sink.type =
com.telefonica.iot.cygnus.sinks.NGSICartoDBSink
cygnus-ngsi.sinks.cartodb-sink.channel = cartodb-channel
cygnus-ngsi.sinks.cartodb-sink.keys_conf_file =
/opt/apache-flume/conf/cartodb_keys.conf
cygnus-ngsi.channels.cartodb-channel.type = memory
cygnus-ngsi.channels.cartodb-channel.capacity = 1000
cygnus-ngsi.channels.cartodb-channel.transactionCapacity = 100
Let me know if you want more info.
On Mon, Dec 4, 2017 at 1:13 PM Help-Desk <jira-help-desk at jira.fiware.org>
wrote:
>
>
Joaquin Salvachua - 04/Dec/17 1:12 PM
------------------
Hello,
I am checking the issue with your development, but I need a little more of information, for trying to replicate the error, Can you send me your cygnus agent configuration file?
FW External User - 27/Nov/17 5:58 AM
------------------
Comment by peter at familiohq.com :
Dear FIWARE,
Did you have a chance to look into this issue further? I'm a bit blocked in
my further development.
On Thu, Nov 23, 2017 at 3:02 PM Help-Desk <jira-help-desk at jira.fiware.org>
wrote:
>
>
FW External User - 23/Nov/17 11:09 AM
------------------
Comment by jsalvachua at dit.upm.es :
Dear Peter,
we will have a look at it. Perhaps we may ask you a bit about your configuration.
Best Regards
Joaquín
> El 23 nov 2017, a las 14:53, Peter Theill <peter at familiohq.com> escribió:
>
> I've setup Orion and Cygnus so whenever a new entity is created in Orion it's transferring its data to Carto using the Cygnus CartoDbSink.
>
> However I'm having some issues with this implementation. I'm getting the data correctly from Orion to Cygnus but when the sink tries to forward those data to Carto I'm getting this error message (also attached for better formatting) when trying to persist it in Carto
>
> cygnus_1 | time=2017-11-23T13:40:11.175Z | lvl=INFO | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[294] : [NGSIRestHandler] Starting internal transaction (efab5d17-12c3-4716-8b1b-15352ec6ae18)
>
> cygnus_1 | time=2017-11-23T13:40:11.176Z | lvl=INFO | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[312] : [NGSIRestHandler] Received data ({"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"type":"Event","isPattern":"false","id":"getif","attributes":[{"name":"location","type":"geo:point","value":"79.00555, -148.99808"}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[320] : [NGSIRestHandler] Parsed NotifyContextRequest: {"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555, -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[345] : [NGSIRestHandler] NGSI event created for ContextElementResponse: {"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555, -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[350] : [NGSIRestHandler] Header added to NGSI event (fiware-service: default)
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[353] : [NGSIRestHandler] Header added to NGSI event (fiware-servicepath: /)
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[356] : [NGSIRestHandler] Header added to NGSI event (fiware-correlator: d3be117c-d053-11e7-ad11-0242ac120009)
>
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[359] : [NGSIRestHandler] Header added to NGSI event (transaction-id: efab5d17-12c3-4716-8b1b-15352ec6ae18)
>
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[385] : [NGSIRestHandler] NGSI events put in the channel, ids=1516398619
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=intercept | msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[84] : [gi] Event intercepted, id=1516398619
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=intercept | msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[117] : [gi] Adding flume event header (notified-entity: getif_Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=intercept | msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[120] : [gi] Adding flume event header (grouped-entity: getif_Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=intercept | msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[123] : [gi] Adding flume event header (grouped-servicepath: /)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=intercept | msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[127] : [gi] Event put in the channel, id=1516398619
>
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO | corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=httpRequestSend.cpp[620]:httpRequestSendWithCurl | msg=Notification Successfully Sent to http://staging.familiohq.com:5050/notify <http://staging.familiohq.com:5050/notify>
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO | corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending | subsrv=pending | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd | msg=Transaction ended
>
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[554] : Batch completed
>
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=persistBatch | msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[381] : [cartodb-sink] Processing sub-batch regarding the default_/_getif_Event destination
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[570] : [cartodb-sink] Processing context element (id=getif, type=Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=aggregate | msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[592] : [cartodb-sink] Processing context attribute (name=location, type=geo:point)
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=INFO | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=persistRawAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[616] : [cartodb-sink] Persisting data at NGSICartoDBSink. Schema (default), Table (x002fxffffgetifxffffx0045vent), Data (('2017-11-23T13:40:11.241Z','/','getif','Event',ST_SetSRID(ST_MakePoint(79.00555,-148.99808), 4326)))
>
> cygnus_1 | time=2017-11-23T13:40:11.255Z | lvl=ERROR | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run | msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver event. Exception follows.
>
> cygnus_1 | java.lang.NullPointerException
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistRawAggregation(NGSICartoDBSink.java:625)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistBatch(NGSICartoDBSink.java:417)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:558)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
> cygnus_1 | time=2017-11-23T13:40:16.256Z | lvl=ERROR | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run | msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when transaction is OPEN!
>
> cygnus_1 | at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
> cygnus_1 | time=2017-11-23T13:40:18.648Z | lvl=DEBUG | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run | msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[126] : Checking file:/opt/apache-flume/conf/agent.conf for changes
>
> cygnus_1 | time=2017-11-23T13:40:18.654Z | lvl=WARN | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=<init> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[49] : No grouping rules have been read. Details: /usr/cygnus/conf/grouping_rules.conf (No such file or directory)
>
> cygnus_1 | time=2017-11-23T13:40:21.257Z | lvl=ERROR | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run | msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when transaction is OPEN!
>
> cygnus_1 | at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
> cygnus_1 | time=2017-11-23T13:40:26.258Z | lvl=ERROR | corr=d3be117c-d053-11e7-ad11-0242ac120009 | trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run | msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when transaction is OPEN!
>
> cygnus_1 | at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
>
>
> Are you able to help me figure out why I can't send this information to carto?
>
>
>
> --
> Peter Theill
> CTO
> +45 6171 5096 <tel:61%2071%2050%2096>
> peter at familiohq.com <mailto:peter at familiohq.com>
>
> Familio
> Artillerivej 86
> DK-2300 Copenhagen S
> https://familiohq.com/ <https://familiohq.com/><carto-cygnus-log.txt>
------------------------
Issue id: HELP-12977
Description:
I've setup Orion and Cygnus so whenever a new entity is created in Orion
it's transferring its data to Carto using the Cygnus CartoDbSink.
However I'm having some issues with this implementation. I'm getting the
data correctly from Orion to Cygnus but when the sink tries to forward
those data to Carto I'm getting this error message (also attached for
better formatting) when trying to persist it in Carto
cygnus_1 | time=2017-11-23T13:40:11.175Z | lvl=INFO |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[294] :
[NGSIRestHandler] Starting internal transaction
(efab5d17-12c3-4716-8b1b-15352ec6ae18)
cygnus_1 | time=2017-11-23T13:40:11.176Z | lvl=INFO |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[312] :
[NGSIRestHandler] Received data
({"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"type":"Event","isPattern":"false","id":"getif","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
-148.99808"}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[320] :
[NGSIRestHandler] Parsed NotifyContextRequest:
{"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
-148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}
cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[345] :
[NGSIRestHandler] NGSI event created for ContextElementResponse:
{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
-148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}
cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[350] :
[NGSIRestHandler] Header added to NGSI event (fiware-service: default)
cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[353] :
[NGSIRestHandler] Header added to NGSI event (fiware-servicepath: /)
cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[356] :
[NGSIRestHandler] Header added to NGSI event (fiware-correlator:
d3be117c-d053-11e7-ad11-0242ac120009)
cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[359] :
[NGSIRestHandler] Header added to NGSI event (transaction-id:
efab5d17-12c3-4716-8b1b-15352ec6ae18)
cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=getEvents |
msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[385] :
[NGSIRestHandler] NGSI events put in the channel, ids=1516398619
cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=intercept |
msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[84] :
[gi] Event intercepted, id=1516398619
cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=intercept |
msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[117] :
[gi] Adding flume event header (notified-entity: getif_Event)
cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=intercept |
msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[120] :
[gi] Adding flume event header (grouped-entity: getif_Event)
cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=intercept |
msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[123] :
[gi] Adding flume event header (grouped-servicepath: /)
cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=intercept |
msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[127] :
[gi] Event put in the channel, id=1516398619
orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
subsrv=pending | comp=Orion |
op=httpRequestSend.cpp[620]:httpRequestSendWithCurl | msg=Notification
Successfully Sent to http://staging.familiohq.com:5050/notify
orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
subsrv=pending | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd |
msg=Transaction ended
cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=processNewBatches |
msg=com.telefonica.iot.cygnus.sinks.NGSISink[554] : Batch completed
cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=persistBatch |
msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[381] : [cartodb-sink]
Processing sub-batch regarding the default_/_getif_Event destination
cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=aggregate |
msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[570]
: [cartodb-sink] Processing context element (id=getif, type=Event)
cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=aggregate |
msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[592]
: [cartodb-sink] Processing context attribute (name=location,
type=geo:point)
cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=INFO |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=persistRawAggregation |
msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[616] : [cartodb-sink]
Persisting data at NGSICartoDBSink. Schema (default), Table
(x002fxffffgetifxffffx0045vent), Data
(('2017-11-23T13:40:11.241Z','/','getif','Event',ST_SetSRID(ST_MakePoint(79.00555,-148.99808),
4326)))
cygnus_1 | time=2017-11-23T13:40:11.255Z | lvl=ERROR |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-*15352ec6ae18
| srv=default | subsrv=/ | comp=cygnus-ngsi | op=run |
msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
event. Exception follows.*
*cygnus_1 | java.lang.NullPointerException*
*cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistRawAggregation(NGSICartoDBSink.java:625)*
*cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistBatch(NGSICartoDBSink.java:417)*
*cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:558)*
*cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)*
*cygnus_1 | at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)*
*cygnus_1 | at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)*
*cygnus_1 | at java.lang.Thread.run(Thread.java:748)*
cygnus_1 | time=2017-11-23T13:40:16.256Z | lvl=ERROR |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=run |
msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
event. Exception follows.
cygnus_1 | java.lang.IllegalStateException: begin() called when
transaction is OPEN!
cygnus_1 | at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
cygnus_1 | at
org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
cygnus_1 | at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
cygnus_1 | at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
cygnus_1 | at java.lang.Thread.run(Thread.java:748)
cygnus_1 | time=2017-11-23T13:40:18.648Z | lvl=DEBUG | corr=N/A
| trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run |
msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[126]
: Checking file:/opt/apache-flume/conf/agent.conf for changes
cygnus_1 | time=2017-11-23T13:40:18.654Z | lvl=WARN | corr=N/A
| trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=<init> |
msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[49] : No
grouping rules have been read. Details:
/usr/cygnus/conf/grouping_rules.conf (No such file or directory)
cygnus_1 | time=2017-11-23T13:40:21.257Z | lvl=ERROR |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=run |
msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
event. Exception follows.
cygnus_1 | java.lang.IllegalStateException: begin() called when
transaction is OPEN!
cygnus_1 | at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
cygnus_1 | at
org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
cygnus_1 | at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
cygnus_1 | at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
cygnus_1 | at java.lang.Thread.run(Thread.java:748)
cygnus_1 | time=2017-11-23T13:40:26.258Z | lvl=ERROR |
corr=d3be117c-d053-11e7-ad11-0242ac120009 |
trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
comp=cygnus-ngsi | op=run |
msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
event. Exception follows.
cygnus_1 | java.lang.IllegalStateException: begin() called when
transaction is OPEN!
cygnus_1 | at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
cygnus_1 | at
org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
cygnus_1 | at
com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
cygnus_1 | at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
cygnus_1 | at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
cygnus_1 | at java.lang.Thread.run(Thread.java:748)
Are you able to help me figure out why I can't send this information to
carto?
--
*Peter Theill*
CTO
+45 6171 5096 <61%2071%2050%2096>
peter at familiohq.com
*Familio*
Artillerivej 86
DK-2300 Copenhagen S
https://familiohq.com/
__________________________________________________________________________________________
You can get more information about our cookies and privacy policies on the following links:
- http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Privacy_Policy
- http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Cookies_Policy_FIWARE
Fiware-tech-help mailing list
Fiware-tech-help at lists.fiware.org
https://lists.fiware.org/listinfo/fiware-tech-help
[Created via e-mail received from: Peter Theill <peter at familiohq.com>]
FIWARE Chapter:
FIWARE GEri:
Status: Answered
---------------------
This email was generated by FIWARE JIRA following an email received into the Main Help Desk.
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy