Hi Andres,
Anything news on this? Also are you able to provide me with your
"agent.conf" and "cartodb_keys.conf" file (removing your actual key is
fine) so I'm able to compare it with mine to see if there is anything
different? Maybe also your "docker-compose.yml" file if you're using Docker
to setup orion/cygnus.
Thanks!
On Mon, Dec 11, 2017 at 4:07 PM Help-Desk <jira-help-desk at jira.fiware.org>
wrote:
>
>
> From FIWARE JIRA - Main Help Desk ----
>
>
> -------------------------------------------------------------------------------
> Comments:
>
>
> Andres Muñoz - Today 4:06 PM
> ------------------
> Hello,
>
> I am trying to replicate the error with the settings of your agent.conf
> file, please give a little more of time.
>
>
> FW External User - Today 10:51 AM
> ------------------
> Comment by peter at familiohq.com :
>
> Hi guys,
>
> Have you been able to look at my issue? I would appreciate your help with
> this issue.
>
>
>
> On Mon, Dec 4, 2017 at 1:53 PM Peter Theill <peter at familiohq.com> wrote:
>
> > Sure, this is my file agent.conf
> >
> > cygnus-ngsi.sources = http-source
> >
> > cygnus-ngsi.sinks = cartodb-sink
> >
> > cygnus-ngsi.channels = cartodb-channel
> >
> >
> > cygnus-ngsi.sources.http-source.channels = cartodb-channel
> >
> > cygnus-ngsi.sources.http-source.type =
> > org.apache.flume.source.http.HTTPSource
> >
> > cygnus-ngsi.sources.http-source.port = 5050
> >
> > cygnus-ngsi.sources.http-source.handler =
> > com.telefonica.iot.cygnus.handlers.NGSIRestHandler
> >
> > cygnus-ngsi.sources.http-source.handler.notification_target = /notify
> >
> > cygnus-ngsi.sources.http-source.handler.default_service = default
> >
> > cygnus-ngsi.sources.http-source.handler.default_service_path = /
> >
> > cygnus-ngsi.sources.http-source.interceptors = ts gi
> >
> > cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
> >
> > cygnus-ngsi.sources.http-source.interceptors.gi.type =
> > com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
> >
> > cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file
> =
> > /usr/cygnus/conf/grouping_rules.conf
> >
> >
> > cygnus-ngsi.sinks.cartodb-sink.type =
> > com.telefonica.iot.cygnus.sinks.NGSICartoDBSink
> >
> > cygnus-ngsi.sinks.cartodb-sink.channel = cartodb-channel
> >
> > cygnus-ngsi.sinks.cartodb-sink.keys_conf_file =
> > /opt/apache-flume/conf/cartodb_keys.conf
> >
> >
> > cygnus-ngsi.channels.cartodb-channel.type = memory
> >
> > cygnus-ngsi.channels.cartodb-channel.capacity = 1000
> >
> > cygnus-ngsi.channels.cartodb-channel.transactionCapacity = 100
> >
> >
> > Let me know if you want more info.
> >
> > On Mon, Dec 4, 2017 at 1:13 PM Help-Desk <jira-help-desk at jira.fiware.org
> >
> > wrote:
> >
> >>
> >>
> >
>
>
>
>
> FW External User - 04/Dec/17 1:54 PM
> ------------------
> Comment by peter at familiohq.com :
>
> Sure, this is my file agent.conf
>
> cygnus-ngsi.sources = http-source
>
> cygnus-ngsi.sinks = cartodb-sink
>
> cygnus-ngsi.channels = cartodb-channel
>
>
> cygnus-ngsi.sources.http-source.channels = cartodb-channel
>
> cygnus-ngsi.sources.http-source.type =
> org.apache.flume.source.http.HTTPSource
>
> cygnus-ngsi.sources.http-source.port = 5050
>
> cygnus-ngsi.sources.http-source.handler =
> com.telefonica.iot.cygnus.handlers.NGSIRestHandler
>
> cygnus-ngsi.sources.http-source.handler.notification_target = /notify
>
> cygnus-ngsi.sources.http-source.handler.default_service = default
>
> cygnus-ngsi.sources.http-source.handler.default_service_path = /
>
> cygnus-ngsi.sources.http-source.interceptors = ts gi
>
> cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
>
> cygnus-ngsi.sources.http-source.interceptors.gi.type =
> com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
>
> cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file =
> /usr/cygnus/conf/grouping_rules.conf
>
>
> cygnus-ngsi.sinks.cartodb-sink.type =
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink
>
> cygnus-ngsi.sinks.cartodb-sink.channel = cartodb-channel
>
> cygnus-ngsi.sinks.cartodb-sink.keys_conf_file =
> /opt/apache-flume/conf/cartodb_keys.conf
>
>
> cygnus-ngsi.channels.cartodb-channel.type = memory
>
> cygnus-ngsi.channels.cartodb-channel.capacity = 1000
>
> cygnus-ngsi.channels.cartodb-channel.transactionCapacity = 100
>
>
> Let me know if you want more info.
>
> On Mon, Dec 4, 2017 at 1:13 PM Help-Desk <jira-help-desk at jira.fiware.org>
> wrote:
>
> >
> >
>
>
>
>
> Joaquin Salvachua - 04/Dec/17 1:12 PM
> ------------------
> Hello,
>
> I am checking the issue with your development, but I need a little more of
> information, for trying to replicate the error, Can you send me your cygnus
> agent configuration file?
>
>
> FW External User - 27/Nov/17 5:58 AM
> ------------------
> Comment by peter at familiohq.com :
>
> Dear FIWARE,
>
> Did you have a chance to look into this issue further? I'm a bit blocked in
> my further development.
>
>
>
> On Thu, Nov 23, 2017 at 3:02 PM Help-Desk <jira-help-desk at jira.fiware.org>
> wrote:
>
> >
> >
>
>
>
>
> FW External User - 23/Nov/17 11:09 AM
> ------------------
> Comment by jsalvachua at dit.upm.es :
>
> Dear Peter,
>
> we will have a look at it. Perhaps we may ask you a bit about your
> configuration.
>
> Best Regards
>
> Joaquín
>
>
> > El 23 nov 2017, a las 14:53, Peter Theill <peter at familiohq.com>
> escribió:
> >
> > I've setup Orion and Cygnus so whenever a new entity is created in Orion
> it's transferring its data to Carto using the Cygnus CartoDbSink.
> >
> > However I'm having some issues with this implementation. I'm getting the
> data correctly from Orion to Cygnus but when the sink tries to forward
> those data to Carto I'm getting this error message (also attached for
> better formatting) when trying to persist it in Carto
> >
> > cygnus_1 | time=2017-11-23T13:40:11.175Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[294] :
> [NGSIRestHandler] Starting internal transaction
> (efab5d17-12c3-4716-8b1b-15352ec6ae18)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.176Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[312] :
> [NGSIRestHandler] Received data
> ({"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"type":"Event","isPattern":"false","id":"getif","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808"}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
> >
> > cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[320] :
> [NGSIRestHandler] Parsed NotifyContextRequest:
> {"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}
> >
> > cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[345] :
> [NGSIRestHandler] NGSI event created for ContextElementResponse:
> {"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}
> >
> > cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[350] :
> [NGSIRestHandler] Header added to NGSI event (fiware-service: default)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[353] :
> [NGSIRestHandler] Header added to NGSI event (fiware-servicepath: /)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[356] :
> [NGSIRestHandler] Header added to NGSI event (fiware-correlator:
> d3be117c-d053-11e7-ad11-0242ac120009)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[359] :
> [NGSIRestHandler] Header added to NGSI event (transaction-id:
> efab5d17-12c3-4716-8b1b-15352ec6ae18)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[385] :
> [NGSIRestHandler] NGSI events put in the channel, ids=1516398619
> >
> > cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[84] :
> [gi] Event intercepted, id=1516398619
> >
> > cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[117] :
> [gi] Adding flume event header (notified-entity: getif_Event)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[120] :
> [gi] Adding flume event header (grouped-entity: getif_Event)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[123] :
> [gi] Adding flume event header (grouped-servicepath: /)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[127] :
> [gi] Event put in the channel, id=1516398619
> >
> > orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z |
> lvl=INFO | corr=N/A | trans=1511444367-204-00000000002 | from=pending |
> srv=pending | subsrv=pending | comp=Orion |
> op=httpRequestSend.cpp[620]:httpRequestSendWithCurl | msg=Notification
> Successfully Sent to http://staging.familiohq.com:5050/notify <
> http://staging.familiohq.com:5050/notify>
> > orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z |
> lvl=INFO | corr=N/A | trans=1511444367-204-00000000002 | from=pending |
> srv=pending | subsrv=pending | comp=Orion |
> op=logMsg.h[1916]:lmTransactionEnd | msg=Transaction ended
> >
> > cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=processNewBatches |
> msg=com.telefonica.iot.cygnus.sinks.NGSISink[554] : Batch completed
> >
> > cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistBatch |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[381] : [cartodb-sink]
> Processing sub-batch regarding the default_/_getif_Event destination
> >
> > cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[570]
> : [cartodb-sink] Processing context element (id=getif, type=Event)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[592]
> : [cartodb-sink] Processing context attribute (name=location,
> type=geo:point)
> >
> > cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistRawAggregation |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[616] : [cartodb-sink]
> Persisting data at NGSICartoDBSink. Schema (default), Table
> (x002fxffffgetifxffffx0045vent), Data
> (('2017-11-23T13:40:11.241Z','/','getif','Event',ST_SetSRID(ST_MakePoint(79.00555,-148.99808),
> 4326)))
> >
> > cygnus_1 | time=2017-11-23T13:40:11.255Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> >
> > cygnus_1 | java.lang.NullPointerException
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistRawAggregation(NGSICartoDBSink.java:625)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistBatch(NGSICartoDBSink.java:417)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:558)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> >
> > cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> >
> > cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> >
> > cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> >
> > cygnus_1 | time=2017-11-23T13:40:16.256Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> >
> > cygnus_1 | java.lang.IllegalStateException: begin() called
> when transaction is OPEN!
> >
> > cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> >
> > cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> >
> > cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> >
> > cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> >
> > cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> >
> > cygnus_1 | time=2017-11-23T13:40:18.648Z | lvl=DEBUG |
> corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[126]
> : Checking file:/opt/apache-flume/conf/agent.conf for changes
> >
> > cygnus_1 | time=2017-11-23T13:40:18.654Z | lvl=WARN |
> corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=<init>
> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[49] : No
> grouping rules have been read. Details:
> /usr/cygnus/conf/grouping_rules.conf (No such file or directory)
> >
> > cygnus_1 | time=2017-11-23T13:40:21.257Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> >
> > cygnus_1 | java.lang.IllegalStateException: begin() called
> when transaction is OPEN!
> >
> > cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> >
> > cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> >
> > cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> >
> > cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> >
> > cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> >
> > cygnus_1 | time=2017-11-23T13:40:26.258Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
> >
> > cygnus_1 | java.lang.IllegalStateException: begin() called
> when transaction is OPEN!
> >
> > cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> >
> > cygnus_1 | at
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
> >
> > cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
> >
> > cygnus_1 | at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> >
> > cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> >
> > cygnus_1 | at java.lang.Thread.run(Thread.java:748)
> >
> >
> >
> > Are you able to help me figure out why I can't send this information to
> carto?
> >
> >
> >
> > --
> > Peter Theill
> > CTO
> > +45 6171 5096 <61%2071%2050%2096> <tel:61%2071%2050%2096>
> > peter at familiohq.com <mailto:peter at familiohq.com>
> >
> > Familio
> > Artillerivej 86
> > DK-2300 Copenhagen S
> > https://familiohq.com/ <https://familiohq.com/><carto-cygnus-log.txt>
>
>
>
> ------------------------
> Issue id: HELP-12977
> Description:
> I've setup Orion and Cygnus so whenever a new entity is created in Orion
> it's transferring its data to Carto using the Cygnus CartoDbSink.
>
> However I'm having some issues with this implementation. I'm getting the
> data correctly from Orion to Cygnus but when the sink tries to forward
> those data to Carto I'm getting this error message (also attached for
> better formatting) when trying to persist it in Carto
>
> cygnus_1 | time=2017-11-23T13:40:11.175Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[294] :
> [NGSIRestHandler] Starting internal transaction
> (efab5d17-12c3-4716-8b1b-15352ec6ae18)
>
> cygnus_1 | time=2017-11-23T13:40:11.176Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[312] :
> [NGSIRestHandler] Received data
>
> ({"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"type":"Event","isPattern":"false","id":"getif","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
> -148.99808"}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]})
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[320] :
> [NGSIRestHandler] Parsed NotifyContextRequest:
>
> {"subscriptionId":"5a16cd57fde41d458740a0b4","originator":"localhost","contextResponses":[{"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
>
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[345] :
> [NGSIRestHandler] NGSI event created for ContextElementResponse:
>
> {"contextElement":{"id":"getif","type":"Event","isPattern":"false","attributes":[{"name":"location","type":"geo:point","value":"79.00555,
>
> -148.99808","metadatas":[]}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[350] :
> [NGSIRestHandler] Header added to NGSI event (fiware-service: default)
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[353] :
> [NGSIRestHandler] Header added to NGSI event (fiware-servicepath: /)
>
> cygnus_1 | time=2017-11-23T13:40:11.239Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[356] :
> [NGSIRestHandler] Header added to NGSI event (fiware-correlator:
> d3be117c-d053-11e7-ad11-0242ac120009)
>
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[359] :
> [NGSIRestHandler] Header added to NGSI event (transaction-id:
> efab5d17-12c3-4716-8b1b-15352ec6ae18)
>
> cygnus_1 | time=2017-11-23T13:40:11.240Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=getEvents |
> msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[385] :
> [NGSIRestHandler] NGSI events put in the channel, ids=1516398619
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[84] :
> [gi] Event intercepted, id=1516398619
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[117] :
> [gi] Adding flume event header (notified-entity: getif_Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[120] :
> [gi] Adding flume event header (grouped-entity: getif_Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[123] :
> [gi] Adding flume event header (grouped-servicepath: /)
>
> cygnus_1 | time=2017-11-23T13:40:11.241Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=intercept |
> msg=com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor[127] :
> [gi] Event put in the channel, id=1516398619
>
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
> corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
> subsrv=pending | comp=Orion |
> op=httpRequestSend.cpp[620]:httpRequestSendWithCurl | msg=Notification
> Successfully Sent to http://staging.familiohq.com:5050/notify
>
> orion_1 | time=Thursday 23 Nov 13:40:11 2017.243Z | lvl=INFO |
> corr=N/A | trans=1511444367-204-00000000002 | from=pending | srv=pending |
> subsrv=pending | comp=Orion | op=logMsg.h[1916]:lmTransactionEnd |
> msg=Transaction ended
>
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=processNewBatches |
> msg=com.telefonica.iot.cygnus.sinks.NGSISink[554] : Batch completed
>
> cygnus_1 | time=2017-11-23T13:40:11.244Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistBatch |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[381] : [cartodb-sink]
> Processing sub-batch regarding the default_/_getif_Event destination
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[570]
> : [cartodb-sink] Processing context element (id=getif, type=Event)
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=DEBUG |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=aggregate |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink$CartoDBAggregator[592]
> : [cartodb-sink] Processing context attribute (name=location,
> type=geo:point)
>
> cygnus_1 | time=2017-11-23T13:40:11.254Z | lvl=INFO |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=persistRawAggregation |
> msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[616] : [cartodb-sink]
> Persisting data at NGSICartoDBSink. Schema (default), Table
> (x002fxffffgetifxffffx0045vent), Data
>
> (('2017-11-23T13:40:11.241Z','/','getif','Event',ST_SetSRID(ST_MakePoint(79.00555,-148.99808),
> 4326)))
>
> cygnus_1 | time=2017-11-23T13:40:11.255Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-*15352ec6ae18
> | srv=default | subsrv=/ | comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.*
>
> *cygnus_1 | java.lang.NullPointerException*
>
> *cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistRawAggregation(NGSICartoDBSink.java:625)*
>
> *cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSICartoDBSink.persistBatch(NGSICartoDBSink.java:417)*
>
> *cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:558)*
>
> *cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)*
>
> *cygnus_1 | at
>
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)*
>
> *cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)*
>
> *cygnus_1 | at java.lang.Thread.run(Thread.java:748)*
>
> cygnus_1 | time=2017-11-23T13:40:16.256Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
>
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at
>
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at
>
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
> cygnus_1 | time=2017-11-23T13:40:18.648Z | lvl=DEBUG | corr=N/A
> | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=run |
>
> msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[126]
> : Checking file:/opt/apache-flume/conf/agent.conf for changes
>
> cygnus_1 | time=2017-11-23T13:40:18.654Z | lvl=WARN | corr=N/A
> | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=<init> |
> msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[49] : No
> grouping rules have been read. Details:
> /usr/cygnus/conf/grouping_rules.conf (No such file or directory)
>
> cygnus_1 | time=2017-11-23T13:40:21.257Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
>
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at
>
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at
>
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
> cygnus_1 | time=2017-11-23T13:40:26.258Z | lvl=ERROR |
> corr=d3be117c-d053-11e7-ad11-0242ac120009 |
> trans=efab5d17-12c3-4716-8b1b-15352ec6ae18 | srv=default | subsrv=/ |
> comp=cygnus-ngsi | op=run |
> msg=org.apache.flume.SinkRunner$PollingRunner[160] : Unable to deliver
> event. Exception follows.
>
> cygnus_1 | java.lang.IllegalStateException: begin() called when
> transaction is OPEN!
>
> cygnus_1 | at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>
> cygnus_1 | at
>
> org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
>
> cygnus_1 | at
>
> com.telefonica.iot.cygnus.sinks.NGSISink.processNewBatches(NGSISink.java:482)
>
> cygnus_1 | at
> com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:370)
>
> cygnus_1 | at
>
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
> cygnus_1 | at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
> cygnus_1 | at java.lang.Thread.run(Thread.java:748)
>
>
> Are you able to help me figure out why I can't send this information to
> carto?
>
>
> --
> *Peter Theill*
> CTO
> +45 6171 5096 <61%2071%2050%2096> <61%2071%2050%2096>
> peter at familiohq.com
>
> *Familio*
> Artillerivej 86
> DK-2300 Copenhagen S
> https://familiohq.com/
>
>
> __________________________________________________________________________________________
>
> You can get more information about our cookies and privacy policies on the
> following links:
> -
> http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Privacy_Policy
> -
> http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Cookies_Policy_FIWARE
>
> Fiware-tech-help mailing list
> Fiware-tech-help at lists.fiware.org
> https://lists.fiware.org/listinfo/fiware-tech-help
>
>
> [Created via e-mail received from: Peter Theill <peter at familiohq.com>]
>
> FIWARE Chapter:
> FIWARE GEri:
>
> Status: In Progress
>
> ---------------------
> This email was generated by FIWARE JIRA following an email received into
> the Main Help Desk.
>
--
*Peter Theill*
CTO
+45 6171 5096 <61%2071%2050%2096>
peter at familiohq.com
*Familio*
Artillerivej 86
DK-2300 Copenhagen S
https://familiohq.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-tech-help/attachments/20171220/79f16815/attachment-0001.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy