One more thing, Francisco, how can I find the Cygnus version?
Best regards
2015-09-14 18:22 GMT+02:00 Carlos Corrales <carlos at secmotic.com>:
> Hi everyone,
>
> Thanks for your responses, we have been testing the whole day and trying
> to find out the issue. It is not a problem with neither the password, nor
> the oauth2 token. We are able to access using ssh user/password and the
> curl command proposed by Pasquale with Oauth2 token. This is what I
> received using the Oauth2 token curl proposed by Pasquale
>
> curl -X GET "
> http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado"
> -H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
>
> {"FileStatuses":{"FileStatus":[{"pathSuffix":".staging","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"700","accessTime":0,"modificationTime":1437059783102,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433847949370,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras2","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1434651036673,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras_fiware","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1434710282953,"blockSize":0,"replication":0},{"pathSuffix":"input","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433840978953,"blockSize":0,"replication":0},{"pathSuffix":"input_data","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433841088243,"blockSize":0,"replication":0},{"pathSuffix":"org42","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1435749061128,"blockSize":0,"replication":0},{"pathSuffix":"prueba","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433842740482,"blockSize":0,"replication":0}]}}
>
>
> Everything seems to be correct. However, using the same token and updating
> ORION, I receive the following error in Cygnus:
>
> time=2015-09-14T18:10:44.419CEST | lvl=ERROR |
> trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] :
> Persistence error (The org42/idelgado_tablanull directory could not be
> created in HDFS. HttpFS response: 401 Unauthorized)
> time=2015-09-14T18:10:44.419CEST | lvl=WARN |
> trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event
> TTL has expired, it is no more re-injected in the channel (id=1444248959,
> ttl=0)
>
> However, if I try to access the directory shown in the error, I'm able to
> do so
>
> ssh idelgado at cosmos.lab.fi-ware.org
> hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
> Found 1 items
> -rw-r----- 3 idelgado idelgado 1416 2015-07-01 13:11
> /user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt
>
> My cygnus.conf file content is:
>
> #
> ============================================
>
> # OrionHDFSSink
> configuration
>
> # channel name from where to read notification
> events
> cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
> # sink class, must not be
> changed
>
> cygnusagent.sinks.hdfs-sink.type =
> es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
> # Comma-separated list of FQDN/IP address regarding the Cosmos Namenode
> endpoints
> cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
> # port of the Cosmos service listening for persistence operations; 14000
> for httpfs, 50070 for webhdfs and free c\
> hoice for
> inifinty
>
> cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
> # default username allowed to write in
> HDFS
> cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
> # default password for the default
> username
>
> #cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX
>
> cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i
> # HDFS backend type (webhdfs, httpfs or
> infinity)
> cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
> # how the attributes are stored, either per row either per column (row,
> column)
> cygnusagent.sinks.hdfs-sink.attr_persistence = column
> # prefix for the database and table names, empty if no prefix is
> desired
> cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
> # Hive FQDN/IP address of the Hive
> server
>
> cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
> # Hive port for Hive external table
> provisioning
>
> cygnusagent.sinks.hdfs-sink.hive_port = 10000
>
> Could you please help us?
>
> Thanks in advance!
>
>
>
>
>
> 2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <
> manuel.escrichevicente at telefonica.com>:
>
>> In this case I see both Pasquale (Coach) and Francisco (GE Owner) have
>> reacted to the request.
>>
>> In the normal flow GE owners enter into action upon request from Coaches.
>>
>> Kind regards,
>>
>> Manuel
>>
>>
>>
>> *From:* JUAN JOSE HIERRO SUREDA
>> *Sent:* lunes, 14 de septiembre de 2015 0:37
>> *To:* Carlos Corrales
>> *Cc:* fiware-finodex-coaching at lists.fi-ware.org; jose at secmotic.com;
>> FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
>> *Subject:* Re: [Fiware-finodex-coaching] Problems with COSMOS persistance
>>
>>
>>
>>
>> I guess that the FINODEX coach will forward the ticket to the FIWARE GE
>> owner (or eventually, if he hadn't know where to redirect it, he would
>> redirect the original question to the fiware-tech-help mailing list)
>>
>> Point is that one way or the other, it would arrive at the inbox of the
>> FIWARE GE owner who has to take care of the ticket.
>>
>> You may have sent the ticket directly to the fiware-tech-help mailing
>> list ... however, I guess that the FINODEX coach team wanted to be informed
>> to keep track of the evolution.
>>
>> Manuel, in copy, is the master of all the helpdesk processes and he can
>> clarify what the actual workflow is.
>>
>> Best regards,
>>
>> -- Juanjo
>>
>> ______________________________________________________
>>
>>
>>
>> Coordinator and Chief Architect, FIWARE platform
>>
>> CTO Industrial IoT, Telefónica
>>
>>
>>
>> email: juanjose.hierro at telefonica.com
>>
>> twitter: @JuanjoHierro
>>
>>
>>
>> You can follow FIWARE at:
>>
>> website: http://www.fiware.org
>>
>> twitter: @FIWARE
>>
>> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242
>>
>> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932
>>
>> On 13/09/15 16:36, Carlos Corrales wrote:
>>
>> Hi Juanjo,
>>
>> Thanks for you feedback, we sent it today because we wanted you to answer
>> it tomorrow morning :).
>>
>>
>>
>> As you can see, a ticket has been created in Jira (
>> https://jira.fiware.org/browse/HELC-1025). I used the
>> fiware-finodex-coaching list, because we are a FINODEX project and they
>> told us to do so. Should we create another ticket in the fiware-tech-help
>> list also?
>>
>> Best regards
>>
>>
>>
>> 2015-09-13 16:30 GMT+02:00 Juanjo Hierro <juanjose.hierro at telefonica.com
>> >:
>>
>> Hi,
>>
>> I copy Francisco (Paco) Romero who should be able to provide some help
>> on this. I guess he won't be able to work on the matter until tomorrow
>> since support is not granted out of office working hours.
>>
>> Nevertheless, I believe that it is rather important that you open an
>> issue so that a ticket on our help-desk system is created and then
>> follow-up of support is feasible.
>>
>> You may create a ticket in our helpdesk simply by sending an email to
>> either fiware-lab-help (if it happens to be an issue which is mostly
>> related to FIWARE Lab or the global COSMOS instance setup on the FIWARE
>> Lab) or fiware-tech-help (if it happens to be an issue mostly related with
>> the technology, i.e., with COSMOS in general and not the particular
>> instance of COSMOS running on the FIWARE Lab).
>>
>> Best regards,
>>
>> -- Juanjo
>>
>> ______________________________________________________
>>
>>
>>
>> Coordinator and Chief Architect, FIWARE platform
>>
>> CTO Industrial IoT, Telefónica
>>
>>
>>
>> email: juanjose.hierro at telefonica.com
>>
>> twitter: @JuanjoHierro
>>
>>
>>
>> You can follow FIWARE at:
>>
>> website: http://www.fiware.org
>>
>> twitter: @FIWARE
>>
>> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242
>>
>> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932
>>
>> On 13/09/15 15:20, Carlos Corrales wrote:
>>
>> Hi everyone,
>>
>> I'm Carlos Corrales, part of the development team at SAPIN (FINODEX012).
>> We have been working hard in the application and now that it's finished and
>> we are doing tests with final users, we are improving our code. A couple of
>> months ago we were able to write data to our HDFS in the COSMOS Fiware
>> machine (cosmos.lab.fi-ware.org) with our user (idelgado). As we are not
>> using Cosmos for production right now (it will be used in the future), we
>> left it aside. Now that we are checking that everything works as expected,
>> we can see that our Cygnus connector is not able to write properly in HDFS.
>> We haven't changed anything in the configuration, but we receive the
>> following error when trying to write (flume.log file)
>>
>> time=2015-09-13T15:11:36.744CEST | lvl=INFO |
>> trans=1442149853-658-0000000000 | function=persist | comp=Cygnus |
>> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink[367] :
>> [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file
>> (idelgado_SAPINnull), Data
>> ({"recvTime":"2015-09-13T15:11:00.413","age":"25",
>> "age_md":[],"exercisetype":"Light", "exercisetype_md":[],"hr":"36",
>> "hr_md":[],"id_session":"1234", "id_session_md":[]})
>> time=2015-09-13T15:11:36.803CEST | lvl=ERROR |
>> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
>> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] :
>> Persistence error (The unknownt/idelgado_SAPINnull directory could not be
>> created in HDFS. HttpFS response: 401 Unauthorized)
>> time=2015-09-13T15:11:36.804CEST | lvl=INFO |
>> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
>> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[150] : An event
>> was put again in the channel (id=1444248959, ttl=0)
>> time=2015-09-13T15:11:36.804CEST | lvl=INFO |
>> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
>> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[175] : Finishing
>> transaction (1442149853-658-0000000000)
>>
>> As you can see, the error is 401 Unauthorized. However, we can access the
>> Cosmos machine using SSH, and the user/password in the cygnus.conf file. We
>> also checked that the IP address of the Cosmos machine hasn't changed.
>>
>> Could you please help us? We don't know what else to check...
>>
>> Best regards
>>
>>
>> --
>>
>> Carlos Corrales
>>
>> Cofounder Secmotic Innovation S.L.
>>
>> http://www.secmotic.com
>>
>>
>>
>> _______________________________________________
>>
>> Fiware-finodex-coaching mailing list
>>
>> Fiware-finodex-coaching at lists.fiware.org
>>
>> https://lists.fiware.org/listinfo/fiware-finodex-coaching-new
>>
>>
>> ------------------------------
>>
>>
>> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
>> puede contener información privilegiada o confidencial y es para uso
>> exclusivo de la persona o entidad de destino. Si no es usted. el
>> destinatario indicado, queda notificado de que la lectura, utilización,
>> divulgación y/o copia sin autorización puede estar prohibida en virtud de
>> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
>> que nos lo comunique inmediatamente por esta misma vía y proceda a su
>> destrucción.
>>
>> The information contained in this transmission is privileged and
>> confidential information intended only for the use of the individual or
>> entity named above. If the reader of this message is not the intended
>> recipient, you are hereby notified that any dissemination, distribution or
>> copying of this communication is strictly prohibited. If you have received
>> this transmission in error, do not read it. Please immediately reply to the
>> sender that you have received this communication in error and then delete
>> it.
>>
>> Esta mensagem e seus anexos se dirigem exclusivamente ao seu
>> destinatário, pode conter informação privilegiada ou confidencial e é para
>> uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o
>> destinatário indicado, fica notificado de que a leitura, utilização,
>> divulgação e/ou cópia sem autorização pode estar proibida em virtude da
>> legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos
>> o comunique imediatamente por esta mesma via e proceda a sua destruição
>>
>>
>>
>>
>> --
>>
>> Carlos Corrales
>>
>> Cofounder Secmotic Innovation S.L.
>>
>> http://www.secmotic.com
>>
>>
>>
>> ------------------------------
>>
>> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
>> puede contener información privilegiada o confidencial y es para uso
>> exclusivo de la persona o entidad de destino. Si no es usted. el
>> destinatario indicado, queda notificado de que la lectura, utilización,
>> divulgación y/o copia sin autorización puede estar prohibida en virtud de
>> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
>> que nos lo comunique inmediatamente por esta misma vía y proceda a su
>> destrucción.
>>
>> The information contained in this transmission is privileged and
>> confidential information intended only for the use of the individual or
>> entity named above. If the reader of this message is not the intended
>> recipient, you are hereby notified that any dissemination, distribution or
>> copying of this communication is strictly prohibited. If you have received
>> this transmission in error, do not read it. Please immediately reply to the
>> sender that you have received this communication in error and then delete
>> it.
>>
>> Esta mensagem e seus anexos se dirigem exclusivamente ao seu
>> destinatário, pode conter informação privilegiada ou confidencial e é para
>> uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o
>> destinatário indicado, fica notificado de que a leitura, utilização,
>> divulgação e/ou cópia sem autorização pode estar proibida em virtude da
>> legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos
>> o comunique imediatamente por esta mesma via e proceda a sua destruição
>>
>
>
>
> --
> Carlos Corrales
> Cofounder Secmotic Innovation S.L.
> http://www.secmotic.com
>
>
--
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-finodex-coaching/attachments/20150914/7ab2764b/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy