Hi everyone,
Thanks for your responses, we have been testing the whole day and trying to
find out the issue. It is not a problem with neither the password, nor the
oauth2 token. We are able to access using ssh user/password and the curl
command proposed by Pasquale with Oauth2 token. This is what I received
using the Oauth2 token curl proposed by Pasquale
curl -X GET "
http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/idelgado?op=liststatus&user.name=idelgado"
-H "X-Auth-Token: rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i"
{"FileStatuses":{"FileStatus":[{"pathSuffix":".staging","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"700","accessTime":0,"modificationTime":1437059783102,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433847949370,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras2","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1434651036673,"blockSize":0,"replication":0},{"pathSuffix":"cuentapalabras_fiware","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1434710282953,"blockSize":0,"replication":0},{"pathSuffix":"input","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433840978953,"blockSize":0,"replication":0},{"pathSuffix":"input_data","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433841088243,"blockSize":0,"replication":0},{"pathSuffix":"org42","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1435749061128,"blockSize":0,"replication":0},{"pathSuffix":"prueba","type":"DIRECTORY","length":0,"owner":"idelgado","group":"idelgado","permission":"740","accessTime":0,"modificationTime":1433842740482,"blockSize":0,"replication":0}]}}
Everything seems to be correct. However, using the same token and updating
ORION, I receive the following error in Cygnus:
time=2015-09-14T18:10:44.419CEST | lvl=ERROR |
trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] :
Persistence error (The org42/idelgado_tablanull directory could not be
created in HDFS. HttpFS response: 401 Unauthorized)
time=2015-09-14T18:10:44.419CEST | lvl=WARN |
trans=1442246992-282-0000000000 | function=process | comp=Cygnus |
msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[153] : The event
TTL has expired, it is no more re-injected in the channel (id=1444248959,
ttl=0)
However, if I try to access the directory shown in the error, I'm able to
do so
ssh idelgado at cosmos.lab.fi-ware.org
hadoop fs -ls /user/idelgado/org42/idelgado_tablanull
Found 1 items
-rw-r----- 3 idelgado idelgado 1416 2015-07-01 13:11
/user/idelgado/org42/idelgado_tablanull/idelgado_tablanull.txt
My cygnus.conf file content is:
#
============================================
# OrionHDFSSink
configuration
# channel name from where to read notification
events
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
# sink class, must not be
changed
cygnusagent.sinks.hdfs-sink.type =
es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
# Comma-separated list of FQDN/IP address regarding the Cosmos Namenode
endpoints
cygnusagent.sinks.hdfs-sink.cosmos_host = 130.206.80.46
# port of the Cosmos service listening for persistence operations; 14000
for httpfs, 50070 for webhdfs and free c\
hoice for
inifinty
cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
# default username allowed to write in
HDFS
cygnusagent.sinks.hdfs-sink.cosmos_default_username = idelgado
# default password for the default
username
#cygnusagent.sinks.hdfs-sink.cosmos_default_password = XXXXXXX
cygnusagent.sinks.hdfs-sink.oauth2_token = rxcEgpDHqHAqhWnwUB93JlAVJ7uz9i
# HDFS backend type (webhdfs, httpfs or
infinity)
cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
# how the attributes are stored, either per row either per column (row,
column)
cygnusagent.sinks.hdfs-sink.attr_persistence = column
# prefix for the database and table names, empty if no prefix is
desired
cygnusagent.sinks.hdfs-sink.naming_prefix = idelgado_tabla
# Hive FQDN/IP address of the Hive
server
cygnusagent.sinks.hdfs-sink.hive_host = 130.206.80.46
# Hive port for Hive external table
provisioning
cygnusagent.sinks.hdfs-sink.hive_port = 10000
Could you please help us?
Thanks in advance!
2015-09-14 10:26 GMT+02:00 MANUEL ESCRICHE VICENTE <
manuel.escrichevicente at telefonica.com>:
> In this case I see both Pasquale (Coach) and Francisco (GE Owner) have
> reacted to the request.
>
> In the normal flow GE owners enter into action upon request from Coaches.
>
> Kind regards,
>
> Manuel
>
>
>
> *From:* JUAN JOSE HIERRO SUREDA
> *Sent:* lunes, 14 de septiembre de 2015 0:37
> *To:* Carlos Corrales
> *Cc:* fiware-finodex-coaching at lists.fi-ware.org; jose at secmotic.com;
> FRANCISCO ROMERO BUENO; MANUEL ESCRICHE VICENTE; JUAN JOSE HIERRO SUREDA
> *Subject:* Re: [Fiware-finodex-coaching] Problems with COSMOS persistance
>
>
>
>
> I guess that the FINODEX coach will forward the ticket to the FIWARE GE
> owner (or eventually, if he hadn't know where to redirect it, he would
> redirect the original question to the fiware-tech-help mailing list)
>
> Point is that one way or the other, it would arrive at the inbox of the
> FIWARE GE owner who has to take care of the ticket.
>
> You may have sent the ticket directly to the fiware-tech-help mailing
> list ... however, I guess that the FINODEX coach team wanted to be informed
> to keep track of the evolution.
>
> Manuel, in copy, is the master of all the helpdesk processes and he can
> clarify what the actual workflow is.
>
> Best regards,
>
> -- Juanjo
>
> ______________________________________________________
>
>
>
> Coordinator and Chief Architect, FIWARE platform
>
> CTO Industrial IoT, Telefónica
>
>
>
> email: juanjose.hierro at telefonica.com
>
> twitter: @JuanjoHierro
>
>
>
> You can follow FIWARE at:
>
> website: http://www.fiware.org
>
> twitter: @FIWARE
>
> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242
>
> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932
>
> On 13/09/15 16:36, Carlos Corrales wrote:
>
> Hi Juanjo,
>
> Thanks for you feedback, we sent it today because we wanted you to answer
> it tomorrow morning :).
>
>
>
> As you can see, a ticket has been created in Jira (
> https://jira.fiware.org/browse/HELC-1025). I used the
> fiware-finodex-coaching list, because we are a FINODEX project and they
> told us to do so. Should we create another ticket in the fiware-tech-help
> list also?
>
> Best regards
>
>
>
> 2015-09-13 16:30 GMT+02:00 Juanjo Hierro <juanjose.hierro at telefonica.com>:
>
> Hi,
>
> I copy Francisco (Paco) Romero who should be able to provide some help
> on this. I guess he won't be able to work on the matter until tomorrow
> since support is not granted out of office working hours.
>
> Nevertheless, I believe that it is rather important that you open an
> issue so that a ticket on our help-desk system is created and then
> follow-up of support is feasible.
>
> You may create a ticket in our helpdesk simply by sending an email to
> either fiware-lab-help (if it happens to be an issue which is mostly
> related to FIWARE Lab or the global COSMOS instance setup on the FIWARE
> Lab) or fiware-tech-help (if it happens to be an issue mostly related with
> the technology, i.e., with COSMOS in general and not the particular
> instance of COSMOS running on the FIWARE Lab).
>
> Best regards,
>
> -- Juanjo
>
> ______________________________________________________
>
>
>
> Coordinator and Chief Architect, FIWARE platform
>
> CTO Industrial IoT, Telefónica
>
>
>
> email: juanjose.hierro at telefonica.com
>
> twitter: @JuanjoHierro
>
>
>
> You can follow FIWARE at:
>
> website: http://www.fiware.org
>
> twitter: @FIWARE
>
> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242
>
> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932
>
> On 13/09/15 15:20, Carlos Corrales wrote:
>
> Hi everyone,
>
> I'm Carlos Corrales, part of the development team at SAPIN (FINODEX012).
> We have been working hard in the application and now that it's finished and
> we are doing tests with final users, we are improving our code. A couple of
> months ago we were able to write data to our HDFS in the COSMOS Fiware
> machine (cosmos.lab.fi-ware.org) with our user (idelgado). As we are not
> using Cosmos for production right now (it will be used in the future), we
> left it aside. Now that we are checking that everything works as expected,
> we can see that our Cygnus connector is not able to write properly in HDFS.
> We haven't changed anything in the configuration, but we receive the
> following error when trying to write (flume.log file)
>
> time=2015-09-13T15:11:36.744CEST | lvl=INFO |
> trans=1442149853-658-0000000000 | function=persist | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink[367] :
> [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file
> (idelgado_SAPINnull), Data
> ({"recvTime":"2015-09-13T15:11:00.413","age":"25",
> "age_md":[],"exercisetype":"Light", "exercisetype_md":[],"hr":"36",
> "hr_md":[],"id_session":"1234", "id_session_md":[]})
> time=2015-09-13T15:11:36.803CEST | lvl=ERROR |
> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[140] :
> Persistence error (The unknownt/idelgado_SAPINnull directory could not be
> created in HDFS. HttpFS response: 401 Unauthorized)
> time=2015-09-13T15:11:36.804CEST | lvl=INFO |
> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[150] : An event
> was put again in the channel (id=1444248959, ttl=0)
> time=2015-09-13T15:11:36.804CEST | lvl=INFO |
> trans=1442149853-658-0000000000 | function=process | comp=Cygnus |
> msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[175] : Finishing
> transaction (1442149853-658-0000000000)
>
> As you can see, the error is 401 Unauthorized. However, we can access the
> Cosmos machine using SSH, and the user/password in the cygnus.conf file. We
> also checked that the IP address of the Cosmos machine hasn't changed.
>
> Could you please help us? We don't know what else to check...
>
> Best regards
>
>
> --
>
> Carlos Corrales
>
> Cofounder Secmotic Innovation S.L.
>
> http://www.secmotic.com
>
>
>
> _______________________________________________
>
> Fiware-finodex-coaching mailing list
>
> Fiware-finodex-coaching at lists.fiware.org
>
> https://lists.fiware.org/listinfo/fiware-finodex-coaching-new
>
>
> ------------------------------
>
>
> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
> puede contener información privilegiada o confidencial y es para uso
> exclusivo de la persona o entidad de destino. Si no es usted. el
> destinatario indicado, queda notificado de que la lectura, utilización,
> divulgación y/o copia sin autorización puede estar prohibida en virtud de
> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
> que nos lo comunique inmediatamente por esta misma vía y proceda a su
> destrucción.
>
> The information contained in this transmission is privileged and
> confidential information intended only for the use of the individual or
> entity named above. If the reader of this message is not the intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. If you have received
> this transmission in error, do not read it. Please immediately reply to the
> sender that you have received this communication in error and then delete
> it.
>
> Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário,
> pode conter informação privilegiada ou confidencial e é para uso exclusivo
> da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário
> indicado, fica notificado de que a leitura, utilização, divulgação e/ou
> cópia sem autorização pode estar proibida em virtude da legislação vigente.
> Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique
> imediatamente por esta mesma via e proceda a sua destruição
>
>
>
>
> --
>
> Carlos Corrales
>
> Cofounder Secmotic Innovation S.L.
>
> http://www.secmotic.com
>
>
>
> ------------------------------
>
> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
> puede contener información privilegiada o confidencial y es para uso
> exclusivo de la persona o entidad de destino. Si no es usted. el
> destinatario indicado, queda notificado de que la lectura, utilización,
> divulgación y/o copia sin autorización puede estar prohibida en virtud de
> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
> que nos lo comunique inmediatamente por esta misma vía y proceda a su
> destrucción.
>
> The information contained in this transmission is privileged and
> confidential information intended only for the use of the individual or
> entity named above. If the reader of this message is not the intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. If you have received
> this transmission in error, do not read it. Please immediately reply to the
> sender that you have received this communication in error and then delete
> it.
>
> Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário,
> pode conter informação privilegiada ou confidencial e é para uso exclusivo
> da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário
> indicado, fica notificado de que a leitura, utilização, divulgação e/ou
> cópia sem autorização pode estar proibida em virtude da legislação vigente.
> Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique
> imediatamente por esta mesma via e proceda a sua destruição
>
--
Carlos Corrales
Cofounder Secmotic Innovation S.L.
http://www.secmotic.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-finodex-coaching/attachments/20150914/da68f8d7/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy