Hi Francisco, sorry for the delay in the response but in these days i'm not going to work to finish my thesis, I graduate in March. I have received an answer on StackOverflow about the problem of the hive tables in Cygnus. This is the link of the response: http://stackoverflow.com/questions/27918545/automatic-creation-of-the-hive-tables-on-cosmos?noredirect=1#comment45506173_27918545 It should be useful for you. Kind regagrds, Antonio 2015-02-06 8:57 GMT+01:00 FRANCISCO ROMERO BUENO < francisco.romerobueno at telefonica.com>: > Hi Antonio, > > > > Could you perform the following test? Start Cygnus as always, and then create a new fresh entity at Orion and a subscription regarding this new entity. Finally, send me the entire log, please. The best log would be that created in debug mode, so start Cygnus with -Dflume.root.logger=DEBUG,LOGFILE > > > > Thanks! > > Francisco > > > > *De:* antonio caristia [mailto:a.caristia at gmail.com] > *Enviado el:* jueves, 29 de enero de 2015 17:50 > > *Para:* FRANCISCO ROMERO BUENO > *Asunto:* Re: Automatic creation of the hive tables on cosmos > > > > Hi Francisco, > > I have used a server in python that receives the subscriptions from orion > and creates the tables > > using hive api. > > At this moment i use this workaround, problably i'm not able to configure > correctly cygnus. > > If you have some ideas to help me, you can send me an email to this > address :) > > Thanks for your reply. > > > > Best regards, > > > > Antonio > > > > > > 2015-01-29 15:43 GMT+01:00 FRANCISCO ROMERO BUENO < > francisco.romerobueno at telefonica.com>: > > Hi Antonio, > > > > Sorry for the delay in continuing the conversation. > > > > I’ve seen that these tables, associated to your username, have been > created: > > > > gioakbombaci_pm10_142245780725_observations > > gioakbombaci_pm10_142245909633_observations > > gioakbombaci_pm10_142245960442_observations > > gioakbombaci_pm10_142246020579_observations > > gioakbombaci_pm10_142246134747_observations > > gioakbombaci_pm10_142246257881_observations > > gioakbombaci_pm10_142246363018_observations > > gioakbombaci_pm10_142246475644_observations > > gioakbombaci_pm10_14224658677_observations > > gioakbombaci_pm10_142246691926_observations > > gioakbombaci_pm10_14224683607_observations > > gioakbombaci_pressure_142245780725_observations > > gioakbombaci_pressure_142245909633_observations > > gioakbombaci_pressure_142245960442_observations > > gioakbombaci_pressure_142246020579_observations > > gioakbombaci_pressure_142246134747_observations > > gioakbombaci_pressure_142246257881_observations > > gioakbombaci_pressure_142246363018_observations > > gioakbombaci_pressure_142246475644_observations > > gioakbombaci_pressure_14224658677_observations > > gioakbombaci_pressure_142246691926_observations > > gioakbombaci_pressure_14224683607_observations > > gioakbombaci_temperature_142245780725_observations > > gioakbombaci_temperature_142245909633_observations > > gioakbombaci_temperature_142245960442_observations > > gioakbombaci_temperature_142246020579_observations > > gioakbombaci_temperature_142246134747_observations > > gioakbombaci_temperature_142246257881_observations > > gioakbombaci_temperature_142246363018_observations > > gioakbombaci_temperature_142246475644_observations > > gioakbombaci_temperature_14224658677_observations > > gioakbombaci_temperature_142246691926_observations > > gioakbombaci_temperature_14224683607_observations > > > > Did you created them manually or were they created by Cygnus? > > > > Regards, > > Francisco > > > > *De:* antonio caristia [mailto:a.caristia at gmail.com] > *Enviado el:* lunes, 19 de enero de 2015 11:13 > > > *Para:* FRANCISCO ROMERO BUENO > *Asunto:* Re: Automatic creation of the hive tables on cosmos > > > > Hi Francisco, > > > > have you any news for my issue? > > If it is not possible to resolve at this moment, can you give me some info > how to create > > the tables manually or with bash/python scripts? > > > > I can't test my widgets without data to query. > > > > Thanks in advance, best regards. > > > > Antonio > > > > 2015-01-15 16:57 GMT+01:00 antonio caristia <a.caristia at gmail.com>: > > Hi Francisco, > > > > I have modified cygnus.conf and created matching_table.conf on > [MY_CYGNUS_HOME]/conf. > > I have no more the error in flume.log but cygnys does not create the > tables yet. > > I have attached my flume.log and cygnus.conf. > > > > Regards, > > > > Antonio > > > > 2015-01-15 12:08 GMT+01:00 FRANCISCO ROMERO BUENO < > francisco.romerobueno at telefonica.com>: > > Hi Antonio, > > > > A couple of questions: > > > > · Where do you want to persist your context data? Only > Cosmos/HDFS for Hive querying? Because you have also activated the MySQL > and CKAN sinks, which is leading to lot of errors due these sinks are not > properly configured. Just edit these lines: > > o cygnusagent.sinks = hdfs-sink # i.e. remove mysql-sink ckan-sink > > o cygnusagent.channels = hdfs-channel @ i.e. remove mysql-channel > ckan-channel > > o The OrionMySQLSink, OrionCKANSink, ckan-channel and mysql-channel > configuration parts may be removed as well. > > · Do you have such /usr/cygnus/conf/matching_table.conf file? > This is the default location for the destination extractor matching rules, > when installing through the RPM. How did you installed Cygnus? If you > installed from sources, just create an empty matching_table.conf file under > [YOUR_CYGNUS_HOME]/conf folder. > > > > I’m not sure if these issues are related or not to the Hive table problem, > but fixing them will lead to a more clean log file J > > > > Regards, > > Francisco > > > > *De:* antonio caristia [mailto:a.caristia at gmail.com] > *Enviado el:* jueves, 15 de enero de 2015 10:50 > > > *Para:* FRANCISCO ROMERO BUENO > *Asunto:* Re: Automatic creation of the hive tables on cosmos > > > > Hi Francisco, > > > > I do not use any matching rules, in the log file i have this no such file > error. > > > > [root at orionold logs]# cat flume.log |grep table > > 09 Jan 2015 10:30:09,306 ERROR [lifecycleSupervisor-1-2] > (es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor.initialize:84) > - *Runtime error (File not found. > Details=/usr/cygnus/conf/matching_table.conf (No such file or directory))* > > 09 Jan 2015 11:04:15,009 INFO > [SinkRunner-PollingRunner-DefaultSinkProcessor] > (es.tid.fiware.fiwareconnectors.cygnus.backends.hdfs.HDFSBackend.provisionHiveTable:135) > - Creating Hive external table=gioakbombaci_def_serv_Taxi1_Taxi_column > > 13 Jan 2015 12:33:42,158 ERROR [lifecycleSupervisor-1-3] > (es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor.initialize:84) > - *Runtime error (File not found. > Details=/usr/cygnus/conf/matching_table.conf (No such file or directory))* > > 13 Jan 2015 12:36:18,190 ERROR [lifecycleSupervisor-1-2] > (es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor.initialize:84) > - *Runtime error (File not found. > Details=/usr/cygnus/conf/matching_table.conf (No such file or directory))* > > 13 Jan 2015 12:57:12,137 ERROR [lifecycleSupervisor-1-2] > (es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor.initialize:84) > - *Runtime error (File not found. > Details=/usr/cygnus/conf/matching_table.conf (No such file or directory))* > > > > > > This is the url > https://drive.google.com/file/d/0B5QWQPdKi7qvbzE5aVkyZi03NUU/view?usp=sharing on > my google drive where you can find flume.log > > > > The Entity is Taxi1, the type is Taxi. > > These are the attributes: > > 1. name = temperature, type = centigrade > 2. name = pressure, type = mmHg > 3. name = PM10, type = concentration > 4. name = taxiPosition, type = coords > 5. name = Service1, type = text > > Thanks for your help, best regards. > > > > Antonio > > > > 2015-01-15 9:27 GMT+01:00 FRANCISCO ROMERO BUENO < > francisco.romerobueno at telefonica.com>: > > Hi Antonio, > > > > The configuration seems to be OK. Can you send me the related logs? > > > > In addition, which is the name and type of the NGIS entity whose data is > being persisted through Cygnus? Are you using any matching rule at the > destination extractor matching table? > > > > Regards, > > Francisco > > > > > > *De:* antonio caristia [mailto:a.caristia at gmail.com] > *Enviado el:* martes, 13 de enero de 2015 16:21 > *Para:* FRANCISCO ROMERO BUENO > *Asunto:* Re: Automatic creation of the hive tables on cosmos > > > > Hi Francisco, > > thanks for your reply. > > I use cygnus 0.6, i have attached my cygnus.conf. > > > > Thanks for your help, best regards. > > > > Antonio > > > > 2015-01-13 13:13 GMT+01:00 FRANCISCO ROMERO BUENO < > francisco.romerobueno at telefonica.com>: > > Hi Antonio, > > > > Sorry for the delay, but I was on vacations J > > > > Can you send me the Cygnus configuration you are using? Which Cygnus > version indeed? > > > > Regards, > > Francisco > > > > *De:* antonio caristia [mailto:a.caristia at gmail.com] > *Enviado el:* viernes, 09 de enero de 2015 12:08 > *Para:* FRANCISCO ROMERO BUENO > *Asunto:* Automatic creation of the hive tables on cosmos > > > > Hi Francisco, > > I have an Orion context broker and i use cygnus to send data on my cosmos > account. > > Cygnus create correctly the txt files with the data on my cosmos account, > but it does not create > > the tables for hive. I would query my data through an hive client. > > Is there a configuration paramater to set in cygnus.conf to do that? > > How can i solve? > > > > Thanks in advance. > > Best regards, > > > > Antonio > > > ------------------------------ > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener información privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilización, > divulgación y/o copia sin autorización puede estar prohibida en virtud de > la legislación vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma vía y proceda a su > destrucción. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, > pode conter informação privilegiada ou confidencial e é para uso exclusivo > da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário > indicado, fica notificado de que a leitura, utilização, divulgação e/ou > cópia sem autorização pode estar proibida em virtude da legislação vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destruição > > > > > ------------------------------ > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener información privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilización, > divulgación y/o copia sin autorización puede estar prohibida en virtud de > la legislación vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma vía y proceda a su > destrucción. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, > pode conter informação privilegiada ou confidencial e é para uso exclusivo > da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário > indicado, fica notificado de que a leitura, utilização, divulgação e/ou > cópia sem autorização pode estar proibida em virtude da legislação vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destruição > > > > > ------------------------------ > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener información privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilización, > divulgación y/o copia sin autorización puede estar prohibida en virtud de > la legislación vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma vía y proceda a su > destrucción. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, > pode conter informação privilegiada ou confidencial e é para uso exclusivo > da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário > indicado, fica notificado de que a leitura, utilização, divulgação e/ou > cópia sem autorização pode estar proibida em virtude da legislação vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destruição > > > > > > > ------------------------------ > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener información privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilización, > divulgación y/o copia sin autorización puede estar prohibida en virtud de > la legislación vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma vía y proceda a su > destrucción. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, > pode conter informação privilegiada ou confidencial e é para uso exclusivo > da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário > indicado, fica notificado de que a leitura, utilização, divulgação e/ou > cópia sem autorização pode estar proibida em virtude da legislação vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destruição > > > > ------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener información privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilización, > divulgación y/o copia sin autorización puede estar prohibida en virtud de > la legislación vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma vía y proceda a su > destrucción. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, > pode conter informação privilegiada ou confidencial e é para uso exclusivo > da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário > indicado, fica notificado de que a leitura, utilização, divulgação e/ou > cópia sem autorização pode estar proibida em virtude da legislação vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destruição > -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.fiware.org/private/fiware-lab-help/attachments/20150222/528b1f75/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy