[Fiware-lab-help] [FIWARE-JIRA] (HELP-12848) Hive - JSonSerde library - Cygnus

[Andres Muñoz (JIRA)]

    [ https://jira.fiware.org/browse/HELP-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=101060#comment-101060 ] 

Andres Muñoz commented on HELP-12848:
-------------------------------------

This issue is related to the use of the old Fiware Cosmos platform, Thus, using the new cloud platform this issue is solved

> [Fiware-lab-help] Hive - JSonSerde library - Cygnus
> ---------------------------------------------------
>
>                 Key: HELP-12848
>                 URL: https://jira.fiware.org/browse/HELP-12848
>             Project: Help-Desk
>          Issue Type: extRequest
>          Components: FIWARE-LAB-HELP
>            Reporter: FW External User
>            Assignee: Andres Muñoz
>         Attachments: [FIWARE-JIRA] (HELP-12848) [Fiware-general-help] [SPAM] [FIWARE-JIRA] (HELP-12870) [SPAM] Website Job Application, [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus, [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus, [FIWARE-JIRA] (HELP-12848) [Fiware-lab-help] Hive - JSonSerde library - Cygnus
>
>
> Hi ,
> I have configured the Cygnus instance with the following configuration for Hive:
> cygnus-ngsi.sinks.hdfs-sink.hive.host = cosmos.lab.fiware.org
> cygnus-ngsi.sinks.hdfs-sink.hive.server_version = 2
> cygnus-ngsi.sinks.hdfs-sink.file_format = json-column
> cygnus-ngsi.sinks.hdfs-sink.csv_separator = ,
> cygnus-ngsi.sinks.hdfs-sink.hdfs_host = storage.cosmos.lab.fiware.org
> cygnus-ngsi.sinks.hdfs-sink.hdfs_port = 14000
> cygnus-ngsi.sinks.hdfs-sink.hdfs_username = <username>
> cygnus-ngsi.sinks.hdfs-sink.hdfs_password = <password>
> cygnus-ngsi.sinks.hdfs-sink.oauth2_token = <token>
> cygnus-ngsi.sinks.hdfs-sink.hive = true
> After table creation, I have received the below error message:
> Error message:
> SQL Error [1] [08S01]: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
> SQL Error [1] [08S01]: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
>   Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
>   Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
> Ddl hive table:
> create external table if not exists default.mellusom_defaultx002fx0053x0043x0053x002d004_x0053martx0043ityx0053ensor_column
> (
> recvx0054ime string,
> fiwarex0053ervicex0050ath string,
> entityx0049d string,
> entityx0054ype string,
> brightness string,
> brightness_md array<struct<name:string,type:string,value:string>>,
> city string,city_md array<struct<name:string,type:string,value:string>>,
> continent string,continent_md array<struct<name:string,type:string,value:string>>,
> country string,country_md array<struct<name:string,type:string,value:string>>,
> location string,location_md array<struct<name:string,type:string,value:string>>,
> owner string,owner_md array<struct<name:string,type:string,value:string>>,
> region string,region_md array<struct<name:string,type:string,value:string>>,
> temperature_c string,temperature_c_md array<struct<name:string,type:string,value:string>>,
> zip string,zip_md array<struct<name:string,type:string,value:string>>)
> row format serde 'org.openx.data.jsonserde.JsonSerDe' with serdeproperties ("dots.in.keys" = "true")
> location '/user/mellusom/default/SCS-004_SmartCitySensor'
> I solved the issue regarding the error message using this link:
> https://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Where-to-put-json-serde-1-3-7-jar-with-dependencies-jar-file/td-p/48073<http://cp.mcafee.com/d/avndygsd3gOrhvKYY-U-NsTsSyyPtBBBVBMSyyOyevKUYyrhhp76zBWXVEVd79J55VNdNZcQszzrItxrWmHOSHr7PPZ1lGNcWuvaDiCrJm9DjPVkWkPuE9LK-qen-LOrz1EVd7bPTnKnjpujVxUsUOUVR4mlHFGTppVkffGhBrwqrhdI6XYCyOzsQsIL8I6XCOsVHkiP5RmUiCmWNZmQfzLbVd_1DpS9BNw1MGn45JIjYK5V6vM1ivNYLeFtpKpshFm1uDNgYKynpbzyUzIiVykOpt3RLwxYbi-xshS9tSh_Vd0KJz3RNP9IhHjV4orqluJIo_fUR1jIz6N_w09JAsYYyCr1vF6y0bOc-q82VEw0woH0o-a7Cy0-h2eNb6y0Kq818IhwTgSOedBjl2gIAwSd_>
> and after that I have tried to deploy on my local Hive istance the jar library recovered by http://www.congiu.net/hive-json-serde/<http://cp.mcafee.com/d/FZsS93gwrhvKYY-U-NsTsSyyPtBBBVBMSyyOyevKUYyrhhp76zBWXVEVd79J55VNdNZcQszzrItxrWmHOSHr7PPZ1lGNcWuvaDiCrJm9DjPVkWkPuE9LK-qen-LOrz1EVd7bPTnKnjpujVxUsUOUVR4mlHFGTppVkffGhBrwqrjdI6XYCyOzsQsIL8I6XCM0lGM8OrU6UjApYK8QH0LjUEvxIhHjV4orqluJIo_fUR1jIz6N_w09JAsYYyCr1vF6y0bOc-q82VEw0woH0o-a7Cy0-h2eNb6y0Kq818IhwTgSOedFPnyNnzG> (version 1.3.8)
> and now It's correctly working in local.
> Best regards.
> Francesco Paparella
> Ricerca e Sviluppo
> Corvallis S.p.A. a Socio Unico
> Attività di direzione e coordinamento Corvallis Holding SpA
> Gruppo Corvallis
> Viale Aldo Moro, 36
> 73100 Lecce
> Italia
> Telefono         +39 0832 069901
> Telefax   +39 0832 069911
> francesco.paparella at corvallis.it<mailto:francesco.paparella at corvallis.it>
> www.corvallis.it<http://www.corvallis.it/>
> Le informazioni contenute in questo messaggio di posta elettronica e negli eventuali allegati sono riservate e confidenziali e sono indirizzate esclusivamente al destinatario. Si prega di non fare copia, inoltrare a terzi o conservare tale messaggio se non si è il legittimo destinatario dello stesso. Qualora questo messaggio sia stato ricevuto per errore, si prega di rinviarlo al mittente e di cancellarlo permanentemente dal proprio computer.
> The information contained in this message and in any attachment is intended exclusively for the recipient. If you are not the intended recipient you are hereby notified not to copy, save, disclose, or distribute it to any third party. If you erroneously received this message you are kindly requested  to return it to the sender and eliminate it permanently from your computer.
> __________________________________________________________________________________________
> You can get more information about our cookies and privacy policies on the following links:
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Privacy_Policy
> - http://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Cookies_Policy_FIWARE
> Fiware-lab-help mailing list
> Fiware-lab-help at lists.fiware.org
> https://lists.fiware.org/listinfo/fiware-lab-help
> [Created via e-mail received from: Paparella Francesco <Francesco.Paparella at corvallis.it>]



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)