[Backlogmanager] [FIWARE-JIRA] (HELP-13395) [fiware-stackoverflow] How to store data in MySql using cygnus?

[Fernando Lopez (JIRA)]

     [ https://jira.fiware.org/browse/HELP-13395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fernando Lopez deleted HELP-13395:
----------------------------------


> [fiware-stackoverflow] How to store data in MySql using cygnus?
> ---------------------------------------------------------------
>
>                 Key: HELP-13395
>                 URL: https://jira.fiware.org/browse/HELP-13395
>             Project: Help-Desk
>          Issue Type: Monitor
>            Reporter: Backlog Manager
>              Labels: fiware, fiware-cygnus, fiware-orion, mysql
>
> Created question in FIWARE Q/A platform on 21-04-2015 at 17:04
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/29776890/how-to-store-data-in-mysql-using-cygnus
> +Question:+
> How to store data in MySql using cygnus?
> +Description:+
> I have read all the documentation about how cygnus works, I specifically tested this one successfully. I also finished reading this tutorial, but I am sure I haven't configured something correctly.
> in cygnus_instance_1.conf I created:
> CYGNUS_USER=root
> CONFIG_FOLDER=/usr/cygnus/conf
> CONFIG_FILE=/usr/cygnus/conf/agent_1.conf
> AGENT_NAME=cygnusagent
> LOGFILE_NAME=cygnus.log
> ADMIN_PORT=8081
> and in agent_1.conf I created:
> #=============================================
> # To be put in APACHE_FLUME_HOME/conf/cygnus.conf
> #
> # General configuration template explaining how to setup a sink of each of the available types (HDFS, CKAN, MySQL).
> #=============================================
> # The next tree fields set the sources, sinks and channels used by Cygnus. You could use different names than the
> # ones suggested below, but in that case make sure you keep coherence in properties names along the configuration file.
> # Regarding sinks, you can use multiple types at the same time; the only requirement is to provide a channel for each
> # one of them (this example shows how to configure 3 sink types at the same time). Even, you can define more than one
> # sink of the same type and sharing the channel in order to improve the performance (this is like having
> # multi-threading).
> cygnusagent.sources = http-source
> cygnusagent.sinks = hdfs-sink mysql-sink ckan-sink
> cygnusagent.channels = hdfs-channel mysql-channel ckan-channel
> #=============================================
> # source configuration
> # channel name where to write the notification events
> cygnusagent.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel
> # source class, must not be changed
> cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
> # listening port the Flume source will use for receiving incoming notifications
> cygnusagent.sources.http-source.port = 5050
> # Flume handler that will parse the notifications, must not be changed
> cygnusagent.sources.http-source.handler = es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler
> # URL target
> cygnusagent.sources.http-source.handler.notification_target = /notify
> # Default service (service semantic depends on the persistence sink)
> cygnusagent.sources.http-source.handler.default_service = def_serv
> # Default service path (service path semantic depends on the persistence sink)
> cygnusagent.sources.http-source.handler.default_service_path = def_servpath
> # Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
> cygnusagent.sources.http-source.handler.events_ttl = 10
> # Source interceptors, do not change
> cygnusagent.sources.http-source.interceptors = ts de
> # Interceptor type, do not change
> cygnusagent.sources.http-source.interceptors.ts.type = timestamp
> # Destination extractor interceptor, do not change
> cygnusagent.sources.http-source.interceptors.de.type = es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor$Builder
> # Matching table for the destination extractor interceptor, put the right absolute path to the file if necessary
> # See the doc/design/interceptors document for more details
> cygnusagent.sources.http-source.interceptors.de.matching_table = /usr/cygnus/conf/matching_table.conf
> # ============================================
> # OrionHDFSSink configuration
> # channel name from where to read notification events
> cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
> # sink class, must not be changed
> cygnusagent.sinks.hdfs-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionHDFSSink
> # Comma-separated list of FQDN/IP address regarding the Cosmos Namenode endpoints
> # If you are using Kerberos authentication, then the usage of FQDNs instead of IP addresses is mandatory
> cygnusagent.sinks.hdfs-sink.cosmos_host = x1.y1.z1.w1,x2.y2.z2.w2
> # port of the Cosmos service listening for persistence operations; 14000 for httpfs, 50070 for webhdfs and free choice for inifinty
> cygnusagent.sinks.hdfs-sink.cosmos_port = 14000
> # default username allowed to write in HDFS
> cygnusagent.sinks.hdfs-sink.cosmos_default_username = cosmos_username
> # default password for the default username
> cygnusagent.sinks.hdfs-sink.cosmos_default_password = xxxxxxxxxxxxx
> # HDFS backend type (webhdfs, httpfs or infinity)
> cygnusagent.sinks.hdfs-sink.hdfs_api = httpfs
> # how the attributes are stored, either per row either per column (row, column)
> cygnusagent.sinks.hdfs-sink.attr_persistence = column
> # Hive FQDN/IP address of the Hive server
> cygnusagent.sinks.hdfs-sink.hive_host = x.y.z.w
> # Hive port for Hive external table provisioning
> cygnusagent.sinks.hdfs-sink.hive_port = 10000
> # Kerberos-based authentication enabling
> cygnusagent.sinks.hdfs-sink.krb5_auth = false
> # Kerberos username
> cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_user = krb5_username
> # Kerberos password
> cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_password = xxxxxxxxxxxxx
> # Kerberos login file
> cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_login_conf_file = /usr/cygnus/conf/krb5_login.conf
> # Kerberos configuration file
> cygnusagent.sinks.hdfs-sink.krb5_auth.krb5_conf_file = /usr/cygnus/conf/krb5.conf
> # ============================================
> # OrionCKANSink configuration
> # channel name from where to read notification events
> cygnusagent.sinks.ckan-sink.channel = ckan-channel
> # sink class, must not be changed
> cygnusagent.sinks.ckan-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionCKANSink
> # the CKAN API key to use
> cygnusagent.sinks.ckan-sink.api_key = ckanapikey
> # the FQDN/IP address for the CKAN API endpoint
> cygnusagent.sinks.ckan-sink.ckan_host = x.y.z.w
> # the port for the CKAN API endpoint
> cygnusagent.sinks.ckan-sink.ckan_port = 80
> # Orion URL used to compose the resource URL with the convenience operation URL to query it
> cygnusagent.sinks.ckan-sink.orion_url = http://localhost:1026
> # how the attributes are stored, either per row either per column (row, column)
> cygnusagent.sinks.ckan-sink.attr_persistence = row
> # enable SSL for secure Http transportation; 'true' or 'false'
> cygnusagent.sinks.ckan-sink.ssl = false
> # ============================================
> # OrionMySQLSink configuration
> # channel name from where to read notification events
> cygnusagent.sinks.mysql-sink.channel = mysql-channel
> # sink class, must not be changed
> cygnusagent.sinks.mysql-sink.type = es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionMySQLSink
> # the FQDN/IP address where the MySQL server runs 
> cygnusagent.sinks.mysql-sink.mysql_host = localhost
> # the port where the MySQL server listes for incomming connections
> cygnusagent.sinks.mysql-sink.mysql_port = 3306
> # a valid user in the MySQL server
> cygnusagent.sinks.mysql-sink.mysql_username = root
> # password for the user above
> cygnusagent.sinks.mysql-sink.mysql_password = klasika
> # how the attributes are stored, either per row either per column (row, column)
> cygnusagent.sinks.mysql-sink.attr_persistence = column
> #=============================================
> # hdfs-channel configuration
> # channel type (must not be changed)
> cygnusagent.channels.hdfs-channel.type = memory
> # capacity of the channel
> cygnusagent.channels.hdfs-channel.capacity = 1000
> # amount of bytes that can be sent per transaction
> cygnusagent.channels.hdfs-channel.transactionCapacity = 100
> #=============================================
> # ckan-channel configuration
> # channel type (must not be changed)
> cygnusagent.channels.ckan-channel.type = memory
> # capacity of the channel
> cygnusagent.channels.ckan-channel.capacity = 1000
> # amount of bytes that can be sent per transaction
> cygnusagent.channels.ckan-channel.transactionCapacity = 100
> #=============================================
> # mysql-channel configuration
> # channel type (must not be changed)
> cygnusagent.channels.mysql-channel.type = memory
> # capacity of the channel
> cygnusagent.channels.mysql-channel.capacity = 1000
> # amount of bytes that can be sent per transaction
> cygnusagent.channels.mysql-channel.transactionCapacity = 100
> Although I dont use OrionHDFSSink and OrionCKANSink, I  didnt touch those configurations because I really am not sure weather I should.
> When I finally subscribeContext and target cygnus @ default port 5050, I get a normal response, but nothing is created in my database
> What am I doing wrong here?



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)