[Fiware-tech-help] Migrating previously collected datasets to FIWARE backend

[Akil Rilwan]

Dear FIWARE Tech Help,

I'm faced with the task of migrating previously collected environmental
datasets (weather, airquality, noise etc) from sensors deployed in
different locations, and stored in several tables of MySQL database, to my
instance of fiware Orion CB, and thus persisted to fiware backend.


The challenges are many:

   - the data isn't stored in fiware standards, so must be transformed
   according to the fiware data models.
   - not all tables are a good candidate of being transformed to an Entity.
   - some Entities need have field values from several tables as
   attributes. For instance, defining AirQualityObserved Entity-type would
   have attributes from these tables: airquality, co, co2, no2 and
   deployment. So mapping these attributes to a particular Entity-type is a
   challenge.

As this is a one-time upload (not live data), I am thinking to approach the
task by creating a Python script that "pretends" to be a contextProducer to
the Orion instance, sending data (say every 5sec) until the last record.

My greatest challenge however is the data fields --> Entity's
attributes mapping
that actually need be combined from several tables. I have not come across
any previous solution to similar problem, to get me going.

Is there any recommendations from FIWARE Foundation for situations similar
to this? Any help would be appreciated.

Many thanks,

Akil.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-tech-help/attachments/20190415/cc915d4e/attachment.html>