Hi, thanks for your feedback.
As of now I have followed all the steps.
First thing after the monasca-setup is to configtest the monasca-agent, but the only way is to do it through service monasca-agent configtest, which on Centos 7 and others systemd systems is not available.
This same check is used by fiware-check-monitoring.sh, and I think that many other checks are related to a python path that does not exist in Centos (dist-packages vs site-packages)
(monasca)[root at oslXXX monasca ]#
service monasca-agent configtest
The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.
(monasca)[root at oslXXX monasca ]#
[root@ oslXXX ~(admin-spaintenerife)]# ceilometer resource-list -q resource_id=SpainTenerife
+---------------+----------+---------+------------+
| Resource ID | Source | User ID | Project ID |
+---------------+----------+---------+------------+
| SpainTenerife | fihealth | None | None |
+---------------+----------+---------+------------+
[root at oslXXX ~(admin-spaintenerife)]# HOST_RESOURCE_ID=$(nova host-list | awk '/compute/ {print $2 "_" $2; exit}')
[root at oslXXX ~(admin-spaintenerife)]# echo $HOST_RESOURCE_ID
osleskvm085.F.q.d.n.net_osleskvm085.F.q.d.n.net
[root at oslXXX ~(admin-spaintenerife)]# ceilometer meter-list -q resource_id=$HOST_RESOURCE_ID
+------+------+------+-------------+---------+------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+------+------+------+-------------+---------+------------+
+------+------+------+-------------+---------+------------+
[root at oslXXX ~(admin-spaintenerife)]#
[root at oslXXX ~(admin-spaintenerife)]# HOST_RESOURCE_ID=osleskvm020.F.q.d.n.net_osleskvm020.F.q.d.n.net
[root at oslXXX ~(admin-spaintenerife)]# ceilometer meter-list -q resource_id=$HOST_RESOURCE_ID
+------+------+------+-------------+---------+------------+
| Name | Type | Unit | Resource ID | User ID | Project ID |
+------+------+------+-------------+---------+------------+
+------+------+------+-------------+---------+------------+
[root at oslXXX ~(admin-spaintenerife)]#
The fiware-check-monitoring.sh also encounters several situations that come from the diferent python path hierarchy on the different OS .
And from that I cannot be sure if our installation is really broken or the monitor is not fitted, see for yourself:
[root@ oslXXX ~(admin-spaintenerife)]# /root/ceilometer-fiware/ceilometer-plugin-fiware/tools/fiware-check-monitoring.sh
Considering measurements within last 60 minutes
Check Python interpreter... Python 2.7.5 at /usr/bin/python
Check Monasca Agent installation path... /opt/monasca
Check Monasca Agent configuration... Run `service monasca-agent configtest' to check errors
Check Monasca Agent configuration region... OK
Check Monasca Agent configuration hostname... control3
Check Monasca Agent logfile... /var/log/monasca/agent/forwarder.log
Check Monasca API URL... http://130.206.112.3:8070/v2.0
Check Monasca Keystone URL... http://cloud.lab.fiware.org:4731/v3
Check Monasca Agent username... ceilometer-spaintenerife
Check Monasca Agent password... ***************
Check Monasca Agent credentials for 'monasca_user' role... User roles: admin
Check Monasca Agent polling frequency... 30 seconds (consider a higher value)
Check Ceilometer polling frequency... 600 seconds
Check Ceilometer central agent logfile... Not found
Check Ceilometer entry points central node... sed: can't read /usr/lib/python2.7/dist-packages/ceilometer-*.egg-info/entry_points.txt: No such file or directory
sed: can't read /usr/lib/python2.7/dist-packages/ceilometer-*.egg-info/entry_points.txt: No such file or directory
sed: can't read /usr/lib/python2.7/dist-packages/ceilometer-*.egg-info/entry_points.txt: No such file or directory
Could not find all entry points at /usr/lib/python2.7/dist-packages/ceilometer-*.egg-info/entry_points.txt
Check Ceilometer storage driver for Monasca... Could not load class (please check installation details)
Check Ceilometer publisher for Monasca... Could not load class (please check installation details)
Check Ceilometer region pollster class... ceilometer.region.region.RegionPollster
Check last poll from region pollster... grep: /var/log/ceilometer/ceilometer-agent-central.log: No such file or directory
Could not find polling today at /var/log/ceilometer/ceilometer-agent-central.log
Check Monasca metrics for region... Failed
Check Monasca recent metadata for region... OK (5: latitude longitude location cpu_allocation_ratio ram_allocation_ratio)
Check Monasca recent measurements for region.used_ip... No measurements found
Check Monasca recent measurements for region.pool_ip... No measurements found
Check Monasca recent measurements for region.allocated_ip... No measurements found
Check Monasca recent measurements for region.sanity_status... No measurements found
Check last poll from image pollster... grep: /var/log/ceilometer/ceilometer-agent-central.log: No such file or directory
Could not find polling today at /var/log/ceilometer/ceilometer-agent-central.log
Check Monasca metrics for image... Failed
Check Monasca recent measurements for image... No measurements found
Check list of compute nodes... osleskvm085.F.q.d.n.net osleskvm080.F.q.d.n.net osleskvm010.F.q.d.n.net osleskvm020.F.q.d.n.net osleskvm045.F.q.d.n.net osleskvm055.F.q.d.n.net
Check execution of remote commands at compute nodes... OK (ssh)
Check Ceilometer polling frequency at compute node osleskvm085.F.q.d.n.net... 600 seconds
Check Ceilometer polling frequency at compute node osleskvm080.F.q.d.n.net... 600 seconds
Check Ceilometer polling frequency at compute node osleskvm010.F.q.d.n.net... 600 seconds
Check Ceilometer polling frequency at compute node osleskvm020.F.q.d.n.net... 600 seconds
Check Ceilometer polling frequency at compute node osleskvm045.F.q.d.n.net... 600 seconds
Check Ceilometer polling frequency at compute node osleskvm055.F.q.d.n.net... 600 seconds
Check Ceilometer entry points at compute node osleskvm085.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer entry points at compute node osleskvm080.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer entry points at compute node osleskvm010.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer entry points at compute node osleskvm020.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer entry points at compute node osleskvm045.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer entry points at compute node osleskvm055.F.q.d.n.net... Could not find 'compute.info' entry point
Check Ceilometer host pollster class at compute node osleskvm085.F.q.d.n.net... Could not load class (please check installation)
Check Ceilometer host pollster class at compute node osleskvm080.F.q.d.n.net... Could not load class (please check installation)
Check Ceilometer host pollster class at compute node osleskvm010.F.q.d.n.net... ceilometer.compute.pollsters.host.HostPollster
Check Ceilometer host pollster class at compute node osleskvm020.F.q.d.n.net... ceilometer.compute.pollsters.host.HostPollster
Check Ceilometer host pollster class at compute node osleskvm045.F.q.d.n.net... ceilometer.compute.pollsters.host.HostPollster
Check Ceilometer host pollster class at compute node osleskvm055.F.q.d.n.net... Could not load class (please check installation)
Check last poll from host pollster at compute node osleskvm085.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check last poll from host pollster at compute node osleskvm080.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check last poll from host pollster at compute node osleskvm010.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check last poll from host pollster at compute node osleskvm020.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check last poll from host pollster at compute node osleskvm045.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check last poll from host pollster at compute node osleskvm055.F.q.d.n.net... Could not find polling today at /var/log/ceilometer/ceilometer-agent-compute.log
Check Monasca recent measurements for compute.node.cpu.percent... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.cpu.now... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.cpu.max... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.cpu.tot... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.ram.now... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.ram.max... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.ram.tot... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.disk.now... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.disk.max... Failed (0 metrics out of 6 compute nodes)
Check Monasca recent measurements for compute.node.disk.tot... Failed (0 metrics out of 6 compute nodes)
Check Monasca metrics for nova-api... Failed
Check Monasca metrics for nova-cert... Failed
Check Monasca metrics for nova-conductor... Failed
Check Monasca metrics for nova-consoleauth... Failed
Check Monasca metrics for nova-novncproxy... Failed
Check Monasca metrics for nova-objectstore... Failed
Check Monasca metrics for nova-scheduler... Failed
Check Monasca metrics for neutron-dhcp-agent... Failed
Check Monasca metrics for neutron-l3-agent... Failed
Check Monasca metrics for neutron-metadata-agent... Failed
Check Monasca metrics for neutron-openvswitch-agent... Failed
Check Monasca metrics for neutron-server... Failed
Check Monasca metrics for cinder-api... Failed
Check Monasca metrics for cinder-scheduler... Failed
Check Monasca metrics for glance-api... Failed
Check Monasca metrics for glance-registry... Failed
Check Monasca recent measurements for nova-api... No measurements found
Check Monasca recent measurements for nova-cert... No measurements found
Check Monasca recent measurements for nova-conductor... No measurements found
Check Monasca recent measurements for nova-consoleauth... No measurements found
Check Monasca recent measurements for nova-novncproxy... No measurements found
Check Monasca recent measurements for nova-objectstore... No measurements found
Check Monasca recent measurements for nova-scheduler... No measurements found
Check Monasca recent measurements for neutron-dhcp-agent... No measurements found
Check Monasca recent measurements for neutron-l3-agent... No measurements found
Check Monasca recent measurements for neutron-metadata-agent... No measurements found
Check Monasca recent measurements for neutron-openvswitch-agent... No measurements found
Check Monasca recent measurements for neutron-server... No measurements found
Check Monasca recent measurements for cinder-api... No measurements found
Check Monasca recent measurements for cinder-scheduler... No measurements found
Check Monasca recent measurements for glance-api... No measurements found
Check Monasca recent measurements for glance-registry... No measurements found
Check Monasca metrics for active VMs... Failed
Check Monasca recent measurements for active VM 9389066f-b30c-4a4e-8e1d-996f0eaed8bf... Failed
[root@ oslXXX ~(admin-spaintenerife)]#
My monasca-setup output , just so you can check and compare with what should have done in an Ubuntu installation:
Maybe there is something relevant on the output.
(monasca)[root at oslXXX monasca ]# monasca-setup \
> --username=YYYYYYYYYY \
> --password=XXXXXXXXXXXXXXX \
> --project_name=service \
> --keystone_url=http://cloud.lab.fiware.org:4731/v3 \
> --monasca_url=http://130.206.112.3:8070/v2.0 \
> --dimensions=region:SpainTenerife
WARNING: CentOS Linux is not a support Linux distribution
Created symlink from /etc/systemd/system/multi-user.target.wants/monasca-agent.service to /etc/systemd/system/monasca-agent.service.
INFO: Enabled monasca-agent service via systemd
INFO: Configuring base Agent settings.
INFO: Configuring Cinder
INFO: Monitoring the cinder-scheduler block-storage process.
INFO: Monitoring the cinder-api block-storage process.
INFO: Configuring an http_check for the block-storage API.
INFO: Configuring Swift
INFO: Monitoring the swift-proxy-server object-storage process.
INFO: Configuring an http_check for the object-storage API.
INFO: Configuring MySQL
INFO: Watching the mysqld process.
WARNING: Unable to log into the mysql database; the mysql plugin is not configured.
INFO: Configuring HAProxy
INFO: Enabling HAProxy process watching
INFO: Enabling the HAProxy check plugin
INFO: Configuring Ovsvapp
INFO: Monitoring the ovsdb-server OVSvApp-ServiceVM process.
INFO: Monitoring the ovs-vswitchd OVSvApp-ServiceVM process.
INFO: Configuring Neutron
INFO: Monitoring the neutron-server networking process.
INFO: Monitoring the neutron-openvswitch-agent networking process.
INFO: Monitoring the neutron-rootwrap networking process.
INFO: Monitoring the neutron-dhcp-agent networking process.
INFO: Monitoring the neutron-metadata-agent networking process.
INFO: Monitoring the neutron-l3-agent networking process.
INFO: Monitoring the neutron-ns-metadata-proxy networking process.
INFO: Configuring an http_check for the networking API.
INFO: Configuring Ceilometer
INFO: Monitoring the ceilometer-agent-central telemetry process.
INFO: Monitoring the ceilometer-agent-notification telemetry process.
INFO: Monitoring the ceilometer-collector telemetry process.
INFO: Monitoring the ceilometer-alarm-notifier telemetry process.
INFO: Monitoring the ceilometer-alarm-evaluator telemetry process.
INFO: Monitoring the ceilometer-api telemetry process.
INFO: Configuring System
INFO: Configured network
INFO: Configured disk
INFO: Configured load
INFO: Configured memory
INFO: Configured cpu
INFO: Configuring Heat
INFO: Monitoring the heat-api-cfn orchestration process.
INFO: Monitoring the heat-engine orchestration process.
INFO: Configuring an http_check for the orchestration API.
INFO: Configuring Apache
INFO: Watching the apache webserver process.
WARNING: Unable to read /root/.apache.cnf
WARNING: No url specified, using default url http://localhost/server-status?auto
ERROR: Error [Errno 111] Connection refused received when accessing url http://localhost/server-status?auto.
Please ensure the Apache web server is running and your configuration information is correct.
*** The Apache plugin is not configured ***
Please correct and re-run monasca-setup.
INFO: Configuring Nova
INFO: Monitoring the nova-conductor compute process.
INFO: Monitoring the nova-cert compute process.
INFO: Monitoring the nova-scheduler compute process.
INFO: Monitoring the nova-novncproxy compute process.
INFO: Monitoring the nova-consoleauth compute process.
INFO: Monitoring the nova-api compute process.
INFO: Configuring an http_check for the compute API.
INFO: Configuring Glance
INFO: Monitoring the glance-registry image-service process.
INFO: Monitoring the glance-api image-service process.
INFO: Configuring an http_check for the image-service API.
INFO: Postfix found but the required sudo access is not configured.
Refer to plugin documentation for more detail
INFO: Starting monasca-agent service via systemd
(monasca)[root at oslXXX monasca ]#
(monasca)[root at oslXXX monasca ]# systemctl status monasca-agent.service
● monasca-agent.service - Monasca Agent
Loaded: loaded (/etc/systemd/system/monasca-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2016-07-04 12:05:51 WEST; 4min 48s ago
Main PID: 12900 (supervisord)
CGroup: /system.slice/monasca-agent.service
└─12900 /opt/monasca/bin/python /opt/monasca/bin/supervisord -c /etc/monasca/agent/supervisor.conf -n
Jul 04 12:05:57 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:05:57,306 INFO exited: statsd (exit status 1; not expected)
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,310 INFO spawned: 'statsd' with pid 13514
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,312 INFO spawned: 'forwarder' with pid 13515
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,314 INFO spawned: 'collector' with pid 13516
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,697 INFO exited: collector (exit status 1; not expected)
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,709 INFO gave up: collector entered FATAL state, too many start retries too quickly
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,710 INFO exited: statsd (exit status 1; not expected)
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,729 INFO gave up: statsd entered FATAL state, too many start retries too quickly
Jul 04 12:06:00 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:00,788 INFO exited: forwarder (exit status 1; not expected)
Jul 04 12:06:01 oslXXX.F.q.d.n.net supervisord[12900]: 2016-07-04 12:06:01,789 INFO gave up: forwarder entered FATAL state, too many start retries too quickly
(monasca)[root at oslXXX monasca ]#
Reporting differences between Ubuntu and Centos like this can be hard for you guys, if there is any other way that makes it easy for you just let me know.
Best regards
Marc
[cid:image001.png at 01D03B02.754194F0]
Joan Marc Riera Duocastella
Managed Services
08018 Barcelona (Spain)
Tlph: +34-680 232 285
e-mail: joan-marc.riera at atos.net<mailto:joan-marc.riera at atos.net>
[cid:image002.gif at 01D03B02.754194F0]
From: PABLO RODRIGUEZ ARCHILLA [mailto:pablo.rodriguezarchilla at telefonica.com]
Sent: Monday, July 04, 2016 12:57 PM
To: Riera Duocastella, Joan Marc
Cc: fiware-lab-federation-nodes at lists.fiware.org
Subject: RE: [Fiware-lab-federation-nodes] Ceilometer-plugin-fiware documentation is inaccurate
Hi, Marc.
See my comments inline
De: fiware-lab-federation-nodes-bounces at lists.fiware.org<mailto:fiware-lab-federation-nodes-bounces at lists.fiware.org> [mailto:fiware-lab-federation-nodes-bounces at lists.fiware.org] En nombre de Riera Duocastella, Joan Marc
Enviado el: lunes, 4 de julio de 2016 11:56
Para: fiware-lab-federation-nodes at lists.fiware.org<mailto:fiware-lab-federation-nodes at lists.fiware.org>
Asunto: [Fiware-lab-federation-nodes] Ceilometer-plugin-fiware documentation is inaccurate
Hello guys,
Our installation is Kilo based on Centos 7.2 .
I’m trying to install the https://github.com/SmartInfrastructures/ceilometer-plugin-fiware on our installation and I find myself lost. Seems based on Ubuntu and Fuel, and I did not use neither of them.
Yes, Fuel is the “reference” installation. But anyway we’re interested in supporting other distros, so your feedback is very valuable in order to make Monitoring also suitable for CentOS
Most of the tests, at the bottom of the site, do not work until you have most of the elements completely installed.
Can we have some example of the verification at the end of each step ?
Verification step follows the installation on purpose, given that it is intended as an overall check process in which some actual measurements are gathered, this requiring all components working together.
In which steps are you having problems? I’d suggest contacting Daniele Santoro from Create-Net for all related to pollsters:
https://github.com/SmartInfrastructures/ceilometer-plugin-fiware#central-agent-pollsters
https://github.com/SmartInfrastructures/ceilometer-plugin-fiware#compute-agent-pollsters
and me for all related Monasca:
https://github.com/SmartInfrastructures/ceilometer-plugin-fiware#monasca
We have all the health checks on green status and the system its stable. https://fi-health.lab.fiware.org/ , so at this point installing a monitoring system should be this difficult , and should not be risky as I feel it is. You will say it’s safe, but it will be me who have to rebuild everything if python goes south.
We’ve tried to ease installation as much as possible, facing the inherent problems of using components from a variety of
contributors to the OpenStack community.
At this point I have to install the monasca Agent and I don’t have virtualenv, luckily it does not have dependencies but if someone else is walking the same path I will gladly exchange the experience with them.
The need for a virtualenv raised when some Python package versions conflicted the ones installed by Fuel on Ubuntu. Using your words: “someone took this same path before, found a wall and jumped over it using virtualenv as pole vault”.
Could you clarify here the steps you’ve followed, and the problems you found, if any? I didn’t catch you in this point.
If you are on centos 7 and facing trouble, do not hesitate to send me a mail and at least we will share the pain, and if we are lucky will find a fix.
Anyway, as asked above, some tests after each step will nicely enhance the experience of installing this monitoring tool.
Maybe you’re right, but we have deliberately opted for a post-installation verification procedure (such “test-after-step” verification implied some problems)
In any case, the aim is the same: help IO’s carry out the installation and configuration with minimal effort on their side. Therefore, we are open to your feedback related to errors found, incompatibilities with CentOS, documentation not clear or missing, etc.
Regards.
--
Pablo Rodríguez Archilla
FIWARE Initiative
(+34) 958 53 79 02
Product Innovation
pablo.rodriguezarchilla at telefonica.com
Telefónica I+D
Edificio I+D Armilla (Parque Tecnológico de la Salud), Granada, Spain
Best regards
Marc
[cid:image001.png at 01D03B02.754194F0][cid:image005.jpg at 01D1D5F5.7AF40010]
Joan Marc Riera Duocastella
Managed Services
08018 Barcelona (Spain)
Tlph: +34-680 232 285
e-mail: joan-marc.riera at atos.net<mailto:joan-marc.riera at atos.net>
[cid:image002.gif at 01D03B02.754194F0][cid:image006.jpg at 01D1D5F5.7AF40010]
________________________________
Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.
The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.
Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-lab-federation-nodes/attachments/20160704/36c05217/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 402 bytes
Desc: image003.png
URL: <https://lists.fiware.org/private/fiware-lab-federation-nodes/attachments/20160704/36c05217/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 4566 bytes
Desc: image004.png
URL: <https://lists.fiware.org/private/fiware-lab-federation-nodes/attachments/20160704/36c05217/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 1514 bytes
Desc: image005.jpg
URL: <https://lists.fiware.org/private/fiware-lab-federation-nodes/attachments/20160704/36c05217/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1888 bytes
Desc: image006.jpg
URL: <https://lists.fiware.org/private/fiware-lab-federation-nodes/attachments/20160704/36c05217/attachment-0001.jpg>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy