[Backlogmanager] [FIWARE-JIRA] (HELP-15185) [fiware-stackoverflow] How can I improve socket hang up when connecting many devices?

[Fernando Lopez (JIRA)]

     [ https://jira.fiware.org/browse/HELP-15185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fernando Lopez updated HELP-15185:
----------------------------------
    Description: 
Created question in FIWARE Q/A platform on 17-12-2018 at 16:12
{color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/53818381/how-can-i-improve-socket-hang-up-when-connecting-many-devices


+Question:+
How can I improve socket hang up when connecting many devices?

+Description:+

I am testing to connect many devices to FIWARE in the following environment.
Each component is deployed in a container on a physical server.

+-------------------------------------------------+
|Comet - Cygnus - Orion - IoTAgentJSON - Mosquitto| - device*N
+-------------------------------------------------+

Under the condition that each device transmits data at 1 msg/sec, the following error occurs at IoTAgent when the number of devices is 350.(That is, 350 msg/sec)

{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.NGSIService | srv=n/a | subsrv=n/a | msg=Error found executing update action in Context Broker: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.81037597Z"}
{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.Alarms | srv=n/a | subsrv=n/a | msg=Raising [ORION-ALARM]: {\"code\":\"ECONNRESET\"} | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810440213Z"}
{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentJSON.MQTTBinding | srv=n/a | subsrv=n/a | msg=MEASURES-002: Couldn't send the updated values to the Context Broker due to an error: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810526916Z"}



Question 1: Where is the cause? IoTAgent? Or Orion?


Error found executing update action in Context Broker: Error: socket hang up but there is no error log displayed in Orion.


Question 2: How can I improve the processing performance of FIWARE?


Do you need the scale of IoTAgent?
Do you need to consider Orion's parameters?


I need to consider values ​​such as reqPoolSize and maxConnections with reference to the following URL?
https://fiware-orion.readthedocs.io/en/master/admin/perf_tuning/#http-server-tuning

Do you need the scale of Orion?


How to scale Orion GE? 



Question 3: Is there a batch operation on IoT Agent?


On the following page, you should do a batch operation instead of opening a new connection for each entity, but is there such a function in IoTAgent?
ECONNRESET when opening a large number of connection in small time period



  was:

Created question in FIWARE Q/A platform on 17-12-2018 at 16:12
{color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/53818381/how-can-i-improve-socket-hang-up-when-connecting-many-devices


+Question:+
How can I improve socket hang up when connecting many devices?

+Description:+

I am testing to connect many devices to FIWARE in the following environment.
Each component is deployed in a container on a physical server.

+-------------------------------------------------+
|Comet - Cygnus - Orion - IoTAgentJSON - Mosquitto| - device*N
+-------------------------------------------------+

Under the condition that each device transmits data at 1 msg/sec, the following error occurs at IoTAgent when the number of devices is 350.(That is, 350 msg/sec)

{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.NGSIService | srv=n/a | subsrv=n/a | msg=Error found executing update action in Context Broker: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.81037597Z"}
{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.Alarms | srv=n/a | subsrv=n/a | msg=Raising [ORION-ALARM]: {\"code\":\"ECONNRESET\"} | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810440213Z"}
{"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentJSON.MQTTBinding | srv=n/a | subsrv=n/a | msg=MEASURES-002: Couldn't send the updated values to the Context Broker due to an error: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810526916Z"}



Question 1: Where is the cause? IoTAgent? Or Orion?


Error found executing update action in Context Broker: Error: socket hang up but there is no error log displayed in Orion.


Question 2: How can I improve the processing performance of FIWARE?


Do you need the scale of IoTAgent?
Do you need to consider Orion's parameters?


I need to consider values ​​such as reqPoolSize and maxConnections with reference to the following URL?
https://fiware-orion.readthedocs.io/en/master/admin/perf_tuning/#http-server-tuning

Do you need the scale of Orion?


How to scale Orion GE? 



Question 3: Is there a batch operation on IoT Agent?


On the following page, you should do a batch operation instead of opening a new connection for each entity, but is there such a function in IoTAgent?
ECONNRESET when opening a large number of connection in small time period



     HD-Enabler: Orion

> [fiware-stackoverflow] How can I improve socket hang up when connecting many devices?
> -------------------------------------------------------------------------------------
>
>                 Key: HELP-15185
>                 URL: https://jira.fiware.org/browse/HELP-15185
>             Project: Help-Desk
>          Issue Type: Monitor
>          Components: FIWARE-TECH-HELP
>            Reporter: Backlog Manager
>            Assignee: Jose Manuel Cantera
>              Labels: fiware, fiware-orion
>
> Created question in FIWARE Q/A platform on 17-12-2018 at 16:12
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/53818381/how-can-i-improve-socket-hang-up-when-connecting-many-devices
> +Question:+
> How can I improve socket hang up when connecting many devices?
> +Description:+
> I am testing to connect many devices to FIWARE in the following environment.
> Each component is deployed in a container on a physical server.
> +-------------------------------------------------+
> |Comet - Cygnus - Orion - IoTAgentJSON - Mosquitto| - device*N
> +-------------------------------------------------+
> Under the condition that each device transmits data at 1 msg/sec, the following error occurs at IoTAgent when the number of devices is 350.(That is, 350 msg/sec)
> {"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.NGSIService | srv=n/a | subsrv=n/a | msg=Error found executing update action in Context Broker: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.81037597Z"}
> {"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentNGSI.Alarms | srv=n/a | subsrv=n/a | msg=Raising [ORION-ALARM]: {\"code\":\"ECONNRESET\"} | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810440213Z"}
> {"log":"time=2018-12-16T14:57:24.810Z | lvl=ERROR | corr=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | trans=ec11c37f-5194-4cb3-8d79-e04a2d1e745c | op=IoTAgentJSON.MQTTBinding | srv=n/a | subsrv=n/a | msg=MEASURES-002: Couldn't send the updated values to the Context Broker due to an error: Error: socket hang up | comp=IoTAgent\n","stream":"stdout","time":"2018-12-16T14:57:24.810526916Z"}
> Question 1: Where is the cause? IoTAgent? Or Orion?
> Error found executing update action in Context Broker: Error: socket hang up but there is no error log displayed in Orion.
> Question 2: How can I improve the processing performance of FIWARE?
> Do you need the scale of IoTAgent?
> Do you need to consider Orion's parameters?
> I need to consider values ​​such as reqPoolSize and maxConnections with reference to the following URL?
> https://fiware-orion.readthedocs.io/en/master/admin/perf_tuning/#http-server-tuning
> Do you need the scale of Orion?
> How to scale Orion GE? 
> Question 3: Is there a batch operation on IoT Agent?
> On the following page, you should do a batch operation instead of opening a new connection for each entity, but is there such a function in IoTAgent?
> ECONNRESET when opening a large number of connection in small time period



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)