[Fiware-fic3-coaching] Context broker load test

[Federico Lopez]

Thank you very much!!



2015-04-15 16:40 GMT+02:00 Franck Le Gall <franck.le-gall at eglobalmark.com>:

>  Dear Federico,
>
>
>
> Following our discussion, below are some load tests kindly provided by
> Leandro Guillen (IMDEA) on the ORION Broker
>
>
>
> It basically says that Orion can handle around 1500 transactions per
> second when run in this particular machine.
>
>
>
> I will be pleased to hear more from load tests you are going to run on
> your system
>
>
>
> Kind regards
>
> Franck
>
>
>
> *Scenario*
>
> *User impact Vs CB Behaivour *
>
> *Sistem Under Test*
>
> *Type*
>
> *Virtual*
>
> *OPs*
>
> *Base*
>
> How affects the use of serveral users at the same time to the Time
> response and the TPS avg.
> Total Threads = XML threads + JSON threads
>
> Orion
>
> BIG (XXL)
>
> (NO) CPU 16 Cores MEM 16 GB
>
> Append + Update + Query
>
> Comment:
> The Average Throughput is similar to lastest version (0.10.1).
> In this case, we add a append request and we add validations by status
> code and the response body in all requests (append, update and query)
>
> *Test exec*
>
> ~/test/apache-jmeter-2.11/bin/jmeter.sh -n -t
> ~/JMXs/orionPerfTest_v1.2.jmx -p ~/test/jmeter.prop
> -JPROJECT="Scenarios_0.11.0" -JTEST_NAME="HILOS" -JHOST=10.95.20.2
> -JPORT=1026 -JHILOS=???? -JITERATIONS=1 -JRAMPUP=50 -JRUNTIME=180
>
> *Threads*
>
> *Metrics*
>
> *Results without verbose logs*
>
> *Latest Version*
>
> *Comments*
>
> *Results with verbose logs*
>
> *Lastest Version *
>
> *1*
>
> Average Response Time, ms:
>
> 1
>
> 1
>
> Maximum Response Time, ms:
>
> 104
>
> 96
>
> Average Throughput (TPS):
>
> *1323.17*
>
> *1329.87*
>
> Average Virtual Users:
>
> 1.98895
>
> 1.98895
>
> Max Virtual Users:
>
> 2
>
> 2
>
> Url:
>
> *https://loadosophia.org/gui/125418 <https://loadosophia.org/gui/125418>*
>
> *5*
>
> Average Response Time, ms:
>
> 5
>
> 5
>
> Maximum Response Time, ms:
>
> 105
>
> 105
>
> Average Throughput (TPS):
>
> *1452.97*
>
> *1477.12*
>
> Average Virtual Users:
>
> 8.19545
>
> 8.1448
>
> Max Virtual Users:
>
> 10
>
> 10
>
> Url:
>
> *https://loadosophia.org/gui/125420 <https://loadosophia.org/gui/125420>*
>
> *50*
>
> Average Response Time, ms:
>
> 54
>
> 52
>
> Maximum Response Time, ms:
>
> 286
>
> 318
>
> Average Throughput (TPS):
>
> *1452.47*
>
> *1506.84*
>
> Average Virtual Users:
>
> 78.2609
>
> 78.655
>
> Max Virtual Users:
>
> 100
>
> 100
>
> Url:
>
> *https://loadosophia.org/gui/125426 <https://loadosophia.org/gui/125426>*
>
> *250*
>
> Average Response Time, ms:
>
> 256
>
> 244
>
> Maximum Response Time, ms:
>
> 1706
>
> 1308
>
> Average Throughput (TPS):
>
> *1452.37*
>
> *1514.26*
>
> Average Virtual Users:
>
> 389.931
>
> 389.81
>
> Max Virtual Users:
>
> 500
>
> 500
>
> Url:
>
> *https://loadosophia.org/gui/125428 <https://loadosophia.org/gui/125428>*
>
> *500*
>
> Average Response Time, ms:
>
> 547
>
> 519
>
> *500 - Recheck with verbose logs*
>
> 1416
>
> Maximum Response Time, ms:
>
> 2816
>
> 2643
>
> 6169
>
> Average Throughput (TPS):
>
> *1402.55*
>
> *1511.81*
>
> *635.641*
>
> Average Virtual Users:
>
> 777.474
>
> 776.612
>
> 779.444
>
> Max Virtual Users:
>
> 1000
>
> 1000
>
> 1006
>
> Url:
>
> *https://loadosophia.org/gui/125460 <https://loadosophia.org/gui/125460>*
>
> *https://loadosophia.org/gui/125773 <https://loadosophia.org/gui/125773>*
>
> *750*
>
> Average Response Time, ms:
>
> 830
>
>
>
> *Error rate: 0.38% Caused by: java.net.ConnectException: Connection timed
> out See in Extra 1 page "Aggregate Report for 750 Threads"*
>
> *750 - Recheck with verbose logs*
>
> 1718
>
> Maximum Response Time, ms:
>
> 146957
>
> 148008
>
> Average Throughput (TPS):
>
> *1458.46*
>
> *664.524*
>
> Average Virtual Users:
>
> 1234.48
>
> 1230.91
>
> Max Virtual Users:
>
> 1734
>
> 1721
>
> Url:
>
> *https://loadosophia.org/gui/125768 <https://loadosophia.org/gui/125768>*
>
> *https://loadosophia.org/gui/125777 <https://loadosophia.org/gui/125777>*
>
> *1000*
>
> Average Response Time, ms:
>
> 1040
>
> 1042
>
>
>
> *Error rate: 0.73% Caused by: java.net.ConnectException: Connection timed
> out See in Extra 1 page "Aggregate Report for 1000 Threads"*
>
> *1000 - Recheck with verbose logs*
>
> 2463
>
> Maximum Response Time, ms:
>
> 155109
>
> 155261
>
> 156280
>
> Average Throughput (TPS):
>
> *1551.09*
>
> *1678.95*
>
> *649.606*
>
> Average Virtual Users:
>
> 1586.16
>
> 1794.97
>
> 1691.05
>
> Max Virtual Users:
>
> 2467
>
> 2475
>
> 2465
>
> Url:
>
> *https://loadosophia.org/gui/125467 <https://loadosophia.org/gui/125467>*
>
> *https://loadosophia.org/gui/125778 <https://loadosophia.org/gui/125778>*
>
> *1500*
>
> Average Response Time, ms:
>
> 1633
>
> 1574
>
>
>
> *Error rate: 1.63% Caused by: java.net.ConnectException: Connection timed
> out See in Extra 1 page "Aggregate Report for 1500 Threads"*
>
> *1500 - Recheck with verbose logs*
>
> 3142
>
> 3177
>
> Maximum Response Time, ms:
>
> 163372
>
> 162967
>
> 164486
>
> 164091
>
> Average Throughput (TPS):
>
> *1504.44*
>
> *1710.55*
>
> *736.204*
>
> 812.517
>
> Average Virtual Users:
>
> 2529.98
>
> 2792.82
>
> 2516.73
>
> 2781.36
>
> Max Virtual Users:
>
> 3926
>
> 3940
>
> 3928
>
> 3941
>
> Url:
>
> *https://loadosophia.org/gui/125729 <https://loadosophia.org/gui/125729>*
>
> *https://loadosophia.org/gui/125744 <https://loadosophia.org/gui/125744>*
>
> *2000*
>
> Average Response Time, ms:
>
>
>
>
> *Error rate: 42.33% Caused by: java.net.ConnectException: Connection timed
> out See in Extra 1 page "Aggregate Report for 2000 Threads" Max Used: CPU
> 21% and MEM 8Gb*
>
> *2000 - Recheck with verbose logs*
>
> Maximum Response Time, ms:
>
> Average Throughput (TPS):
>
> Average Virtual Users:
>
> Max Virtual Users:
>
> Url:
>
> *https://loadosophia.org/gui/125753 <https://loadosophia.org/gui/125753>*
>
> *https://loadosophia.org/gui/125784 <https://loadosophia.org/gui/125784>*
>
> *3000*
>
> Average Response Time, ms:
>
> Maximum Response Time, ms:
>
> Average Throughput (TPS):
>
> 0
>
> Average Virtual Users:
>
> Max Virtual Users:
>
> Url:
>
> *5000*
>
> Average Response Time, ms:
>
> Maximum Response Time, ms:
>
> Average Throughput (TPS):
>
> Average Virtual Users:
>
> Max Virtual Users:
>
> Url:
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-fic3-coaching/attachments/20150415/c18776e9/attachment.html>