[Backlogmanager] [FIWARE-JIRA] (HELP-8770) [fiware-stackoverflow] Fiware Orion context Broker Out of Memory Issue

[Fernando Lopez (JIRA)]

     [ https://jira.fiware.org/browse/HELP-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fernando Lopez reassigned HELP-8770:
------------------------------------

    Assignee: Backlog Manager

> [fiware-stackoverflow] Fiware Orion context Broker Out of Memory Issue
> ----------------------------------------------------------------------
>
>                 Key: HELP-8770
>                 URL: https://jira.fiware.org/browse/HELP-8770
>             Project: Help-Desk
>          Issue Type: Monitor
>          Components: FIWARE-TECH-HELP
>            Reporter: Backlog Manager
>            Assignee: Backlog Manager
>              Labels: fiware, fiware-orion
>
> Created question in FIWARE Q/A platform on 14-04-2015 at 15:04
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/29629170/fiware-orion-context-broker-out-of-memory-issue
> +Question:+
> Fiware Orion context Broker Out of Memory Issue
> +Description:+
> I have deployed Orion Context Broker v0.20 on OpenStack - CentOS 6.5 with 4GB RAM | 2 VCPU | 5.0GB Disk.
> It works as expected until the multi-threaded HTTP request (for 100 users) test with Apache JMeter 2.13.
> The service shuts down with; 
>   Out of memory: Kill process 5640 (contextBroker) score 946 or
>   sacrifice child Killed process 5640, UID 501, (contextBroker)
>   total-vm:15377736kB, anon-rss:3822
> The problem seems to be the libmicrohttpd.
>  libmicrohttpd invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
> libmicrohttpd cpuset=/ mems_allowed=0
> Pid: 2551, comm: libmicrohttpd Not tainted 2.6.32-431.el6.x86_64 #1
> Call Trace:
>  [<ffffffff810d05b1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
>  [<ffffffff81122960>] ? dump_header+0x90/0x1b0
>  [<ffffffff8122798c>] ? security_real_capable_noaudit+0x3c/0x70
>  [<ffffffff81122de2>] ? oom_kill_process+0x82/0x2a0
>  [<ffffffff81122d21>] ? select_bad_process+0xe1/0x120
>  [<ffffffff81123220>] ? out_of_memory+0x220/0x3c0
>  [<ffffffff8112fb3c>] ? __alloc_pages_nodemask+0x8ac/0x8d0
>  [<ffffffff81167b9a>] ? alloc_pages_vma+0x9a/0x150
>  [<ffffffff81149c3c>] ? do_wp_page+0x52c/0x920
>  [<ffffffff8100bc2e>] ? invalidate_interrupt1+0xe/0x20
>  [<ffffffff8100bc2e>] ? invalidate_interrupt1+0xe/0x20
>  [<ffffffff8114a82d>] ? handle_pte_fault+0x2cd/0xb00
>  [<ffffffff8104eeb7>] ? pte_alloc_one+0x37/0x50
>  [<ffffffff8100bc2e>] ? invalidate_interrupt1+0xe/0x20
>  [<ffffffff8114b28a>] ? handle_mm_fault+0x22a/0x300
>  [<ffffffff8104a8d8>] ? __do_page_fault+0x138/0x480
>  [<ffffffff8105dbef>] ? finish_task_switch+0x4f/0xf0
>  [<ffffffff81527910>] ? thread_return+0x4e/0x76e
>  [<ffffffff8152d45e>] ? do_page_fault+0x3e/0xa0
>  [<ffffffff8152a815>] ? page_fault+0x25/0x30
> Mem-Info:
> Node 0 DMA per-cpu:
> CPU    0: hi:    0, btch:   1 usd:   0
> CPU    1: hi:    0, btch:   1 usd:   0
> Node 0 DMA32 per-cpu:
> CPU    0: hi:  186, btch:  31 usd:  30
> CPU    1: hi:  186, btch:  31 usd:  30
> Node 0 Normal per-cpu:
> CPU    0: hi:  186, btch:  31 usd:  48
> CPU    1: hi:  186, btch:  31 usd:  51
> active_anon:771153 inactive_anon:199950 isolated_anon:352
>  active_file:17 inactive_file:48 isolated_file:38
>  unevictable:0 dirty:0 writeback:612 unstable:0
>  free:21204 slab_reclaimable:2119 slab_unreclaimable:5685
>  mapped:90 shmem:41 pagetables:3131 bounce:0
> Node 0 DMA free:15740kB min:248kB low:308kB high:372kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15352kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
> lowmem_reserve[]: 0 3512 4017 4017
> Node 0 DMA32 free:60708kB min:58868kB low:73584kB high:88300kB active_anon:2857740kB inactive_anon:593508kB active_file:72kB inactive_file:48kB unevictable:0kB isolated(anon):640kB isolated(file):0kB present:3596504kB mlocked:0kB dirty:0kB writeback:2444kB mapped:88kB shmem:0kB slab_reclaimable:44kB slab_unreclaimable:400kB kernel_stack:328kB pagetables:868kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1248 all_unreclaimable? no
> lowmem_reserve[]: 0 0 505 505
> Node 0 Normal free:8368kB min:8464kB low:10580kB high:12696kB active_anon:231444kB inactive_anon:201556kB active_file:0kB inactive_file:144kB unevictable:0kB isolated(anon):896kB isolated(file):24kB present:517120kB mlocked:0kB dirty:0kB writeback:4kB mapped:272kB shmem:164kB slab_reclaimable:8432kB slab_unreclaimable:22340kB kernel_stack:1616kB pagetables:11656kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:312 all_unreclaimable? no
> lowmem_reserve[]: 0 0 0 0
> Node 0 DMA: 3*4kB 2*8kB 2*16kB 2*32kB 2*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15740kB
> Node 0 DMA32: 3*4kB 7*8kB 4*16kB 7*32kB 3*64kB 0*128kB 29*256kB 51*512kB 26*1024kB 0*2048kB 0*4096kB = 60708kB
> Node 0 Normal: 346*4kB 188*8kB 101*16kB 47*32kB 15*64kB 5*128kB 2*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 8120kB
> 1824 total pagecache pages
> 1705 pages in swap cache
> Swap cache stats: add 68397, delete 66692, find 1718/1909
> Free swap  = 0kB
> Total swap = 262136kB
> 1048575 pages RAM
> 34863 pages reserved
> 1067 pages shared
> 988252 pages non-shared
> Has anyone faced this issue or have suggestion to workaround it?
> The JMeter Test Plan;
> <?xml version="1.0" encoding="UTF-8"?>
> <jmeterTestPlan version="1.2" properties="2.8" jmeter="2.13 r1665067">
>   <hashTree>
>     <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="HTTP Request" enabled="true">
>       <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
>         <collectionProp name="Arguments.arguments"/>
>       </elementProp>
>       <stringProp name="HTTPSampler.domain">ORION_IP_ADDRESS</stringProp>
>       <stringProp name="HTTPSampler.port">1026</stringProp>
>       <stringProp name="HTTPSampler.connect_timeout"></stringProp>
>       <stringProp name="HTTPSampler.response_timeout"></stringProp>
>       <stringProp name="HTTPSampler.protocol"></stringProp>
>       <stringProp name="HTTPSampler.contentEncoding"></stringProp>
>       <stringProp name="HTTPSampler.path">/v1/contextTypes</stringProp>
>       <stringProp name="HTTPSampler.method">GET</stringProp>
>       <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
>       <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
>       <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
>       <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
>       <boolProp name="HTTPSampler.monitor">false</boolProp>
>       <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
>     </HTTPSamplerProxy>
>     <hashTree/>
>   </hashTree>
> </jmeterTestPlan>



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)