[Backlogmanager] [FIWARE-JIRA] (HELP-15923) FIWARE.Question.Tech.Why docker keeps dumping unnecessary files?.

[Mohamed Sadiq (JIRA)]

     [ https://jira.fiware.org/browse/HELP-15923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mohamed Sadiq reassigned HELP-15923:
------------------------------------

    Assignee: Backlog Manager

> FIWARE.Question.Tech.Why docker keeps dumping unnecessary files?.
> -----------------------------------------------------------------
>
>                 Key: HELP-15923
>                 URL: https://jira.fiware.org/browse/HELP-15923
>             Project: Help-Desk
>          Issue Type: Monitor
>          Components: FIWARE-TECH-HELP
>            Reporter: Backlog Manager
>            Assignee: Backlog Manager
>              Labels: docker, fiware, fiware-orion
>
> Created question in FIWARE Q/A platform on 15-06-2019 at 16:06
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/56611482/why-docker-keeps-dumping-unnecessary-files
> +Question:+
> Why docker keeps dumping unnecessary files?
> +Description:+
> I am running dockerised fiware-orion on remote host. I know that I have nearly 200GB of storage allocated, but realised docker is consuming so much of disk space dumping unnecessary files.
> So every few days I have to check to ensure I have some storage enough for orion to run fine. Whenever entity-query returns empty, I know that docker has completely comsumed storage 100%. So I have to clean disk and start services running again. On average, it happens once a week.
> $df -h
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/vda1       197G  187G     0 100% /
> devtmpfs        1.4G     0  1.4G   0% /dev
> tmpfs           1.4G   12K  1.4G   1% /dev/shm
> tmpfs           1.4G  138M  1.3G  10% /run
> tmpfs           1.4G     0  1.4G   0% /sys/fs/cgroup
> /dev/dm-16       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/a0f3c5ab84aa06d5b2db00c4324dd6bf7141500ff4c83e23e9aba7c7268bcad4
> tmpfs           285M     0  285M   0% /run/user/0
> /dev/dm-3        10G  526M  9.5G   6% /var/lib/docker/devicemapper/mnt/0ef509f0c144e458c0c2f9bdbf6941e11d292a032953d48fb316c90caa05f21a
> shm              64M     0   64M   0% /var/lib/docker/containers/71eea4f1af5b284059599eb07cc2aa2d73bdeccc9885952f2062763c81528d8f/mounts/shm
> /dev/dm-4        10G   99M  9.9G   1% /var/lib/docker/devicemapper/mnt/3e108b5c2f45bcfe8c5f7ad28e7f8510ee666d5b4cd728a4b899857750440f1b
> shm              64M     0   64M   0% /var/lib/docker/containers/7e9c401ec8993d7f24870692f84926bf39e4673ec4f23e1d9a7b7147cbfdf9b7/mounts/shm
> /dev/dm-2        10G  266M  9.8G   3% /var/lib/docker/devicemapper/mnt/4d1e151d4270c38e96103713ddc9a3fc00b4e9411ddd89e3791c319508f12449
> shm              64M  8.0K   64M   1% /var/lib/docker/containers/b6138e616e81c3e731499fc14793786fe60f334c6281f8116e8029d1cf561b6a/mounts/shm
> /dev/dm-5        10G  325M  9.7G   4% /var/lib/docker/devicemapper/mnt/e1c6b5e20fed47cf38fb287ec424dac4a5c65c660b40de68fd604360e09082e4
> shm              64M     0   64M   0% /var/lib/docker/containers/634ca1304587c7ad37b763512e7feece81e197a1229e4176213a9b70bb2984d0/mounts/shm
> /dev/dm-6        10G   10G   20K 100% /var/lib/docker/devicemapper/mnt/38fc5fa4880f505e58067e90674e1c560378a75c9cc993dd6a673d1dd674fdff
> shm              64M     0   64M   0% /var/lib/docker/containers/3a10f4a0e3123f7877a10f933d0d18df86b85b16b449457505f59e83d3e22192/mounts/shm
> /dev/dm-8        10G  803M  9.3G   8% /var/lib/docker/devicemapper/mnt/456c4aa19f3ad57d3e7abc4a69e972371cfec2b67faeee4e548f18b2affe42f8
> shm              64M     0   64M   0% /var/lib/docker/containers/e8964627c7b008829d4745c46d0210fe97d54f36693085145184243e17adba91/mounts/shm
> /dev/dm-7        10G  1.2G  8.9G  12% /var/lib/docker/devicemapper/mnt/39e19f7dc51ed511d887932e13903129d1a3f13a6aee6c0e69c8e9e63dacfa6c
> shm              64M     0   64M   0% /var/lib/docker/containers/1c2e3b465890a5cd010ab73bfc7f73c992e9abb49043357dc4916adeb74c8bed/mounts/shm
> /dev/dm-10       10G  146M  9.9G   2% /var/lib/docker/devicemapper/mnt/5913150a306d91b81d12cd5f759aeb1cd6a192ca7dcaffbd8581d4e8735cc2d1
> shm              64M     0   64M   0% /var/lib/docker/containers/6414fa2c750fcb723c836772d112589a67dff7ee636aa8e03c78e288b5d77ae5/mounts/shm
> /dev/dm-11       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/2e557f90bc0dcc7e6f85a7536b2b480ae1bd61dcaf778f3fa3a0db5f93503fcf
> shm              64M     0   64M   0% /var/lib/docker/containers/e9857010411222e00b1c990fdd15d61aa03576e9fe4b7829c98e140a33572dc2/mounts/shm
> /dev/dm-9        10G  276M  9.8G   3% /var/lib/docker/devicemapper/mnt/a57a351c9bfd8a30db66a7666ab940690a4c7431b26e6771c8bd381097d179a4
> shm              64M     0   64M   0% /var/lib/docker/containers/dcab63511f6bbd121213ed1fd667547c0339f70568d64747b5c280185951c510/mounts/shm
> Disk clean:
> $docker volume ls -qf dangling=true | xargs docker volume rm 2>&1 | cat;
> 0a72c1d34b80bae0d14fbfd2c2fcf6fc85225c91a68a9c2f2ddcc638d019222e
> 0d2dd091a7a95dd939a7563f1529cde23567370ae0cc26c7cc7a83be6e288559
> 0e6c34eced17c7caf0c4972ab4541de315e8894234373643a633fde866c6a8e5
> 0e048ac11dcc838dc8eae4ce0bb33d5d24196a55a9d25ec74f2146e6c43cb021
> 0e70f251dfde36c48682fa031bdadbb54e5c8b2e2721efb4953111f1df103825
> 0f3d5dee1af5f69de58978a463a7215463556a34dfbb66a495a8dbf0feff2143
> 0f9c3f1d8c22fe5569eae4642c53f75ed79178f6233d2783a321cfeb396de9a9
> 1a83079ea34fa69cb31d68634ab453afb7c38407782185bd0800c4b63b61372e
> 1abac004cb400cf115a642b4008947d7fbb680ceb1bde951fc1ecac830553325
> 1b1bdd6311c6e49190e88232eca570d5ce77b0b81e6c20acd4a0f144f35e8ba0
> 1b90f899753f729e8f6e9317b023bc2b7bfa3445eb4ca05d40da9462f21cf122
> 1b332a0c0ea724fc9c10d8686641000762de479540074160f8d60dd947fb2e6a
> 1bb70a93caeb9bd106536d7add316d8771e1362d40fa00e5888f59711435bc4f
> 1d4712cb10539a47d344fdd711d9b905f01e44dbb47b2f7e2b0210a385311393
> 1e1a194980567c64108fbfd2214602690e30df23595fee8dbdcb1d5fedea961d
> 1f47735dbeb3f6608d7e506b687f36f884848d0f7c253768c9d97108c1f81c8d
> 2a194cb16e57845fff9f20118bbcd7f482878bbb3572f2aac45b07470a4e60e5
> 2acaeef75b19885c0163ec426fe4adb613267b60000021b596db1fafb9d98a16
> 2aff76459463fd4aa6940558ac18cc169f66ce70e682ccd4ce8758e62f98cbe0
> 2ba3c5dc2357aeaa54b1dbb1e923fce8109d57fed0f4e5f8798632751672e403
> 2c7cfce56be947e574b28b1d434cdbdecbebc6b2f0fc3d1c22a869b4e92d0ee7
> 2d566a5614991a077f3b010a70736f42e6d04e825646b9d3c509c759f8f37766
> 2e10db3cb23b09c8d895f01c90054538427cc6399872899d1c8d2bb4ef43711f
> 2e2801a798e33ed3cfd5f0f6c5fc43253150ac0ac8d19494114b2b9004f65d1c
> 2f82f3ab86a609e67286666e75f1a43496842d0708b7aea36ec40a2cd533f117
> 2f79104fb3c570b228b781ce0b23f790f685bb8d452aa734c52425586484cae2
> 3a7fff8d67681e6a701b61c14e74cdcf77fdfb830cfa31ed156865aef57df07d
> 3a5723321d452c33e35d3002193398e2a7a10890bd4adbe6593231633e649bf3
> 3aa1fd9bad1b75e09e13741ace2c9ff5cd8a630cfc49124fbcab85aa5b943222
> .
> .
> .
> Disk space (after disk clean):
> $df -h
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/vda1       197G   38G  150G  20% /
> devtmpfs        1.4G     0  1.4G   0% /dev
> tmpfs           1.4G     0  1.4G   0% /dev/shm
> tmpfs           1.4G  137M  1.3G  10% /run
> tmpfs           1.4G     0  1.4G   0% /sys/fs/cgroup
> /dev/dm-16       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/a0f3c5ab84aa06d5b2db00c4324dd6bf7141500ff4c83e23e9aba7c7268bcad4
> tmpfs           285M     0  285M   0% /run/user/0
> /dev/dm-11       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/2e557f90bc0dcc7e6f85a7536b2b480ae1bd61dcaf778f3fa3a0db5f93503fcf
> shm              64M     0   64M   0% /var/lib/docker/containers/e9857010411222e00b1c990fdd15d61aa03576e9fe4b7829c98e140a33572dc2/mounts/shm
> Disk space(one hour later):
> $df -h
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/vda1       197G   39G  149G  21% /
> devtmpfs        1.4G     0  1.4G   0% /dev
> tmpfs           1.4G   12K  1.4G   1% /dev/shm
> tmpfs           1.4G  138M  1.3G  10% /run
> tmpfs           1.4G     0  1.4G   0% /sys/fs/cgroup
> /dev/dm-16       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/a0f3c5ab84aa06d5b2db00c4324dd6bf7141500ff4c83e23e9aba7c7268bcad4
> tmpfs           285M     0  285M   0% /run/user/0
> /dev/dm-11       10G   49M   10G   1% /var/lib/docker/devicemapper/mnt/2e557f90bc0dcc7e6f85a7536b2b480ae1bd61dcaf778f3fa3a0db5f93503fcf
> shm              64M     0   64M   0% /var/lib/docker/containers/e9857010411222e00b1c990fdd15d61aa03576e9fe4b7829c98e140a33572dc2/mounts/shm
> /dev/dm-2        10G   99M  9.9G   1% /var/lib/docker/devicemapper/mnt/735a5508654bfdb6c0752838c5ab65ff7bf1b82a7329ea20ee215651a5ae39e2
> shm              64M     0   64M   0% /var/lib/docker/containers/95dd176c4df52b02fc8b8a78ebdbab444a7be1213a2cfc54beee134f450bad23/mounts/shm
> /dev/dm-4        10G  526M  9.5G   6% /var/lib/docker/devicemapper/mnt/1a749000c102efd084e52034a44cec2e8524ccdb75bcb27972f2f05dcf083ea8
> shm              64M     0   64M   0% /var/lib/docker/containers/a8178e76fb4107f90fe1187698d58e362969515343b0adf7c428436fe9d482cb/mounts/shm
> /dev/dm-1        10G  383M  9.7G   4% /var/lib/docker/devicemapper/mnt/1dc589a616892704fd55587a7d558307db703155a4fd743219addc7b5a65d962
> /dev/dm-3        10G  266M  9.8G   3% /var/lib/docker/devicemapper/mnt/7a501e67c52c5d2dbd77fd29cc68946469c3a609c2fe356b7bba7fde82f148a1
> shm              64M     0   64M   0% /var/lib/docker/containers/495918e5611c7a416ef20401f5701e1570b94f540eb5d764c47c6a74f73a169f/mounts/shm
> shm              64M  8.0K   64M   1% /var/lib/docker/containers/374f9bb469ec5fe42b7e7772a5d4c580aa36178f64efac8b3fba02f148b42de6/mounts/shm
> /dev/dm-6        10G  502M  9.6G   5% /var/lib/docker/devicemapper/mnt/b58cbb82569f79848c432dae10c8e549ac2e783e1ba065def8a033cf94a625c8
> shm              64M     0   64M   0% /var/lib/docker/containers/98dd55e27406e756b870361e0e25cc7cca4f57a9e95161aa8a3359a13a477e58/mounts/shm
> /dev/dm-7        10G  325M  9.7G   4% /var/lib/docker/devicemapper/mnt/cfcfd404d2fa0e0fc770d34e1b6b0cd27bbef84390bfad02368760c4da2abe87
> shm              64M     0   64M   0% /var/lib/docker/containers/a1bb4f9e726902446809a85d17553b93f773d8013e227faa0a254d90f1821359/mounts/shm
> /dev/dm-9        10G  1.2G  8.9G  12% /var/lib/docker/devicemapper/mnt/1529c11ac704d149b94723287a390dd38d9373c58ed5ed0b21d8d0585a0a7ac7
> shm              64M     0   64M   0% /var/lib/docker/containers/8594cfcef1fe0cf3fa75bac83ce5ef18ef096cad6debe0957384fc424f3edc03/mounts/shm
> /dev/dm-5        10G  276M  9.8G   3% /var/lib/docker/devicemapper/mnt/88b94454d368ae2066fde71dd0062252b09fb87c8e4a3c8fda351013ea7a7182
> shm              64M     0   64M   0% /var/lib/docker/containers/e1900109937266b65090c366c60a70581c0584474346083c8a9b971831b3edaf/mounts/shm
> /dev/dm-10       10G  803M  9.3G   8% /var/lib/docker/devicemapper/mnt/9a84f16db1fe54d4d0d6719ab1a48c6d88d53880c65345c010bf8fc4164166d6
> shm              64M     0   64M   0% /var/lib/docker/containers/10db801e75d33f20f07954848ad2c8c3a4213d59c0046469c8a773fb1e0f172a/mounts/shm
> /dev/dm-8        10G  146M  9.9G   2% /var/lib/docker/devicemapper/mnt/dd643159c8d53030e6f9bac07353fabb4d279015b7f565c2f05fd60b074109d1
> shm              64M     0   64M   0% /var/lib/docker/containers/47ac2d9eb4beeddc4908ceb2c415abf9311cd4eac8d40825049c7965fd8644e4/mounts/shm
> Isn't there a way to avoid this issue? I know this is completely docker issue, but added fiware-orion tag so if anyone had similar experience.



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)