[Backlogmanager] [FIWARE-JIRA] (HELP-20975) [fiware-stackoverflow] Data loss when using Fiware Orion Broker, QuantumLeap and CrateDB

[Ken Zangelin (JIRA)]

     [ https://jira.fiware.org/browse/HELP-20975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ken Zangelin updated HELP-20975:
--------------------------------
    Status: In Progress  (was: Open)

> [fiware-stackoverflow] Data loss when using Fiware Orion Broker, QuantumLeap and CrateDB
> ----------------------------------------------------------------------------------------
>
>                 Key: HELP-20975
>                 URL: https://jira.fiware.org/browse/HELP-20975
>             Project: Help-Desk
>          Issue Type: Monitor
>          Components: FIWARE-TECH-HELP
>            Reporter: Backlog Manager
>            Assignee: Ken Zangelin
>              Labels: cratedb, fiware, fiware-orion
>
> Created question in FIWARE Q/A platform on 15-03-2023 at 18:03
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/75748890/data-loss-when-using-fiware-orion-broker-quantumleap-and-cratedb
> +Question:+
> Data loss when using Fiware Orion Broker, QuantumLeap and CrateDB
> +Description:+
> I'm using Fiware Orion Broker, QuantumLeap and CrateDB, with the aim of recording all temporal data in cratedb.
> My docker-compose configuration is this:
> orion:
>     image: fiware/orion:${ORION_VERSION}
>     hostname: orion
>     container_name: fiware-orion
>     depends_on:
>         - mongo-db
>     networks:
>         - fiware
>     expose:
>         - "${ORION_PORT}"
>     ports:
>         - "${ORION_PORT}:${ORION_PORT}"
>     command: -dbhost mongo-db
>     healthcheck:
>         test: curl --fail -s http://orion:${ORION_PORT}/version || exit 1
>         interval: 5s
> mongo-db:
>     image: mongo:latest
>     hostname: mongo-db
>     container_name: db-mongo
>     expose:
>         - "${MONGO_DB_PORT}"
>     ports:
>         - "${MONGO_DB_PORT}:${MONGO_DB_PORT}"
>     networks:
>         - fiware
>     volumes:
>         -  ./volumes/mongo-db:/data
>     healthcheck:
>         test: |
>             host=`hostname --ip-address || echo '127.0.0.1'`; 
>             mongo --quiet $host/test --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' && echo 0 || echo 1
>         interval: 5s
> quantumleap:
>     image: orchestracities/quantumleap:latest
>     hostname: quantumleap
>     container_name: fiware-quantumleap
>     ports:
>         - "${QUANTUMLEAP_PORT}:${QUANTUMLEAP_PORT}"
>     depends_on:
>         - crate-db
>         - redis-db
>     environment:
>         - CRATE_HOST=crate-db
>         - LOGLEVEL=WARNING
>     healthcheck:
>         test: curl --fail -s http://quantumleap:${QUANTUMLEAP_PORT}/version || exit 1
>     networks:
>         - fiware
> crate-db:
>     image: crate:latest
>     hostname: crate-db
>     container_name: db-crate
>     ports:
>         - "4200:4200"
>         - "4300:4300"
>     command: crate -Cauth.host_based.enabled=false  -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*" -Cdiscovery.type=single-node
>     environment:
>         - CRATE_HEAP_SIZE=2g 
>     volumes:
>         - ./volumes/crate-db:/data
>     networks:
>         - fiware
> I'm running performance tests through Apache JMeter, and consecutive requests are sent for 1 minutes to evaluate their performance, but I'm having a problem where some data is not being registered in the CrateDB, that is, in the last test I did, it was done about 18000 requests, and in CrateDB only about 10000 are registered.
> I also tried using the TimescaleDB database in QuantumLeap, but the same problem happens, so I assume that the problem is not with the database.
> Does anyone know what the problem could be?



--
This message was sent by Atlassian JIRA
(v6.4.1#64016)