[ https://jira.fiware.org/browse/HELP-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fernando Lopez reassigned HELP-19044:
-------------------------------------
Assignee: Miguel Ángel Pedraza
> [fiware-stackoverflow] CYGNUS_MONGO_DATA_EXPIRATION doesn't work?
> ---------------------------------------------------------------------
>
> Key: HELP-19044
> URL: https://jira.fiware.org/browse/HELP-19044
> Project: Help-Desk
> Issue Type: Monitor
> Components: FIWARE-TECH-HELP
> Reporter: Backlog Manager
> Assignee: Miguel Ángel Pedraza
> Labels: fiware, fiware-cygnus, fiware-orion, fiware-sth-comet, mongodb
>
> Created question in FIWARE Q/A platform on 14-12-2021 at 21:12
> {color: red}Please, ANSWER this question AT{color} https://stackoverflow.com/questions/70355924/cygnus-mongo-data-expiration-doesnt-work
> +Question:+
> CYGNUS_MONGO_DATA_EXPIRATION doesn't work?
> +Description:+
> I installed Orion, MongoDB, Cygnus and sth-comet with docker-compose. I set CYGNUS_MONGO_DATA_EXPIRATION=60 for Cygnus.
> I created a subscription to an Orion entity, but the historical data doesn't expire. I want the old data gone.
> dockyer-compose.yaml is as follows:
> version: "3.5"
> services:
> # Databases - Orion uses Mongo-DB, Cygnus is persisting to Postgres
> # Modified to use password authentication between mongoDB and Fiware Orion to improve security in docker.
> mongo-db:
> image: mongo:4.4
> hostname: mongo-db
> container_name: mongo-db
> ports:
> - "27017:27017"
> #command: --nojournalS
> environment:
> - "MONGO_INITDB_ROOT_USERNAME=${MONGOUSERNAME}" # Define the mongoDB user name
> - "MONGO_INITDB_ROOT_PASSWORD=${MONGOPASSWORD}" # Define passwords for security measures within docker
> - "MONGO_INITDB_DATABASE=admin"
> volumes:
> - ./db:/data/db
> - ./configdb:/data/configdb
> orion:
> image: fiware/orion:3.3.1
> hostname: orion
> container_name: fiware-orion
> depends_on:
> - mongo-db
> environment:
> - "ORION_MONGO_HOST=mongo-db:27017"
> - "ORION_MONGO_USER=${MONGOUSERNAME}" # Define the mongoDB user name
> - "ORION_MONGO_PASSWORD=${MONGOPASSWORD}" # Define passwords for security measures within docker
> - "ORION_MONGO_AUTH_SOURCE=admin"
> - "ORION_PORT=1026"
> - "ORION_LOG_LEVEL=INFO"
> ports:
> - "1026:1026"
> command: -dbhost mongo-db
> networks:
> #- proxynet #Comment out the openwhisk network as it is not necessary when building the development server.
> - default
> # Cygnus is configured to write context data to PostgeSQL
> cygnus:
> image: fiware/cygnus-ngsi:2.14.0
> hostname: cygnus
> container_name: fiware-cygnus
> networks:
> - default
> depends_on:
> - mongo-db
> ports:
> - "5055:5055"
> - "5080:5080"
> environment:
> - "CYGNUS_MONGO_HOSTS=mongo-db:27017" # Hostname of the PostgreSQL server used to persist historical context data
> - "CYGNUS_MONGO_USER=${MONGOUSERNAME}"
> - "CYGNUS_MONGO_PASS=${MONGOPASSWORD}"
> - "CYGNUS_MONGO_AUTH_SOURCE=admin"
> - "CYGNUS_MONGO_SERVICE_PORT=5055"
> - "CYGNUS_LOG_LEVEL=DEBUG"
> - "CYGNUS_SERVICE_PORT=5055" # Notification Port that Cygnus listens to for Postgres subscriptions
> - "CYGNUS_API_PORT=5080" # Port that Cygnus listens on for operational reasons
> - "CYGNUS_ORION_SSL=true"
> - "CYGNUS_ORION_HOST=orion"
> - "CYGNUS_ORION_PORT=1026"
> - "CYGNUS_MONGO_DATA_EXPIRATION=60"
> sth-comet:
> image: fiware/sth-comet
> hostname: sth-comet
> container_name: fiware-sth-comet
> depends_on:
> - mongo-db
> networks:
> - default
> ports:
> - "8666:8666"
> environment:
> - "STH_HOST=0.0.0.0"
> - "STH_PORT=8666"
> - "DB_PREFIX=sth_"
> - "DB_USERNAME=${MONGOUSERNAME}"
> - "DB_PASSWORD=${MONGOPASSWORD}"
> - "DB_AUTH_SOURCE=admin"
> - "DB_URI=mongo-db:27017"
> - "LOGOPS_LEVEL=DEBUG"
> volumes:
> mongo-db: ~
> networks:
> default:
> ipam:
> config:
> - subnet: 172.18.1.0/24
> Prior to docker-compose up -d, I made the following directories:
> db
> config
> Next, I set a subscription like this:
> curl -X POST http://localhost:1026/v2/subscriptions -H 'Content-Type: application/json' -d @- <<EOF
> {
> "description": "Notify all context changes to Cygnus",
> "subject": {
> "entities": [
> {
> "idPattern": ".*"
> }
> ]
> },
> "notification": {
> "http": {
> "url": "http://cygnus:5055/notify"
> }
> }
> }
> EOF
> The oriong's log is:
> time=2021-12-14T00:32:01.654Z | lvl=INFO | corr=4130a13c-5c75-11ec-b540-0242ac120103 | trans=1639441850-794-00000000001 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: POST /v2/subscriptions, request payload (217 bytes): { "description": "Notify all context changes to Cygnus", "subject": { "entities": [ { "idPattern": ".*" } ] }, "notification": { "http": { "url": "http://cygnus:5055/notify" } }}, response code: 201
> Then, I inserted the following entity:
> curl localhost:1026/v2/entities -s -S -H 'Content-Type: application/json' -d @- <<EOF
> {"id":"Room1","type":"Room","temperature":{"value":23,"type":"Float"},"pressure":{"value":720,"type":"Integer"}}
> EOF
> Orion said(I'm sorry that I lost the head of the log line):
> subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: POST /v2/entities, request payload (112 bytes): {"id":"Room1","type":"Room","temperature":{"value":23,"type":"Float"},"pressure":{"value":720,"type":"Integer"}}, response code: 201
> Then, I updated the entity:
> curl localhost:1026/v2/entities/Room1/attrs -s -S -H 'Content-Type: application/json' -X PATCH -d @- <<EOF
> {"temperature":{"value":24,"type":"Float"},"pressure":{"value":730,"type":"Float"}}
> EOF
> Orion said:
> time=2021-12-14T00:37:12.558Z | lvl=INFO | corr=fa8479ba-5c75-11ec-8b92-0242ac120103 | trans=1639441850-794-00000000007 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: PATCH /v2/entities/Room1/attrs, request payload (83 bytes): {"temperature":{"value":24,"type":"Float"},"pressure":{"value":730,"type":"Float"}}, response code: 204
> time=2021-12-14T00:37:12.591Z | lvl=INFO | corr=fa8479ba-5c75-11ec-8b92-0242ac120103; cbnotif=1 | trans=1639441850-794-00000000008 | from=172.18.1.1 | srv=<none> | subsrv=/ | comp=Orion | op=logTracing.cpp[65]:logInfoHttpNotification | msg=Notif delivered (subId: 61b7e60138b3ce79144d71ab): POST cygnus:5055/notify, response code: 200
> I waited some minutes and updated the entity again:
> curl localhost:1026/v2/entities/Room1/attrs -s -S -H 'Content-Type: application/json' -X PATCH -d @- <<EOF
> {"temperature":{"value":25,"type":"Float"},"pressure":{"value":740,"type":"Float"}}
> EOF
> The log of orion is:
> time=2021-12-14T00:40:15.524Z | lvl=INFO | corr=6793036e-5c76-11ec-8fa2-0242ac120103 | trans=1639441850-794-00000000009 | from=172.18.1.1 | srv=<none> | subsrv=<none> | comp=Orion | op=logTracing.cpp[148]:logInfoRequestWithPayload | msg=Request received: PATCH /v2/entities/Room1/attrs, request payload (83 bytes): {"temperature":{"value":25,"type":"Float"},"pressure":{"value":740,"type":"Float"}}, response code: 204
> time=2021-12-14T00:40:15.558Z | lvl=INFO | corr=6793036e-5c76-11ec-8fa2-0242ac120103; cbnotif=1 | trans=1639441850-794-00000000010 | from=172.18.1.1 | srv=<none> | subsrv=/ | comp=Orion | op=logTracing.cpp[65]:logInfoHttpNotification | msg=Notif delivered (subId: 61b7e60138b3ce79144d71ab): POST cygnus:5055/notify, response code: 200
> Lastly, I printed out the historical data:
> It shows the historical table contains the old updating. How can I eliminate the old ones automatically?
> The log of cygnus is(It's so big that I made an excerpt of the log):
> https://gist.github.com/ishidakei/b3aea7b287899c472b74a438ce4451c8
--
This message was sent by Atlassian JIRA
(v6.4.1#64016)
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy