[Masterdata] real time data in the EU portal?

[Alberto Abella]

OMG.... I guess they need to have different resources for real-time
info and for historical databases, otherwise they are wasting some
efforts...

On Tue, 2020-10-27 at 18:14 +0100, Johnny Choque wrote:
> - Short answer: Non real-time harvesting 
> 
> - Long answer: (from EDP Data Supplier Guidelines)
> Due to the high volume of metadata that will be harvested from a growing list of data suppliers and the required runtime for the harvesting processes, each data supplier site will probably not be harvested on a daily basis. Hereto the harvesting processes have to be clustered and scheduled on a fixed time schedule (e.g. during the night) in order to avoid any load impacts on the harvested sites during regular business hours usage by their users.
> Log files which are written by the harvesting processes can be used to identify the optimal time to run the harvesters. In addition log-based evaluation methods will be used to monitor the performance of each harvester. If the performance of a harvester decreases due to resource problems, a ticket will be posted to the data source via the helpdesk.
> 
> -----Mensaje original-----
> De: masterdata-bounces at lists.fiware.org <masterdata-bounces at lists.fiware.org> En nombre de Alberto Abella
> Enviado el: martes, 27 de octubre de 2020 16:49
> Para: masterdata proposal list <masterdata at lists.fiware.org>
> Asunto: [Masterdata] real time data in the EU portal?
> 
> I was wondering whast are the capabilities of data ingestion of the EU open data portal, because if it is same approach as Spanish open data portal one real-time data is just something hard to be implemented for the debugging mechanisms and other restrictions imposed by the portal . 
> Could the same happen in the EU portal?
> 
> It is not critical that real-time data ingestion for the overall project but it would be interesting to know what are the EU portal capabilities, specially for the integration.
> 
> Best
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/masterdata/attachments/20201027/73526099/attachment.html>