 |
Firstly, in API block, there are four function blocks: Authentication/Authorization, Config Manager, Source Manager, and Job Manager. More specifically, Config Manager is responsible for managing the configuration information such as default data storage, mapping rules from different standards to the datahub¡¯s standard. Source Manager is responsible for managing the source information such as name, access endpoint, and standard type. Job Manager is used for managing the job information such as started time, ended time, and current status
Secondly, Handler block consists of four function blocks: Authorizer, Cleaner, Elastic Indexer, and Queue Manager. While the first three blocks are mainly responsible for improving security and performance, Queue Manager is used to manage the queue information in Data Queue for interconnection.
Thirdly, there are three function blocks in Harvester block: Object Collector, Metadata Importer, and Data Downloader. In particular, Object Collecting takes charge of requesting the metadata from external systems based on the registered source information. Metadata Importer is responsible for mapping and storing the metadata into database while Download Importer is responsible for downloading and storing the data content. It means that database is not only responsible for storing the internal metadata such as source, jobm and config but also for storing the metadata collected from external systems.
|
IEEE/ICACT20220341 Slide.10
[Big Slide]
[YouTube]