Throughout this documentation, we can refer to Central EGA as
CentralEGA, and any Local EGA instance as
LocalEGA. When two or more Local EGA instances are involved,
we will use
LEGA<i> for Local EGA instance
The Local EGA project is divided into several microservices.
|db||A Postgres database with appropriate schema|
|mq||A RabbitMQ message broker with appropriate accounts, exchanges, queues and bindings. We use a federated queue to get messages from CentralEGA's broker and shovels to send answers back.|
|inbox||SFTP server, acting as a dropbox, where user credentials come from Central EGA|
|keyserver||Handles the encryption/decryption keys|
|Ingesters||Split the Crypt4GH header and move the remainder to the storage backend. No cryptographic task, nor connection to the keyserver.|
|Verifiers||Connect to the keyserver (via SSL) and decrypt the stored files and checksum them against their embedded checksum.|
|archive||Storage backend: as a regular file system or as a S3 object store.|
|ID Mapper||Handles the so-called Stable ID filename mappings from CentralEGA.. store.|
The workflow consists of two ordered parts:
The user first logs onto the Local EGA’s inbox and uploads its files. He/She then goes to the Central EGA’s interface to prepare a submission. Upon completion, the files are ingested into the archive and become searchable by the Central EGA’s engine.
More concretely, Central EGA contains a database of users. The Central EGA’ ID is used to authenticate the user against either their EGA password or an RSA key.
For every uploaded file, Central EGA receives a notification that the file has landed. The file is checksumed and presented in the Central EGA’s interface in order for the user to double-check that it was properly uploaded.
☞ More details about the Inbox login system.
When a submission is ready, Central EGA triggers an ingestion process
on the user-chosen Local EGA instance. The uploaded file must be
encrypted in the
Crypt4GH file format using that Local EGA’s public PGP
key. Central EGA’s interface is updated with progress notifications
whether the ingestion was successful, or whether there was an error.
☞ More details about the Ingestion Procedure.