Throughout this documentation, we can refer to Central EGA as CEGA, or CentralEGA, and any Local EGA instance as LEGA, or LocalEGA. When two or more Local EGA instances are involved, we will use LEGA<i> for Local EGA instance <i>.

Local EGA

The Local EGA project is divided into several microservices.

Service Description Status
db A Postgres database with appropriate schema
mq A RabbitMQ message broker with appropriate accounts, exchanges, queues and bindings. We use a federated queue to get messages from CentralEGA's broker and shovels to send answers back.
inbox SFTP server, acting as a dropbox, where user credentials come from Central EGA
keyserver Handles the encryption/decryption keys
Ingesters Split the Crypt4GH header and move the remainder to the storage backend. No cryptographic task, nor connection to the keyserver.
Verifiers Connect to the keyserver (via SSL) and decrypt the stored files and checksum them against their embedded checksum.
archive Storage backend: as a regular file system or as a S3 object store.
ID Mapper Handles the so-called Stable ID filename mappings from CentralEGA.. store.

The workflow consists of two ordered parts:

The user first logs onto the Local EGA’s inbox and uploads its files. He/She then goes to the Central EGA’s interface to prepare a submission. Upon completion, the files are ingested into the archive and become searchable by the Central EGA’s engine.

More concretely, Central EGA contains a database of users. The Central EGA’ ID is used to authenticate the user against either their EGA password or an RSA key.

For every uploaded file, Central EGA receives a notification that the file has landed. The file is checksumed and presented in the Central EGA’s interface in order for the user to double-check that it was properly uploaded.

☞ More details about the Inbox login system.

When a submission is ready, Central EGA triggers an ingestion process on the user-chosen Local EGA instance. The uploaded file must be encrypted in the Crypt4GH file format using that Local EGA’s public PGP key. Central EGA’s interface is updated with progress notifications whether the ingestion was successful, or whether there was an error.

☞ More details about the Ingestion Procedure.

General Architecture and Connected Components


Codacy Badge | Build Status | Version 1.1 | Generated May 21, 2019