# End-to-End Data Flow

This page traces the three core data flows through the quicproquo system:
registration, group creation, and message exchange. Each flow is illustrated
with an ASCII sequence diagram showing control-plane (AS) and data-plane (DS)
traffic.

Throughout these flows the server is **MLS-unaware** -- it stores and forwards
opaque byte blobs without parsing their MLS content.

---

## 1. Registration Flow

Before a client can join any MLS group, it must generate an Ed25519 identity
keypair and upload at least one KeyPackage to the Authentication Service. Peers
fetch these KeyPackages to add the client to groups.

### Sequence Diagram

```text
  Client (Alice)                           NodeService (AS)
  ──────────────                           ────────────────
        │                                         │
        │  1. Generate Ed25519 identity keypair    │
        │     (IdentityKeypair::generate)          │
        │                                         │
        │  2. Generate MLS KeyPackage              │
        │     (GroupMember::generate_key_package)   │
        │     - Creates HPKE init keypair          │
        │     - Embeds Ed25519 pk in credential    │
        │     - Signs leaf node with Ed25519 sk    │
        │     - TLS-encodes the KeyPackage         │
        │                                         │
        │  3. QUIC connect + TLS 1.3 handshake     │
        │ ────────────────────────────────────────>│
        │                                         │
        │  4. uploadKeyPackage(identityKey, pkg)   │
        │ ────────────────────────────────────────>│
        │                                         │  5. Validate:
        │                                         │     - identityKey == 32 bytes
        │                                         │     - package non-empty, <= 1 MB
        │                                         │     - auth version allowed
        │                                         │
        │                                         │  6. Compute SHA-256(package)
        │                                         │
        │                                         │  7. Append to per-identity queue:
        │                                         │     keyPackages[identityKey].push(pkg)
        │                                         │
        │                                         │  8. Flush keypackages.bin to disk
        │                                         │
        │              fingerprint (SHA-256)       │
        │ <────────────────────────────────────────│
        │                                         │
        │  9. Compare local fingerprint with       │
        │     server-returned fingerprint          │
        │     (tamper detection)                   │
        │                                         │
```

### Key Points

- **KeyPackages are single-use** (RFC 9420 requirement). Each `fetchKeyPackage`
  call atomically removes and returns one package. The client should upload
  multiple KeyPackages if it expects to be added to several groups.

- The `identityKey` used as the AS index is the **raw 32-byte Ed25519 public
  key**, not a fingerprint or hash. Peers must know Alice's public key out-of-
  band (QR code, directory, etc.) to fetch her KeyPackage.

- The HPKE init private key generated during `generate_key_package` is stored
  in the client's `DiskKeyStore`. The **same `GroupMember` instance** (or a
  restored instance with the same key store) must later call `join_group` to
  decrypt the Welcome message.

- The optional hybrid public key (`uploadHybridKey`) can also be uploaded
  during registration for post-quantum envelope encryption.

---

## 2. Group Creation Flow

Alice creates a new MLS group, fetches Bob's KeyPackage from the AS, adds Bob
to the group (producing a Commit and a Welcome), and delivers the Welcome to
Bob via the DS.

### Sequence Diagram

```text
  Alice                       NodeService (AS+DS)                    Bob
  ─────                       ──────────────────                    ───
    │                                 │                               │
    │  1. create_group("my-group")    │                               │
    │     (local MLS operation --     │                               │
    │      Alice is sole member,      │                               │
    │      epoch 0)                   │                               │
    │                                 │                               │
    │  2. fetchKeyPackage(bob_pk)     │                               │
    │ ───────────────────────────────>│                               │
    │                                 │  3. Pop bob's KeyPackage      │
    │                                 │     from queue (atomic)       │
    │          bob_kp bytes           │                               │
    │ <───────────────────────────────│                               │
    │                                 │                               │
    │  4. add_member(bob_kp)          │                               │
    │     Local MLS operations:       │                               │
    │     a. Deserialise & validate   │                               │
    │        Bob's KeyPackage         │                               │
    │     b. Produce Commit message   │                               │
    │        (adds Bob to ratchet     │                               │
    │         tree, advances epoch)   │                               │
    │     c. Produce Welcome message  │                               │
    │        (encrypted to Bob's      │                               │
    │         HPKE init key, contains │                               │
    │         group secrets + tree)   │                               │
    │     d. merge_pending_commit()   │                               │
    │        (Alice advances to       │                               │
    │         epoch 1 locally)        │                               │
    │                                 │                               │
    │  5. enqueue(bob_pk, welcome)    │                               │
    │ ───────────────────────────────>│                               │
    │                                 │  6. Append welcome to         │
    │                                 │     deliveries[(ch, bob_pk)]  │
    │                                 │                               │
    │                                 │  7. Notify bob_pk waiters     │
    │                                 │                               │
    │                                 │                               │
    │                                 │  8. Bob connects and fetches  │
    │                                 │ <─────────────────────────────│
    │                                 │     fetch(bob_pk)             │
    │                                 │                               │
    │                                 │  9. Drain bob's queue         │
    │                                 │     (returns [welcome])       │
    │                                 │                               │
    │                                 │        [welcome_bytes]        │
    │                                 │ ─────────────────────────────>│
    │                                 │                               │
    │                                 │                               │  10. join_group(welcome)
    │                                 │                               │      - Decrypt Welcome with
    │                                 │                               │        HPKE init private key
    │                                 │                               │      - Extract ratchet tree
    │                                 │                               │        from GroupInfo ext
    │                                 │                               │      - Initialise MlsGroup
    │                                 │                               │        at epoch 1
    │                                 │                               │
    │                                 │                               │  Bob is now a group member
    │                                 │                               │
```

### Key Points

- The **Commit** message is relevant for groups with more than two members. In
  the two-party case, Alice is the sole existing member and merges the commit
  herself. In a multi-member group, the Commit would be sent to all existing
  members via the DS so they can advance their epoch.

- The **Welcome** message is encrypted to Bob's HPKE init key (derived from
  the KeyPackage). Only the `GroupMember` instance that generated that
  KeyPackage holds the corresponding private key.

- The `use_ratchet_tree_extension = true` MLS config embeds the full ratchet
  tree in the Welcome's `GroupInfo` extension. This means Bob does not need a
  separate tree fetch -- `new_from_welcome` extracts it automatically.

- The DS routes solely by `recipientKey` (Bob's Ed25519 public key). It does
  not parse the Welcome, the Commit, or any MLS structure.

---

## 3. Message Exchange Flow

After both Alice and Bob are group members, they exchange MLS Application
messages through the DS.

### Sequence Diagram

```text
  Alice                       NodeService (DS)                      Bob
  ─────                       ──────────────────                    ───
    │                                 │                               │
    │  ─── Alice sends a message to Bob ───                          │
    │                                 │                               │
    │  1. send_message("hello bob")   │                               │
    │     MLS create_message():       │                               │
    │     - Derive message key from   │                               │
    │       epoch secret + gen counter│                               │
    │     - Encrypt plaintext with    │                               │
    │       AES-128-GCM               │                               │
    │     - Produce MlsMessageOut     │                               │
    │       (PrivateMessage variant)  │                               │
    │     - TLS-encode to bytes       │                               │
    │                                 │                               │
    │  2. enqueue(bob_pk, ciphertext) │                               │
    │ ───────────────────────────────>│                               │
    │                                 │  3. Store in bob's queue      │
    │                                 │  4. Notify bob_pk waiters     │
    │                                 │                               │
    │                                 │          (time passes)        │
    │                                 │                               │
    │                                 │  5. Bob polls for messages    │
    │                                 │ <─────────────────────────────│
    │                                 │     fetchWait(bob_pk, 30000)  │
    │                                 │                               │
    │                                 │  6. Drain bob's queue         │
    │                                 │        [ciphertext]           │
    │                                 │ ─────────────────────────────>│
    │                                 │                               │
    │                                 │                               │  7. receive_message(ct)
    │                                 │                               │     MLS process_message():
    │                                 │                               │     - Identify sender from
    │                                 │                               │       PrivateMessage header
    │                                 │                               │     - Derive decryption key
    │                                 │                               │       from epoch secret
    │                                 │                               │     - Decrypt AES-128-GCM
    │                                 │                               │     - Return plaintext:
    │                                 │                               │       "hello bob"
    │                                 │                               │
    │  ─── Bob replies to Alice ───                                   │
    │                                 │                               │
    │                                 │                               │  8. send_message("hello alice")
    │                                 │                               │     (same MLS encrypt flow)
    │                                 │                               │
    │                                 │  9. enqueue(alice_pk, ct)     │
    │                                 │ <─────────────────────────────│
    │                                 │ 10. Store + notify            │
    │                                 │                               │
    │ 11. fetch(alice_pk)             │                               │
    │ ───────────────────────────────>│                               │
    │         [ciphertext]            │                               │
    │ <───────────────────────────────│                               │
    │                                 │                               │
    │ 12. receive_message(ct)         │                               │
    │     -> "hello alice"            │                               │
    │                                 │                               │
```

### Key Points

- **MLS provides forward secrecy**: each message is encrypted with a key
  derived from the current epoch secret and a per-sender generation counter.
  Compromising a future key does not reveal past messages.

- **The DS is a dumb relay**: it does not decrypt, inspect, or reorder
  messages. It stores opaque byte blobs in a FIFO queue keyed by recipient.

- **Long-polling** via `fetchWait` avoids the need for persistent connections
  or WebSocket-style push. The client specifies a timeout in milliseconds; the
  server blocks up to that duration using `tokio::sync::Notify`. The `recv
  --stream` CLI flag loops `fetchWait` indefinitely for continuous message
  reception.

- **Channel-aware routing** is supported: the `channelId` field in `enqueue`
  and `fetch` allows scoping queues by channel (e.g., a 16-byte UUID for
  1:1 conversations). When `channelId` is empty, messages go to the default
  (legacy) queue.

---

## Control-Plane vs. Data-Plane Summary

```text
┌─────────────────────────────────────────────────────────────────────┐
│                        Control Plane (AS)                           │
│                                                                     │
│  uploadKeyPackage    ────>  Store KeyPackage for identity           │
│  fetchKeyPackage     <────  Pop and return one KeyPackage           │
│  uploadHybridKey     ────>  Store hybrid PQ public key              │
│  fetchHybridKey      <────  Return hybrid PQ public key             │
│                                                                     │
│  Traffic: Infrequent. Once per group join (upload before,           │
│           fetch during group add).                                  │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│                        Data Plane (DS)                              │
│                                                                     │
│  enqueue             ────>  Append payload to recipient queue       │
│  fetch               <────  Drain and return all queued payloads    │
│  fetchWait           <────  Long-poll drain with timeout            │
│                                                                     │
│  Traffic: High-frequency. Every MLS message (Welcome, Commit,       │
│           Application) flows through the DS.                        │
└─────────────────────────────────────────────────────────────────────┘
```

The separation means the AS can be rate-limited or placed behind stricter
access controls without affecting message throughput on the DS.

---

## State Transitions

The following diagram summarises the client-side state machine across all three
flows:

```text
                      ┌──────────────┐
                      │   No State    │
                      └──────┬───────┘
                             │
                 IdentityKeypair::generate()
                             │
                             ▼
                      ┌──────────────┐
                      │   Identity    │  Ed25519 keypair exists
                      │   Generated   │  No KeyPackage, no group
                      └──────┬───────┘
                             │
              generate_key_package() + uploadKeyPackage()
                             │
                             ▼
                      ┌──────────────┐
                      │  Registered   │  KeyPackage on AS
                      │              │  HPKE init key in DiskKeyStore
                      └──────┬───────┘
                             │
              ┌──────────────┴──────────────┐
              │                             │
      create_group()                join_group(welcome)
              │                             │
              ▼                             ▼
       ┌─────────────┐              ┌──────────────┐
       │ Group Owner  │              │ Group Member  │
       │ (epoch 0)    │              │ (epoch N)     │
       └──────┬──────┘              └──────┬───────┘
              │                             │
       add_member()                         │
              │                             │
              ▼                             ▼
       ┌──────────────────────────────────────────┐
       │          Active Group Member              │
       │                                          │
       │  send_message()  -> enqueue via DS       │
       │  receive_message() <- fetch from DS       │
       │                                          │
       │  Epoch advances on each Commit           │
       └──────────────────────────────────────────┘
```

---

## Further Reading

- [Architecture Overview](overview.md) -- system diagram and two-service model
- [Service Architecture](service-architecture.md) -- RPC method details and long-polling internals
- [GroupMember Lifecycle](../internals/group-member-lifecycle.md) -- detailed MLS state machine
- [KeyPackage Exchange Flow](../internals/keypackage-exchange.md) -- single-use semantics and AS internals
- [MLS (RFC 9420)](../protocol-layers/mls.md) -- key schedule, ratchet tree, and ciphersuite details
- [Forward Secrecy](../cryptography/forward-secrecy.md) -- how MLS provides forward secrecy
- [Post-Compromise Security](../cryptography/post-compromise-security.md) -- group healing after key compromise
