Architecture

End-to-end prompt encryption

The Privatemode API uses end-to-end encryption to secure user data, ensuring that every component involved in processing an API call only handles encrypted information and can't access the original data. Prompts are encrypted on the client side, decrypted within runtime-encrypted workers, and re-encrypted before being returned to the client.

Secure key exchange

Privatemode's key exchange protocol has two main goals:

  1. Only initiate key exchange with verified and trusted AI workers.
  2. Establish an end-to-end confidential channel on application level between the client and the AI workers.

Workflow

  1. Coordinator verification: The client connects to the Privatemode backend to verify the identity and integrity of the Contrast Coordinator through remote attestation. This ensures the client is talking to a genuine, untampered Privatemode deployment.
  2. Root certificate retrieval: Upon successful verification, the backend provides the Coordinator Mesh CA certificate to the client. This certificate acts as the trust anchor for the entire deployment.
  3. Key agreement: The client agrees on a symmetric encryption key with the secret service using HPKE with the hybrid post-quantum KEM X25519MLKEM768:
    • The client generates an ephemeral key pair and sends the public key to the secret service through the Privatemode HTTP API.
    • The secret service derives a shared secret for the client's public key and returns the encapsulated key together with its certificate, signing both with its private key. The certificate is issued by the Mesh CA.
    • The client verifies the certificate against the Mesh CA, checks the signature, and derives the same shared secret from the encapsulated key.
  4. Key distribution to workers: After the Coordinator verifies the AI workers through attestation, the secret service securely distributes the encryption keys to the appropriate AI workers. These workers use the keys to decrypt the prompts, process the data, and then re-encrypt the results before sending them back to the client.

The resulting flow is illustrated below:

flowchart LR
  A[Client software]-- "verifies" -->B[Coordinator]
  B[Coordinator]-- "verifies" -->C[secret service]
  B[Coordinator]-- "verifies" -->D[AI workers]
  A[Client software]-- "agrees on keys" -->C[secret service]
  C[secret service]-- "sends keys" -->D[AI workers]

Encryption

The prompt and response encryption uses the exchanged symmetric key with Authenticated Encryption implemented through AES-GCM.

Prompts are encrypted by the client-side Privatemode proxy and decrypted by a server-side encryption proxy hosted on the worker. Responses are handled accordingly where encryption is done by the server-side encryption proxy and decryption of the responses is performed by the client-side Privatemode proxy.

Workflow

  1. Request encryption: The client encrypts all request fields, except metadata required for routing and billing, like token length or model name, keeping them accessible to the service provider. The encrypted fields encode the key ID which maps to the used key.

    The following request fields aren't encrypted:

    • model
    • stream_options
    • max_tokens
    • max_completion_tokens
    • n
    • stream

  2. Request decryption: The server-side proxy decodes the encrypted fields with the key that maps to the encoded key ID. This doesn't affect the low-level runtime encryption provided by the Confidential Computing Environment.

  3. Prompt processing: The decrypted fields are securely transmitted to the inference server locally via a TCP socket.

  4. Response encryption: The response from the inference server is returned through the same socket. The server-side proxy then encrypts the response and sends it back to the client-side Privatemode proxy.

    The following response fields aren't encrypted:

    • id
    • usage

Service provider isolation

The Coordinator and AI workers in Privatemode are designed to operate independently of us as the service provider, meaning that we've by no means access to your encryption keys. This security is reinforced during the remote attestation process, where the client not only verifies the Coordinator's integrity but also its identity. By inspecting the open-source code, clients can confirm that the Coordinator is configured to prevent any unauthorized access by the service provider, ensuring that all encryption keys remain secure and exclusively controlled by the client.