Getting started

FAQ

Questions

  1. How's the Privatemode API different from other GenAI APIs?
  2. Can you see my prompts and the corresponding responses?
  3. You run your service on Scaleway and Lyceum. Can't they see my data?
  4. You use the OpenAI API standard. Is any data transferred to OpenAI?
  5. Which data is processed by Edgeless Systems?
  6. Is privatemode.ai down?
  7. Why can't Privatemode reveal data across tenants or users?
  8. Is Privatemode secure against quantum computers and "store now, decrypt later" attacks?
  9. Can I run Privatemode on my own infrastructure?

How's the Privatemode API different from other GenAI APIs?

In contrast to other GenAI APIs, the Privatemode API offers dependable privacy and security properties. Your prompts and responses always stay hidden from any third party, even from us, Edgeless Systems.

With other GenAI APIs, data can be accessed by third parties at various points when interacting with the AI. For example:

  • Infrastructure providers like Microsoft or AWS have privileged access and control over the underlying hardware and system software. This means they can potentially access your prompts and responses.
  • GenAI service providers like OpenAI also have the ability to access your data.

With the Privatemode API, your prompts and responses remain confidential. By design, neither Scaleway and Lyceum Technology as the infrastructure providers nor Edgeless Systems as your service provider can access or leak your prompts or responses.

To ensure this, the Privatemode API leverages confidential computing, a cutting-edge technology that provides runtime encryption and protection for data. The Privatemode API applies confidential computing end-to-end and make this verifiable from the outside. With the help of confidential computing-based remote attestation, it's possible to verify the integrity and authenticity of the software of the entire Privatemode software stack.

This makes the Privatemode API fundamentally different from other GenAI APIs. You fully own your prompts and responses.

Feel free to dive deeper into Privatemode's security properties.

Can you see my prompts and the corresponding responses?

No. By leveraging confidential computing and providing hardware-enforced end-to-end encryption, Privatemode ensures that Edgeless Systems can't access your prompts or responses.

You run your service on Scaleway and Lyceum. Can't they see my data?

This is important to us: by design, our infrastructure providers can't access any of your prompts or responses.

The Privatemode API is based on confidential computing, and this is where it truly shines. Thanks to its strong, hardware-enforced confidentiality, even the infrastructure providers that control the hardware and system software can't access any of your data.

You use the OpenAI API standard. Is any data transferred to OpenAI?

No. The Privatemode API doesn't use any OpenAI services. It only adheres to the common OpenAI interface definitions (prompt and response format) to provide a convenient development experience and ensure easy code portability.

Which data is processed by Edgeless Systems?

Prompts and responses are always encrypted and inaccessible to third parties. For operating the service, metadata such as IP address, timestamp, API key, and token usage is stored for up to 90 days. Token usage for each API key is permanently stored for billing purposes. See Trust and compliance for the full list.

Is privatemode.ai down?

You can check the status of the Privatemode API on the status page.

Why can't Privatemode reveal data across tenants or users?

Privatemode doesn't save prompts or responses after an inference request is completed. The underlying inference engine, vLLM, uses batching to process multiple prompts simultaneously. However, vLLM ensures that requests remain isolated in memory, preventing any cross-tenant interference or data leakage during execution.

Privatemode retains transient inference state in a prompt cache in worker memory. Every cache lookup key includes a secret salt. By default, the Privatemode proxy generates a new random salt for each request, preventing cache reuse. For better performance, clients can opt into cache reuse by using the same salt across requests. Requests with different salts can't reuse each other's cache entries or observe whether a prompt prefix produced a cache hit.

See Prompt cache security for the security design and Secure prompt caching for fast AI inference for a detailed implementation overview.

Is Privatemode secure against quantum computers and "store now, decrypt later" (SNDL) attacks?

Yes. Privatemode is designed to be quantum-resistant and protects against "store now, decrypt later" (SNDL) attacks. We use symmetric memory encryption (AES) for data in-use and quantum-resistant hybrid key exchange (X25519MLKEM768) for data in-transit.

Can I run Privatemode on my own infrastructure?

Privatemode is a managed service. The inference runs on confidential-computing infrastructure in the EU that Edgeless Systems operates, and the inference stack isn't available for installation on your own hardware.

What you run yourself is the client side: the Privatemode proxy or the SDK on your machines or in your network, and optionally a self-hosted web app. Because the client verifies the deployment through remote attestation, you get hardware-enforced control over your data without operating GPUs. You can also reproduce the reference values from the public source code to verify the service independently.

If your requirements go beyond the managed service, contact Edgeless Systems.