Your AI vendor is a dependency, not a destiny.

That sentence is the entire page. Everything below is the decomposition — because an abstraction you cannot break down into concrete, configured, diagrammed parts is just a slogan. We will take it apart into six layers, and for each one show the actual code, the actual config, and the exact place a private key or a tenant ID physically lives.

Here is the full request lifecycle — one user request, from the edge of the network to the model and back. Watch where identity gets attached, where the encryption changes shape, and where the data finally comes to rest. Every box below has its own section.

If the image has not rendered yet, here is the same lifecycle as plain text — this is the spine of the whole page:

   User / Agent
        │   TLS 1.3  (encryption in transit — Layer 5)
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ EDGE / LOAD BALANCER          terminates public TLS   │
 └──────────────────────────────────────────────────────┘
        │   mTLS  (client certificate from here inward)
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ CONTROLLER  — identity injected ONCE  (Layer 4)       │
 │   tenant_id · workspace_id · role                     │
 │   read from the verified token → request-scoped ctx   │
 └──────────────────────────────────────────────────────┘
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ INPUT GUARDRAILS  (Layer 3)                           │
 │   prompt-injection · jailbreak · PII scan             │
 └──────────────────────────────────────────────────────┘
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ MODEL GATEWAY  — one neutral interface  (Layer 1)     │
 │   decision matrix picks a provider      (Layer 2)     │
 │   ┌─────────┐ ┌────────┐ ┌─────────┐ ┌──────────────┐ │
 │   │Anthropic│ │ OpenAI │ │ Bedrock │ │ Vertex / ... │ │
 │   └─────────┘ └────────┘ └─────────┘ └──────────────┘ │
 └──────────────────────────────────────────────────────┘
        │   mTLS egress — client cert, private key
        ▼   stays in the pod / HSM, never on the wire
   [  LLM PROVIDER  ]
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ OUTPUT GUARDRAILS  (Layer 3)                          │
 │   PII leak · policy violation · hallucination check   │
 └──────────────────────────────────────────────────────┘
        ▼
 ┌──────────────────────────────────────────────────────┐
 │ PERSIST + AUDIT  (Layer 6)                            │
 │   AES-256-GCM at rest · per-tenant data key           │
 │   KMS holds the root key · append-only audit log      │
 └──────────────────────────────────────────────────────┘

Before the layers, the motivation — because if you do not feel the problem, the solution looks like over-engineering.

When you call an LLM provider, you are taking a hard dependency on a company you do not control. That company can — and routinely does — have an outage, change its prices, deprecate the exact model your prompts were tuned for, change its data-handling terms, or lose a compliance certification your customer requires. None of those are hypothetical. All of them happened to someone in the last year.

This is not anti-vendor. You will still pick the best model for each job — that is Layer 2. The point is that the choice stays yours, reversible, and cheap to change. The same instinct that says “do not hard-code your database vendor into every query” says “do not hard-code your model vendor into every prompt.”

The model gateway is one internal module with one job: expose a provider-neutral way to call an LLM. Application code calls the gateway. The gateway — and only the gateway — knows that Anthropic, OpenAI, Bedrock, and Vertex exist.

Step one is a neutral request and response shape. It is deliberately small — the common denominator of what every provider can do:

// The shape the WHOLE application speaks. No vendor words in here.
export interface ModelRequest {
  messages: { role: 'system' | 'user' | 'assistant'; content: string }[]
  maxTokens: number
  temperature?: number
  // a capability tag, NOT a vendor model name — Layer 2 turns this into a real model
  task: 'cheap-extract' | 'general' | 'hard-reasoning' | 'long-context'
}

export interface ModelResponse {
  text: string
  provider: string          // which vendor actually served it (for logs/audit)
  model: string             // the concrete model id that was used
  usage: { inputTokens: number; outputTokens: number; costUsd: number }
}

// Every vendor adapter implements exactly this. That's the entire seam.
export interface ModelProvider {
  readonly name: string
  complete(req: ModelRequest): Promise<ModelResponse>
}

Step two is one thin adapter per vendor. Each adapter translates the neutral shape into that vendor's wire format and back. This is the only file in the codebase that imports a vendor SDK:

import Anthropic from '@anthropic-ai/sdk'
import { ModelProvider, ModelRequest, ModelResponse } from '../types'
import { resolveModel } from '../decision-matrix'

export class AnthropicAdapter implements ModelProvider {
  readonly name = 'anthropic'
  private client = new Anthropic()  // reads its key from the KMS-backed secret mount

  async complete(req: ModelRequest): Promise<ModelResponse> {
    const model = resolveModel('anthropic', req.task)   // Layer 2 decides the concrete model
    const system = req.messages.find(m => m.role === 'system')?.content
    const turns  = req.messages.filter(m => m.role !== 'system')

    const res = await this.client.messages.create({
      model,
      max_tokens: req.maxTokens,
      temperature: req.temperature ?? 0.7,
      system,
      messages: turns as Anthropic.MessageParam[],
    })

    const text = res.content.filter(b => b.type === 'text').map(b => b.text).join('')
    return {
      text,
      provider: this.name,
      model,
      usage: {
        inputTokens: res.usage.input_tokens,
        outputTokens: res.usage.output_tokens,
        costUsd: priceFor(model, res.usage),
      },
    }
  }
}

// gateway/adapters/openai.ts, bedrock.ts, vertex.ts are the same shape —
// import the vendor SDK, map the neutral request in, map the response out.

Step three is the gateway itself. It owns the cross-cutting concerns so every call gets them for free — provider selection, fallback, retries, timeouts, cost accounting, and tenant tagging:

import { ModelRequest, ModelResponse, ModelProvider } from './types'
import { AnthropicAdapter } from './adapters/anthropic'
import { OpenAIAdapter } from './adapters/openai'
import { BedrockAdapter } from './adapters/bedrock'
import { VertexAdapter } from './adapters/vertex'
import { chooseProvider } from './decision-matrix'   // Layer 2
import { tenantContext } from '../context/tenant'    // Layer 4
import { auditLog } from '../audit'                  // Layer 6

const providers: Record<string, ModelProvider> = {
  anthropic: new AnthropicAdapter(),
  openai:    new OpenAIAdapter(),
  bedrock:   new BedrockAdapter(),
  vertex:    new VertexAdapter(),
}

export async function complete(req: ModelRequest): Promise<ModelResponse> {
  const { tenantId, workspaceId } = tenantContext.get()   // ambient — see Layer 4

  // Layer 2 returns an ordered list: [primary, ...fallbacks]
  const order = chooseProvider(req, { tenantId })

  let lastErr: unknown
  for (const name of order) {
    try {
      const res = await providers[name].complete(req)
      await auditLog({ tenantId, workspaceId, provider: name, model: res.model, usage: res.usage })
      return res
    } catch (err) {
      lastErr = err            // outage / rate limit / deprecation → try the next provider
    }
  }
  throw new Error('All providers failed: ' + String(lastErr))
}

The gateway can reach four providers. Which one should serve a given request? That is not a vibe — it is a checklist. This is what the LinkedIn mock interview was really asking: what criteria do you use to choose a model, and what are the guardrails on that choice?

Here is the matrix. Every row is a question the router asks before a single token is spent:

In code, the matrix is a pure function: request in, ordered provider list out. Pure means it is trivially testable — and a model-selection decision you cannot unit-test is a decision you cannot defend in an audit.

import { ModelRequest } from './types'
import { getTenantPolicy } from '../tenancy/policy'   // per-tenant residency + compliance rules
import { circuitState } from './health'

// Returns an ORDERED list: [primary, ...fallbacks]. Never a single value.
export function chooseProvider(
  req: ModelRequest,
  ctx: { tenantId: string },
): string[] {
  const policy = getTenantPolicy(ctx.tenantId)   // e.g. { residency: 'us-gov', certs: ['fedramp'] }

  // 1. HARD FILTER — drop any provider this tenant is not allowed to use.
  //    Residency and compliance are guardrails, not preferences: they cannot be traded away.
  let eligible = ALL_PROVIDERS.filter(p =>
    p.regions.includes(policy.residency) &&
    policy.certs.every(c => p.certifications.includes(c)),
  )

  // 2. HEALTH FILTER — drop providers whose circuit breaker is currently open.
  eligible = eligible.filter(p => circuitState(p.name) !== 'open')

  // 3. RANK the survivors by capability fit, then latency, then cost.
  const ranked = eligible.sort((a, b) =>
    capabilityScore(b, req.task) - capabilityScore(a, req.task) ||
    a.p50LatencyMs - b.p50LatencyMs ||
    costFor(a, req) - costFor(b, req),
  )

  if (ranked.length === 0) {
    throw new Error(`No compliant provider for tenant ${ctx.tenantId}`)  // fail closed
  }
  return ranked.map(p => p.name)   // primary first, fallbacks after
}

Guardrails are the code that sits between the user and the model, and again between the model and anything the model's output touches. The model is powerful and gullible; the guardrails are the seatbelt.

In the lifecycle, guardrails wrap the gateway call — nothing reaches a provider unchecked, and nothing leaves the model unchecked:

import { complete } from '../gateway'
import { ModelRequest, ModelResponse } from '../gateway/types'
import { scanInput, scanOutput, GuardrailError } from './checks'
import { tenantContext } from '../context/tenant'

export async function guardedComplete(req: ModelRequest): Promise<ModelResponse> {
  const { tenantId } = tenantContext.get()

  // ── INPUT GATE ────────────────────────────────────────────────
  const inVerdict = await scanInput(req, tenantId)
  if (inVerdict.blocked) {
    throw new GuardrailError('input', inVerdict.reason)   // never reaches a provider
  }

  // ── THE MODEL CALL (Layers 1 + 2) ─────────────────────────────
  const res = await complete(inVerdict.sanitizedRequest)

  // ── OUTPUT GATE ───────────────────────────────────────────────
  const outVerdict = await scanOutput(res, tenantId)
  if (outVerdict.blocked) {
    throw new GuardrailError('output', outVerdict.reason)  // never reaches the user
  }
  return outVerdict.sanitizedResponse
}

This is the layer the mock interview pushed hardest on, and rightly so. In a multi-tenant system, the catastrophic failure is not downtime — it is Tenant A seeing Tenant B's data. And here is the uncomfortable truth about how that happens:

The rule: inject, don't pass

Do not pass tenantId, workspaceId, and role as ordinary function arguments that every layer has to remember to forward. Instead, the controller establishes them once, at the very edge of the request, from the verified token — and injects them into a request-scoped context that every lower layer reads automatically.

Step one — the controller reads identity from the verified token and injects it. This is the only place identity is set:

import { Router } from 'express'
import { verifyToken } from '../auth'
import { tenantContext } from '../context/tenant'
import { guardedComplete } from '../guardrails/wrap'

export const chatRouter = Router()

chatRouter.post('/chat', async (req, res, next) => {
  // 1. Verify the token. This is the trust boundary — nothing before it is trusted.
  const claims = await verifyToken(req.headers.authorization)
  //    claims = { tenantId, workspaceId, role, userId }  — cryptographically verified

  // 2. INJECT identity into the request-scoped context, then run the handler INSIDE it.
  //    Everything downstream of tenantContext.run() can read this — and nothing can
  //    run a query without it, because the data layer refuses to (see step 3).
  await tenantContext.run(claims, async () => {
    const answer = await guardedComplete({
      messages: req.body.messages,
      maxTokens: 1024,
      task: 'general',
    })
    res.json({ answer: answer.text })
  })
})

Step two — the context itself. In Node this is AsyncLocalStorage (Python: contextvars; Java: a request-scoped bean). It is “ambient” — available to any code running inside the request, without being handed down explicitly:

import { AsyncLocalStorage } from 'node:async_hooks'

export interface TenantClaims {
  tenantId: string
  workspaceId: string
  role: 'admin' | 'member' | 'readonly'
  userId: string
}

const als = new AsyncLocalStorage<TenantClaims>()

export const tenantContext = {
  run: <T>(claims: TenantClaims, fn: () => Promise<T>) => als.run(claims, fn),

  // Throws if called outside a request scope. That throw is a FEATURE:
  // it means no code path can quietly run "tenant-less".
  get(): TenantClaims {
    const claims = als.getStore()
    if (!claims) throw new Error('No tenant context — query attempted outside a request scope')
    return claims
  },
}

Step three — the data layer reads the tenant from the context itself. There is no public way to query without a tenant filter, because the function that would let you do that does not exist:

import { pool } from './pool'
import { tenantContext } from '../context/tenant'

// Every query goes through here. There is no exported "raw query" function.
export async function scopedQuery<T>(sql: string, params: unknown[] = []): Promise<T[]> {
  const { tenantId } = tenantContext.get()    // ambient — cannot be forgotten, cannot be faked

  const client = await pool.connect()
  try {
    // Wall 1 (application): bind the tenant into a session variable for THIS connection.
    await client.query('SET LOCAL app.tenant_id = $1', [tenantId])
    // Wall 2 (database): Postgres row-level security reads that same variable — see below.
    const result = await client.query(sql, params)
    return result.rows as T[]
  } finally {
    client.release()
  }
}

-- Row-level security: even a query with NO tenant filter only sees its own tenant's rows.
-- This is the independent second wall — it holds even if application code has a bug.
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

-- Now: SELECT * FROM documents  -- returns ONLY the current tenant's rows.
-- The filter is no longer the developer's responsibility — it is the database's.

And the model call inherits all of this for free: back in Layer 1, the gateway read tenantContext.get() to tag every provider call and every audit row with the tenant. Prompts, logs, and cost accounting stay partitioned by the same identity that partitions the database — because it is the same identity, injected once.

Encryption in transit means: while data is moving across a network, anyone who taps the wire sees ciphertext. The statement is easy. The engineering is in the details — which hops, what kind of TLS, and where the keys physically live. Let us not leave any of that hand-wavy.

Every hop, no exceptions

There is no hop — however “internal” — that travels in plaintext. The network perimeter is not a substitute for encryption; that assumption is exactly what zero-trust architecture exists to kill.

HOP                                   PROTECTION         WHO PROVES IDENTITY
─────────────────────────────────────────────────────────────────────────────
browser  → load balancer              TLS 1.3            server only
load balancer → controller            mTLS               server + client
controller → gateway / guardrails     mTLS               server + client
gateway  → LLM provider (egress)      mTLS               server + client
service  → database                   TLS 1.3 + cert     server (+ client cert)

The public edge uses ordinary TLS 1.3: the browser checks the server's certificate, the browser stays anonymous (the user authenticates with a token, separately). From the load balancer inward, every hop upgrades to mTLS — mutual TLS — where the client also presents a certificate and the server refuses the connection without one.

How mTLS actually works — and where each private key lives

This is the question the interview asked directly. In a client-certificate exchange there are two key pairs, and the entire security of the scheme rests on where the private halves live and never leave:

The mTLS handshake, step by step — note that the private keys are used but never sent:

CLIENT                                                    SERVER
  │                                                          │
  │ ── 1. ClientHello ─────────────────────────────────────▶ │
  │                                                          │
  │ ◀── 2. ServerHello + server's PUBLIC certificate ──────  │
  │        + "I require a client certificate too"            │
  │                                                          │
  │   3. verify server cert against the trusted CA bundle    │
  │      (is this really who I meant to call?)               │
  │                                                          │
  │ ── 4. client's PUBLIC certificate ─────────────────────▶ │
  │ ── 5. a signature made WITH the client's PRIVATE key ──▶ │
  │      (the private key never leaves — only the signature) │
  │                                                          │
  │                          6. verify client cert against   │
  │                             the trusted CA bundle, and   │
  │                             verify the signature         │
  │                                                          │
  │ ◀════ 7. encrypted channel open — both sides proven ════▶│

And the configuration — concretely, this is all it is. Issue a cert + key pair per service from an internal certificate authority, mount them, point the client and server at the file paths, and pin the CA bundle so only internally-issued certificates are trusted:

# The cert + key pair is issued by the internal CA (cert-manager / Vault PKI)
# and delivered as a secret. The pod MOUNTS it — the key is never in the image.
volumes:
  - name: mtls-certs
    secret:
      secretName: gateway-client-cert   # contains tls.crt, tls.key, ca.crt
      defaultMode: 0400                 # read-only, owner-only

containers:
  - name: model-gateway
    volumeMounts:
      - name: mtls-certs
        mountPath: /etc/mtls
        readOnly: true                  # the workload cannot even modify it

import { Agent } from 'undici'
import { readFileSync } from 'node:fs'

// The private key is READ from the mount at startup, held in memory, and used to
// sign handshakes. It is never logged, never serialized, never sent.
export const mtlsAgent = new Agent({
  connect: {
    cert: readFileSync('/etc/mtls/tls.crt'),   // our PUBLIC certificate — ok to present
    key:  readFileSync('/etc/mtls/tls.key'),   // our PRIVATE key — used in memory only
    ca:   readFileSync('/etc/mtls/ca.crt'),    // pin: trust ONLY internally-issued certs
    rejectUnauthorized: true,                  // refuse any peer we cannot verify
  },
})

// Every egress call from the gateway uses this agent. In a service mesh
// (Istio, Linkerd) the sidecar does all of the above and the app code stays clean —
// but the model is identical: mounted key, public cert on the wire, pinned CA.

Encryption at rest means: while data sits in storage — the database, the object store, the backups, the logs — it is ciphertext. A stolen disk or a leaked snapshot is useless without the keys. Again the statement is simple; the engineering is in which keys, held where, and who can ask them to do what.

Envelope encryption — the professional pattern

Naive encryption uses one key for everything; if it leaks, everything is exposed, and rotating it means re-encrypting the world. Envelope encryption fixes both problems with a hierarchy of keys:

┌─────────────────────────────────────────────────────────────────┐
│  KMS  (hardware security module)                                 │
│                                                                   │
│   ROOT KEY  ── never leaves the KMS hardware. Ever.               │
│      │        the application cannot read it — it can only ASK   │
│      │        the KMS to use it, and every ask is logged.        │
│      ▼                                                            │
│   wraps (encrypts) each tenant's DATA KEY                         │
└──────┬────────────────────────────────────────────────────────────┘
       │   the app stores only the WRAPPED (encrypted) data key
       ▼
 ┌───────────────────────┐   ┌───────────────────────┐
 │ Tenant A   data key   │   │ Tenant B   data key   │   ... one per tenant
 │ (stored wrapped)      │   │ (stored wrapped)      │
 └──────────┬────────────┘   └──────────┬────────────┘
            │ to read Tenant A's data:  │
            │  1. ask KMS to unwrap A's data key (in memory only)
            │  2. AES-256-GCM decrypt the rows with it
            │  3. discard the unwrapped key
            ▼
   ciphertext rows in Postgres / object store / backups

So “where does the key live” has a precise answer at this layer:

• The root key lives inside the KMS / HSM hardware and never comes out. The application never holds it — it can only request operations (“unwrap this data key”), and each request is authenticated, authorized, and audit-logged.
• Each tenant data key is stored only in its wrapped (encrypted) form, next to the data. On its own it is useless — it has to be unwrapped by the KMS, in memory, for each use, then discarded.
• The application holds neither key at rest. At most it holds an unwrapped data key in memory for the duration of one operation.

import { KMSClient, DecryptCommand } from '@aws-sdk/client-kms'
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto'
import { tenantContext } from '../context/tenant'
import { getWrappedDataKey } from './keystore'

const kms = new KMSClient()

// Decrypt a stored field for the CURRENT tenant (identity comes from Layer 4).
export async function decryptField(ciphertext: Buffer, iv: Buffer, tag: Buffer): Promise<string> {
  const { tenantId } = tenantContext.get()

  // 1. Fetch this tenant's data key — but it is WRAPPED; we cannot use it yet.
  const wrapped = await getWrappedDataKey(tenantId)

  // 2. Ask the KMS to unwrap it. The ROOT key does the work inside the KMS;
  //    we get back the data key in memory only. We never see the root key.
  const { Plaintext: dataKey } = await kms.send(new DecryptCommand({ CiphertextBlob: wrapped }))

  // 3. AES-256-GCM decrypt the field with the unwrapped data key.
  const decipher = createDecipheriv('aes-256-gcm', dataKey as Buffer, iv)
  decipher.setAuthTag(tag)
  const plain = Buffer.concat([decipher.update(ciphertext), decipher.final()])

  // 4. Discard the unwrapped key — it lived in memory for one operation.
  ;(dataKey as Buffer).fill(0)
  return plain.toString('utf8')
}

The layers of at-rest encryption stack — each is independent, like the tenancy walls:

Walk a single request through everything we built. This is the animated diagram at the top, narrated:

1. Edge. The request arrives over TLS 1.3 (Layer 5). The load balancer terminates public TLS and re-originates the connection as mTLS for everything inward.
2. Controller. The token is verified. Identity — tenant_id, workspace_id, role — is injected once into the request-scoped context (Layer 4). Nothing downstream has to be handed identity; it is ambient and enforced.
3. Input guardrails. The request is scanned for prompt injection, jailbreaks, and PII (Layer 3). If it fails, it never reaches a provider.
4. Gateway + decision matrix. The gateway (Layer 1) asks the decision matrix (Layer 2) for an ordered provider list — hard-filtered by the tenant's residency and compliance policy, then ranked by capability, latency, and cost. It calls the primary provider through a vendor adapter.
5. Egress. The call leaves over mTLS with a client certificate; the private key that proves the gateway's identity was mounted into the pod and never touches the wire (Layer 5). If the provider is down, the gateway falls through to the next one on the list — no code change, no incident.
6. Output guardrails. The response is scanned for PII leaks, policy violations, and hallucinations before the user ever sees it (Layer 3).
7. Persist + audit. Anything stored is written as AES-256-GCM ciphertext under the tenant's own data key, whose root key never leaves the KMS (Layer 6). An append-only audit row — tenant, workspace, provider, model, token usage, cost — is written under the same injected identity.