Lattice

Lattice is a static retriever — a token lookup table rather than a transformer. It embeds text in microseconds on a CPU and adds a few megabytes to a deployment. It scores materially below a real dense embedder on retrieval, which is the trade it exists to make.

It is an explicit serving leg. A namespace that selects Lattice never falls back to another leg, and an unconfigured artifact is a validation error rather than a silent substitution.

Provisioning

Generate a deployment artifact with the upstream Lattice slicer, place its model.safetensors and tokenizer.json together, and set LAYER_LATTICE_MODEL_PATH to the model file before starting the gateway. The supported model id is erikkaum/lattice-retrieval; the requested embed.dims must match the loaded artifact, and only text modality is supported.

uv run slicer slice \
  --dim 512 \
  --quant int4_row \
  --output-dir /var/lib/hevlayer/lattice
export LAYER_LATTICE_MODEL_PATH=/var/lib/hevlayer/lattice/model.safetensors

"text": {
  "type": "string",
  "embed": {
    "model": "erikkaum/lattice-retrieval",
    "dims": 512,
    "serving": { "prefer": "lattice" }
  }
}

prefer: lattice selects the Lattice artifact. prefer: local also resolves to it when the declared model is erikkaum/lattice-retrieval.

The recommended operating point is an int4-per-row, 512-dimensional artifact. Int4 quantizes the model’s lookup-table weights only. Layer writes the resulting normalized vectors as [512]f32; Turbopuffer’s int8 minimum for quantized vector storage is a separate choice and is not used by this path.

End-to-end example

Declare the Lattice profile on a string attribute, write rows, and query with Embed. The gateway embeds both sides in-process — no external inference provider is involved.

Write two rows into a namespace whose text attribute carries the profile above:

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/write" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "title": "Planet",
       "text": "Jupiter is the biggest planet in the Solar System."},
      {"id": "photo-1", "title": "Photosynthesis",
       "text": "Plants turn sunlight, water, and carbon dioxide into food."}
    ],
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "erikkaum/lattice-retrieval",
          "dims": 512,
          "serving": { "prefer": "lattice" }
        }
      }
    }
  }'

Query by meaning rather than exact phrase:

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"]
  }'

{
  "rows": [
    { "id": "planet-1", "$dist": 0.137, "title": "Planet",
      "text": "Jupiter is the biggest planet in the Solar System." }
  ],
  "performance": {
    "embedding_tokens": 7,
    "embedding_ms": 1   // in-process lookup — no network hop to a provider
  }
}

A live example of exactly this contract is the Wikipedia × Lattice demo: all 283,997 Simple English Wikipedia articles (1.74M paragraph rows) embedded through Lattice and searched on Turbopuffer, with the performance echo displayed beside each result. Source at github.com/hev/wiki.

Limits

• Text only. An image modality on a Lattice profile is a validation error.
• No revision pins or instructions. Those extensions require a GPU-served profile.
• embed.dims must equal the sliced artifact’s dimension. A mismatch is a validation error at write time, not a silent reshape.
• A directory that fails to load stops the gateway at startup rather than serving a namespace that cannot embed.