API

Embed

Add an embed block to a string attribute in your schema and Layer computes that attribute’s vector on every write. Put ["Embed", "..."] in rank_by and Layer computes the query vector with the same model. Your application never loads a model or sends a vector of its own.

Where the vector comes from

embed.serving.prefer picks the provider that runs the model:

`embed.serving.prefer`	Who computes the vector
`local`	A CPU model bundled with Layer, running beside the gateway. No key, no GPU, no account.
`worker`	A GPU worker in your own cluster, serving an open-weights checkpoint you declared.
`turbopuffer`	Turbopuffer’s hosted embedding service, from its managed model menu.

The provider is independent of the store the vector is written to, so any store that accepts embed can use any configured provider.

Layer never substitutes one provider for another. A profile that selects a provider the deployment has not configured fails validation at write time, and a provider that fails at request time returns an error instead of falling back to a different model.

native is accepted as an alias for turbopuffer, autoscaler for worker, and lattice for local on a Lattice profile.

All three providers are available. Layer validates a turbopuffer profile and forwards it unchanged to the upstream POST /v2/namespaces/{ns} and POST /v2/namespaces/{ns}/query, so Turbopuffer computes the vector and it never passes through Layer. local and worker compute the vector in Layer and write an ordinary ANN column.

CPU models

Layer bundles three CPU embedding models, small enough to run on a laptop under Docker. Their weights are baked into the gateway image, so there is nothing to download or configure before the first write. They use the same schema as the larger GPU and hosted models, with lower retrieval quality.

Model	Dims	Modality	Use it for
`sentence-transformers/all-MiniLM-L6-v2`	384	text	The default. 22M parameters, symmetric, no prefixes.
`BAAI/bge-small-en-v1.5`	384	text	Better retrieval for 50% more parameters. Try it when MiniLM’s results are weak.
`openai/clip-vit-base-patch32`	512	image	Text-to-image search. Both towers run on CPU.

Text

This write creates the articles namespace with an embedded text attribute and upserts two rows. Layer embeds each row’s text with MiniLM before storing it.

await client.write_namespace("articles", {
    "upsert_rows": [
        {"id": "planet-1", "title": "Planet",
         "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
         "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "text": {
            "type": "string",
            "embed": {
                "model": "sentence-transformers/all-MiniLM-L6-v2",
                "dims": 384,
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "articles", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "planet-1", "title": "Planet",
            "text": "Jupiter is the biggest planet in the Solar System."},
        {"id": "photo-1", "title": "Photosynthesis",
            "text": "Plants turn sunlight, water, and carbon dioxide into food."},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "text": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":   "sentence-transformers/all-MiniLM-L6-v2",
                "dims":    384,
                "serving": map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("articles", {
  upsert_rows: [
    { id: "planet-1", title: "Planet",
      text: "Jupiter is the biggest planet in the Solar System." },
    { id: "photo-1", title: "Photosynthesis",
      text: "Plants turn sunlight, water, and carbon dioxide into food." },
  ],
  distance_metric: "cosine_distance",
  schema: {
    text: {
      type: "string",
      embed: {
        model: "sentence-transformers/all-MiniLM-L6-v2",
        dims: 384,
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/write" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "planet-1", "title": "Planet",
       "text": "Jupiter is the biggest planet in the Solar System."},
      {"id": "photo-1", "title": "Photosynthesis",
       "text": "Plants turn sunlight, water, and carbon dioxide into food."}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "text": {
        "type": "string",
        "embed": {
          "model": "sentence-transformers/all-MiniLM-L6-v2",
          "dims": 384,
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query by meaning with Embed. Layer reads the model from the text schema and embeds the query string with it:

response = await client.query_namespace("articles", {
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"],
})
print(response.rows, response.performance)

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy:            []any{"text", "ANN", []any{"Embed", "largest planet in the solar system"}},
    TopK:              3,
    IncludeAttributes: []string{"title", "text"},
})

const response = await client.queryNamespace("articles", {
  rank_by: ["text", "ANN", ["Embed", "largest planet in the solar system"]],
  top_k: 3,
  include_attributes: ["title", "text"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]],
    "top_k": 3,
    "include_attributes": ["title", "text"]
  }'

{
  "rows": [
    { "id": "planet-1", "$dist": 0.137, "title": "Planet",
      "text": "Jupiter is the biggest planet in the Solar System." }
  ],
  "performance": {
    "embedding_tokens": 7,
    "embedding_ms": 9
  }
}

The two text models are embedded differently, and Layer handles both. bge expects an instruction prefix on queries and MiniLM expects none, so Layer adds the prefix only when it embeds a query. bge uses CLS pooling and MiniLM uses mean pooling. Both outputs are L2-normalized, so cosine_distance scores are comparable across rows.

Images

With modality: image on a CLIP model, Layer embeds each attribute value with the image tower at write time and embeds Embed query text with the text tower, against the same vector column. Both towers run on CPU beside the gateway.

await client.write_namespace("photos", {
    "upsert_rows": [
        {"id": "commons-131", "title": "Sunset at Porto Covo",
         "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    ],
    "distance_metric": "cosine_distance",
    "schema": {
        "image_url": {
            "type": "string",
            "embed": {
                "model": "openai/clip-vit-base-patch32",
                "dims": 512,
                "modality": "image",
                "serving": {"prefer": "local"},
            },
        },
    },
})

_, err := client.WriteNamespace(ctx, "photos", hevlayer.TurbopufferWriteRequest{
    "upsert_rows": []map[string]any{
        {"id": "commons-131", "title": "Sunset at Porto Covo",
            "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"},
    },
    "distance_metric": "cosine_distance",
    "schema": map[string]any{
        "image_url": map[string]any{
            "type": "string",
            "embed": map[string]any{
                "model":    "openai/clip-vit-base-patch32",
                "dims":     512,
                "modality": "image",
                "serving":  map[string]any{"prefer": "local"},
            },
        },
    },
})

await client.writeNamespace("photos", {
  upsert_rows: [
    { id: "commons-131", title: "Sunset at Porto Covo",
      image_url: "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" },
  ],
  distance_metric: "cosine_distance",
  schema: {
    image_url: {
      type: "string",
      embed: {
        model: "openai/clip-vit-base-patch32",
        dims: 512,
        modality: "image",
        serving: { prefer: "local" },
      },
    },
  },
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/write" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "upsert_rows": [
      {"id": "commons-131", "title": "Sunset at Porto Covo",
       "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"}
    ],
    "distance_metric": "cosine_distance",
    "schema": {
      "image_url": {
        "type": "string",
        "embed": {
          "model": "openai/clip-vit-base-patch32",
          "dims": 512,
          "modality": "image",
          "serving": { "prefer": "local" }
        }
      }
    }
  }'

Query the image column with text:

response = await client.query_namespace("photos", {
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"],
})

response, err := client.QueryNamespace(ctx, "photos", &hevlayer.QueryRequest{
    RankBy:            []any{"image_url", "ANN", []any{"Embed", "sunset over water"}},
    TopK:              16,
    IncludeAttributes: []string{"title", "image_url"},
})

const response = await client.queryNamespace("photos", {
  rank_by: ["image_url", "ANN", ["Embed", "sunset over water"]],
  top_k: 16,
  include_attributes: ["title", "image_url"],
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]],
    "top_k": 16,
    "include_attributes": ["title", "image_url"]
  }'

{
  "rows": [
    { "id": "commons-131", "$dist": 0.412, "title": "Sunset at Porto Covo",
      "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" }
  ],
  "performance": {
    "embedding_tokens": 5,
    "embedding_ms": 134
  }
}

Write responses report embedding_images instead of embedding_tokens. CPU CLIP handles query-time text embeds and write-time image embeds for small and medium corpora. A bulk image backfill needs a GPU. The provider is set per namespace, so a backfill can run on a GPU while queries embed on CPU.

The lens demo runs this setup: Wikimedia Commons Quality images embedded and searched on CPU, with the gateway’s performance echo shown beside each result. Source at github.com/hev/lens.

Image input

An image profile accepts an HTTP(S) URL or a base64 image string and fetches at most 20 MiB per URL. A rate-limited image host returns HTTP 429 with error upstream_error; a server error from the image host returns HTTP 503 with error service_unavailable. Layer preserves the host’s Retry-After response header for both. Malformed URLs or base64, non-retryable 4xx responses, oversize content, and undecodable images return 422 validation_error.

Changing models requires a re-index

The two text models both emit 384 dimensions, so switching between them leaves the namespace schema valid. It still requires re-embedding every row.

Different models place vectors in unrelated spaces. If documents were embedded with MiniLM and queries are embedded with bge, the distances compare incompatible vectors and the ranking is meaningless, and Layer returns no error. The same applies when moving from a CPU model to a hosted or GPU model. Re-index from your source data so every vector comes from the new model.

Bringing your own weights

You can mount a model directory over the bundled menu to add or override checkpoints. The mount is optional and the bundled models need none.

For a single static retriever that trades retrieval quality for microsecond embeds and a few megabytes of deployment size, see Lattice.

Query with `Embed`

["Embed", text] computes a query vector from text using the model settings of the attribute being ranked. When rank_by names the source attribute, Layer reads the model from its schema:

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10,
})
print(response.rows)

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"text", "ANN", []any{"Embed", "chest pain radiating to left arm"}},
    TopK:   10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]],
    "top_k": 10
  }'

When rank_by names the derived vector attribute embed_<attr>, pass the model explicitly. Omitting it returns 422 with a model name must be provided.

response = await client.query_namespace("clinical-notes", {
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
        "model": "acme/clinical-retrieval-v3",
    }]],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{
    RankBy: []any{"embed_text", "ANN", []any{"Embed", "chest pain radiating to left arm",
        map[string]any{"model": "acme/clinical-retrieval-v3"}}},
    TopK: 10,
})

const response = await client.queryNamespace("clinical-notes", {
  rank_by: ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
    model: "acme/clinical-retrieval-v3",
  }]],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", {
      "model": "acme/clinical-retrieval-v3"
    }]],
    "top_k": 10
  }'

Query vectors are cached for 60 seconds by default. Set LAYER_EMBED_CACHE_TTL_MS to change the TTL. Instruction prefixes are part of the cache key. A missing provider returns 503 service_unavailable.

`Embed` with `Auto`

An inline Embed lets query routing run a semantic or fused query in one request:

response = await client.query_namespace("articles", {
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
        "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}],
    }],
    "top_k": 10,
})

response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{
    RankBy: []any{"title", "Auto", "how plants turn sunlight into food", map[string]any{
        "vector": []any{"Embed", "how plants turn sunlight into food",
            map[string]any{"field": "text"}},
    }},
    TopK: 10,
})

const response = await client.queryNamespace("articles", {
  rank_by: ["title", "Auto", "how plants turn sunlight into food", {
    vector: ["Embed", "how plants turn sunlight into food", { field: "text" }],
  }],
  top_k: 10,
});

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "rank_by": ["title", "Auto", "how plants turn sunlight into food", {
      "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}]
    }],
    "top_k": 10
  }'

The first tuple field (title) is the attribute for the lexical legs. field inside Embed names the attribute whose schema supplies the embedding profile and vector column (text above). Omit field when both are the same attribute.

Layer chooses the route before it resolves Embed, so a short input routed to hybrid_text never calls the embedding provider. A semantic or fused response reports the routing decision (routing.policy: "v1", routing.executed: true) and merges embedding measurements into performance. Without a vector or an inline Embed, the router returns routing.executed: false and leaves embedding to the caller.

Model settings

The full embed block:

Field	Meaning
`model`	A provider-namespaced Hugging Face repo id.
`dims`	The vector dimension. Must match what the model emits.
`attribute`	Where the vector is stored. Defaults to `embed_<attr>`, and may not be the source attribute.
`serving.prefer`	`local`, `worker`, or `turbopuffer`. See where the vector comes from.
`modality`	`text` (the default) or `image`. `image` requires a CLIP-family model.
`revision`	Pins a checkpoint revision, including a finetuned one.
`instructions.document` / `instructions.query`	Prefixes for asymmetric retrieval models. Both affect the query-cache key.
`chunk`	Splits the attribute into one row per chunk on write. See chunking long documents.

The first write that establishes an embedded schema must also set distance_metric.

revision, instructions, and chunk are Layer extensions. Layer never forwards them upstream, and they require a GPU-served profile, except that a CLIP image profile may use local.

Chunking long documents

A single vector for a long document matches queries poorly, so long sources should be split into chunks and each chunk embedded as its own row.

Split documents in your indexer and write one row per chunk. Store the parent document’s id as an ordinary attribute so each hit can be traced back to its source.

Performance accounting

Write and query responses report embedding work under performance:

{
  "rows": [ /* ... */ ],
  "performance": {
    "embedding_tokens": 8,
    "embedding_ms": 42
  }
}

Queries omit embedding_tokens on a cache hit. Layer merges provider measurements into the same object and exports the work as hevlayer_embed_tokens_total and hevlayer_embed_compute_seconds_total, labeled by namespace, store kind, model, and serving mode.