# Layer — Community Edition Search endpoint: https://hevlayer.com/api/ask/ce # Agents Source: https://hevlayer.com/docs/ce/agents import Edition from "../../components/docs/Edition.astro"; import Callout from "../../components/docs/Callout.astro"; Use the docs from your coding agent through the same edition-specific corpus as the site. Search results cite heading anchors in the selected view. ## Local docs From the repository's `site/` directory: ```sh pnpm run ask --edition ce overview pnpm run ask --edition ce search "write and query rows" pnpm run ask --edition ce section get "quickstart#2-write-rows" ``` Use `--edition pro` to select the full documentation. Local retrieval needs no API key. It generates the edition projections from the committed digest and source corpus, preserving untouched full-corpus summaries and glossary. ## Hosted docs Point the ask CLI at `https://hevlayer.com/api/ask/ce` for Community Edition or `https://hevlayer.com/api/ask/pro` for the full docs. For example: ```sh layer ask --endpoint https://hevlayer.com/api/ask/ce tree ``` ## Text exports [CE index](/docs/ce/llms.txt) and [CE full text](/docs/ce/llms-full.txt) contain only the CE view. The full-product equivalents are [full index](/docs/pro/llms.txt) and [full text](/docs/pro/llms-full.txt). Keep the edition in copied citations so readers and agents open the same view. # Standalone configuration Source: https://hevlayer.com/docs/ce/configuration import FeatureGate from "../../components/docs/FeatureGate.astro"; The [Community Edition repository](https://github.com/hev/layer/blob/v0.6/docs/setup.md) contains installation, CI, and standalone configuration instructions. Start with the [quickstart](/docs/ce/quickstart) to write and query your first rows. # Layer CLI Source: https://hevlayer.com/docs/ce/cli import Edition from "../../components/docs/Edition.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; The CLI connects to the standalone gateway. Set its base URL to your local service and its key to the gateway bearer token — your Turbopuffer API key. ## Install From the repository root: ```sh go build -o layer ./apps/layer-cli ``` ## Configuration `layer` reads named environments from `~/.hevlayer/config.toml`. The directory is created with mode `0700`; the config file is written with mode `0600`. ```toml active = "local" [envs.local] base_url = "http://localhost:8080" api_key = "tpuf_..." ``` Resolution order is: | Priority | Source | | --- | --- | | 1 | Explicit flags such as `--base-url` and `--api-key` | | 2 | `LAYER_BASE_URL`, `LAYER_API_KEY`, and the `HEVLAYER_` twins | | 3 | Environment selected by `--env` or `LAYER_ENV` | | 4 | Active environment in `~/.hevlayer/config.toml` | | 5 | Built-in base URL default | A shell exporting `LAYER_BASE_URL` or `LAYER_API_KEY` keeps the env-var-only behavior and does not need a config file. `--env` and `LAYER_ENV` select an environment for one invocation without changing the active environment. | Flag | Environment | Default | | --- | --- | --- | | `--base-url` | `LAYER_BASE_URL`, `HEVLAYER_BASE_URL` | `https://aws-us-east-1.hevlayer.com` | | `--api-key` | `LAYER_API_KEY`, `HEVLAYER_API_KEY` | none | | `--env` | `LAYER_ENV` | active config env | | `-o`, `--output` | none | `table` | Output formats are `table`, `json`, and `names`. ## Environments ```sh layer env add local --base-url http://localhost:8080 --api-key "$TURBOPUFFER_API_KEY" layer env use local layer env ls layer env show local -o json ``` `env add` prompts for missing values on a TTY. On a non-TTY, the required values must be supplied by flags. API keys are masked in `env ls` and `env show`. ## Initialize a Namespace For a Turbopuffer namespace: ```sh layer init products --shards 8 layer init products --shards 8 --watch=false layer init products --shards 8 --poll-interval 5s ``` `init` calls `POST /v2/namespaces/{namespace}/init` through the selected gateway environment. The command creates or reattaches to the namespace shard marker, starts the shard backfill, and watches by default until `shard_lag_rows` reaches `0` and scatter/gather is active. Re-running with the same shard count is idempotent; requesting a different shard count returns a conflict message instead of changing the marker. ## TUI The environment and index views read the standalone gateway. Function, pipeline, and key views require the full runtime. Snapshot history columns also require the full runtime. ## VectorStores `layer vectorstore list` and `layer vectorstore get NAME` read standalone connection metadata. Configure the connection on the gateway through [configuration](https://github.com/hev/layer/blob/v0.6/docs/setup.md). ## Inspect an index `layer index list` and `layer index get NAME` read namespace metadata. Snapshot-history columns require Pro. ## Delete an index `layer index delete NAME` requests namespace deletion. For normal local shutdown, use `docker compose down`. ## Ask the docs From the `site/` directory, use `pnpm run ask --edition ce search "query"`. For deployed docs, use `layer ask --endpoint https://hevlayer.com/api/ask/ce tree`. Full documentation is available at `/api/ask/pro`. # Demos Source: https://hevlayer.com/docs/ce/demos import Edition from "../../components/docs/Edition.astro"; Every demo below is a live app built on Layer that reimplements nothing. Each composes shipped gateway features — [routing](/docs/ce/api/query#query-routing), [hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion), fuzzy matching, [local embedding serving](/docs/ce/api/embed), [pipelines](/docs/ce/api/pipelines), [snapshots](/docs/ce/api/snapshots), and the [function runtime](/docs/ce/kubernetes/function-crd) — over a different corpus, and makes the gateway's behavior legible in the UI. They are also the fastest way to see what the gateway does without standing up a cluster. | Demo | What it shows | Corpus | | --- | --- | --- | | [shelf](https://shelf.hevlayer.com) | The query router, made legible | Books | | [chart](https://chart.hevlayer.com) | Query routing on clinical search, with a number | PMC-Patients case reports | | [hybrid-text](https://hybrid-text.hevlayer.com) | Hybrid text fusion, proven with qrels | BEIR/SciFact abstracts | | [wiki](https://wiki.hevlayer.com) | Auto routing plus CPU-only Lattice embedding, at scale | Simple English Wikipedia | | [lens](https://lens.hevlayer.com) | Text-to-image search with zero GPUs | Wikimedia Commons Quality images | | [shop](https://shop.hevlayer.com) | Everything together — an end-to-end app | Amazon product catalog | These deployed applications demonstrate retrieval behavior. Running their full workloads can require external stores, model artifacts, and data preparation; they are separate from the local Compose quickstart. ## shelf — book search that shows its routing **Live:** [shelf.hevlayer.com](https://shelf.hevlayer.com) · **Source:** [github.com/hev/shelf](https://github.com/hev/shelf) One search box, three routes. Type an author, a title, or a vibe; the gateway's `Auto` rank expression picks keyword (`hybrid_text`), `semantic`, or a `fused` blend from the shape of the query, and shelf renders that decision as a badge with the reason. The routing policy keys on token count, so the canned chips visibly change route as the query gets longer. This is the text-native routing showcase: it makes the [query router](/docs/ce/api/query#query-routing) decision the hero, not a footnote. Built on the [query router](/docs/ce/api/query#query-routing) (`Auto`), [hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion), and fuzzy matching. ## chart — clinical patient-notes search that shows its routing **Live:** [chart.hevlayer.com](https://chart.hevlayer.com) The same routing hero on the corpus with the sharpest bimodal query distribution there is: clinicians search both by exact token (`metformin 500mg`, `CABG`, `aspirn`) and by clinical picture (`elderly woman with progressive dyspnea and bilateral lower-extremity edema`). chart is the first Layer demo with real relevance judgments — PMC-Patients ReCDS qrels — so the routing and hybrid claims are measured, not asserted. Behind the search box, an open-weight Gemma cascade (vLLM, scale-to-zero on the GPU pool) reads each note once and extracts clinical events and facet labels: the [function runtime](/docs/ce/kubernetes/function-crd) showcase. The corpus is published, de-identified case reports (PMC-Patients, CC-BY-NC-SA). It is a search demo — not raw EHR, and not clinical advice. Built on the [query router](/docs/ce/api/query#query-routing), [hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion) with fuzzy matching, [pipelines](/docs/ce/api/pipelines), the [function runtime](/docs/ce/kubernetes/function-crd), and [snapshots](/docs/ce/api/snapshots). ## hybrid-text — hybrid text fusion over SciFact **Live:** [hybrid-text.hevlayer.com](https://hybrid-text.hevlayer.com) · **Source:** [github.com/hev/hybrid-text-fusion-demo](https://github.com/hev/hybrid-text-fusion-demo) The eval-shaped sibling of the routing demos, over ~5,000 scientific abstracts from BEIR/SciFact. One query string fans out into a full-input BM25 leg plus one fuzzy leg per token, fused by reciprocal rank fusion — so results survive typos and morphological variants without losing BM25's signal. It is purely lexical: no embeddings, no GPU, no vector index. SciFact ships qrels, so the UI flags known-relevant abstracts and the demo scores nDCG@10 / recall@10; every search also shows its gateway round-trip time and a fusion inspector (tokens, legs, RRF constant). Built on [hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion) and fuzzy matching. ## wiki — all of Simple English Wikipedia, routed and embedded on CPU **Live:** [wiki.hevlayer.com](https://wiki.hevlayer.com) · **Source:** [github.com/hev/wiki](https://github.com/hev/wiki) The routing hero at corpus scale: one `Auto` query over all 283,997 Simple English Wikipedia articles (1.74M paragraph rows) routes each search to full-text, semantic, or a fused RRF blend, and the UI renders the gateway's `routing` echo beside every result. The semantic leg is the [Lattice](/docs/ce/api/lattice) showcase — the whole corpus embedded through `prefer: lattice`, an ~8 MB int4 lookup-table artifact served in-process on the gateway CPU, with `performance.embedding_ms` and `embedding_tokens` echoed whenever the chosen route embeds. No GPU anywhere in the write or query path. Built on the [query router](/docs/ce/api/query#query-routing), [hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion), and [local embedding serving](/docs/ce/api/lattice) (`prefer: lattice`). ## lens — text-to-image search with zero GPUs **Live:** [lens.hevlayer.com](https://lens.hevlayer.com) · **Source:** [github.com/hev/lens](https://github.com/hev/lens) Cross-modal search over Wikimedia Commons [Quality images](https://commons.wikimedia.org/wiki/Commons:Quality_images): type `sunset over water`, get sunsets. The schema is two lines — a string `image_url` attribute with a [local CLIP](/docs/ce/api/embed#images) embedding profile. The gateway fetches each image and runs CLIP's image tower in-process on CPU at write time, then embeds query text with the same checkpoint's text tower at query time. The app posts writes and queries and renders the echo; it contains no embedding, tokenizer, or image-preprocessing code, and there is no GPU worker or autoscaler pool anywhere in the path. Every result pairs the fixed serving contract (`prefer: local`, gateway CPU) with the live `performance.embedding_ms` echo, and carries its Commons attribution and license. Built on [local CLIP serving](/docs/ce/api/embed#images) (`serving.prefer: local`, `modality: image`) and [full-runtime schema configuration](/docs/pro/kubernetes/index-crd#schema-attribute-embedding). # Concepts Source: https://hevlayer.com/docs/ce/concepts import Edition from "../../components/docs/Edition.astro"; import Callout from "../../components/docs/Callout.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; import StoreMatrixLink from "../../components/docs/StoreMatrixLink.astro"; ## Wire protocol matching Layer accepts the Turbopuffer HTTP wire protocol: the methods and paths a client calls, the JSON request fields it sends, and the response shapes and status codes it reads. For supported operations, an application can point its Turbopuffer client at Layer's base URL and keep the same request body. See the [API reference](/docs/ce/api/introduction) for authentication and client setup. With Turbopuffer as the store, Layer forwards native requests after gateway validation. With another store, Layer translates supported operations into that store's native calls. A valid but unsupported feature should return `422 UnsupportedByStore`, naming the store, route, and feature, rather than silently dropping part of the request. Matching the wire does not promise identical index internals, latency, scores, or ranking across stores. In particular, full-text ranking is backend-specific. Layer's additional request fields, routes, and response metadata are documented as gateway enhancements. ### How we validate it - **API and client contracts.** The SDK harness compares the gateway OpenAPI operations and generated Python client with the upstream API, and checks captured HTTP requests against documented examples using a mock server. These checks catch route, field, and serialization drift; they do not prove that a real backend returns the right results. - **Backend acceptance.** Store-specific suites send requests through a real gateway and backend using generated clients. They check supported operations and explicit rejection of unsupported requests. - **Documented examples.** A committed selection of upstream examples runs against a real gateway and store. Each request is classified as **ok**, **unsupported**, **fail**, or **blocked** by a prerequisite. A baseline change fails the check for review; matching a baseline can still preserve known failures. This is a selected test corpus, not proof that every upstream request or combination works. The capability matrix is generated from backend declarations and checked for source drift. It states the contract; acceptance results are evidence of behavior. Both are needed to assess compatibility. ## Gateway enhancements Layer adds retrieval operations around the store while keeping one client endpoint. [Hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion) combines retrieval legs, [query routing](/docs/ce/api/query#query-routing) selects a strategy, [scans](/docs/ce/api/scans) select or count matching rows, and [federated queries](/docs/ce/api/federated-query) combine named namespaces. The API reference calls out each backend's limits at the relevant feature. The [Layer clients](/docs/ce/api/introduction#install) expose these additions; plain HTTP can call the same API. Native requests and enhanced requests can share the gateway endpoint. Where Layer needs bookkeeping attributes, it reserves the `_hevlayer_*` prefix. Treat these fields as read-only; the [document model](/docs/ce/document-model) defines the contract. ## Gateway and store The gateway receives writes and queries over HTTP and executes them against the selected store. Local Compose fronts your existing Turbopuffer account. See [configuration](https://github.com/hev/layer/blob/v0.6/docs/setup.md). ## Namespaces and rows A namespace groups rows addressed by ID. A row contains attributes and can include vectors. The first [write](/docs/ce/api/write) creates a namespace. ## Retrieval [Query routing](/docs/ce/api/query#query-routing) chooses a ranking strategy. [Scans](/docs/ce/api/scans) select rows or aggregate matching values, while [federation](/docs/ce/api/federated-query) merges results across explicit namespaces. Support depends on the backing store and request shape. ## Scatter/gather Turbopuffer namespaces can use `_hevlayer_shard` hash buckets. Initialization stamps rows and backfills existing rows. Scatter/gather starts after `layer.shard_lag_rows` reaches zero; the single-namespace path serves queries while backfill runs. See [CLI initialization](/docs/ce/cli#initialize-a-namespace). ## Glossary | Concept | Meaning | | --- | --- | | Wire protocol | The HTTP methods, paths, request fields, response shapes, and status codes exchanged by client and server. | | Wire feature | An individual operation or option whose backend support is declared in the capability matrix. | | Gateway | The Layer service that receives client requests, validates them, and executes them against the configured stores. | | [VectorStore](/docs/ce/api/vectorstores) | A serving connection to the backend that stores and queries rows. | | [Warehouse](/docs/ce/api/warehouses) | An upstream source connection, separate from the store serving retrieval requests. | | Namespace | A named collection of rows addressed through `/v2/namespaces/{namespace}`. | | Document / row | An ID and application attributes, optionally including vectors. | | [Scan](/docs/ce/api/scans) | Row selection that returns matching IDs, field values, or a count; supported selectors depend on the backend. | | Shard | A hash bucket within a namespace, identified by the reserved `_hevlayer_shard` attribute. | | Scatter/gather | Running subqueries across shards or namespaces and combining their results into one response. | | Leg | One subquery contributing to a hybrid or federated result. | | RRF | Reciprocal rank fusion: combining ranked lists using each result's position in its input lists. | | Tokenizer policy | The rules that turn input text into retrieval tokens, including word boundaries, case normalization, and token limits. | | Route | A retrieval strategy, such as `hybrid_text`, `semantic`, or `fused`, selected by the query router where supported. | | Routing policy | The deterministic, versioned rules used to select an `Auto` route. | | Deferral | An `Auto` response with `executed: false`: the application must supply an embedding before the selected route can execute. | # Document model Source: https://hevlayer.com/docs/ce/document-model import Edition from "../../components/docs/Edition.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; A row has an `id`, application attributes, and optionally a vector. Supply these fields in [write requests](/docs/ce/api/write). ## Reserved attributes The `_hevlayer_*` prefix belongs to the gateway. Do not set these attributes in client writes. Select your backend to see which attributes Layer manages. | Attribute | Type | Purpose | | --- | --- | --- | | `_hevlayer_upserted_at` | integer (epoch ms) | Server-stamped on row-producing writes through Layer. Records when the gateway received the write. | | `_hevlayer_shard` | integer | Hash bucket assigned at write time (`xxh64(id) % shard_count`), present only on sharded namespaces. Lets the gateway [scatter/gather](/docs/ce/concepts#scattergather) a query across the shards of one namespace. | Initialize a namespace with [`layer init --shards N`](/docs/ce/cli#initialize-a-namespace) to stamp new writes and backfill existing rows. Scatter/gather starts after `layer.shard_lag_rows` reaches zero. # FAQ Source: https://hevlayer.com/docs/ce/faq import Edition from "../../components/docs/Edition.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; ## Do I need an account? You need a Turbopuffer API key. The [CE quickstart](/docs/ce/quickstart) runs the gateway locally in front of your Turbopuffer account without signup or a license key. ## Where are my rows stored? In the Turbopuffer account behind your key, or in the existing store you configure. `docker compose down` stops the local services. ## What is the source license? The gateway uses BSL 1.1. See [licensing](/docs/ce/licensing) for the source license and trademark links. ## Who built Layer? [Adam Hevenor](https://hevmind.com/about). Layer is a hev mind product. # Failure Modes Source: https://hevlayer.com/docs/ce/failure-modes import Edition from "../../components/docs/Edition.astro"; import Callout from "../../components/docs/Callout.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; ## Read A gateway connection failure returns an error to the client. SDKs do not retry directly against the backing store. Check `/health`, gateway logs, and connectivity to the configured database or upstream service. A backing-store failure fails the request. Unsupported request shapes return `422 UnsupportedByStore`. ## Write Success requires the backing-store write to succeed. An invalid Turbopuffer key fails against Turbopuffer. ## Client failures Check the base URL, authentication settings, and error body. Retrying a timed-out write may repeat a request that reached the store; use stable row IDs and the store's write semantics when designing retries. See [health and metrics](/docs/ce/observability) for gateway diagnostics. # Introduction Source: https://hevlayer.com/docs/ce import Edition from "../../components/docs/Edition.astro"; import Diagram from "../../components/docs/Diagram.astro"; import { layerMapDiagram, layerMapDiagramNarrow, ceGatewayDiagram, ceGatewayDiagramNarrow, ceGatewayDiagramLabel } from "../../lib/diagrams"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; Layer Community Edition is the same standalone retrieval gateway included in Pro, ready to run on your laptop or in your pipeline. Run it in front of Turbopuffer to add [query routing and hybrid fusion](/docs/ce/api/query), [scans and facets](/docs/ce/api/scans), and [federated queries](/docs/ce/api/federated-query). {ceGatewayDiagram} ## Start locally The [quickstart](/docs/ce/quickstart) runs the gateway with Docker Compose in front of your Turbopuffer account and returns your first search result. Data stays in Turbopuffer; the gateway is the API entry point. ## Connect and query Use the [Python, Go, or TypeScript clients](/docs/ce/api/introduction), or curl. [Write](/docs/ce/api/write) rows and vectors, [query](/docs/ce/api/query) with vector or text ranking, [scan](/docs/ce/api/scans) matching rows, and [federate](/docs/ce/api/federated-query) across named namespaces. [Configuration](https://github.com/hev/layer/blob/v0.6/docs/setup.md) describes standalone connections and authentication. ## Observe Use [health and metrics](/docs/ce/observability) to check the gateway and [failure modes](/docs/ce/failure-modes) to diagnose request errors. # Guarantees Source: https://hevlayer.com/docs/ce/guarantees import Edition from "../../components/docs/Edition.astro"; import Callout from "../../components/docs/Callout.astro"; import StoreMatrixLink from "../../components/docs/StoreMatrixLink.astro"; ## Commitments - Successful writes require success from the selected backing store. - Unsupported store operations fail explicitly; the gateway does not silently replace them with a narrower request. - Client-visible behavior is documented per store and API. - Gateway or backing-store failures remain visible to callers. See [failure modes](/docs/ce/failure-modes). You operate the deployment and its backups. The local Compose volume holds rows; the bundle does not configure an external backup service. # Licensing Source: https://hevlayer.com/docs/ce/licensing import Edition from "../../components/docs/Edition.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; Community Edition is source-available under the Business Source License 1.1. Read the [LICENSE](https://github.com/hev/layer/blob/v0.6/LICENSE) for its permissions, restrictions, and conversion date. Trademark rights are covered separately by [TRADEMARKS.md](https://github.com/hev/layer/blob/v0.6/TRADEMARKS.md). The local [Compose quickstart](/docs/ce/quickstart) needs no signup or runtime license key. # Health and metrics Source: https://hevlayer.com/docs/ce/observability ## Health ```sh curl --fail http://localhost:8080/health ``` The health route requires no bearer token. Use Compose health checks and gateway logs to diagnose startup and database connectivity. ## Metrics `GET /metrics` exposes Prometheus-compatible gateway metrics without auth. The metrics catalog and proxy routes under `/v2/metrics` provide names and query access; proxy queries require a configured metrics service. The local Compose bundle does not include that service. ```sh curl --fail http://localhost:8080/metrics ``` ## Telemetry The standalone gateway sends anonymous startup and aggregate feature-use telemetry. Disable it with `LAYER_TELEMETRY=off` or `DO_NOT_TRACK=1`. The local Compose bundle disables it by default. Telemetry excludes query text, vectors, namespace names, document IDs, bearer tokens, API keys, and document contents. See the generated [telemetry reference](https://github.com/hev/layer/blob/v0.6/docs/telemetry.md). # Limits Source: https://hevlayer.com/docs/ce/limits import Edition from "../../components/docs/Edition.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; Limits depend on the selected store and operation. An unsupported request returns an error rather than a partial translation. - Batch queries accept up to 16 ranked legs per request. - Live values scans retain up to 1,000,000 distinct values and report `truncated: true` when the result exceeds the cap. - Row sizes, attribute shapes, vector dimensions, and indexing throughput follow the selected backend's limits. Scan and federation concurrency also consume gateway memory and upstream capacity. Review [scans](/docs/ce/api/scans) and [federated query](/docs/ce/api/federated-query) before increasing fan-out. # Changelog Source: https://hevlayer.com/docs/ce/roadmap import FeatureGate from "../../components/docs/FeatureGate.astro"; The [published release history](https://github.com/hev/layer/releases) records released versions and their artifacts. Production documentation follows the released tag; development previews follow their version branch. Surfaces marked **in preview** are deliberately absent from this page. They are documented and usable, but they sit outside the release line and carry no compatibility promise, so they are not part of a version's contract. ## Edge `edge` is the opt-in development build. It tracks the `v0.6` branch and is republished as the branch moves; pin a released tag for anything you depend on. ### Running edge ```sh git clone --branch v0.6 https://github.com/hev/layer.git cd layer export GATEWAY_IMAGE=hevlayer/layer-gateway:edge export TURBOPUFFER_API_KEY="tpuf_..." docker compose up -d --wait curl --fail http://localhost:8080/health ``` The [quickstart](/docs/ce/quickstart) covers the rest. ## 0.6 ### Community Edition - 📦 One-bundle Community Edition — a single Compose file and a public image, run with a Turbopuffer key - 🧑‍💻 [`layer` CLI](/docs/ce/cli) on Community Edition — keyless `index get`, `env add`, and inspection ### Gateway and pipelines - 🚦 Higher fan-out for pinned namespaces — 4× the default concurrency - 📄 REST source pagination — `searchAfter` over enumerated snapshots - 🦾 arm64 REST worker - 🗑️ Namespace deletion without S3 ### Fixes - 🔒 Scoped keys confined to their namespaces - 🧮 Document-cache scans agree with counts - 🪣 Standalone gateway runs without an object store ## 0.5 ### Embedding - 🧮 [Gateway-side query embedding](/docs/ce/api/embed) — `embed:` attributes resolved on the wire - 🎛️ [Serving preference](/docs/ce/api/embed) — per-attribute choice of where embedding runs - 🤝 Native Turbopuffer embeddings wired through the gateway - 🪟 Lattice embedding provider — pinned artifact mounted by Helm - 🖼️ Local CLIP provider — multimodal without a GPU worker - 🧰 BYO models — the blended embedding policy removed - 💸 Native embed cost attribution ### Install and CLI - 🧙 `layer install aws` — guided Terraform + Helm install wizard - ♻️ Profile-aware [`layer install`](/docs/ce/cli) lifecycle — the install script deleted - 🗄️ Document cache Helm profiles - 🚀 [Quickstart](/docs/ce/quickstart) from a clone — the bundled Compose flow ### Licensing - 🎟️ [Team tier gates](/docs/ce/licensing) — custom InfraRules and multiple vector stores - 🔏 Authenticated `/v2/license` probe - ⏳ Trial licenses skip post-expiry grace ### Pipelines - 📚 Enumerated REST full snapshots - 🧬 Idempotent sweeps — content hashes keep Function-written attributes ### Stores and gateway - 🗃️ Kubernetes vector stores preferred in pro - 🧊 Turbopuffer base64 vectors accepted - 🎯 Namespace delete routed by vector store - 🔓 Segment lock convoys resolved - 📴 Standalone gateway runs with no object store ## 0.4 ### API hardening - 🧩 Finalize CRDs - 🚆 Wire-compatible pass-through reads and writes - 🏷️ Naming things ### Lifecycle and operability - 🎚️ [Autoscaling compute](/docs/ce/kubernetes/scaling-crd) for pipelines and UDFs - 🗄️ [Document cache endpoint](/docs/ce/api/query#fetch) for multi-stage pipelines - 📸 [Index snapshot history](/docs/ce/api/snapshots) - 🧨 Coordinated delete - ⛵ [Helm and Terraform install](/docs/ce/install) scripts - 🔐 [Scoped API keys](/docs/ce/api/keys) — minted [`ApiKey` resources](/docs/ce/kubernetes/apikey-crd) - 🔑 License key validation - 🧾 Key audit logs — lifecycle events to S3 - 🏭 [Warehouse CRD](/docs/ce/kubernetes/warehouse-crd) — declared Snowflake and Hugging Face sources - 🏗️ Production cluster cutover — lean topology live in prod ### Surfaces - 🪟 [Dashboard MVP](/docs/ce/dashboard) — CRD management and observability - 📚 Documentation site - 🧰 Official Python, Go, and TypeScript clients ### Search - 🎯 [Stable reads](/docs/ce/api/query#stable-reads) during heavy writes - 🕰️ Temporal queries — `as_of` / `between` selector on reads - 🚦 [Ready signal](/docs/ce/api/namespace-metadata) — namespace reports when every row is indexed - 📜 Precomputed facet listings in [snapshots](/docs/ce/api/snapshots) - 🪙 Precomputed facet counts in [snapshots](/docs/ce/api/snapshots) - 🪃 [Scans](/docs/ce/api/scans) — row selection by filter, `fts`, `hybrid_text`, or `ann` - 🆔 Search by id via document-cached vector - 🪢 [Hybrid text fusion](/docs/ce/api/query#hybrid-text-fusion) - 🧭 [Query routing](/docs/ce/api/query#query-routing) - 📰 [Search history](/docs/ce/api/search-history) saved to S3 - 🔥 Trending searches - 🗂️ [Enhanced namespace metadata](/docs/ce/api/namespace-metadata) ## Proposals Unreleased design proposals are tracked separately from published release notes. A proposal does not establish store or edition support; use the selected edition's API reference. # Tradeoffs Source: https://hevlayer.com/docs/ce/tradeoffs import Edition from "../../components/docs/Edition.astro"; import StoreNote from "../../components/docs/StoreNote.astro"; import FeatureGate from "../../components/docs/FeatureGate.astro"; The gateway adds a network hop and performs validation, routing, and result merging for Layer-specific requests. Wider scans and federated queries can increase upstream work and gateway memory use. Turbopuffer sharding and write timestamps add reserved indexed attributes. # Quickstart Source: https://hevlayer.com/docs/ce/quickstart import Edition from "../../components/docs/Edition.astro"; import CodeTabs from "../../components/docs/CodeTabs.astro"; import Preview from "../../components/docs/Preview.astro"; Run Layer on your laptop in front of your existing Turbopuffer account. You'll need Docker with Compose, Git, `curl`, and a Turbopuffer API key. ## 1. Clone and start Replace `tpuf_...` with your key before running this block. ```sh git clone --branch v0.6.0 https://github.com/hev/layer.git cd layer export GATEWAY_IMAGE=hevlayer/layer-gateway:0.6.0 export TURBOPUFFER_API_KEY="tpuf_..." docker compose up -d --wait curl --fail http://localhost:8080/health ```

Started without a key, Compose runs on a local Postgres store instead.

The gateway listens on `localhost:8080`. The `0.6.0` tag is the released image; set `GATEWAY_IMAGE=hevlayer/layer-gateway:edge` only to opt into the development build. Allow a few minutes for the first image downloads. For CI setup and standalone configuration, see the [repository setup guide](https://github.com/hev/layer/blob/v0.6/docs/setup.md). ## 2. Write rows The first write creates the namespace. Run this in the same shell as step 1; the authorization header carries your Turbopuffer key. ```sh curl --fail-with-body http://localhost:8080/v2/namespaces/products \ -H "Authorization: Bearer $TURBOPUFFER_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "distance_metric": "cosine_distance", "schema": {"title": {"type": "string", "full_text_search": true}}, "upsert_rows": [ {"id": "earbuds", "title": "wireless earbuds", "vector": [1, 0, 0]}, {"id": "speaker", "title": "portable speaker", "vector": [0, 1, 0]} ] }' ``` ## 3. Query ```sh curl --fail-with-body http://localhost:8080/v2/namespaces/products/query \ -H "Authorization: Bearer $TURBOPUFFER_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"rank_by": ["vector", "ANN", [1, 0, 0]], "top_k": 1, "include_attributes": true}' ``` The first row has `id: "earbuds"` and `$dist: 0`. Continue with the [API docs](/docs/ce/api/introduction) for client SDKs and query options, or explore the [demos](/demos) to see what you can build. # Data connections Source: https://hevlayer.com/docs/ce/api/data-supply ## VectorStores See [VectorStores](/docs/ce/api/vectorstores) to list and inspect serving connections. ## Warehouses See [Warehouses](/docs/ce/api/warehouses) to list and inspect upstream source connections. # Introduction Source: https://hevlayer.com/docs/ce/api/introduction import Edition from "../../../components/docs/Edition.astro"; import StoreMatrixLink from "../../../components/docs/StoreMatrixLink.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; import StoreNote from "../../../components/docs/StoreNote.astro"; import Upstream from "../../../components/docs/Upstream.astro"; import FeatureGate from "../../../components/docs/FeatureGate.astro"; Native requests are wire-compatible with the upstream Turbopuffer endpoints and forwarded as-is after validation. This documentation covers only Layer enhancements. Use the Layer clients or plain HTTP against your gateway URL. The local [quickstart](/docs/ce/quickstart) writes and queries rows through the gateway. ## Install There are four ways to call Layer: the Python client, the Go client, the TypeScript client, and the REST API itself. The clients are generated from `apps/layer-gateway/openapi.yaml`, so all four expose the same operations — every endpoint page on this site shows them side by side. Anything the clients can do, plain HTTP can do. ```sh pip install hevlayer # Python 3.11+ go get github.com/hev/layer-go # Go 1.22+ npm install hevlayer # Node 18+ ``` Point a client at the gateway: ```python import os from hevlayer import AsyncHevlayer client = AsyncHevlayer( base_url=os.environ["LAYER_GATEWAY_URL"], api_key=os.environ.get("LAYER_GATEWAY_API_KEY", ""), ) ``` ```go import ( "os" hevlayer "github.com/hev/layer-go" ) client := hevlayer.NewClient( hevlayer.WithBaseURL(os.Getenv("LAYER_GATEWAY_URL")), hevlayer.WithAPIKey(os.Getenv("LAYER_GATEWAY_API_KEY")), ) ``` ```typescript import { Hevlayer } from "hevlayer"; const client = new Hevlayer({ baseUrl: process.env.LAYER_GATEWAY_URL, apiKey: process.env.LAYER_GATEWAY_API_KEY, }); ``` ```bash curl "$LAYER_GATEWAY_URL/v2/namespaces" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" ``` Code examples across these pages assume this `client` — and in Go, a `ctx context.Context`. The cURL tab on each page is the bare REST contract; any HTTP stack works the same way. ## Authentication When fronting Turbopuffer with `deriveFromStore`, use the upstream API key as the gateway bearer token. Standalone [configuration](https://github.com/hev/layer/blob/v0.6/docs/setup.md) controls the connection and inbound authentication mode. ```sh curl --fail-with-body http://localhost:8080/v2/namespaces \ -H "Authorization: Bearer $TURBOPUFFER_API_KEY" ``` Client examples using `LAYER_GATEWAY_URL` refer to the gateway base URL. `LAYER_GATEWAY_API_KEY` is the bearer token for a keyed connection. ## Gateway failures SDKs return connection errors when the gateway is unreachable. They do not retry directly against a backing store. Store failures and unsupported request shapes remain visible to callers. ## Gateway operations [Write rows](/docs/ce/api/write), [query](/docs/ce/api/query), [scan](/docs/ce/api/scans), [embed](/docs/ce/api/embed), and [federate](/docs/ce/api/federated-query). ## Response metadata Layer returns retrieval metadata in [response headers](/docs/ce/api/response-headers). Generated clients expose those headers in their typed responses where supported. # Federated query Source: https://hevlayer.com/docs/ce/api/federated-query import Edition from "../../../components/docs/Edition.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; A **federated query** runs one query across a **set** of namespaces. `POST /v2/query` is namespace-less: the per-namespace [query endpoint](/docs/ce/api/query) names its namespace in the path, while this endpoint takes the set in the body, so the namespace is no longer a path parameter. It has no upstream equivalent — turbopuffer's multi-query is single-namespace, so the fan-out and the merge are a Layer composition over per-namespace reads. One ranking is spread across every namespace in the set and merged into a single ranked list — "search my whole feed / my whole library." For several **independent** rankings instead, query each namespace separately; a federated query always returns one fused list. ## Fan-out and fuse Supply one ranking and a `namespaces` set. The gateway runs the ranking against each namespace and returns one merged `rows` list, each row tagged with the namespace it came from. ```python response = await client.query({ "namespaces": ["moment-pod-changelog", "moment-pod-latent-space", "moment-pod-no-priors"], "rank_by": ["text", "Auto", "evaluating RAG systems"], "top_k": 12, "filters": ["published_at", "Gte", 1740000000], "include_attributes": ["text", "show", "source_url", "start_sec"], }) ``` ```go response, err := client.Query(ctx, &hevlayer.FederatedQueryRequest{ Namespaces: []string{"moment-pod-changelog", "moment-pod-latent-space", "moment-pod-no-priors"}, RankBy: []any{"text", "Auto", "evaluating RAG systems"}, TopK: 12, Filters: []any{"published_at", "Gte", 1740000000}, IncludeAttributes: []string{"text", "show", "source_url", "start_sec"}, }) ``` ```typescript const response = await client.query({ namespaces: ["moment-pod-changelog", "moment-pod-latent-space", "moment-pod-no-priors"], rank_by: ["text", "Auto", "evaluating RAG systems"], top_k: 12, filters: ["published_at", "Gte", 1740000000], include_attributes: ["text", "show", "source_url", "start_sec"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "namespaces": ["moment-pod-changelog", "moment-pod-latent-space", "moment-pod-no-priors"], "rank_by": ["text", "Auto", "evaluating RAG systems"], "top_k": 12, "filters": ["published_at", "Gte", 1740000000], "include_attributes": ["text", "show", "source_url", "start_sec"] }' ``` `rank_by` accepts the same vocabulary as a single-namespace query, including the Layer [`HybridText`](/docs/ce/api/query#hybrid-text-fusion) and [`Auto`](/docs/ce/api/query#query-routing) expressions. The query text is identical across the fan-out, so `Auto` routes **once** and the chosen route runs against every namespace — a single `routing` block is echoed. `filters` apply to every namespace. Name the explicit set to search, or pass `namespaces: ["*"]` to expand the authenticated key's namespace allowlist at request time. Omitting `namespaces` is equivalent to `["*"]`. ### Response One fused list. Each row carries `$namespace` (its origin) and `$rank` (its position within that namespace's results — the key the merge orders on). A `merge` block names the strategy, and a `namespaces` block reports each namespace's freshness and how many rows it returned to the merge. ```json { "rows": [ { "id": "ep123#7", "$namespace": "moment-pod-latent-space", "$rank": 1, "$score": 11.4, "show": "Latent Space", "text": "..." } ], "merge": { "method": "rank-interleave", "route": "fused" }, "routing": { "route": "fused", "policy": "v1", "tokens": 3, "executed": true }, "hybrid": { "tokens": ["evaluating", "rag", "systems"], "tokens_dropped": 0, "fuzziness": "auto", "rank_constant": 60, "legs": 4, "per_leg_limit": 60 }, "namespaces": [ { "namespace": "moment-pod-latent-space", "stable_as_of": 1747300000123, "matched": 14 }, { "namespace": "moment-pod-changelog", "stable_as_of": 1747299999001, "matched": 9 }, { "namespace": "moment-pod-no-priors", "stable_as_of": 1747300000050, "matched": 0 } ] } ``` | Field | Meaning | | --- | --- | | `$namespace` | The namespace this row came from. | | `$rank` | The row's 1-based rank **within its namespace**. The merge orders on this on a text route; see [Merge](#merge). | | `$score` / `$dist` | The row's native score within its namespace, carried as provenance. Comparable **within** a namespace, not across the fused list — do not threshold on it. On a vector route `$dist` is the merged ordering key and *is* globally comparable. | | `merge` | The merge applied. `method` is `"distance"` on a vector route, `"rank-interleave"` on a text route; `route` is currently always `"fused"` (reserved for future route-specific provenance). | | `namespaces` | Per-namespace echo: each reached namespace's `stable_as_of` watermark and `matched` — the rows it **returned to the merge** (up to [`per_namespace_limit`](#fusion-options)), not its surviving share of the final `top_k`. A namespace that matched nothing reports `matched: 0`. | `routing` and `hybrid` echo exactly as they do for a single-namespace [`Auto`](/docs/ce/api/query#query-routing) / [`HybridText`](/docs/ce/api/query#response) query, since the route runs once for the whole fan-out. ### Fusion options | Option | Default | Meaning | | --- | --- | --- | | `fusion.per_namespace_limit` | `clamp(2 × top_k, 10, 100)` | How many rows each namespace returns to the merge. Shallower than the single-namespace `per_leg_limit`: across a wide fan-out most namespaces contribute nothing to the final `top_k`, so deep per-namespace retrieval is wasted. Integer > 0. | | `fusion.rank_constant` | `60` | Reserved for route-dependent fusion tuning. It is accepted for forward compatibility and is inert in the current gateway. | ## Filters One `filters` expression is applied to **every** namespace in the set, and each namespace evaluates it independently against its own schema. So a filter only behaves uniformly when the attributes it references are part of a **shared filterable schema** across the set — present in every namespace, with the same type, and declared filterable. This is the filter analog of the [shared embedding space](#vector-merge-requires-a-matching-embedding-space) a vector merge needs: a federated query is only as coherent as the contract its namespaces share. A namespace that cannot evaluate the filter — the attribute is absent, has an incompatible type, or is not filterable there — is reported as `filter_schema_mismatch`, not as `matched: 0`. This is a per-namespace validity failure: the namespace did not participate in the filtered ranking because it could not evaluate the predicate. In default best-effort mode, the namespace is omitted from `rows`, listed in `errors`, and the response carries `x-layer-partial: true`. With `strict: true`, the whole request fails with `422`. This docs contract selects `422` for the strict-mode failure because the namespace set and filter expression are valid only if every required namespace can evaluate the filter. Keep filters to attributes declared filterable in every namespace in the set. Using the same embedding model does not imply a shared filterable schema; scope the set to namespaces that support the filters you send. ## Merge A federated query merges by the quantity the route makes comparable across namespaces: - **Vector route** (`ANN` over a query vector) — distances are comparable across namespaces that share an embedding space, so the gateway merges by `$dist` and the fused order is the exact global nearest-neighbor ranking. `merge.method` is `"distance"`. - **Text route** (`BM25`, `HybridText`, or a routed `fused`/`hybrid_text`) — BM25 and hybrid scores are **not** comparable across corpora (different term statistics; a hybrid `$score` is already a per-namespace fusion). The gateway merges by **rank-interleave**: rows are ordered by their `$rank` within their namespace, ties broken by `id`. `merge.method` is `"rank-interleave"`. No single pod dominates the head of the list; each contributes its best matches in rank order. The fused list exposes `$rank` as the ordering key and carries each row's native `$score` only as provenance. `top_k` truncates the merged list. Each namespace returns [`fusion.per_namespace_limit`](#fusion-options) rows to the merge. ### Vector merge requires a matching embedding space Merging by distance is only meaningful when every namespace in the set embeds into the same geometry: model, output dimension, normalization, and distance metric must match. A standalone namespace without an embedding profile uses the best-effort downgrade below, or fails when `strict: true`. The gateway compares those profiles before a vector fan-out uses distance merge: - All reached namespaces have the same profile → `merge.method: "distance"`. - A profile is missing or differs → best-effort responses downgrade to `merge.method: "rank-interleave"` and include `merge.downgraded_reason` (`"missing_embedding_profile"` or `"embedding_profile_mismatch"`). - With `strict: true`, a missing or mismatched profile is a `422`. A text fan-out is rank-interleave regardless, so embedding profiles do not affect it. ## Consistency Each namespace has its own [stable-read](/docs/ce/api/query#stable-reads) watermark; there is no single consistent cut across independent namespaces. The `namespaces` block reports each namespace's `stable_as_of`, and the response header `x-layer-stable-as-of` carries the **minimum** across the reached namespaces — the most conservative answer to "as of when." Layer does not manufacture a global watermark. ## Partial failure Across many namespaces, one may be deleted, time out, error upstream, or fail to evaluate the [filter](#filters). Unlike single-namespace [multi-query](/docs/ce/api/query#batch-query), the fan-out defaults to **best-effort**: it returns the namespaces that succeeded, lists the rest in an `errors` block, and sets `x-layer-partial: true`. A feed search does not blank because one pod is briefly unavailable. ```json { "rows": [ "..." ], "errors": [ { "namespace": "moment-pod-no-priors", "error": "Upstream error: namespace not found" } ] } ``` Availability failures carry a human-readable upstream message in `error`, not a stable machine code — match on `namespace`, not on the `error` string. Filter schema mismatches are the exception: they carry the stable `code: "filter_schema_mismatch"` plus a human-readable `detail` so clients can separate "namespace unavailable" from "namespace cannot evaluate this filter." ```json { "rows": [ { "id": "ep123#7", "$namespace": "moment-pod-latent-space", "$rank": 1 } ], "errors": [ { "namespace": "moment-pod-no-priors", "code": "filter_schema_mismatch", "error": "filter schema mismatch", "detail": "filter attribute published_at is absent, not filterable, or has an incompatible type" } ] } ``` Set `"strict": true` to opt into fail-fast: any namespace error fails the whole request. Reach for it when a missing namespace would make the result misleading rather than merely thinner. ## Pinned namespace concurrency Federated queries run up to `64` pinned and ready namespace legs concurrently, configured with `LAYER_PINNED_FEDERATED_QUERY_NAMESPACE_THREADS` (clamped to `1..512`). Other namespace legs retain `LAYER_FEDERATED_QUERY_NAMESPACE_THREADS` (default `16`). Mixed requests use separate lane limits with an overall ceiling of `512` when a pinned lane is present. Readiness uses the same [metadata observations and expiry as scans](/docs/ce/api/scans#pinning-and-fan-out). HybridText retains its `15`-token expansion limit regardless of pinning. Pinning changes scheduling capacity, not token selection or ranking semantics. ## Limits | Limit | Value | | --- | --- | | Namespaces per request | 512. Over the cap → `422` naming the excess. | | Pagination | Not supported. `cursor` is rejected — a fused cursor across independent namespaces does not form the monotone bands pagination relies on. | ## Validation | Condition | Status | | --- | --- | | A namespace read fails upstream | 200 with `errors`, or 502 with `strict: true` | | A namespace cannot evaluate `filters` (attribute absent, wrong type, or not filterable) | 200 with `errors[].code: "filter_schema_mismatch"` and `x-layer-partial: true`, or 422 with `strict: true` | | `namespaces` is empty | 422 | | `namespaces: ["*"]` mixed with other names | 422 | | `namespaces` set exceeds the cap | 422 | | Vector route missing/mismatched embedding profiles with `strict: true` | 422 | | `cursor` present | 422 | | `rank_by` expression invalid | 422 (same checks as a single-namespace query) | # Embed Source: https://hevlayer.com/docs/ce/api/embed import Edition from "../../../components/docs/Edition.astro"; import StoreNote from "../../../components/docs/StoreNote.astro"; import PostgresWarning from "../../../components/docs/PostgresWarning.astro"; import Upstream from "../../../components/docs/Upstream.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; import StoreMatrixLink from "../../../components/docs/StoreMatrixLink.astro"; The `embed` schema block and the `Embed` rank expression are Turbopuffer's, on every backend. Layer takes the same request and response shapes whichever store is behind it, so a client written against Turbopuffer's embedding documentation works unchanged through Layer. This documentation covers only Layer enhancements. Add an `embed` block to a string attribute in your schema and Layer computes that attribute's vector on every write. Put `["Embed", "..."]` in `rank_by` and Layer computes the query vector with the same model. Your application never loads a model or sends a vector of its own. ## Where the vector comes from `embed.serving.prefer` picks the provider that runs the model: | `embed.serving.prefer` | Who computes the vector | | --- | --- | | `local` | A CPU model bundled with Layer, running beside the gateway. No key, no GPU, no account. | | `worker` | A GPU worker in your own cluster, serving an open-weights checkpoint you declared. | | `turbopuffer` | Turbopuffer's hosted embedding service, from its managed model menu. | The provider is independent of the store the vector is written to, so any store that accepts `embed` can use any configured provider. Layer never substitutes one provider for another. A profile that selects a provider the deployment has not configured fails validation at write time, and a provider that fails at request time returns an error instead of falling back to a different model. `native` is accepted as an alias for `turbopuffer`, `autoscaler` for `worker`, and `lattice` for `local` on a [Lattice](/docs/ce/api/lattice) profile. All three providers are available. Layer validates a `turbopuffer` profile and forwards it unchanged to the upstream `POST /v2/namespaces/{ns}` and `POST /v2/namespaces/{ns}/query`, so Turbopuffer computes the vector and it never passes through Layer. `local` and `worker` compute the vector in Layer and write an ordinary ANN column. ## CPU models Layer bundles three CPU embedding models, small enough to run on a laptop under Docker. Their weights are baked into the gateway image, so there is nothing to download or configure before the first write. They use the same schema as the larger GPU and hosted models, with lower retrieval quality. | Model | Dims | Modality | Use it for | | --- | --- | --- | --- | | `sentence-transformers/all-MiniLM-L6-v2` | 384 | text | The default. 22M parameters, symmetric, no prefixes. | | `BAAI/bge-small-en-v1.5` | 384 | text | Better retrieval for 50% more parameters. Try it when MiniLM's results are weak. | | `openai/clip-vit-base-patch32` | 512 | image | Text-to-image search. Both towers run on CPU. | ### Text This write creates the `articles` namespace with an embedded `text` attribute and upserts two rows. Layer embeds each row's `text` with MiniLM before storing it. ```python await client.write_namespace("articles", { "upsert_rows": [ {"id": "planet-1", "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System."}, {"id": "photo-1", "title": "Photosynthesis", "text": "Plants turn sunlight, water, and carbon dioxide into food."}, ], "distance_metric": "cosine_distance", "schema": { "text": { "type": "string", "embed": { "model": "sentence-transformers/all-MiniLM-L6-v2", "dims": 384, "serving": {"prefer": "local"}, }, }, }, }) ``` ```go _, err := client.WriteNamespace(ctx, "articles", hevlayer.TurbopufferWriteRequest{ "upsert_rows": []map[string]any{ {"id": "planet-1", "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System."}, {"id": "photo-1", "title": "Photosynthesis", "text": "Plants turn sunlight, water, and carbon dioxide into food."}, }, "distance_metric": "cosine_distance", "schema": map[string]any{ "text": map[string]any{ "type": "string", "embed": map[string]any{ "model": "sentence-transformers/all-MiniLM-L6-v2", "dims": 384, "serving": map[string]any{"prefer": "local"}, }, }, }, }) ``` ```typescript await client.writeNamespace("articles", { upsert_rows: [ { id: "planet-1", title: "Planet", text: "Jupiter is the biggest planet in the Solar System." }, { id: "photo-1", title: "Photosynthesis", text: "Plants turn sunlight, water, and carbon dioxide into food." }, ], distance_metric: "cosine_distance", schema: { text: { type: "string", embed: { model: "sentence-transformers/all-MiniLM-L6-v2", dims: 384, serving: { prefer: "local" }, }, }, }, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/write" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upsert_rows": [ {"id": "planet-1", "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System."}, {"id": "photo-1", "title": "Photosynthesis", "text": "Plants turn sunlight, water, and carbon dioxide into food."} ], "distance_metric": "cosine_distance", "schema": { "text": { "type": "string", "embed": { "model": "sentence-transformers/all-MiniLM-L6-v2", "dims": 384, "serving": { "prefer": "local" } } } } }' ``` Query by meaning with `Embed`. Layer reads the model from the `text` schema and embeds the query string with it: ```python response = await client.query_namespace("articles", { "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]], "top_k": 3, "include_attributes": ["title", "text"], }) print(response.rows, response.performance) ``` ```go response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{ RankBy: []any{"text", "ANN", []any{"Embed", "largest planet in the solar system"}}, TopK: 3, IncludeAttributes: []string{"title", "text"}, }) ``` ```typescript const response = await client.queryNamespace("articles", { rank_by: ["text", "ANN", ["Embed", "largest planet in the solar system"]], top_k: 3, include_attributes: ["title", "text"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]], "top_k": 3, "include_attributes": ["title", "text"] }' ``` ```jsonc { "rows": [ { "id": "planet-1", "$dist": 0.137, "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System." } ], "performance": { "embedding_tokens": 7, "embedding_ms": 9 } } ``` The two text models are embedded differently, and Layer handles both. bge expects an instruction prefix on queries and MiniLM expects none, so Layer adds the prefix only when it embeds a query. bge uses CLS pooling and MiniLM uses mean pooling. Both outputs are L2-normalized, so `cosine_distance` scores are comparable across rows. ### Images With `modality: image` on a CLIP model, Layer embeds each attribute value with the image tower at write time and embeds `Embed` query text with the text tower, against the same vector column. Both towers run on CPU beside the gateway. ```python await client.write_namespace("photos", { "upsert_rows": [ {"id": "commons-131", "title": "Sunset at Porto Covo", "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"}, ], "distance_metric": "cosine_distance", "schema": { "image_url": { "type": "string", "embed": { "model": "openai/clip-vit-base-patch32", "dims": 512, "modality": "image", "serving": {"prefer": "local"}, }, }, }, }) ``` ```go _, err := client.WriteNamespace(ctx, "photos", hevlayer.TurbopufferWriteRequest{ "upsert_rows": []map[string]any{ {"id": "commons-131", "title": "Sunset at Porto Covo", "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"}, }, "distance_metric": "cosine_distance", "schema": map[string]any{ "image_url": map[string]any{ "type": "string", "embed": map[string]any{ "model": "openai/clip-vit-base-patch32", "dims": 512, "modality": "image", "serving": map[string]any{"prefer": "local"}, }, }, }, }) ``` ```typescript await client.writeNamespace("photos", { upsert_rows: [ { id: "commons-131", title: "Sunset at Porto Covo", image_url: "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" }, ], distance_metric: "cosine_distance", schema: { image_url: { type: "string", embed: { model: "openai/clip-vit-base-patch32", dims: 512, modality: "image", serving: { prefer: "local" }, }, }, }, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/write" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upsert_rows": [ {"id": "commons-131", "title": "Sunset at Porto Covo", "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg"} ], "distance_metric": "cosine_distance", "schema": { "image_url": { "type": "string", "embed": { "model": "openai/clip-vit-base-patch32", "dims": 512, "modality": "image", "serving": { "prefer": "local" } } } } }' ``` Query the image column with text: ```python response = await client.query_namespace("photos", { "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]], "top_k": 16, "include_attributes": ["title", "image_url"], }) ``` ```go response, err := client.QueryNamespace(ctx, "photos", &hevlayer.QueryRequest{ RankBy: []any{"image_url", "ANN", []any{"Embed", "sunset over water"}}, TopK: 16, IncludeAttributes: []string{"title", "image_url"}, }) ``` ```typescript const response = await client.queryNamespace("photos", { rank_by: ["image_url", "ANN", ["Embed", "sunset over water"]], top_k: 16, include_attributes: ["title", "image_url"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/photos/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["image_url", "ANN", ["Embed", "sunset over water"]], "top_k": 16, "include_attributes": ["title", "image_url"] }' ``` ```jsonc { "rows": [ { "id": "commons-131", "$dist": 0.412, "title": "Sunset at Porto Covo", "image_url": "https://upload.wikimedia.org/…/640px-Porto_Covo.jpg" } ], "performance": { "embedding_tokens": 5, "embedding_ms": 134 } } ``` Write responses report `embedding_images` instead of `embedding_tokens`. CPU CLIP handles query-time text embeds and write-time image embeds for small and medium corpora. A bulk image backfill needs a GPU. The provider is set per namespace, so a backfill can run on a GPU while queries embed on CPU. The [lens demo](https://lens.hevlayer.com) runs this setup: Wikimedia Commons Quality images embedded and searched on CPU, with the gateway's `performance` echo shown beside each result. Source at [github.com/hev/lens](https://github.com/hev/lens). ### Image input An image profile accepts an HTTP(S) URL or a base64 image string and fetches at most 20 MiB per URL. A rate-limited image host returns HTTP `429` with error `upstream_error`; a server error from the image host returns HTTP `503` with error `service_unavailable`. Layer preserves the host's `Retry-After` response header for both. Malformed URLs or base64, non-retryable 4xx responses, oversize content, and undecodable images return `422 validation_error`. ### Changing models requires a re-index The two text models both emit 384 dimensions, so switching between them leaves the namespace schema valid. It still requires re-embedding every row. Different models place vectors in unrelated spaces. If documents were embedded with MiniLM and queries are embedded with bge, the distances compare incompatible vectors and the ranking is meaningless, and Layer returns no error. The same applies when moving from a CPU model to a hosted or GPU model. Re-index from your source data so every vector comes from the new model. ### Bringing your own weights You can mount a model directory over the bundled menu to add or override checkpoints. The mount is optional and the bundled models need none. For a single static retriever that trades retrieval quality for microsecond embeds and a few megabytes of deployment size, see [Lattice](/docs/ce/api/lattice). ## Query with `Embed` `["Embed", text]` computes a query vector from `text` using the model settings of the attribute being ranked. When `rank_by` names the source attribute, Layer reads the model from its schema: ```python response = await client.query_namespace("clinical-notes", { "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]], "top_k": 10, }) print(response.rows) ``` ```go response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{ RankBy: []any{"text", "ANN", []any{"Embed", "chest pain radiating to left arm"}}, TopK: 10, }) ``` ```typescript const response = await client.queryNamespace("clinical-notes", { rank_by: ["text", "ANN", ["Embed", "chest pain radiating to left arm"]], top_k: 10, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["text", "ANN", ["Embed", "chest pain radiating to left arm"]], "top_k": 10 }' ``` When `rank_by` names the derived vector attribute `embed_`, pass the model explicitly. Omitting it returns `422` with `a model name must be provided`. ```python response = await client.query_namespace("clinical-notes", { "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", { "model": "acme/clinical-retrieval-v3", }]], "top_k": 10, }) ``` ```go response, err := client.QueryNamespace(ctx, "clinical-notes", &hevlayer.QueryRequest{ RankBy: []any{"embed_text", "ANN", []any{"Embed", "chest pain radiating to left arm", map[string]any{"model": "acme/clinical-retrieval-v3"}}}, TopK: 10, }) ``` ```typescript const response = await client.queryNamespace("clinical-notes", { rank_by: ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", { model: "acme/clinical-retrieval-v3", }]], top_k: 10, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/clinical-notes/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["embed_text", "ANN", ["Embed", "chest pain radiating to left arm", { "model": "acme/clinical-retrieval-v3" }]], "top_k": 10 }' ``` Query vectors are cached for 60 seconds by default. Set `LAYER_EMBED_CACHE_TTL_MS` to change the TTL. Instruction prefixes are part of the cache key. A missing provider returns `503 service_unavailable`. ### `Embed` with `Auto` An inline `Embed` lets [query routing](/docs/ce/api/query#query-routing) run a semantic or fused query in one request: ```python response = await client.query_namespace("articles", { "rank_by": ["title", "Auto", "how plants turn sunlight into food", { "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}], }], "top_k": 10, }) ``` ```go response, err := client.QueryNamespace(ctx, "articles", &hevlayer.QueryRequest{ RankBy: []any{"title", "Auto", "how plants turn sunlight into food", map[string]any{ "vector": []any{"Embed", "how plants turn sunlight into food", map[string]any{"field": "text"}}, }}, TopK: 10, }) ``` ```typescript const response = await client.queryNamespace("articles", { rank_by: ["title", "Auto", "how plants turn sunlight into food", { vector: ["Embed", "how plants turn sunlight into food", { field: "text" }], }], top_k: 10, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["title", "Auto", "how plants turn sunlight into food", { "vector": ["Embed", "how plants turn sunlight into food", {"field": "text"}] }], "top_k": 10 }' ``` The first tuple field (`title`) is the attribute for the lexical legs. `field` inside `Embed` names the attribute whose schema supplies the embedding profile and vector column (`text` above). Omit `field` when both are the same attribute. Layer chooses the route before it resolves `Embed`, so a short input routed to `hybrid_text` never calls the embedding provider. A semantic or fused response reports the routing decision (`routing.policy: "v1"`, `routing.executed: true`) and merges embedding measurements into `performance`. Without a vector or an inline `Embed`, the router returns `routing.executed: false` and leaves embedding to the caller. ## Model settings The full `embed` block: | Field | Meaning | | --- | --- | | `model` | A provider-namespaced Hugging Face repo id. | | `dims` | The vector dimension. Must match what the model emits. | | `attribute` | Where the vector is stored. Defaults to `embed_`, and may not be the source attribute. | | `serving.prefer` | `local`, `worker`, or `turbopuffer`. See [where the vector comes from](#where-the-vector-comes-from). | | `modality` | `text` (the default) or `image`. `image` requires a CLIP-family model. | | `revision` | Pins a checkpoint revision, including a finetuned one. | | `instructions.document` / `instructions.query` | Prefixes for asymmetric retrieval models. Both affect the query-cache key. | | `chunk` | Splits the attribute into one row per chunk on write. See [chunking long documents](#chunking-long-documents). | The first write that establishes an embedded schema must also set `distance_metric`. `revision`, `instructions`, and `chunk` are Layer extensions. Layer never forwards them upstream, and they require a GPU-served profile, except that a CLIP image profile may use `local`. ## Chunking long documents A single vector for a long document matches queries poorly, so long sources should be split into chunks and each chunk embedded as its own row. Split documents in your indexer and write one row per chunk. Store the parent document's id as an ordinary attribute so each hit can be traced back to its source. ## Performance accounting Write and query responses report embedding work under `performance`: ```jsonc { "rows": [ /* ... */ ], "performance": { "embedding_tokens": 8, "embedding_ms": 42 } } ``` Queries omit `embedding_tokens` on a cache hit. Layer merges provider measurements into the same object and exports the work as `hevlayer_embed_tokens_total` and `hevlayer_embed_compute_seconds_total`, labeled by namespace, store kind, model, and serving mode. # Lattice Source: https://hevlayer.com/docs/ce/api/lattice import Preview from "../../../components/docs/Preview.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro";

Lattice is in preview. It is documented and usable, but it sits outside the release line: it is not listed in the changelog, its configuration can change without a deprecation cycle, and it carries no compatibility promise. The supported way to embed text on CPU is the [bundled model menu](/docs/ce/api/embed#cpu-models); Lattice is for workloads that have measured that menu and need something smaller.

Lattice is a static retriever — a token lookup table rather than a transformer. It embeds text in microseconds on a CPU and adds a few megabytes to a deployment. It scores materially below a real dense embedder on retrieval, which is the trade it exists to make. It is an explicit serving leg. A namespace that selects Lattice never falls back to another leg, and an unconfigured artifact is a validation error rather than a silent substitution. ## Provisioning Generate a deployment artifact with the upstream [Lattice slicer](https://github.com/ErikKaum/lattice/tree/main/slicer), place its `model.safetensors` and `tokenizer.json` together, and set `LAYER_LATTICE_MODEL_PATH` to the model file before starting the gateway. The supported model id is `erikkaum/lattice-retrieval`; the requested `embed.dims` must match the loaded artifact, and only text modality is supported. ```bash uv run slicer slice \ --dim 512 \ --quant int4_row \ --output-dir /var/lib/hevlayer/lattice export LAYER_LATTICE_MODEL_PATH=/var/lib/hevlayer/lattice/model.safetensors ``` ```jsonc "text": { "type": "string", "embed": { "model": "erikkaum/lattice-retrieval", "dims": 512, "serving": { "prefer": "lattice" } } } ``` `prefer: lattice` selects the Lattice artifact. `prefer: local` also resolves to it when the declared model is `erikkaum/lattice-retrieval`. The recommended operating point is an int4-per-row, 512-dimensional artifact. Int4 quantizes the model's lookup-table weights only. Layer writes the resulting normalized vectors as `[512]f32`; Turbopuffer's int8 minimum for quantized vector storage is a separate choice and is not used by this path. ## End-to-end example Declare the Lattice profile on a string attribute, write rows, and query with `Embed`. The gateway embeds both sides in-process — no external inference provider is involved. Write two rows into a namespace whose `text` attribute carries the profile above: ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/write" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upsert_rows": [ {"id": "planet-1", "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System."}, {"id": "photo-1", "title": "Photosynthesis", "text": "Plants turn sunlight, water, and carbon dioxide into food."} ], "schema": { "text": { "type": "string", "embed": { "model": "erikkaum/lattice-retrieval", "dims": 512, "serving": { "prefer": "lattice" } } } } }' ``` Query by meaning rather than exact phrase: ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/articles/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["text", "ANN", ["Embed", "largest planet in the solar system"]], "top_k": 3, "include_attributes": ["title", "text"] }' ``` ```jsonc { "rows": [ { "id": "planet-1", "$dist": 0.137, "title": "Planet", "text": "Jupiter is the biggest planet in the Solar System." } ], "performance": { "embedding_tokens": 7, "embedding_ms": 1 // in-process lookup — no network hop to a provider } } ``` A live example of exactly this contract is the [Wikipedia × Lattice demo](https://wiki.hevlayer.com): all 283,997 Simple English Wikipedia articles (1.74M paragraph rows) embedded through Lattice and searched on Turbopuffer, with the `performance` echo displayed beside each result. Source at [github.com/hev/wiki](https://github.com/hev/wiki). ## Limits - Text only. An image modality on a Lattice profile is a validation error. - No [revision pins or instructions](/docs/ce/api/embed#model-settings). Those extensions require a GPU-served profile. - `embed.dims` must equal the sliced artifact's dimension. A mismatch is a validation error at write time, not a silent reshape. - A directory that fails to load stops the gateway at startup rather than serving a namespace that cannot embed. # Response Headers Source: https://hevlayer.com/docs/ce/api/response-headers import Edition from "../../../components/docs/Edition.astro"; Layer keeps turbopuffer-compatible read bodies in the upstream shape and returns Layer-specific metadata in response headers. | Header | Values | Returned by | | --- | --- | --- | | `x-layer-stable-as-of` | Epoch milliseconds | Query, multi-query, scan counts | | `x-layer-next-cursor` | Opaque cursor token | Single-query pagination | | `x-layer-warning` | `vector_attribute_dropped` | Query, fetch | | `traceparent` | W3C trace context | Query, multi-query | The watermark header can be absent before a stable observation exists. SDKs expose these headers as fields where that is more convenient. For example, `query_namespace` returns upstream `rows` and also sets `stable_as_of` / `next_cursor` on the parsed response object when the headers are present. # Query & Fetch Source: https://hevlayer.com/docs/ce/api/query import PostgresWarning from "../../../components/docs/PostgresWarning.astro"; import Edition from "../../../components/docs/Edition.astro"; import StoreMatrixLink from "../../../components/docs/StoreMatrixLink.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; import StoreNote from "../../../components/docs/StoreNote.astro"; import Upstream from "../../../components/docs/Upstream.astro"; import FeatureGate from "../../../components/docs/FeatureGate.astro"; Native queries are wire-compatible with the upstream `POST /v2/namespaces/{ns}/query` endpoint and forwarded as-is after validation. This documentation covers only Layer enhancements. This is Layer's query API. Layer reports its own metadata in `x-layer-*` response headers. ## Stable reads Layer tracks a stable watermark for Turbopuffer-backed namespaces. When one is available, `x-layer-stable-as-of` reports that observation. Store capabilities and watcher state determine whether a watermark filter applies; a cold-start gateway can return a query without this header. This is achieved by querying Turbopuffer with `consistency=eventual` and watching each namespace's `index.status`. While the index is updating, Layer injects a hidden `_hevlayer_upserted_at <= watermark` predicate so the read does not include partially indexed rows. Stable or unknown status runs without that predicate; a 429 can trigger one retry with the watermark filter forced on. Responses report `x-layer-stable-as-of` in epoch milliseconds when the watcher has a watermark. The header is omitted on a cold-start gateway that has not yet observed a stable poll. Query responses carry `next_cursor` and, when another page is available, `x-layer-next-cursor`. Cursor behavior depends on the selected store and query mode; unsupported pagination features return `422 UnsupportedByStore`. ## Query by id Pass `nearest_to_id` in place of `vector` to rank by stored document vectors instead of a raw query vector — exactly one of the two is required. `nearest_to_id` takes an **array of document ids**: the gateway resolves each id's vector (document cache first, the namespace's configured VectorStore on miss with a cache backfill) and averages them component-wise into a single centroid, then ranks nearest neighbors to that centroid. Pass one id to rank by a single document; pass several to get "more like these" over a set of seeds. ```python response = await client.query_namespace("products", { "nearest_to_id": ["asin-B08N5WRWNW", "asin-B07PXGQC1Q"], "top_k": 10, "include_attributes": ["title", "category"], }) ``` ```go response, err := client.QueryNamespace(ctx, "products", &hevlayer.QueryRequest{ NearestToID: []string{"asin-B08N5WRWNW", "asin-B07PXGQC1Q"}, TopK: 10, IncludeAttributes: []string{"title", "category"}, }) ``` ```typescript const response = await client.queryNamespace("products", { nearest_to_id: ["asin-B08N5WRWNW", "asin-B07PXGQC1Q"], top_k: 10, include_attributes: ["title", "category"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "nearest_to_id": ["asin-B08N5WRWNW", "asin-B07PXGQC1Q"], "top_k": 10, "include_attributes": ["title", "category"] }' ``` | Outcome | Status | | --- | --- | | Every id resolved (cache or origin) | 200, ranked results | | Any id has no stored vector anywhere | 404 (names the missing ids) | | `nearest_to_id` empty, or both/neither of `vector` / `nearest_to_id` | 422 | The centroid is an unweighted mean, so seed ids contribute equally regardless of how many you pass. All resolved vectors share the namespace's dimensionality, so no reconciliation is needed across seeds. This fuses the seeds into one ranking; to run several *independent* rankings in a single request, see [batch query](#batch-query). ## Rank expressions Pass `rank_by` with `top_k` when you need an explicit ranking operator instead of the top-level `vector` / `nearest_to_id` shape. Layer handles the portable subset with the same stable-read behavior as vector queries. Native upstream query bodies that omit `top_k` remain pass-through. `rank_by` is mutually exclusive with `vector` and `nearest_to_id`. ## Batch query `nearest_to_id` fuses several seeds into a **single** ranking. To run several **independent** queries in one round trip, each with its own ranking, post a `queries` array. The response is a parallel `results` array — one ranked result set per query, in request order: `{ "results": [{ "rows": ... }] }`. Layer holds every leg on the same stable cut, so a batch reads one consistent view of the index. (The method is `batch_query_namespace`. It is named apart from turbopuffer's own upstream multi-query — a `rerank_by` body, which Layer passes through unchanged, as noted at the end of this section — to keep the two distinct.) ```python batch = await client.batch_query_namespace("products", { "queries": [ {"rank_by": ["vector", "ANN", [0.1, 0.2, 0.3]], "top_k": 10}, {"rank_by": ["title", "BM25", "wireless earbuds"], "top_k": 10}, ], }) # batch.results[0].rows ranked by vector; batch.results[1].rows by text ``` ```go batch, err := client.BatchQueryNamespace(ctx, "products", &hevlayer.BatchQueryRequest{ Queries: []hevlayer.TurbopufferQueryRequest{ {"rank_by": []any{"vector", "ANN", []float64{0.1, 0.2, 0.3}}, "top_k": 10}, {"rank_by": []any{"title", "BM25", "wireless earbuds"}, "top_k": 10}, }, }) ``` ```typescript const batch = await client.batchQueryNamespace("products", { queries: [ { rank_by: ["vector", "ANN", [0.1, 0.2, 0.3]], top_k: 10 }, { rank_by: ["title", "BM25", "wireless earbuds"], top_k: 10 }, ], }); // batch.results[0].rows ranked by vector; batch.results[1].rows by text ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "queries": [ {"rank_by": ["vector", "ANN", [0.1, 0.2, 0.3]], "top_k": 10}, {"rank_by": ["title", "BM25", "wireless earbuds"], "top_k": 10} ] }' ``` All legs in a non-fused batch share one `x-layer-stable-as-of` value. A leg may use native `rank_by`, or the Layer `vector` / `nearest_to_id` single-query shape; `nearest_to_id` is resolved before the leg is sent upstream. Batches must contain 2 to 16 legs. `cursor` is rejected at the top level and per leg because pagination is single-query only. When `rerank_by` is present, Layer treats the request as an upstream fused query and passes the body through unchanged. Reach for a batch query when you genuinely need N rankings — distinct user queries batched into one round trip, or hybrid retrieval fused upstream with RRF. Reach for `nearest_to_id` when many seeds should collapse into one "more like these" ranking. To get typo-tolerant text search without building the fused query yourself, see [hybrid text fusion](#hybrid-text-fusion). Every leg here targets the namespace in the path. To fan one query across a **set** of namespaces — and merge them into a single ranked list — see [federated query](/docs/ce/api/federated-query). ## Hybrid text fusion BM25 misses typos and morphological variants; fuzzy matching alone loses the relevance signal BM25 provides. `HybridText` runs both in one request: the gateway tokenizes your input string, expands it into one BM25 leg plus one fuzzy leg per token, and the effective legs are RRF-fused into one ranking. One expression in, typo-tolerant ranked results out. `HybridText` is a Layer-only `rank_by` spelling on the existing query route — no new endpoint, no client changes beyond the expression. The gateway tokenizes with [`alyze`](https://github.com/turbopuffer/alyze), turbopuffer's own open-source tokenizer and the same code that segmented your text at index time, so query tokens match index terms by construction. The ranked field must be indexed for both full-text and fuzzy matching — declare it `{"type": "string", "full_text_search": true, "fuzzy": true}` in the namespace schema. The BM25 leg uses the full-text index; the per-token fuzzy legs use the fuzzy index. ```python response = await client.query_namespace("support-tickets", { "rank_by": ["content", "HybridText", "conection timout kubernets"], "top_k": 10, "filters": ["tenant", "Eq", "t-42"], "include_attributes": ["content", "title"], }) ``` ```go response, err := client.QueryNamespace(ctx, "support-tickets", &hevlayer.QueryRequest{ RankBy: []any{"content", "HybridText", "conection timout kubernets"}, TopK: 10, Filters: []any{"tenant", "Eq", "t-42"}, IncludeAttributes: []string{"content", "title"}, }) ``` ```typescript const response = await client.queryNamespace("support-tickets", { rank_by: ["content", "HybridText", "conection timout kubernets"], top_k: 10, filters: ["tenant", "Eq", "t-42"], include_attributes: ["content", "title"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/support-tickets/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["content", "HybridText", "conection timout kubernets"], "top_k": 10, "filters": ["tenant", "Eq", "t-42"], "include_attributes": ["content", "title"] }' ``` An optional fourth tuple element tunes the expansion. Defaults: ```json ["content", "HybridText", "conection timout kubernets", { "fuzziness": "auto", "stopwords": "en", "rank_constant": 60, "per_leg_limit": null }] ``` | Option | Default | Meaning | | --- | --- | --- | | `fuzziness` | `"auto"` | Edit-distance tolerance for the fuzzy legs, keyed to each token's length (turbopuffer requires at least 3 query characters per edit). `"auto"` permits up to distance 2: exact for tokens of 3–5 characters, distance 1 for 6–8, distance 2 for 9 or more. Fixed `0`, `1`, or `2` caps the ladder, so `0` is exact-only. | | `stopwords` | `"en"` | Tokens that do not get a fuzzy leg. `"en"` is the built-in English list, `false` gives every token a fuzzy leg, and an array of strings replaces the built-in list (entries are lowercased). The BM25 leg always ranks the full input and is unaffected. | | `rank_constant` | `60` | turbopuffer's RRF constant, passed through verbatim. Integer > 0. | | `per_leg_limit` | `clamp(5 × top_k, 50, 200)` | How deep each leg retrieves before fusion. Integer > 0. | | `threads` | `8` unless configured per namespace | Maximum concurrent upstream requests when the gateway scatter/gathers the expansion across a [sharded](/docs/ce/concepts#scattergather) namespace — the same fan-out control as [scans](/docs/ce/api/scans). Clamped to active shards. No effect on unsharded namespaces, where the expansion is a single fused upstream call. | Set top-level `include_leg_breakdown: true` to return per-row `$fused.legs` attribution. Each leg entry reports the leg label, that row's 1-based rank within the leg, and the leg's raw score or distance. `rank` and `score` are `null` when the row fell outside that leg's `per_leg_limit` cut. Labels are `bm25`, `fuzzy:`, and `semantic` on routed fused queries. ### Tokenization The input string becomes tokens under a fixed, documented policy: 1. Split on Unicode (UAX #29) word boundaries and lowercase, using `alyze` — the code behind turbopuffer's production `word_v4` tokenizer. Punctuation-only tokens never survive the split. 2. Drop tokens shorter than 2 characters. 3. Dedupe. 4. Remove stop words under the `stopwords` option (English by default). Removed tokens are reported in `stopwords_dropped` and do not count against the cap, so on long queries the cap spends its legs on content tokens. 5. Cap at 15 tokens (15 fuzzy legs + 1 BM25 leg = 16, the upstream subquery limit). Tokens cut by the cap are counted in `tokens_dropped`. Stop words only suppress fuzzy legs: the BM25 leg ranks the full input string. Stemming and language detection are not applied. The input must yield at least one token before stop-word removal. An input made only of stop words is valid: it has no fuzzy legs and ranks on the BM25 leg alone. ### Response Results are the RRF-fused list. A `hybrid` block echoes the effective expansion so defaults are never invisible: ```json { "rows": [ { "id": "ticket-4117", "$score": 0.0639, "content": "...", "title": "Connection timeout on Kubernetes ingress" } ], "hybrid": { "tokens": ["conection", "timout", "kubernets"], "tokens_dropped": 0, "stopwords": "en", "stopwords_dropped": [], "fuzziness": "auto", "rank_constant": 60, "legs": 4, "per_leg_limit": 50 }, "next_cursor": null } ``` | Field | Meaning | | --- | --- | | `$score` | RRF score. Comparable **within** a response, not across requests — do not threshold on it. | | `$fused.legs` | Present only when `include_leg_breakdown: true`. Per-leg attribution in effective leg order; each item has `leg`, `rank`, and `score`. | | `tokens` | Tokens that produced fuzzy legs, post-policy. | | `tokens_dropped` | Tokens removed by the 15-token cap (not by the length or punctuation rules, or as stop words). | | `stopwords` | The effective `stopwords` option: `"en"`, `false`, or the caller's list. | | `stopwords_dropped` | Tokens removed as stop words, in input order. Empty when `stopwords` is `false`. | | `legs` | Total effective subqueries in the fused expansion. Normally the fuzzy legs + 1 BM25 leg (plus 1 ANN leg on routed fused queries). On the `surfaced` fallback there is no BM25 leg, so `legs` equals the token count (one fuzzy leg per token). | | `surfaced` | Present and `true` only when the empty-result fallback fired (see [Surfacing fallback](#surfacing-fallback)). Absent on the normal path. | | `next_cursor` | Top-level field (not inside `hybrid`), always present in the body: the next page token, or `null` on the last page. Mirrors the `x-layer-next-cursor` header. Pass a non-null value back as `cursor`. | The `hybrid` block appears only on `HybridText` responses. On sharded namespaces it also reports the effective `threads` fan-out width. Requests without a `HybridText` expression, including native turbopuffer multi-query + `rerank_by` bodies, keep their upstream-shaped responses byte-for-byte. ### Surfacing fallback Every primary leg ranks by BM25 over the full input, which upstream scores at zero — and drops — when no token matches a stored term exactly. A fully-misspelled query therefore fuses to zero rows. When the primary expansion returns nothing, Layer re-runs one fuzzy leg per token, reorders each leg by field/token edit distance, and fuses those instead, so a typo-heavy query still surfaces near matches. The response then carries `"surfaced": true` in the `hybrid` block, and `legs` reflects the surfacing expansion — one fuzzy leg per token, with no BM25 leg. Working queries never reach this path; the fallback is purely additive and absent (`surfaced` omitted) on the normal path. ### Semantics - **Fusion.** RRF uses the effective leg order: BM25 first, then one fuzzy leg per token, then the semantic ANN leg on routed fused queries. `include_leg_breakdown: true` can require one upstream query per leg on unsharded namespaces so Layer can report per-leg ranks. - **One consistency cut.** Request-level `filters` are replicated to every leg, and the [stable-read](#stable-reads) watermark predicate is injected into every leg from a single read — all legs see the same cut. Responses carry `x-layer-stable-as-of` as usual. - **All-or-nothing.** Any leg failure fails the request; Layer does not return a partial fusion over surviving legs. ### Validation All return `422`: | Condition | Why | | --- | --- | | Input yields zero tokens under the policy before stop-word removal | Nothing to expand. An input made only of stop words is not rejected. | | `HybridText` inside a `queries` array | The expansion is already one batch deep by construction. | | `fuzziness` not in `"auto" \| 0 \| 1 \| 2`; `stopwords` not `"en"`, `false`, or an array of strings; `rank_constant` ≤ 0; `per_leg_limit` ≤ 0; `threads` < 1 | Out of range. | To let the gateway pick between hybrid text and semantic retrieval per query, see [query routing](#query-routing). ## Query routing Real search boxes receive both `"timout"` and `"why do pods lose their connection during deploys"`. The first wants [hybrid text fusion](#hybrid-text-fusion); the second wants semantic retrieval — lexical legs add noise on long conversational input, and ANN underperforms on short identifier-shaped tokens. `Auto` is a Layer-only `rank_by` spelling that makes that call per query, so the branch doesn't live ad hoc in your application code. The route is chosen from the shape of the input alone. Supply an inline `Embed` as the vector source and Layer resolves it only after the policy selects `semantic` or `fused`; a `hybrid_text` route never calls the embedding provider. Without either an inline `Embed` or a numeric vector, a vector-needing route returns the routing decision instead of results so the application can embed and re-issue. Short keyword traffic executes immediately and never pays for an embedding. ```python response = await client.query_namespace("support-tickets", { "rank_by": ["title", "Auto", user_input, { "vector": ["Embed", user_input, {"field": "content"}], }], "top_k": 10, "filters": ["tenant", "Eq", "t-42"], }) ``` ```go response, err := client.QueryNamespace(ctx, "support-tickets", &hevlayer.QueryRequest{ RankBy: []any{"title", "Auto", userInput, map[string]any{ "vector": []any{"Embed", userInput, map[string]any{"field": "content"}}, }}, TopK: 10, Filters: []any{"tenant", "Eq", "t-42"}, }) ``` ```typescript const response = await client.queryNamespace("support-tickets", { rank_by: ["title", "Auto", userInput, { vector: ["Embed", userInput, { field: "content" }], }], top_k: 10, filters: ["tenant", "Eq", "t-42"], }); ``` ```bash # One request: title is lexical; content selects the embedding profile. curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/support-tickets/query" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "rank_by": ["title", "Auto", "why do pods lose their connection during deploys", { "vector": ["Embed", "why do pods lose their connection during deploys", { "field": "content" }] }], "top_k": 10, "filters": ["tenant", "Eq", "t-42"] }' ``` ### Routing policy The v1 policy reads the token count of the input under the same [tokenizer policy](#tokenization) as hybrid text fusion: | Tokens | Route | Runs | | --- | --- | --- | | ≤ 2 | `hybrid_text` | The [hybrid text fusion](#hybrid-text-fusion) expansion. | | ≥ 8 | `semantic` | ANN over the supplied query vector. | | 3 – 7 | `fused` | Both, merged upstream by RRF. | Vector availability never changes which route is chosen — only whether it executes in this request. `hybrid_text` always executes and does not resolve an inline `Embed`; `semantic` and `fused` resolve it after routing, execute with a numeric `vector`, and defer only when neither is present. The policy is versioned (`"policy": "v1"`) in the response. ### Options The optional fourth tuple element: | Option | Default | Meaning | | --- | --- | --- | | `route` | `"auto"` | Force `"hybrid_text"`, `"semantic"`, or `"fused"` instead of applying the policy. Used on re-issue after a deferral, and for A/B comparison of strategies on the same input. | | `vector` | — | A numeric query vector, or `["Embed", input, {field?, model?}?]`. `field` selects the source or derived attribute whose embedding profile supplies the model and vector target; it defaults to the Auto lexical field. Layer resolves `Embed` only for `semantic` and `fused`. A derived `embed_` field requires `model`. | | `fuzziness` | `"auto"` | Forwarded to the `HybridText` expansion on the `hybrid_text` and `fused` routes: `"auto"`, `0`, `1`, or `2`. `0` forces exact-only matching. No effect on the `semantic` route. | | `stopwords` | `"en"` | Forwarded to the `HybridText` expansion on the `hybrid_text` and `fused` routes: `"en"`, `false`, or an array of strings. Does not change the token count the routing policy reads. | When the chosen route expands hybrid-text legs, the hybrid defaults apply and the [`hybrid` echo block](#response) appears alongside `routing`. Set top-level `include_leg_breakdown: true` to add `$fused.legs` to each fused row; the fused route includes a final `semantic` leg after the BM25 and fuzzy-token legs. ### Response Every `Auto` response carries a `routing` block: ```json { "rows": [{"id": "ticket-4117", "$score": 0.0639, "title": "..."}], "routing": { "route": "hybrid_text", "policy": "v1", "tokens": 1, "executed": true }, "hybrid": {"tokens": ["timout"], "tokens_dropped": 0, "stopwords": "en", "stopwords_dropped": [], "fuzziness": "auto", "rank_constant": 60, "legs": 2, "per_leg_limit": 50} } ``` | Field | Meaning | | --- | --- | | `route` | The strategy chosen (or forced). | | `policy` | Routing policy version that made the decision. `"forced"` when `route` was supplied. | | `tokens` | Token count the policy read, post tokenizer policy. | | `executed` | `false` on a deferral: the route needs a vector the request didn't supply. `rows` is empty; embed and re-issue with the route forced. | Routed queries follow the same semantics as their underlying strategy: one consistency cut across all legs and all-or-nothing leg failure. When an inline `Embed` is resolved, its `embedding_tokens` and `embedding_ms` measurements are merged into the normal top-level `performance` object. ### Validation All return `422`: | Condition | Why | | --- | --- | | Forced `"semantic"` or `"fused"` without `vector` | Forcing asserts you have the vector; only auto-routing defers. | | Input yields zero tokens under the policy | Nothing to route. | | `vector` dimensionality mismatch | Same check as a plain vector query. | | `Auto` inside a `queries` array | Inherited from [hybrid text fusion](#validation). | ## Counting matches To count how many rows match a full-text or vector query, use [scan](/docs/ce/api/scans) count mode with the `fts` or `ann` selector. Ranked counts share the single `/scans` endpoint with filter counts — `fts` is exact, `ann` is a radius scan flagged `approximate`, and both honor the `exhaustive` flag and the count deadline. ## Fetch `GET /v2/namespaces/{ns}/documents/{id}` fetches a row by ID. `POST /v2/namespaces/{ns}/documents` fetches a batch with a body such as `{"ids":["earbuds","speaker"],"include_attributes":["title"]}`. The standalone gateway reads through to the selected store. A missing single row returns 404; a batch response separates `documents` from `missing` IDs. These routes do not require a managed document cache in the Compose bundle. # Scan Source: https://hevlayer.com/docs/ce/api/scans import PostgresWarning from "../../../components/docs/PostgresWarning.astro"; import Edition from "../../../components/docs/Edition.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; import FeatureGate from "../../../components/docs/FeatureGate.astro"; Live scans select matching IDs, count rows, or aggregate distinct field values against the configured store. Choose `source: "origin"` for standalone scans. ## Routes Create a job with `POST /v2/namespaces/{ns}/scans`. Poll it through `GET /v2/namespaces/{ns}/scans/{id}` and read its result pages through `GET /v2/namespaces/{ns}/scans/{id}/results`. Count mode responds synchronously. ID and values job state is in-memory and resets on gateway restart. ## ID mode ```json {"mode":"ids","source":"origin","filters":["category","Eq","audio"],"page_size":1000} ``` ## Count mode ```json {"mode":"count","source":"origin","filters":["category","Eq","audio"]} ``` ## Values mode ```json {"mode":"values","source":"origin","field":"category","page_size":1000} ``` Values scans return distinct `v` values and their `n` counts. The result is bounded to 1,000,000 distinct values; `truncated: true` indicates that only the most frequent values were retained, each with its exact count. ## Full-text count Count rows matching a BM25 query with the `fts` selector. Full-text counts are exact and always run origin scatter/gather, so `source` must be omitted, `auto`, or `origin`. A `filters` array, when present, is ANDed on as an extra constraint. ```python count = await client.create_scan("products", { "mode": "count", "fts": {"field": "title", "query": "wireless headphones"}, "filters": ["category", "Eq", "Electronics"], "exhaustive": True, }) ``` ```go count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{ Mode: "count", Fts: hevlayer.FtsScan{Field: "title", Query: "wireless headphones"}, Filters: []interface{}{"category", "Eq", "Electronics"}, Exhaustive: true, }) ``` ```typescript const count = await client.createScan("products", { mode: "count", fts: { field: "title", query: "wireless headphones" }, filters: ["category", "Eq", "Electronics"], exhaustive: true, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "mode": "count", "fts": {"field": "title", "query": "wireless headphones"}, "filters": ["category", "Eq", "Electronics"], "exhaustive": true }' ``` ## Hybrid text count Count rows in the keyword/fuzzy neighborhood of a `HybridText` query with the `hybrid_text` selector. The scan tokenizes `query` with the HybridText policy, then evaluates the BM25 leg, one fuzzy leg per token, **and** one surfacing leg per token (the RFC 0057 empty-result fallback's legs), and counts the de-duplicated union of returned row ids. This count is a **superset** of the `hybrid_text` query route's deduped rows: the scan always includes the surfacing legs, whereas the query route only adds them when its primary legs (BM25 + fuzzy) return nothing. On a partial-typo query whose primary legs do match, the scan can therefore count more rows than the route returns. Use this selector for a generous live count next to `hybrid_text` or `auto` results that routed to `hybrid_text`; plain `fts` counts exact BM25 only. ```python count = await client.create_scan("products", { "mode": "count", "hybrid_text": {"field": "title", "query": "wireles headphones"}, "filters": ["category", "Eq", "Electronics"], }) ``` ```go count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{ Mode: "count", HybridText: hevlayer.HybridTextScan{Field: "title", Query: "wireles headphones"}, Filters: []interface{}{"category", "Eq", "Electronics"}, }) ``` ```typescript const count = await client.createScan("products", { mode: "count", hybrid_text: { field: "title", query: "wireles headphones" }, filters: ["category", "Eq", "Electronics"], }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "mode": "count", "hybrid_text": {"field": "title", "query": "wireles headphones"}, "filters": ["category", "Eq", "Electronics"] }' ``` ## Radius count Count rows within `radius` of a query vector with the `ann` selector — a distance-ball scan. `radius` is required and finite (without an upper bound every row is in the ball); `field` defaults to `vector`. Like `fts`, radius counts always run origin scatter/gather. The count is **approximate**: ANN recall means the index's membership of the ball may differ from the true set, independent of saturation, so the response carries `approximate: true`. The radius bound is applied by the gateway to the `$dist` returned by the ranked query. It is not sent upstream as a filter. ```python count = await client.create_scan("products", { "mode": "count", "ann": {"field": "vector", "vector": [0.12, -0.3, 0.88], "radius": 0.25}, }) ``` ```go count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{ Mode: "count", Ann: hevlayer.AnnScan{Field: "vector", Vector: []float64{0.12, -0.3, 0.88}, Radius: 0.25}, }) ``` ```typescript const count = await client.createScan("products", { mode: "count", ann: { field: "vector", vector: [0.12, -0.3, 0.88], radius: 0.25 }, }); ``` ```bash curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \ -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "mode": "count", "ann": {"field": "vector", "vector": [0.12, -0.3, 0.88], "radius": 0.25} }' ``` ```json { "count": 980, "served_by": "origin", "approximate": true, "bounded": false, "timed_out": false, "shards_saturated": 0, "shards_total": 1, "threads": 1, "elapsed_ms": 51 } ``` ### Bounding ranked scans Ranked selectors fan out one turbopuffer query per shard, each capped at `top_k = 10_000`. `threads` bounds fan-out width: how many shard requests can run at once. `exhaustive` and `timeout_seconds` bound depth: what happens when a shard hits that cap and how long recursion can run. - `exhaustive: false` (default) — one scatter/gather. A saturated shard contributes its cap as a lower bound; the response carries `bounded: true` with `shards_saturated > 0`. - `exhaustive: true` — for BM25, recurse on each saturated shard via score-band pagination (`$score < last` with an `id` tiebreak) until every page is short or `timeout_seconds` elapses. ANN radius scans do not push `$dist` filters upstream; the gateway counts returned rows whose `$dist <= radius` and marks the shard exhausted when the first over-radius row appears. If the full page is still inside the radius, the shard remains `bounded`. The same `threads` value applies to the initial round and every exhaustive round over the remaining saturated shards. `bounded` and `approximate` are independent. `bounded` means a shard saturated and the count is a `>=` lower bound for the rows the index returned; `approximate` means the distance ball's membership is itself fuzzy. An `ann` count can be `bounded: false` yet still `approximate: true`. ## Sources Use origin for live store reads. Managed-cache and precomputed snapshot serving are documented in the full view. Origin filters use the selected store's supported filter grammar. ## Pinning and fan-out For Turbopuffer origin scans, namespaces observed with `pinning.ready_replicas > 0` default to 32 concurrent upstream requests, capped at 512. Other namespaces default to 8, capped at 32. Missing, failed, or older-than-two-minute observations use the unpinned limits. A request's `threads` remains bounded by the applicable maximum and active shard count. These controls do not enable pinning. # VectorStores Source: https://hevlayer.com/docs/ce/api/vectorstores import Edition from "../../../components/docs/Edition.astro"; import StoreNote from "../../../components/docs/StoreNote.astro"; ## VectorStores `GET /v2/vectorstores` lists standalone store connections without exposing credential values. `GET /v2/vectorstores/{name}` reads one connection. ```sh curl --fail-with-body http://localhost:8080/v2/vectorstores \ -H "Authorization: Bearer $TURBOPUFFER_API_KEY" ``` The list response has a `vectorstores` array. Entries identify the store name, kind, default selection, endpoint, and inbound authentication mode. Configure connections through [standalone configuration](https://github.com/hev/layer/blob/v0.6/docs/setup.md). For a keyed store, send its configured bearer token. # Write Source: https://hevlayer.com/docs/ce/api/write import PostgresWarning from "../../../components/docs/PostgresWarning.astro"; import Edition from "../../../components/docs/Edition.astro"; import StoreMatrixLink from "../../../components/docs/StoreMatrixLink.astro"; import StoreNote from "../../../components/docs/StoreNote.astro"; import Upstream from "../../../components/docs/Upstream.astro"; import FeatureGate from "../../../components/docs/FeatureGate.astro"; import CodeTabs from "../../../components/docs/CodeTabs.astro"; Writes are wire-compatible with the upstream `POST /v2/namespaces/{ns}` endpoint and forwarded as-is after validation. This documentation covers only Layer enhancements. Layer stamps every row-producing write with `_hevlayer_upserted_at` for supported stores. The stamp is what holds the [read watermark](/docs/ce/api/query); the full set of reserved attributes Layer manages on a row lives in the [document model](/docs/ce/document-model). ## Status Layer validates the body before forwarding and can fail independently of the backing store, so the write path carries a few statuses a plain proxy wouldn't: - **200 OK** — applied to the store and stamped. - **422 Unprocessable Entity** — Layer rejected the body before forwarding: no recognized native write operation, a reserved `_hevlayer_*` attribute name, a removed custom-write key, or a schema type the configured store cannot represent. The body is a Layer error (`{ "error": "validation_error", … }` or `{ "error": "UnsupportedByStore", … }`), not a store one. - **Upstream passthrough** — any non-2xx the backing store returns is relayed verbatim. - **502 Bad Gateway** — Layer could not reach the backing store (`{ "error": "upstream_error", … }`); the write did not apply. The passthrough case includes a failed conditional write (`upsert_condition`, `patch_condition`, `delete_condition`) — turbopuffer's error body comes back untouched.