API

Scan

Live scans select matching IDs, count rows, or aggregate distinct field values against the configured store. Choose source: "origin" for standalone scans.

Routes

Create a job with POST /v2/namespaces/{ns}/scans. Poll it through GET /v2/namespaces/{ns}/scans/{id} and read its result pages through GET /v2/namespaces/{ns}/scans/{id}/results. Count mode responds synchronously. ID and values job state is in-memory and resets on gateway restart.

ID mode

{"mode":"ids","source":"origin","filters":["category","Eq","audio"],"page_size":1000}

Count mode

{"mode":"count","source":"origin","filters":["category","Eq","audio"]}

Values mode

{"mode":"values","source":"origin","field":"category","page_size":1000}

Values scans return distinct v values and their n counts. The result is bounded to 1,000,000 distinct values; truncated: true indicates that only the most frequent values were retained, each with its exact count.

Full-text count

Count rows matching a BM25 query with the fts selector. Full-text counts are exact and always run origin scatter/gather, so source must be omitted, auto, or origin. A filters array, when present, is ANDed on as an extra constraint.

count = await client.create_scan("products", {
    "mode": "count",
    "fts": {"field": "title", "query": "wireless headphones"},
    "filters": ["category", "Eq", "Electronics"],
    "exhaustive": True,
})
count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{
    Mode:       "count",
    Fts:        hevlayer.FtsScan{Field: "title", Query: "wireless headphones"},
    Filters:    []interface{}{"category", "Eq", "Electronics"},
    Exhaustive: true,
})
const count = await client.createScan("products", {
  mode: "count",
  fts: { field: "title", query: "wireless headphones" },
  filters: ["category", "Eq", "Electronics"],
  exhaustive: true,
});
curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "count",
    "fts": {"field": "title", "query": "wireless headphones"},
    "filters": ["category", "Eq", "Electronics"],
    "exhaustive": true
  }'

Hybrid text count

Count rows in the keyword/fuzzy neighborhood of a HybridText query with the hybrid_text selector. The scan tokenizes query with the HybridText policy, then evaluates the BM25 leg, one fuzzy leg per token, and one surfacing leg per token (the RFC 0057 empty-result fallback’s legs), and counts the de-duplicated union of returned row ids.

This count is a superset of the hybrid_text query route’s deduped rows: the scan always includes the surfacing legs, whereas the query route only adds them when its primary legs (BM25 + fuzzy) return nothing. On a partial-typo query whose primary legs do match, the scan can therefore count more rows than the route returns. Use this selector for a generous live count next to hybrid_text or auto results that routed to hybrid_text; plain fts counts exact BM25 only.

count = await client.create_scan("products", {
    "mode": "count",
    "hybrid_text": {"field": "title", "query": "wireles headphones"},
    "filters": ["category", "Eq", "Electronics"],
})
count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{
    Mode:       "count",
    HybridText: hevlayer.HybridTextScan{Field: "title", Query: "wireles headphones"},
    Filters:    []interface{}{"category", "Eq", "Electronics"},
})
const count = await client.createScan("products", {
  mode: "count",
  hybrid_text: { field: "title", query: "wireles headphones" },
  filters: ["category", "Eq", "Electronics"],
});
curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "count",
    "hybrid_text": {"field": "title", "query": "wireles headphones"},
    "filters": ["category", "Eq", "Electronics"]
  }'

Radius count

Count rows within radius of a query vector with the ann selector — a distance-ball scan. radius is required and finite (without an upper bound every row is in the ball); field defaults to vector. Like fts, radius counts always run origin scatter/gather.

The count is approximate: ANN recall means the index’s membership of the ball may differ from the true set, independent of saturation, so the response carries approximate: true.

The radius bound is applied by the gateway to the $dist returned by the ranked query. It is not sent upstream as a filter.

count = await client.create_scan("products", {
    "mode": "count",
    "ann": {"field": "vector", "vector": [0.12, -0.3, 0.88], "radius": 0.25},
})
count, err := client.CreateScan(ctx, "products", &hevlayer.CreateScanRequest{
    Mode: "count",
    Ann:  hevlayer.AnnScan{Field: "vector", Vector: []float64{0.12, -0.3, 0.88}, Radius: 0.25},
})
const count = await client.createScan("products", {
  mode: "count",
  ann: { field: "vector", vector: [0.12, -0.3, 0.88], radius: 0.25 },
});
curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/products/scans" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "count",
    "ann": {"field": "vector", "vector": [0.12, -0.3, 0.88], "radius": 0.25}
  }'
{
  "count": 980,
  "served_by": "origin",
  "approximate": true,
  "bounded": false,
  "timed_out": false,
  "shards_saturated": 0,
  "shards_total": 1,
  "threads": 1,
  "elapsed_ms": 51
}

Bounding ranked scans

Ranked selectors fan out one turbopuffer query per shard, each capped at top_k = 10_000. threads bounds fan-out width: how many shard requests can run at once. exhaustive and timeout_seconds bound depth: what happens when a shard hits that cap and how long recursion can run.

  • exhaustive: false (default) — one scatter/gather. A saturated shard contributes its cap as a lower bound; the response carries bounded: true with shards_saturated > 0.
  • exhaustive: true — for BM25, recurse on each saturated shard via score-band pagination ($score < last with an id tiebreak) until every page is short or timeout_seconds elapses. ANN radius scans do not push $dist filters upstream; the gateway counts returned rows whose $dist <= radius and marks the shard exhausted when the first over-radius row appears. If the full page is still inside the radius, the shard remains bounded.

The same threads value applies to the initial round and every exhaustive round over the remaining saturated shards.

bounded and approximate are independent. bounded means a shard saturated and the count is a >= lower bound for the rows the index returned; approximate means the distance ball’s membership is itself fuzzy. An ann count can be bounded: false yet still approximate: true.

Sources

Use origin for live store reads. Managed-cache and precomputed snapshot serving are documented in the full view. Origin filters use the selected store’s supported filter grammar.

Pinning and fan-out

For Turbopuffer origin scans, namespaces observed with pinning.ready_replicas > 0 default to 32 concurrent upstream requests, capped at 512. Other namespaces default to 8, capped at 32. Missing, failed, or older-than-two-minute observations use the unpinned limits. A request’s threads remains bounded by the applicable maximum and active shard count. These controls do not enable pinning.

esc
↵ ask AI · ↑↓ open hev/ask