Source: opensearch_knn_filter_pushdown · opensearch_knn_filter_pushdown.md · updated 2026-08-07 · 🔒 secret gist

Synced verbatim from gist.github.com/bl9.

Common Filter Push-Down in OpenSearch — RFC, Design, and the k-NN filter() Implementation

Deep dive on:

• RFC: neural-search#1135 — Common Filter Support for Hybrid Query Sub-Queries
• Implementation: k-NN#2599 — Add filter function to KNNQueryBuilder
• Prerequisite: OpenSearch#17409 — Add filter function for AbstractQueryBuilder, BoolQueryBuilder, ConstantScoreQueryBuilder
• Consumer: neural-search#1206 — Add filter function for NeuralQueryBuilder and HybridQueryBuilder

Shipped in OpenSearch 3.0.

Table of Contents

1. TL;DR
2. Background concepts you need first
   • 2.1 The OpenSearch query lifecycle
   • 2.2 QueryBuilder vs Lucene Query
   • 2.3 Hybrid search and why it exists
   • 2.4 Filtering in vector search: pre, post, and efficient
   • 2.5 Why "tail relevant results" get dropped
3. The RFC (neural-search#1135)
   • 3.1 Problem statement
   • 3.2 Scope and non-goals
   • 3.3 High-level design: two options
   • 3.4 Low-level design: three options
   • 3.5 Per-query-type push-down semantics
   • 3.6 The FilterCombinationMode that didn't ship
   • 3.7 Score filters are blocked
4. Architecture: three repos, one contract
   • 4.1 The contract in core
   • 4.2 Core implementations
   • 4.3 Plugin implementations
   • 4.4 Why the plugins duplicate instead of share
5. k-NN PR #2599, line by line
   • 5.1 Files changed
   • 5.2 CHANGELOG.md
   • 5.3 KNNQueryBuilder.filter() — the 40 lines
   • 5.4 The unit test
   • 5.5 The integration test
   • 5.6 The review debate: mutate or rebuild?
6. End-to-end request trace
7. Semantics, edge cases, and real bugs
8. Build it yourself
   • 8.1 Dev environment
   • 8.2 Implementing the change from scratch
   • 8.3 The same feature for a query type that doesn't have it
9. Testing
   • 9.1 Unit tests
   • 9.2 Integration tests
   • 9.3 Manual REST testing on a live cluster
   • 9.4 Verifying push-down actually happened
10. Limitations and follow-ups
11. Reference index

1. TL;DR

• User-facing feature: hybrid query gets a top-level filter field. One filter, applied to every sub-query.
• Mechanism: a new method QueryBuilder#filter(QueryBuilder) on the core query-builder interface. It means "combine this filter into me and give me back the resulting builder."
• Default behaviour: AbstractQueryBuilder wraps this in a bool { must: this, filter: <f> }. Query types that natively understand filters override it to do something smarter.
• k-NN's job (PR #2599): override filter() on KNNQueryBuilder so the filter lands inside the k-NN query rather than around it. That's the whole point — a filter inside a k-NN query participates in ANN traversal; a filter outside it is post-filtering and silently destroys recall.
• Size: 40 lines of production code, 105 lines of tests, 1 CHANGELOG line. The design work is 95% of the value.

2. Background concepts you need first

2.1 The OpenSearch query lifecycle

A _search request goes through roughly this pipeline:

1. Route — REST layer resolves the handler, request lands on a coordinating node.
2. Parse — the JSON query DSL is parsed into a tree of QueryBuilder objects (fromXContent).
3. Search request processors — if a search pipeline defines any, they mutate the SearchRequest here.
4. Rewrite — QueryBuilder#rewrite() runs, possibly several rounds, possibly async (this is where terms lookups, neural → knn model inference, etc. resolve).
5. Serialize + fan out — the rewritten builder tree is written to StreamOutput and shipped to every data node holding a shard.
6. toQuery() — on each shard, QueryBuilder#toQuery(QueryShardContext) produces a Lucene Query.
7. Execute — Lucene collects hits per segment.
8. Post-process — search phase results processors (e.g. normalization/combination for hybrid), then response.

Steps 2 and 3 are the only two places where you can still restructure the query as a builder. That matters enormously, and is exactly the argument the RFC makes.

2.2 QueryBuilder vs Lucene Query

Two different objects, often confused:

KNNQueryBuilder holds a QueryBuilder filter field. At toQuery() time that filter is handed to KNNQueryFactory, which converts it to a Lucene Query, runs it to get a filtered doc-ID bitset, and feeds that bitset into the ANN search. The filter must be inside the builder before toQuery() runs, or the engine never sees it.

2.3 Hybrid search and why it exists

hybrid (from the neural-search plugin) runs N sub-queries in parallel over the same shards, keeps their score lists separate, then a search-phase processor normalizes each list (min-max, L2, z-score) and combines them (arithmetic/geometric/harmonic mean, RRF). This is how you fuse BM25 lexical relevance with dense-vector semantic relevance without one score scale swamping the other.

Canonical shape:

{
  "hybrid": {
    "queries": [
      { "match":   { "text": "wireless earbuds" } },
      { "neural":  { "embedding": { "query_text": "wireless earbuds", "model_id": "abc", "k": 50 } } }
    ]
  }
}

Because sub-queries are independent, a filter has to be applied to each of them independently.

2.4 Filtering in vector search: pre, post, and efficient

Three strategies, with very different cost/recall profiles:

• Pre-filtering (scoring script / exact): compute the filtered doc set first, then brute-force distance over exactly those docs. Perfect recall, O(P) distance computations. Great when P is tiny, terrible when P is large.
• Post-filtering (bool filter or post_filter): run ANN over the whole graph, get k results, then drop the ones that fail the filter. Cheap, but if the filter is at all restrictive you get back far fewer than k results — sometimes zero.
• Efficient / "during-search" filtering: hand the filter's doc-ID bitset to the ANN algorithm so it only accepts in-filter candidates while traversing. This is what knn.filter does.

Engine support, per the k-NN docs: efficient k-NN filtering applies filtering during the vector search, which ensures k results are returned if at least k results exist in total; it is supported by the Lucene engine with HNSW (OpenSearch 2.4+) and the Faiss engine with HNSW (2.9+) or IVF (2.10+). From OpenSearch 3.1, Faiss + HNSW additionally uses the Lucene ACORN filtering optimization during graph traversal when memory-optimized search is enabled.

Both engines adaptively pick a strategy. The algorithm weighs N (docs in the index), P (docs surviving the filter), k, R (results returned by filtered ANN), FT (the knn.advanced.filtered_exact_search_threshold index setting), and MDC (max distance computations allowed in exact search when FT is unset). Practically: if the filtered doc-ID list is at or below k, it does exact search, because returning k neighbors requires at least k distance computations anyway.

2.5 Why "tail relevant results" get dropped

This phrase appears throughout the RFC. Concretely:

• Index has 1M docs. Filter category = "medical" matches 2,000 of them.
• Filter outside (bool { must: knn(k=10), filter: category=medical }): ANN returns the global top 10 by vector distance. If none of those 10 are medical, you get 0 results. If 2 are, you get 2.
• Filter inside (knn { ..., filter: category=medical }): the engine restricts traversal (or switches to exact search over the 2,000) and returns the top 10 medical docs. You get 10.

The "tail" is the relevant-but-not-globally-top documents. Post-filtering throws them away before you ever see them. This is the single technical reason the RFC rejected the simpler designs.

3. The RFC (neural-search#1135)

Authored by bzhangam, opened 22 Jan 2025, labelled v3.0.0, now closed/implemented.

3.1 Problem statement

To apply one filter across a hybrid query today, you repeat it in every sub-query. The neural/knn sub-queries take a filter field; the rest need wrapping in a bool. Verbose, error-prone, and the wrapping form is semantically different from the inline form.

Before:

{
  "hybrid": {
    "queries": [
      { "neural": { "embedding": { "query_text": "...", "model_id": "m1", "k": 50,
                                   "filter": { "term": { "status": "active" } } } } },
      { "bool": { "must": [ { "match": { "title": "..." } } ],
                  "filter": [ { "term": { "status": "active" } } ] } },
      { "bool": { "must": [ { "match": { "body": "..." } } ],
                  "filter": [ { "term": { "status": "active" } } ] } }
    ]
  }
}

After:

{
  "hybrid": {
    "queries": [
      { "neural": { "embedding": { "query_text": "...", "model_id": "m1", "k": 50 } } },
      { "match": { "title": "..." } },
      { "match": { "body": "..." } }
    ],
    "filter": { "term": { "status": "active" } }
  }
}

3.2 Scope and non-goals

Explicitly out of scope in v1:

• Metrics — no per-feature metric API; deferred to a separate project.
• Configurability — one filter, AND-combined, always. No per-sub-query opt-out.
• No change to filter semantics — push-down only relocates the filter; how neural/knn/bool each interpret a filter is untouched.
• No recursive push-down — the filter reaches the top-level sub-query and stops. It does not descend into a sub-query's own inner queries. Deliberate: keeps the transform predictable and cheap.

3.3 High-level design: two options

Option 1 (chosen): push down during parsing. Add a filter field to the hybrid query; apply the push-down in HybridQueryBuilder.fromXContent.

Option 2 (deferred): a new SearchRequestProcessor. Model it on core's FilterQueryRequestProcessor, register it in the search pipeline alongside the normalization processor.

The deciding constraint: KNNQueryBuilder optimizes its filter before Lucene Query conversion, so push-down must happen at step 2 or 3 of the lifecycle. Both options satisfy that; Option 1 is cheaper and friendlier.

3.4 Low-level design: three options

Option 1 (chosen): copy the filter into sub-queries when the type supports it. Types that natively hold a filter (neural, knn, bool, constant_score, hybrid) get special handling; everything else gets bool-wrapped.

Option 2: always bool-wrap. Trivial to implement, but for neural/knn it degrades to post-filtering and loses the tail. Rejected.

Option 3: use the existing post_filter. Zero effort, but it runs after hybrid normalization and combination, and it is post-filtering by definition. Rejected.

3.5 Per-query-type push-down semantics

Worked example — bool sub-query:

// input
{ "hybrid": {
    "queries": [
      { "bool": { "must": [ <q1> ], "filter": [ <f1> ] } },
      <q2>
    ],
    "filter": <F> } }

// after parsing
{ "hybrid": {
    "queries": [
      { "bool": { "must": [ <q1> ], "filter": [ <f1>, <F> ] } },
      { "bool": { "must": [ <q2> ], "filter": [ <F> ] } }
    ] } }

Worked example — constant_score wrapping a neural:

// input
{ "hybrid": {
    "queries": [
      { "constant_score": { "filter": { "neural": { ... } }, "boost": 1.2 } },
      <q1>
    ],
    "filter": <F> } }

// after parsing — F goes INSIDE the neural query, not around the constant_score
{ "hybrid": {
    "queries": [
      { "constant_score": { "filter": { "neural": { ..., "filter": <F> } }, "boost": 1.2 } },
      { "bool": { "must": [ <q1> ], "filter": [ <F> ] } }
    ] } }

3.6 The FilterCombinationMode that didn't ship

The RFC proposed a two-argument signature:

public QueryBuilder filter(QueryBuilder filter, FilterCombinationMode mode);

public enum FilterCombinationMode {
    AND,               // v1: the only mode
    IGNORE_IF_EXISTS,  // future: skip if the sub-query already has a filter
    OR                 // future
}

What actually merged in core is the one-argument form, AND hard-coded. Reasonable call — the enum was speculative, and adding a parameter later is an additive change. Worth knowing if you read the RFC and then the code and wonder where the mode went.

3.7 Score filters are blocked

Hybrid normalizes scores after sub-queries run. A user writing a score-based filter at the hybrid level would reasonably expect it to apply to normalized scores — but push-down applies it pre-normalization, per sub-query, on raw scores. The numbers would be meaningless. The RFC's answer: reject score-based filters at parse time rather than document a footgun.

4. Architecture: three repos, one contract

┌─────────────────────────────────────────────────────────────┐
│ OpenSearch core  (PR #17409)                                │
│                                                             │
│   interface QueryBuilder                                    │
│       QueryBuilder filter(QueryBuilder filter);   ← contract│
│                                                             │
│   AbstractQueryBuilder      → bool{must:this, filter:f}     │
│   BoolQueryBuilder          → filterClauses.add(f)          │
│   ConstantScoreQueryBuilder → recurse into wrapped filter   │
└───────────────┬─────────────────────────┬───────────────────┘
                │ extends                 │ extends
    ┌───────────▼──────────┐   ┌──────────▼──────────────────┐
    │ k-NN  (PR #2599)     │   │ neural-search (PR #1206)    │
    │  KNNQueryBuilder     │   │  NeuralQueryBuilder         │
    │    → rebuild with    │   │    → set/merge queryfilter  │
    │      merged filter   │   │  HybridQueryBuilder         │
    └──────────────────────┘   │    → fan out to sub-queries │
                               └─────────────────────────────┘

Merge order was forced: core (3 Mar 2025) → k-NN (14 Mar 2025) → neural-search. neural-search's HybridQueryBuilder.filter() calls query.filter(f) polymorphically, so KNNQueryBuilder must already implement it or you get the base-class bool wrapping and the whole point is lost.

4.1 The contract in core

server/src/main/java/org/opensearch/index/query/QueryBuilder.java:

QueryBuilder filter(QueryBuilder filter);

Documented semantics: combine the filter with this builder. If the builder already carries a filter, merge and return the builder itself. Otherwise use a bool query to combine builder and filter, and return the bool. A null filter is a no-op.

Note this is an interface method with no default — every implementor must supply one. That's why AbstractQueryBuilder provides the fallback: all ~60 core query types extend it, so they all inherit sane behaviour for free.

4.2 Core implementations

AbstractQueryBuilder — the universal fallback:

public static boolean validateFilterParams(QueryBuilder filter) {
    return filter != null;
}

public QueryBuilder filter(QueryBuilder filter) {
    if (validateFilterParams(filter) == false) {
        return this;
    }
    final BoolQueryBuilder modifiedQB = new BoolQueryBuilder();
    modifiedQB.must(this);
    modifiedQB.filter(filter);
    return modifiedQB;
}

validateFilterParams is public static on purpose: plugin code in other JARs calls it. It is the shared null-guard for every implementation across all three repos.

BoolQueryBuilder — no wrapper needed:

public BoolQueryBuilder filter(QueryBuilder filter) {
    if (validateFilterParams(filter) == false) {
        return this;
    }
    filterClauses.add(filter);
    return this;
}

This is a pre-existing method, not a new one — BoolQueryBuilder.filter(QueryBuilder) has always meant "add a filter clause." PR #17409 only added the null-guard so the same method could satisfy the new interface contract. The signature covariance (BoolQueryBuilder return type) is legal and keeps the existing fluent API source-compatible. Nice bit of API design: zero breakage, new semantics for free.

ConstantScoreQueryBuilder — recurse:

public ConstantScoreQueryBuilder filter(QueryBuilder filter) {
    if (validateFilterParams(filter) == false) {
        return this;
    }
    QueryBuilder filteredFilterBuilder = filterBuilder.filter(filter);
    if (filteredFilterBuilder != filterBuilder) {
        return new ConstantScoreQueryBuilder(filteredFilterBuilder);
    }
    return this;
}

The != identity check is the interesting line: if the inner builder mutated itself in place (bool, neural), return this and allocate nothing; if it returned a different object (knn, or a bool-wrapped leaf), rebuild the wrapper around the new object. This "returns-self-or-new" convention runs through the entire feature.

4.3 Plugin implementations

HybridQueryBuilder (neural-search) — the fan-out:

public QueryBuilder filter(QueryBuilder filter) {
    if (validateFilterParams(filter) == false) {
        return this;
    }
    ListIterator<QueryBuilder> iterator = queries.listIterator();
    while (iterator.hasNext()) {
        QueryBuilder query = iterator.next();
        // set the query again because query.filter(filter) can return new query.
        iterator.set(query.filter(filter));
    }
    return this;
}

The ListIterator.set() is required, not stylistic. query.filter(f) may return a different object; a plain for-each would compute the new builder and throw it away.

And in fromXContent, the parse-time application:

} else if (FILTER_FIELD.match(currentFieldName, parser.getDeprecationHandler())) {
    filter = parseInnerQueryBuilder(parser);
}
...
for (QueryBuilder query : queries) {
    if (filter == null) {
        compoundQueryBuilder.add(query);
    } else {
        compoundQueryBuilder.add(query.filter(filter));
    }
    ...
}

Two details worth noting in current main:

• filter is only accepted as a START_OBJECT. An array ("filter": [...]) or a scalar hits throwUnsupportedFilterParsingException → "[hybrid] query's [filter] field must be a query object". To combine several filters, wrap them in a bool yourself.
• Usage is instrumented: EventStatsManager.increment(EventStatName.HYBRID_QUERY_FILTER_REQUESTS). The RFC listed metrics as out of scope; they were added later.

NeuralQueryBuilder (neural-search) — mutate in place:

public QueryBuilder filter(QueryBuilder filterToBeAdded) {
    if (validateFilterParams(filterToBeAdded) == false) {
        return this;
    }
    if (this.queryfilter == null) {
        this.queryfilter = filterToBeAdded;
    } else {
        this.queryfilter = this.queryfilter.filter(filterToBeAdded);
    }
    return this;
}

Contrast this with the k-NN version below — same problem, opposite style. Same feature, two plugins, two conventions.

4.4 Why the plugins duplicate instead of share

NeuralQueryBuilder and KNNQueryBuilder want nearly identical logic, but they live in different plugins with no shared dependency. The RFC's call: accept the duplication rather than invent a shared library for ~15 lines. Pragmatic, and correct for the size.

5. k-NN PR #2599, line by line

Author: chloewqg · Merged by: junqiu-lei, 14 Mar 2025 · Commit: 89b6883 (single commit, branch chloewqg:local2)

5.1 Files changed

5.2 CHANGELOG.md

One line under ## [Unreleased 3.0] → ### Features. Mandatory in this repo; CI enforces it. Note the merged line is missing its leading * bullet marker — a cosmetic slip that got through review.

5.3 KNNQueryBuilder.filter() — the 40 lines

Inserted right after doXContent, at what was line 381:

/**
 * Add a filter to Neural Query Builder                      // (a)
 * @param filterToBeAdded fiXlter to be added                // (b)
 * @return return itself with underlying filter combined with passed in filter
 */
@Override                                                     // (c)
public QueryBuilder filter(QueryBuilder filterToBeAdded) {    // (d)
    if (validateFilterParams(filterToBeAdded) == false) {     // (e)
        return this;
    }

    if (this.filter == null) {                                // (f)
        return KNNQueryBuilder.builder()                      // (g)
            .fieldName(fieldName)
            .vector(vector)
            .k(k)
            .maxDistance(maxDistance)
            .minScore(minScore)
            .methodParameters(methodParameters)
            .filter(filterToBeAdded)                          // (h)
            .ignoreUnmapped(ignoreUnmapped)
            .rescoreContext(rescoreContext)
            .expandNested(expandNested)
            .build();                                         // (i)
    }

    return KNNQueryBuilder.builder()                          // (j)
        .fieldName(fieldName)
        .vector(vector)
        .k(k)
        .maxDistance(maxDistance)
        .minScore(minScore)
        .methodParameters(methodParameters)
        .filter(filter.filter(filterToBeAdded))               // (k) ← the whole trick
        .ignoreUnmapped(ignoreUnmapped)
        .rescoreContext(rescoreContext)
        .expandNested(expandNested)
        .build();
}

Annotations:

• (a) Javadoc says "Neural Query Builder" — copy-paste from NeuralQueryBuilder. bzhangam flagged it in review; chloewqg said a follow-up PR would fix it. It is still wrong in main today.

• (b) fiXlter — typo, also still present. Two cosmetic defects that survived review and a year of main. Instructive about what code review actually catches.

• (c) @Override is what binds this to the new core interface method. Without core PR #17409 merged first, this line is a compile error. This is the hard dependency between the two repos.

• (d) Parameter deliberately named filterToBeAdded, not filter, because filter is already a field on the class. Avoids this.filter shadowing noise in the body.

• (e) The shared null-guard from AbstractQueryBuilder. Note the == false style — OpenSearch convention, avoids the easily-missed !. Returning this (not null, not a copy) makes filter(null) a true no-op and lets callers like ConstantScoreQueryBuilder use identity comparison to detect "nothing changed."

• (f) Branch on whether a filter already exists.

• (g) No existing filter → rebuild the whole builder rather than assigning to the field. See §5.6.

• (h) The new filter becomes the filter, verbatim. No wrapping. This is what makes it a real push-down: KNNQueryBuilder.filter is handed to KNNQueryFactory at toQuery() time and drives efficient filtering.

• (i) build() runs Builder.validate() — checks fieldName non-empty, vector non-null/non-empty, exactly one of k/minScore/maxDistance, 0 < k <= K_MAX, minScore > 0. This validation-on-construction is the stated justification for the rebuild.

• (j) Existing filter → same rebuild, one field different.

• (k) The recursive delegation. filter.filter(filterToBeAdded) dispatches polymorphically on the existing filter's type:

  • existing filter is a BoolQueryBuilder → appends to its filterClauses, returns the same instance. Result: bool { ..., filter: [old..., new] }. Flat, no extra nesting.
  • existing filter is a TermQueryBuilder (or any leaf) → AbstractQueryBuilder.filter wraps it: bool { must: term, filter: newFilter }.
  • existing filter is a ConstantScoreQueryBuilder → recurses into its inner query.

  Fifteen lines get correct AND-composition for every query type in the system, present and future. That's the payoff of putting the contract in core.

What this method does NOT copy: boost and queryName. The Builder has both fields (queryName(String), boost(float), default DEFAULT_BOOST), and build() applies them via .boost(boost).queryName(queryName). filter() sets neither, so a pushed-down filter silently resets boost to 1.0 and drops the _name. Since AbstractQueryBuilder.equals compares both, this is observable. The same omission exists in doRewrite(), which predates this PR — so it's a pre-existing pattern that was copied, not a new mistake. Still a real bug. See §7.

5.4 The unit test

KNNQueryBuilderTests.testFilter(), +55 lines, three cases:

@SneakyThrows
public void testFilter() {
    // Case 1 — null is a no-op
    KNNQueryBuilder knnQueryBuilder = new KNNQueryBuilder(FIELD_NAME, QUERY_VECTOR, K);
    KNNQueryBuilder updatedKnnQueryBuilder = (KNNQueryBuilder) knnQueryBuilder.filter(null);
    assertEquals(knnQueryBuilder, updatedKnnQueryBuilder);

    // Case 2 — existing filter, AND-combined
    knnQueryBuilder = KNNQueryBuilder.builder()
        .fieldName(FIELD_NAME).vector(QUERY_VECTOR).filter(TERM_QUERY).k(K).build();
    updatedKnnQueryBuilder = (KNNQueryBuilder) knnQueryBuilder.filter(TERM_QUERY);

    BoolQueryBuilder expectedUpdatedQueryFilter = new BoolQueryBuilder();
    expectedUpdatedQueryFilter.must(TERM_QUERY);
    expectedUpdatedQueryFilter.filter(TERM_QUERY);

    assertEquals(knnQueryBuilder.fieldName(),           updatedKnnQueryBuilder.fieldName());
    assertEquals(knnQueryBuilder.vector(),              updatedKnnQueryBuilder.vector());
    assertEquals(knnQueryBuilder.getK(),                updatedKnnQueryBuilder.getK());
    assertEquals(knnQueryBuilder.getMaxDistance(),      updatedKnnQueryBuilder.getMaxDistance());
    assertEquals(knnQueryBuilder.getMinScore(),         updatedKnnQueryBuilder.getMinScore());
    assertEquals(knnQueryBuilder.getMethodParameters(), updatedKnnQueryBuilder.getMethodParameters());
    assertEquals(knnQueryBuilder.isIgnoreUnmapped(),    updatedKnnQueryBuilder.isIgnoreUnmapped());
    assertEquals(knnQueryBuilder.getRescoreContext(),   updatedKnnQueryBuilder.getRescoreContext());
    assertEquals(knnQueryBuilder.getExpandNested(),     updatedKnnQueryBuilder.getExpandNested());
    assertEquals(expectedUpdatedQueryFilter,            updatedKnnQueryBuilder.getFilter());

    // Case 3 — no existing filter, direct assignment
    knnQueryBuilder = KNNQueryBuilder.builder()
        .fieldName(FIELD_NAME).vector(QUERY_VECTOR).k(K).build();
    updatedKnnQueryBuilder = (KNNQueryBuilder) knnQueryBuilder.filter(TERM_QUERY);
    // ... same field-by-field assertions ...
    assertEquals(TERM_QUERY, updatedKnnQueryBuilder.getFilter());
}

Reading it:

• Case 1 passes precisely because filter(null) returns this — reference equality would also pass, but assertEquals documents intent.
• Case 2 is the important one. TERM_QUERY is a leaf, so filter.filter(TERM_QUERY) hits AbstractQueryBuilder.filter and produces bool { must: TERM_QUERY, filter: TERM_QUERY }. The test asserts that exact structure. It also confirms every other field survives the rebuild — which is the real risk of the rebuild-vs-mutate approach.
• Case 3 confirms the filter == null branch assigns without wrapping.
• Gap: because the assertions are field-by-field rather than assertEquals(expectedBuilder, actual), the dropped boost/queryName never gets caught. A whole-object comparison on a builder with a non-default boost would fail today.
• The (KNNQueryBuilder) casts are needed because the interface returns QueryBuilder. Safe here only because the implementation is known to return a KNNQueryBuilder; in general callers must not assume this (the base implementation returns a BoolQueryBuilder).
• The PR also left a commented-out copy of the production code inside the test as an explanatory note. Dead comment; would not survive a stricter review.

New import: org.opensearch.index.query.BoolQueryBuilder.

5.5 The integration test

FaissIT.testQueryWithFilterFunctionAppliedMultipleShards(), +50 lines. New imports: MatchNoneQueryBuilder, RangeQueryBuilder.

@SneakyThrows
public void testQueryWithFilterFunctionAppliedMultipleShards() {
    XContentBuilder builder = XContentFactory.jsonBuilder()
        .startObject()
          .startObject(PROPERTIES_FIELD_NAME)
            .startObject(FIELD_NAME)
              .field(TYPE_FIELD_NAME, KNN_VECTOR_TYPE)
              .field(DIMENSION_FIELD_NAME, "3")
              .startObject(KNNConstants.KNN_METHOD)
                .field(KNNConstants.NAME, METHOD_HNSW)
                .field(KNNConstants.METHOD_PARAMETER_SPACE_TYPE, SpaceType.L2.getValue())
                .field(KNNConstants.KNN_ENGINE, KNNEngine.FAISS.getName())
              .endObject()
            .endObject()
            .startObject(INTEGER_FIELD_NAME)
              .field(TYPE_FIELD_NAME, FILED_TYPE_INTEGER)
            .endObject()
          .endObject()
        .endObject();
    String mapping = builder.toString();

    createIndex(INDEX_NAME, Settings.builder()
        .put("number_of_shards", 10)
        .put("number_of_replicas", 0)
        .put("index.knn", true).build());
    putMappingRequest(INDEX_NAME, mapping);

    addKnnDocWithAttributes("doc1", new float[] { 7.0f, 7.0f, 3.0f },
                            ImmutableMap.of("dateReceived", "2024-10-01"));
    refreshIndex(INDEX_NAME);

    final float[] searchVector = { 6.0f, 7.0f, 3.0f };

    // (1) matching filter → 1 hit
    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("dateReceived").gte("2023-11-01");
    KNNQueryBuilder knnQueryBuilder = new KNNQueryBuilder(FIELD_NAME, searchVector, 1, null);
    KNNQueryBuilder updatedKnnQueryBuilder = (KNNQueryBuilder) knnQueryBuilder.filter(rangeQueryBuilder);
    Response response = searchKNNIndex(INDEX_NAME, updatedKnnQueryBuilder, 10);
    List<KNNResult> knnResults = parseSearchResponse(EntityUtils.toString(response.getEntity()), FIELD_NAME);
    assertEquals(1, knnResults.size());

    // (2) impossible filter → 0 hits
    updatedKnnQueryBuilder = (KNNQueryBuilder) knnQueryBuilder.filter(new MatchNoneQueryBuilder());
    response = searchKNNIndex(INDEX_NAME, updatedKnnQueryBuilder, 10);
    knnResults = parseSearchResponse(EntityUtils.toString(response.getEntity()), FIELD_NAME);
    assertEquals(0, knnResults.size());
}

Reading it:

• Faiss + HNSW + L2, index.knn: true — chosen because Faiss is the engine with the richest efficient-filtering path. The test lives in FaissIT, which extends KNNRestTestCase and drives a real cluster over REST.
• 10 shards, 1 document. Deliberate: the doc lands on one shard, the other nine have empty segments. This exercises the serialize→fan-out→toQuery path on shards where the filter matches nothing, catching NPEs and empty-segment bugs. That's why "MultipleShards" is in the name.
• new KNNQueryBuilder(FIELD_NAME, searchVector, 1, null) — the four-arg constructor with an explicit null filter, so the test starts on the filter == null branch.
• knnQueryBuilder.filter(...) twice on the same object — the second call at (2) reuses the original builder, not the updated one. This only works because filter() returns a new instance and leaves the receiver untouched. A mutating implementation would have made the second query range AND match_none. The immutability contract is load-bearing here, and this test would catch a regression to mutation.
• k = 1 in the query, size = 10 in the search — separates ANN k from result size.
• MatchNoneQueryBuilder is the cleanest way to prove the filter is actually reaching the engine: the doc is a perfect vector match, so a nonzero result would mean the filter was ignored.
• Weakness: with one document, this proves the filter is applied, not that it is applied efficiently. It cannot distinguish push-down from post-filtering. A recall test — 10k docs, restrictive filter, assert k results come back — would be the real proof, and isn't here.

5.6 The review debate: mutate or rebuild?

Worth reading in full on the PR; it's the most substantive part.

bzhangam: why construct a new KNNQueryBuilder instead of assigning this.filter and returning this?

heemin32: no behavioural difference today, but validation lives in build(). If a future filter-related validation is added there, direct field assignment bypasses it.

bzhangam: that's speculative — building for a use case that doesn't exist. If the concern is bypass, mark the field final and force mutation through the builder. Two-way door.

heemin32: final on a QueryBuilder reference still permits mutating the referenced object; it only prevents swapping the instance. If we'd adopt the pattern later anyway, adopt it now.

chloewqg closed it by citing a review on k-NN#2585: query builders are intended to be effectively immutable, with all parameters validated together at construction rather than individually.

Rebuild won. And as §5.5 shows, the integration test then depends on that choice. Note the inconsistency though: NeuralQueryBuilder.filter() in the sibling plugin mutates in place and returns this. Both satisfy the interface — the contract only says "return the resulting builder" — but a caller that assumes one style breaks against the other. HybridQueryBuilder handles this correctly with iterator.set(...); hand-written client code often won't.

6. End-to-end request trace

Request:

POST /products/_search?search_pipeline=nlp-pipeline
{
  "query": {
    "hybrid": {
      "queries": [
        { "match": { "title": "running shoes" } },
        { "knn": { "embedding": { "vector": [0.1, 0.2, 0.3], "k": 50 } } }
      ],
      "filter": { "term": { "in_stock": true } }
    }
  }
}

1. REST → parse. HybridQueryBuilder.fromXContent walks the JSON. queries → two builders. filter → a TermQueryBuilder.
2. Push-down loop. For each sub-query, query.filter(termQuery):
   • MatchQueryBuilder has no override → AbstractQueryBuilder.filter → returns bool { must: match, filter: term }.
   • KNNQueryBuilder overrides → this.filter == null → returns a new KNNQueryBuilder with filter = term.
   • Both results are added to the new HybridQueryBuilder. The top-level filter field is not retained — it has been fully distributed.
3. Stats. HYBRID_QUERY_FILTER_REQUESTS incremented.
4. Rewrite. KNNQueryBuilder.doRewrite rewrites the inner filter; if the rewrite produced a different object it rebuilds the k-NN builder around it. (Same missing boost/queryName copy as filter().)
5. Serialize + fan out. No wire-format change was needed anywhere in this feature: KNNQueryBuilder already serialized its filter field. That's why there's no BWC work in PR #2599.
6. toQuery per shard. In KNNQueryBuilder.doToQuery:
   • Guard: if the engine creates custom segment files and doesn't support filters (nmslib), and filter != null → IllegalArgumentException("Engine [x] does not support filters").
   • KNNQueryFactory.CreateQueryRequest.builder()...filter(this.filter)...build() → KNNQueryFactory.create(...).
7. Engine execution. The factory converts the filter builder to a Lucene Query, runs it to get a doc-ID bitset, and picks pre-filter exact search vs filtered ANN based on N/P/k/FT/MDC. The match sub-query's bool runs conventionally.
8. Normalization + combination. The hybrid search-phase processor normalizes each sub-query's score list and combines them.
9. Response.

The key structural fact: the transform happens once, at parse time, on the coordinating node. By the time anything is serialized or executed, the query is indistinguishable from one where the user typed the filters out longhand. That's the RFC's performance argument, and it holds — there's no per-shard or per-segment cost to the feature itself.

7. Semantics, edge cases, and real bugs

Combination is always AND, always. No opt-out. If a sub-query already filters category=books and you push in_stock=true, you get both. If you want OR or per-sub-query behaviour, don't use the common filter — write the filters inline.

Push-down is one level deep. hybrid.filter reaches each sub-query. It does not descend into a sub-query's inner queries — except via the specific overrides (constant_score recursing into its wrapped query, bool appending to its own filter clauses).

boost and queryName are lost. KNNQueryBuilder.filter() and doRewrite() rebuild via KNNQueryBuilder.builder() without .boost(...) or .queryName(...). A knn sub-query with "boost": 2.0 or "_name": "vec" silently loses both when a hybrid filter is pushed into it. AbstractQueryBuilder.equals compares both, so this is observable and testable. Reproduce: build a KNNQueryBuilder with a non-default boost, call filter(termQuery), assert getBoost(). This is a good first-contribution PR if you want one — the fix is two chained calls plus a test.

Filter type matters for the resulting shape.

Repeated push-downs on a leaf filter nest linearly. Not a correctness problem — Lucene flattens most of this at rewrite — but deeply nested bools show up in _validate/query?explain output and can be confusing when debugging.

nmslib rejects filters entirely. A hybrid filter pushed into a knn sub-query on an nmslib index throws at toQuery time, per-shard, not at parse time. The error surfaces as a shard failure, not a 400. Faiss and Lucene engines are fine.

Score-based filters are rejected at parse time by design (§3.7) — the pre/post normalization mismatch would produce nonsense.

Nested hybrid is not supported; pushing a filter into a nested hybrid errors.

filter must be an object. "filter": [ {...}, {...} ] is rejected. Combine with bool instead.

Mutation-style vs rebuild-style implementations coexist. NeuralQueryBuilder mutates and returns this; KNNQueryBuilder rebuilds and returns a new object; BoolQueryBuilder mutates; AbstractQueryBuilder returns a different type entirely. Always use the return value. Never call q.filter(f) for side effects.

8. Build it yourself

8.1 Dev environment

The k-NN plugin has native (JNI) dependencies, so first build is slow.

# JDK 21 is what current main targets; check build.gradle for the exact version
java -version

git clone https://github.com/opensearch-project/k-NN.git
cd k-NN
git submodule update --init -- jni/external/nmslib
git submodule update --init -- jni/external/faiss

# full build, including native libs — expect 20-40 min cold
./gradlew build

# start a single-node cluster with the plugin installed
./gradlew run
# → http://localhost:9200

If you only need the Java side compiled (no native libs, no ITs), ./gradlew compileJava compileTestJava is much faster for iterating.

For the hybrid-query half you also need the neural-search plugin, which depends on ml-commons. Easiest path for end-to-end manual testing is a Docker OpenSearch 3.x image, which bundles all three.

8.2 Implementing the change from scratch

Assume you're on a main that predates the feature.

Step 1 — verify the core contract exists.

grep -n "QueryBuilder filter(QueryBuilder" \
  ~/OpenSearch/server/src/main/java/org/opensearch/index/query/QueryBuilder.java

If it's absent, your k-NN @Override won't compile and you need OpenSearch ≥ 3.0 (or a locally published snapshot) as your dependency. This is the ordering constraint that forced core → k-NN → neural-search.

Step 2 — add the override to src/main/java/org/opensearch/knn/index/query/KNNQueryBuilder.java, after doXContent. Use the code in §5.3 — but fix the javadoc, and add the two lines the original missed:

/**
 * Combine a filter into this KNN query builder.
 * @param filterToBeAdded filter to be added
 * @return a builder whose filter is the AND-combination of the existing filter and the new one
 */
@Override
public QueryBuilder filter(QueryBuilder filterToBeAdded) {
    if (validateFilterParams(filterToBeAdded) == false) {
        return this;
    }
    final QueryBuilder combined = (this.filter == null)
        ? filterToBeAdded
        : this.filter.filter(filterToBeAdded);

    return KNNQueryBuilder.builder()
        .fieldName(fieldName)
        .vector(vector)
        .k(k)
        .maxDistance(maxDistance)
        .minScore(minScore)
        .methodParameters(methodParameters)
        .filter(combined)
        .ignoreUnmapped(ignoreUnmapped)
        .rescoreContext(rescoreContext)
        .expandNested(expandNested)
        .boost(boost)              // ← not in the merged PR
        .queryName(queryName)      // ← not in the merged PR
        .build();
}

Collapsing the two branches into one combined expression also removes the duplicated 11-line builder chain. Behaviourally identical to the merged version except for the boost/name fix.

Step 3 — imports. org.opensearch.index.query.QueryBuilder is almost certainly already imported. validateFilterParams is inherited from AbstractQueryBuilder, no import needed.

Step 4 — CHANGELOG. One bullet under ### Features in the unreleased section. CI fails without it.

Step 5 — spotless. ./gradlew spotlessApply before committing; the repo enforces formatting.

Step 6 — sign your commits. DCO is enforced: git commit --signoff.

8.3 The same feature for a query type that doesn't have it

If you're adding filter() to some other query builder, the checklist is:

1. Does the builder have a field that already means "filter" (or an inner query you should recurse into)? If not, don't override — the AbstractQueryBuilder bool-wrap is the right answer and you get it for free.
2. If it does, decide mutate-vs-rebuild and be consistent with the rest of your plugin.
3. Start with if (validateFilterParams(f) == false) return this; — the identity return is relied on by ConstantScoreQueryBuilder's != check.
4. Combine an existing filter via existing.filter(f), never by hand-rolling a BoolQueryBuilder. Delegation is what makes the feature compose.
5. If you rebuild, copy every field including boost and queryName.
6. Test: null case, no-existing-filter case, existing-filter case, and a whole-object assertEquals against an expected builder with non-default boost/name.

9. Testing

9.1 Unit tests

# just this test
./gradlew test --tests "org.opensearch.knn.index.query.KNNQueryBuilderTests.testFilter"

# whole class
./gradlew test --tests "*KNNQueryBuilderTests"

Cases to cover, beyond what the PR shipped:

public void testFilter_whenBoostAndNameSet_thenPreserved() {
    KNNQueryBuilder qb = KNNQueryBuilder.builder()
        .fieldName(FIELD_NAME).vector(QUERY_VECTOR).k(K)
        .boost(2.0f).queryName("vec").build();
    KNNQueryBuilder updated = (KNNQueryBuilder) qb.filter(TERM_QUERY);
    assertEquals(2.0f, updated.boost(), 0.0f);   // fails on merged code
    assertEquals("vec", updated.queryName());     // fails on merged code
}

public void testFilter_whenExistingFilterIsBool_thenFlatCombination() {
    BoolQueryBuilder existing = new BoolQueryBuilder().filter(TERM_QUERY);
    KNNQueryBuilder qb = KNNQueryBuilder.builder()
        .fieldName(FIELD_NAME).vector(QUERY_VECTOR).k(K).filter(existing).build();
    KNNQueryBuilder updated = (KNNQueryBuilder) qb.filter(RANGE_QUERY);
    BoolQueryBuilder actual = (BoolQueryBuilder) updated.getFilter();
    assertEquals(2, actual.filter().size());      // flat, not nested
}

public void testFilter_whenCalledTwiceOnSameBuilder_thenOriginalUnchanged() {
    KNNQueryBuilder qb = KNNQueryBuilder.builder()
        .fieldName(FIELD_NAME).vector(QUERY_VECTOR).k(K).build();
    qb.filter(TERM_QUERY);
    assertNull(qb.getFilter());                   // immutability contract
}

9.2 Integration tests

# single IT
./gradlew integTest --tests "org.opensearch.knn.index.FaissIT.testQueryWithFilterFunctionAppliedMultipleShards"

# whole IT class
./gradlew integTest --tests "*FaissIT"

ITs need the native libs built, so run a full ./gradlew build at least once first.

Worth adding — a recall test that actually proves push-down rather than mere application:

// index 10_000 docs; only ~50 have category="rare"
// query with k=10 and filter category=rare
// assert 10 results, NOT ~0
// post-filtering would return close to zero; efficient filtering returns 10

That is the assertion that distinguishes this feature from the naive implementation, and it isn't in the repo.

9.3 Manual REST testing on a live cluster

# 1. index with faiss/hnsw
curl -XPUT "localhost:9200/products" -H 'Content-Type: application/json' -d'
{
  "settings": { "index": { "knn": true, "number_of_shards": 2, "number_of_replicas": 0 } },
  "mappings": { "properties": {
    "embedding": { "type": "knn_vector", "dimension": 3,
                   "method": { "name": "hnsw", "space_type": "l2", "engine": "faiss" } },
    "title":    { "type": "text" },
    "in_stock": { "type": "boolean" }
  } }
}'

# 2. docs
curl -XPOST "localhost:9200/products/_bulk?refresh" -H 'Content-Type: application/x-ndjson' -d'
{"index":{"_id":"1"}}
{"embedding":[1.0,1.0,1.0],"title":"running shoes","in_stock":true}
{"index":{"_id":"2"}}
{"embedding":[1.1,1.0,1.0],"title":"running socks","in_stock":false}
{"index":{"_id":"3"}}
{"embedding":[9.0,9.0,9.0],"title":"hiking boots","in_stock":true}
'

# 3. plain knn with an inline filter — the pre-existing capability
curl -XPOST "localhost:9200/products/_search" -H 'Content-Type: application/json' -d'
{ "query": { "knn": { "embedding": {
      "vector": [1.0,1.0,1.0], "k": 2,
      "filter": { "term": { "in_stock": true } } } } } }'
# expect docs 1 and 3 (doc 2 is nearer than 3 but out of stock)

# 4. search pipeline for hybrid
curl -XPUT "localhost:9200/_search/pipeline/nlp-pipeline" -H 'Content-Type: application/json' -d'
{
  "description": "hybrid normalization",
  "phase_results_processors": [
    { "normalization-processor": {
        "normalization": { "technique": "min_max" },
        "combination":   { "technique": "arithmetic_mean", "parameters": { "weights": [0.3, 0.7] } } } }
  ]
}'

# 5. THE FEATURE — one top-level filter, both sub-queries
curl -XPOST "localhost:9200/products/_search?search_pipeline=nlp-pipeline" \
  -H 'Content-Type: application/json' -d'
{
  "query": {
    "hybrid": {
      "queries": [
        { "match": { "title": "running" } },
        { "knn":   { "embedding": { "vector": [1.0,1.0,1.0], "k": 2 } } }
      ],
      "filter": { "term": { "in_stock": true } }
    }
  }
}'
# doc 2 ("running socks") matches both sub-queries but is out of stock → excluded

9.4 Verifying push-down actually happened

Three ways, in increasing order of reliability:

1. _validate/query?explain — shows the Lucene query the builders produced.

curl -XPOST "localhost:9200/products/_validate/query?explain=true" \
  -H 'Content-Type: application/json' -d'
{ "query": { "hybrid": { "queries": [ { "match": { "title": "running" } } ],
                          "filter": { "term": { "in_stock": true } } } } }'

You should see the match clause wrapped in a boolean with a filter clause — proof the push-down ran at parse time.

2. Profile the query with "profile": true and look for the k-NN query implementation in the breakdown. A KnnFloatVectorQuery/Faiss query carrying a filter looks different from a BooleanQuery containing an unfiltered k-NN clause.

3. The recall test. Index enough documents that a restrictive filter would starve post-filtering, then count results. If you get k back, the filter went in. If you get near zero, it didn't. This is the only test that can't be fooled.

10. Limitations and follow-ups

Still open or unresolved as of the current main:

• Javadoc typos in KNNQueryBuilder.filter() — "Neural Query Builder" and "fiXlter". Flagged in review, promised as a follow-up, never fixed.
• boost/queryName dropped on rebuild, in both filter() and doRewrite().
• No FilterCombinationMode. AND only. OR and IGNORE_IF_EXISTS remain speculative.
• No SearchRequestProcessor variant (RFC high-level Option 2), so you can't set an index-default common filter via a default search pipeline.
• No deep push-down into a sub-query's inner queries.
• No performance/recall benchmark in the repo demonstrating the push-down advantage — the ITs prove correctness, not efficiency.
• Two conflicting conventions for filter() across plugins (mutate vs rebuild), with nothing in the interface javadoc that forces one.
• Duplicated logic between NeuralQueryBuilder and KNNQueryBuilder, accepted deliberately.

11. Reference index

Pull requests and issues

Source files

Docs

• Filtering vector search results
• Efficient k-NN filtering
• Hybrid search
• Efficient filtering in the OpenSearch vector engine (blog)