Zero-downtime reindex in OpenSearch

"Reindex OpenSearch with zero downtime."

What it is

A reindex rebuilds an index with a different mapping, analyser, shard count or document shape. It is necessary because most of an index's structure is immutable after creation: you cannot change a field's type, change an analyser, or change the primary shard count in place. The only path is to build a new index and move to it.

"Zero downtime" means readers and writers never see an error or an empty result set during the switch. The mechanism is an alias: applications read and write through a name that points at an index, and the switch is an atomic repointing of that name.

Commonly confused with a rolling restart or a mapping update. Adding a new field to a mapping is an in-place update and needs none of this. Changing an existing field's type, or the analyser applied to it, or the shard count, needs all of it.

The problem it solves

The failure mode without aliases: applications hardcode products-v1, so switching means a coordinated deploy of every reader and writer at the same moment as the index switch. That is a distributed transaction across your fleet with no rollback, performed under time pressure.

With an alias the switch is one API call, atomic, and reversible in one API call. The whole discipline is: never let an application name an index directly. If your applications currently do, fixing that is step zero and it is worth doing before you need a reindex, not during.

Mechanics

The alias indirection

// Applications only ever see "products". Which concrete index that means
// is an operational detail they never learn.
POST /_aliases
{
  "actions": [
    { "add": { "index": "products-v1", "alias": "products",       "is_write_index": true } },
    { "add": { "index": "products-v1", "alias": "products-read" } }
  ]
}

Separate read and write aliases are worth the small extra complexity, because during the migration you will briefly want writes going to two places while reads stay on one.

The procedure

Step 1: create the new index with the target mapping.

PUT /products-v2
{
  "settings": {
    "number_of_shards": 12,          // sized from projected data: 10-50 GB per shard
    "number_of_replicas": 0,         // ZERO during the bulk load; restore after
    "refresh_interval": "-1",        // no refresh during bulk; restore after
    "index.translog.durability": "async"   // temporary; restore after
  },
  "mappings": { "properties": { "title": { "type": "text", "analyzer": "french" } } }
}

Those three settings are the difference between a reindex that takes two hours and one that takes ten. Replicas and refresh both multiply indexing work, and neither is needed while nobody is reading the index. Restoring them afterwards is a step people forget, and an index left at zero replicas is a single node failure away from data loss.

Step 2: reindex, sliced and throttled.

POST /_reindex?wait_for_completion=false&slices=auto&requests_per_second=5000
{
  "source": { "index": "products-v1", "size": 5000 },
  "dest":   { "index": "products-v2" },
  "script": { "source": "ctx._source.price_minor = (int)(ctx._source.price * 100)" }
}

wait_for_completion=false returns a task id immediately, because a large reindex outlives any HTTP client. slices=auto parallelises across source shards, which is usually a large speedup. requests_per_second is the throttle, and it is the setting that keeps the reindex from destroying the latency of the live cluster.

// Monitor and re-throttle live, without restarting.
GET  /_tasks/{task_id}
POST /_reindex/{task_id}/_rethrottle?requests_per_second=2000

Step 3: handle writes that arrive during the reindex. This is the part that makes it genuinely zero-downtime and the part naive answers skip. Three options:

Approach	How	When
Dual-write	Application writes to both aliases during the migration	Best when you control the writers; simple and explicit
Delta reindex	After the bulk pass, reindex again with a range query on `updated_at` since the bulk started; repeat until the delta is tiny	Requires a reliable modification timestamp
Replay from the source of truth	The search index is a projection; replay the change stream from the database	Cleanest if you already have CDC or an outbox

Delta reindex, concretely:

POST /_reindex
{
  "source": {
    "index": "products-v1",
    "query": { "range": { "updated_at": { "gte": "2026-08-03T09:00:00Z" } } }
  },
  "dest": { "index": "products-v2", "version_type": "external" }
}

version_type: external is doing important work: it makes the copy idempotent by document version, so a delta pass cannot overwrite a newer document with an older one. Without it, repeated delta passes can move data backwards.

Step 4: restore settings and warm.

PUT /products-v2/_settings
{ "number_of_replicas": 1, "refresh_interval": "1s",
  "index.translog.durability": "request" }
POST /products-v2/_forcemerge?max_num_segments=1   // read-heavy indices only

Then wait for green, and warm the caches by replaying a sample of production queries against the new index. Switching to a cold index produces a latency spike that looks like the reindex broke something.

Step 5: verify before switching, not after.

Document count matches (allowing for deletes during the window)
Sample of production queries returns comparable results on both indices
Aggregations agree
Spot-check documents whose mapping changed
Latency on the new index is acceptable under replayed load

Verification is where you catch the analyser you got wrong, and it is much cheaper before the switch than after.

Step 6: the atomic swap.

// Both actions in ONE request. The alias is never absent, never ambiguous.
POST /_aliases
{
  "actions": [
    { "remove": { "index": "products-v1", "alias": "products" } },
    { "add":    { "index": "products-v2", "alias": "products", "is_write_index": true } }
  ]
}

Step 7: keep the old index. Rollback is the same call with the actions reversed, and it takes milliseconds. Delete products-v1 after a bake period measured in days, not minutes. This is the cheapest insurance in the whole procedure and the step most often skipped under deadline pressure.

Shard sizing, because the reindex is when you fix it

A reindex is the only convenient opportunity to change primary shard count, so get it right:

Target 10-50 GB per shard.
600 GB of index data / 30 GB per shard = 20 primary shards.
Add replicas for redundancy and read throughput, not for capacity.

Over-sharding is the most common cluster killer: each shard is a Lucene index
with its own memory, file handles and merge threads, and cluster state grows
with shard count. A thousand tiny shards is slower than fifty right-sized ones.

And the heap rule that goes with it: JVM heap at 50 percent of RAM and below roughly 32 GB, so compressed object pointers remain available. The other half of RAM is the OS page cache, which is what Lucene actually reads through.

A worked example

A product search index. 40 million documents, 600 GB, 3,000 queries per second at peak. Required change: the description field must use a French analyser instead of the default, because a third of the catalogue is French and stemming is wrong for it.

An analyser change requires a reindex; there is no in-place option.

Plan and timings:

Day 1  Create products-v2: 20 shards (600 GB / 30 GB), 0 replicas,
       refresh -1. Confirm the mapping on a 1,000-document sample first,
       because getting the analyser wrong and discovering it after a
       six-hour reindex is the expensive mistake.

Day 1  Bulk reindex, slices=auto, throttled to 4,000 docs/sec so live
       query p99 stays under its SLO. 40M docs / 4,000 = ~2.8 hours.
       Watch cluster CPU and search latency; re-throttle if p99 moves.

Day 1  Application starts dual-writing to products-write-v2 (a second
       alias) at the moment the bulk starts, so nothing is missed.

Day 2  Restore replicas to 1, refresh to 1s. Wait for green.
       Force-merge is skipped: this index takes continuous writes, so
       merging to one segment would be undone immediately.

Day 2  Verify: counts match within the expected delete delta; run 500
       recorded production queries against both indices and diff the
       top-10; confirm French queries now stem correctly (the point of
       the exercise) and English ones are unchanged (the regression risk).

Day 2  Replay production query load against v2 to warm caches and confirm
       p99. THEN swap the alias.

Day 5  Delete products-v1 after three days of bake.

What the verification caught in this shape of migration, and it is the reason step 5 exists: applying a French analyser to a mixed-language field improves French queries and degrades English ones, because French stemming mangles English words. The fix is language detection at index time with per-language subfields (description.fr, description.en) and a query that searches both. Discovering that before the swap costs a day; discovering it after costs a rollback and a public regression.

The other thing to say out loud: the dual-write window means the application is temporarily writing twice, so its write latency rises and a failure to one index must not fail the request. Write to the new index asynchronously and reconcile with the delta pass; a dual-write that hard-fails the user request has made the migration riskier than the thing it was avoiding.

Production evidence

Elasticsearch and OpenSearch both document the alias-swap pattern as the supported way to change a mapping, and both provide _reindex with slicing, throttling, live re-throttling and the task API precisely because large reindexes are routine operations rather than exceptional ones.

Index Lifecycle Management (and OpenSearch's Index State Management) build on the same alias indirection for time-series data: rollover creates a new backing index and repoints the write alias atomically, which is the same mechanism applied continuously rather than once.

Data streams are the modern packaging of that pattern for append-only time-series data, and they exist because alias management by hand was error-prone enough to warrant a first-class abstraction.

The 10 to 50 GB per shard guidance and the 32 GB heap ceiling (for compressed oops) are both in the vendors' own sizing documentation, and over-sharding is explicitly called out there as the most common cause of cluster instability.

The debate

The alternative is rebuild from the source of truth rather than reindexing from the old index. The search index is a projection of a database, so you can drop it and replay.

The case for it: it fixes data quality problems that a reindex faithfully copies. If the old index has documents that were never updated after a bug, reindexing preserves the bug. It also exercises the rebuild path, which you want to know works, because it is your recovery procedure.

The case against: it is much slower (you are re-reading a database and re-running enrichment rather than copying already-processed documents), and it puts load on the primary datastore, which the reindex path does not.

My position: reindex from the old index when the change is structural (mapping, analyser, shard count) and the data is known good, because it is faster and does not touch the database. Rebuild from source when data quality is in question, when the document shape changes enough that a script becomes unreadable, or when you want to rehearse the recovery path. And measure rebuild time either way, because it is your recovery time objective for the search tier.

The full ceremony is the wrong answer for a small index where a few minutes of degraded search is acceptable, in which case reindex and swap without dual-writes and accept the gap; and for an append-only time-series index, where rollover to a new backing index with the new mapping means new data gets the new shape and old data ages out on its own, with no reindex at all. That second case is worth naming unprompted, because it is the cheapest answer when it applies.

Follow-up Q&A

"Reindex OpenSearch with zero downtime." Applications read and write through an alias, never a concrete index name. Create the new index with the target mapping, zero replicas and refresh disabled for the bulk load. Reindex with slices=auto and a requests_per_second throttle so live query latency holds. Handle concurrent writes with dual-writes or repeated delta passes filtered on updated_at with version_type: external so they are idempotent. Restore replicas and refresh, verify counts and sample queries against both indices, warm the caches, then swap the alias atomically in a single _aliases call. Keep the old index for days so rollback is one API call.

"Why zero replicas and refresh disabled during the bulk load?" Both multiply indexing work. Every replica re-indexes every document, so one replica roughly doubles the cost; refresh creates a new searchable segment on a timer, and during a bulk load into an index nobody is reading, that is pure overhead plus merge pressure later. Neither is needed until the index goes live. The important discipline is restoring them before the swap, because an index left at zero replicas is one node failure from data loss.

"How do you handle writes that arrive during the reindex?" Three options. Dual-write from the application to both indices, which is explicit and needs writer changes. Delta reindex: after the bulk pass, run again with a range query on updated_at since the bulk started, repeating until the delta is negligible, using version_type: external so a delta pass cannot overwrite a newer document with an older one. Or replay from the source of truth, which is cleanest if you already have CDC or an outbox, since the index is a projection anyway.

"What do you verify before swapping?" Document counts, allowing for deletes during the window. A sample of recorded production queries run against both indices with the top results diffed, which is where an analyser mistake shows up. Aggregation results. Spot checks on documents whose mapping changed. And latency under replayed load, because a cold index gives you a spike that looks like the reindex broke something. Verification before the swap costs a day; after the swap it costs a rollback and a visible regression.

"When would you not reindex at all?" When the index is append-only time-series data: roll over to a new backing index with the new mapping, and let old data age out under lifecycle management. New documents get the new shape, no reindex is needed, and the only cost is that queries spanning the boundary see two mappings for a while. Also when the change is additive, since adding a field to a mapping is an in-place update. Reindexing is only forced by changing something immutable: a field type, an analyser, or the primary shard count.

Common misconceptions

The most common is that you can change a mapping in place. You can add fields; changing a field's type or analyser is immutable and forces a reindex.

The second is that the alias swap is the risky moment. It is one atomic API call and it is instantly reversible. The risky moments are the mapping you got wrong and the writes you missed, both of which happen well before the swap.

The third is that _forcemerge is a generally good idea. On a read-only index it helps; on an index still taking writes it is undone immediately and you have spent significant I/O for nothing.

Interview delivery note

Say this: "The whole thing rests on applications reading and writing through an alias rather than a concrete index name, so the switch is one atomic API call and the rollback is the same call reversed. Create the new index with the target mapping, zero replicas and refresh off for the bulk load, reindex sliced and throttled so live latency holds, and handle concurrent writes with either dual-writes or repeated delta passes filtered on updated_at with external versioning so they're idempotent. Restore the settings, verify against both indices, warm the caches, then swap."

The depth signal is what you verify and when: "I'd run 500 recorded production queries against both indices and diff the top results before swapping, because that's where an analyser mistake shows up, and finding it after the swap costs a rollback and a public regression." And the unprompted alternative: "and if this were append-only time-series data I wouldn't reindex at all, I'd roll over to a new backing index and let the old data age out."

The Staff & Team Lead Interview Reference