fix(sitesearch): preserve the custom index alias across a crawl (#36983)

[fabrizzio-dotCMS]

Proposed Changes

Fixes Bug 1 of #36983 (QA-G17) — the custom alias of a Site Search index destroyed by a crawl in the OpenSearch read phases — plus the two defects that surfaced while diagnosing it: alias resolution that goes blind in the phase it does not read from, and a readiness report that could not answer the questions support was actually asking it.

---

1 · Bug 1 — the alias is overwritten by a crawl

Root cause. site_search_job_schedule.jsp resolved index aliases through the content-index router (new ESIndexAPI()), which is not site-search .os-aware. In Phases 2/3 the physical index lives in OpenSearch tagged with .os, so the lookup misses and the index selector falls back to the raw internal index name — which is then saved as the job's indexAlias. From there the crawl does the damage: it deletes the old index (taking the real alias with it) and re-applies the job's stored string as the new index's alias, so a dead index's NAME becomes the alias. That is the sitesearch-ph-3 → sitesearch_20260810160529 swap QA reported. #36797 rerouted the Indices tab and 6 other callers to the phase-aware API but missed this JSP.

• site_search_job_schedule.jsp — resolve aliases via the phase-aware, .os-aware Site Search API, as the Indices tab already does. Dropped the now-unused ESIndexAPI import.
• SiteSearchJobImpl — when the stored indexAlias is actually a raw index name, recover that index's real alias (or none) instead of carrying the raw name forward to the publisher. Server-side guard, so jobs already saved with a raw name are repaired on their next run rather than needing to be recreated by hand.
• site_search.jsp — the scheduler's alias field now accepts up to 255 chars (the engine's name limit) instead of 60. A crawl-built name (sitesearch_<timestamp>_<uuid>) is 62 chars, so the old cap rejected it with "Invalid Index alias" and left the index permanently un-schedulable.

2 · The aggregated alias view — Bug 3, and the downgrade case

Testing a downgrade (3 → 2 → 1) surfaced the structural half of the same defect: listIndices() is a union of both engines in the dual-write phases, while getAliasToIndexMap() resolves against a single engine (the read provider). Any index living only on the other engine appears in the list with a blank alias — which is Bug 3, in both of its reported forms:

• Phase 2 + an index created in Phase 0 (Elasticsearch only) → alias invisible.

• Phase 1 + an index created in Phase 3 (OpenSearch only, after a downgrade) → alias invisible.

• SiteSearchAPI#getAliasToIndexMapAllEngines() — resolves over the same provider set listIndices() uses, with the read provider applied last so it wins a mirror desync and the management view never contradicts what a search would hit. Single-provider phases (0 and 3) do not consult the idle engine.

• The single-engine getAliasToIndexMap() stays as is — searching must resolve an alias against the engine that will serve the query. The split is documented so the two do not get merged later.

• Switched to the new view: the Indices tab, the crawl index selector, the Search tab selector (so Bug 2's symptom — raw index IDs in the dropdown — is gone too) and SiteSearchJobImpl, where an invisible alias made the crawl treat an existing index as new and drop its alias.

3 · Making the readiness endpoint answer support's questions

The endpoint is the source of truth support consults during a migration. Three gaps, each found by using it against a live stack:

• Site Search rows now carry the alias, per engine. An operator knows these indices by alias, never by sitesearch_<timestamp>_<uuid>. Per engine on purpose: an index can hold its alias on one side and not the other, and that asymmetry is what must be visible before promoting a phase. An alias that is itself shaped like an index name gets a NOTE — the fingerprint of the defect fixed above, which cannot be repaired retroactively, so this is the only way to find the indices that still need theirs restored.

• Content counts come from a count query, not _stats docs.count. That counter is per-shard and only advances on shard refresh, so it trailed a just-written document by seconds: a technician checking whether a publish reached OpenSearch read the previous number and concluded the write was lost. A source of truth must never report a number the engine can already contradict. getIndicesStats() stays, but only to decide existence — one call per engine over the whole set, so an absent index is never confused with an unreachable engine. Both halves of the report now count the same way.

• New: content coverage against the DATABASE (expectedDocCount, esCoveragePercent, osCoveragePercent). Every completeness signal until now compared one engine against the other, which stops being an answer exactly where it matters most: in Phase 3 there is no second engine, so a mirror that was never rebuilt reads unremarkably. The case that prompted this reads 3.06 instead of looking normal.

  The denominator is O(1): pg_class.reltuples × 1 − null_frac of live_inode from pg_stats — two catalog lookups, no table access. An exact COUNT is a sequential scan (verified with EXPLAIN: no index-only path counts non-null live_inode without walking the table), which on a customer-sized table would mean a multi-second query on every refresh. The estimate measured 689/682 against an exact 686/685 — well inside what the metric claims, since it exists to tell 3% from 97%, never 99% from 100%. A never-analyzed table (reltuples = -1) is reported as unknown, not as an empty index.

  Coverage never changes the verdict: the verdict states the ES↔OS relationship, coverage states completeness against the database. Two facts, reported side by side.

Checklist

• Tests
• Translations
• Security Implications Contemplated (none — alias resolution and reporting only; the readiness endpoint keeps its existing @Hidden + admin-plus-migration-role gate and is absent from the OpenAPI schema, so no regeneration applies; the new SQL reads only PostgreSQL catalog views with no user input)

Additional Info

Tests — 58/58 green

• SiteSearchJobAliasResolutionTest (new, unit) — 4 cases pinning the alias-resolution rules: real alias carried through, raw index name → real alias, raw index name with no alias → no alias, unknown name → new-index alias.
• SiteSearchRouterReconciliationTest (existing) — 4 new cases on the aggregated view: it includes the engine the phase does not read from, the read provider wins a conflicting alias, Phase 0 never consults OpenSearch, and the single-engine method stays on the read provider.
• SiteSearchMirrorReconcilerTest (new, unit) — 6 cases: alias per engine, alias missing on one engine only, no alias at all, index-name-shaped alias flagged without changing the verdict, a legitimate sitesearch-prefixed alias NOT flagged, one alias lookup per engine.
• ContentIndexMirrorReconcilerTest (existing) — 5 new cases: the count comes from the query and not from stats (the stats entries carry a poison count, so a regression fails loudly), a failed count reported as -1, coverage against the database, coverage absent without a denominator, and a complete mirror not flagged.
• MigrationReadinessServiceTest, MigrationReadinessResourceTest, OSSiteSearchAliasMapTest, ESIndexHelperTest — unchanged and green.
• SiteSearchJobImplTest (existing IT, MainSuite1a) — new end-to-end case: after a full crawl the custom alias must follow onto the new index, and the replaced index's name must never become an alias.

Validation run

• dotcms-core install: BUILD SUCCESS · dotcms-integration test-compile: BUILD SUCCESS
• Verified live against a Phase-1 stack: dual-write confirmed document by document on both engines, and the endpoint's numbers reconciled against _count on each cluster.
• The new IT was not executed locally (needs the integration stack); it runs in CI via MainSuite1a.

Docs (OPENSEARCH_MIGRATION.md)

• The two alias views and when to use each; a phase change never builds counterparts retroactively.
• How to read the readiness report: the access gate (CMS admin and the OS_MIGRATION_INDEX_VISIBILITY_ROLE_KEY role, default os_migration_qa — 403 otherwise), the order to read the fields in, a per-field table, and what a count cannot tell you (re-publishing is an update, and a dual-write mirror only receives what changes from that point on).
• Two worked examples: the downgrade case, and activating a pre-migration backup content index — whose OpenSearch counterpart is never built, is silent in Phase 1, masked by the read fallback in Phase 2 and customer-visible as lost content in Phase 3, and which only a full reindex repairs.
• The Site Search crawl inherits the content index. Found during this work: a Phase-3 crawl produced an index with 14 documents instead of ~443, because the bundlers build their corpus from conAPI.searchIndex — a phase-routed search over the content index, which held 21 of 685 live documents on OpenSearch. The crawl reports success, the bundlers swallow search failures at debug level, and the damage outlives its cause: reindexing afterwards does not repair the Site Search index already built from the empty one. States the ordering rule this implies — in Phases 2/3, full content reindex first, Site Search crawl second.

Scope
Bug 1 in full. Bugs 2 and 3 turned out to be the same root cause seen from other screens, and the aggregated alias view fixes their symptoms as well. They are left open in the issue for QA to re-verify against a build rather than closed from here.

Follow-up not included here
A guard on ESSiteSearchPublisher.switchIndex: it currently activates the new index and deletes the old one unconditionally, so a truncated crawl destroys the good copy — with no ES copy to recover from in Phase 3. With the coverage metric now available, that guard has a defensible criterion instead of an invented threshold. Worth its own PR.

🤖 Generated with Claude Code

This PR fixes: #36983

[claude]

Claude finished @fabrizzio-dotCMS's task in 3m 19s —— View job

---

Code Review — PR #37010

• Gather context (diff, changed files)
• Review Java changes (reconcilers, SiteSearchJobImpl, SiteSearchAPI)
• Review the flagged Semgrep injection finding
• Review JSP changes
• Post review

---

New Issues

• 🟠 High: dotCMS/src/main/webapp/html/portlet/ext/sitesearch/site_search_job_schedule.jsp:79 — The new line calls Logger.warn(this.getClass(), …), but this JSP does not import com.dotmarketing.util.Logger, and neither init.jsp (its only include) nor auth_backend_check.jsp imports it. The two sibling JSPs touched in the same PR (site_search_index_stats.jsp:12, test_site_search.jsp:9) both add/carry the explicit <%@page import="com.dotmarketing.util.Logger"%>; this one was missed. JSP translation happens on first request, so this is a translation-time compile error that will break the Schedule Site Search Job dialog — the very screen this PR is fixing. Add the import. Fix this →

Resolved / Not an issue

• ✅ Semgrep CUSTOM_INJECTION-2 on ContentIndexMirrorReconciler.java:247 — False positive. DATABASE_COUNTS_SQL is a fully literal text block with no interpolation, no parameters, and nothing caller-supplied; it's passed verbatim to DotConnect().setSQL(...). No user input reaches it. Safe to triage as /fp in Semgrep.

---

Notes (non-blocking)

• The alias-recovery logic in SiteSearchJobImpl.getIndexMetaData (aliasOf reverse lookup over getAliasToIndexMapAllEngines()) and the read-provider-wins merge in SiteSearchAPIImpl.getAliasToIndexMapAllEngines() both look correct and are well covered by the new unit tests. MigrationPhase.current().isReadEnabled() correctly maps to "OpenSearch serves reads" (phases 2/3), so incompleteContentIndexWarning checks the right engine's percentage.
• The engine-independent DB coverage metric, -1/null handling, and the "advisory, never a gate" warning path all fail safe (failures swallowed, degrade to out-of-sync rather than false-green). Good.
• SiteSearchAPI.java shows as a full-file rewrite (+266/-208) but the only semantic additions are getAliasToIndexMapAllEngines() (default) and defaultIndexName(); the rest is line-ending/whitespace churn. Harmless, but it inflates the diff and makes future git blame noisier.

Once the missing Logger import is added, this is good to merge.

· branch issue-36983-sitesearch-alias-phase-aware

[semgrep-dotcms]

Semgrep found 6 CUSTOM_INJECTION-2 findings:

• dotCMS/src/main/java/com/dotcms/content/index/migration/ContentIndexMirrorReconciler.java
  • L247-253 - Triage
  • L247-252 - Triage
  • L247-251 - Triage
  • L247-250 - Triage
  • L247-249 - Triage
  • L247-248 - Triage

The method identified is susceptible to injection. The input should be validated and properly
escaped.

If this is a critical or high severity finding, please also link this issue in the #security channel in Slack.