Skip to content

Clarify Automatic Migration trouble shooting steps when no integrations shown for translated rules#7123

Open
angorayc with Copilot wants to merge 10 commits into
mainfrom
copilot/add-faq-entry-and-troubleshooting-guide
Open

Clarify Automatic Migration trouble shooting steps when no integrations shown for translated rules#7123
angorayc with Copilot wants to merge 10 commits into
mainfrom
copilot/add-faq-entry-and-troubleshooting-guide

Conversation

Copilot AI commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

This PR tightens wording in the Automatic Migration FAQ and closes a troubleshooting gap where translated rules show no integrations despite correct index mapping. It also makes the troubleshooting flow more actionable by directing users to the correct Kibana UI location before API checks.

  • FAQ wording cleanup

    • Updated the FAQ question in solutions/security/get-started/automatic-migration.md to focus on the actionable symptom:
      • from mixed semantic-search/integration phrasing
      • to “Why are no integrations shown for translated rules?”
  • Troubleshooting flow: operator guidance

    • In solutions/security/get-started/automatic-migration-troubleshooting.md Step 1, added explicit guidance on where to run APIs:
      • Kibana Dev Tools Console link
      • Self-managed path: Management → Dev Tools
      • Elastic Cloud path: Developer tools
  • Troubleshooting flow: post-Step-13 model verification

Copilot AI changed the title Adding FAQ entry and troubleshooting guide for elser_embedding docs(security): add troubleshooting guide for wrong ELSER inference model on elser_embedding field Jun 26, 2026
Copilot AI changed the title docs(security): add troubleshooting guide for wrong ELSER inference model on elser_embedding field Document ELSER inference mismatch causing missing translated-rule integrations Jun 26, 2026
Copilot AI changed the title Document ELSER inference mismatch causing missing translated-rule integrations Clarify Automatic Migration FAQ heading for translated rules Jun 26, 2026
Copilot AI changed the title Clarify Automatic Migration FAQ heading for translated rules Clarify Automatic Migration integrations guidance and add ELSER deployment recovery steps Jun 26, 2026
Copilot AI changed the title Clarify Automatic Migration integrations guidance and add ELSER deployment recovery steps Clarify Automatic Migration FAQ and extend troubleshooting with Dev Tools + ELSER deployment checks Jun 26, 2026
@angorayc angorayc marked this pull request as ready for review June 26, 2026 16:51
@angorayc angorayc requested review from a team as code owners June 26, 2026 16:51
@angorayc angorayc changed the title Clarify Automatic Migration FAQ and extend troubleshooting with Dev Tools + ELSER deployment checks Clarify Automatic Migration trouble shooting steps when no integrations shown for translated rules Jun 26, 2026
@angorayc angorayc requested a review from logeekal June 29, 2026 08:11

@logeekal logeekal left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great initiative angela to add these to the docs. It looks good but i have some small comments.

Comment thread solutions/security/get-started/automatic-migration-troubleshooting.md Outdated
Comment thread solutions/security/get-started/automatic-migration-troubleshooting.md Outdated
- **Semantic search returning no results or wrong results** — Jina produces dense vectors; ELSER produces sparse vectors; they are incompatible vector spaces.
- **`function_score` queries with `min_score` returning zero hits** — scores computed against mismatched embeddings are meaningless and rarely cross any threshold.

The root cause is that when the index was created, the ELSER inference endpoint was not available, so Elasticsearch fell back to whichever inference endpoint was configured. The field name `elser_embedding` is just a label; what matters is the `inference_id` baked into the index mapping at creation time.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we show here the example of the wrong inference model ID and query to check it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +67 to +72
PUT _inference/sparse_embedding/elser-2-elasticsearch
{
"service": "elser",
"service_settings": { "num_allocations": 1, "num_threads": 1 }
}
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure about this? Should we direct user to go to Trained Models -> Deploy Elser?

@angorayc angorayc Jun 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@florent-leborgne florent-leborgne removed the request for review from a team June 30, 2026 09:40
…and steps for ELSER inference endpoint configuration.
@github-actions

Copy link
Copy Markdown
Contributor

Elastic Docs AI PR menu

Check the box to run an AI review for this pull request.

  • Review docs changes (docs-review). Status: not started.

Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team.

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@angorayc angorayc requested a review from a team as a code owner July 1, 2026 13:03
kibanamachine added a commit to elastic/kibana that referenced this pull request Jul 3, 2026
… to use ELSER as the default inference modal (#275766) (#276269)

# Backport

This will backport the following commits from `main` to `9.4`:
- [[Automatic Migration] Set automatic migration integration index to
use ELSER as the default inference modal
(#275766)](#275766)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Angela
Chuang","email":"6295984+angorayc@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-07-03T14:50:08Z","message":"[Automatic
Migration] Set automatic migration integration index to use ELSER as the
default inference modal (#275766)\n\n##
Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations
can be created with\nelser_embedding bound to a non-ELSER inference
endpoint, causing\ntranslated rules to show no recommended
integrations.\n\n### Checklist\n\nCheck the PR satisfies following
conditions. \n\nReviewers should verify this PR satisfies this list as
well.\n\n\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials
-\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","backport:version","Team:Automatic
Migrations","v9.5.0","v9.4.2","v9.4.3","v9.4.4"],"title":"[Automatic
Migration] Set automatic migration integration index to use ELSER as the
default inference
modal","number":275766,"url":"https://github.com/elastic/kibana/pull/275766","mergeCommit":{"message":"[Automatic
Migration] Set automatic migration integration index to use ELSER as the
default inference modal (#275766)\n\n##
Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations
can be created with\nelser_embedding bound to a non-ELSER inference
endpoint, causing\ntranslated rules to show no recommended
integrations.\n\n### Checklist\n\nCheck the PR satisfies following
conditions. \n\nReviewers should verify this PR satisfies this list as
well.\n\n\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials
-\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5"}},"sourceBranch":"main","suggestedTargetBranches":["9.4"],"targetPullRequestStates":[{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/275766","number":275766,"mergeCommit":{"message":"[Automatic
Migration] Set automatic migration integration index to use ELSER as the
default inference modal (#275766)\n\n##
Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations
can be created with\nelser_embedding bound to a non-ELSER inference
endpoint, causing\ntranslated rules to show no recommended
integrations.\n\n### Checklist\n\nCheck the PR satisfies following
conditions. \n\nReviewers should verify this PR satisfies this list as
well.\n\n\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials
-\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n-
[x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5"}},{"branch":"9.4","label":"v9.4.2","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Angela Chuang <6295984+angorayc@users.noreply.github.com>
Comment on lines +153 to +171
"_source": {
"excludes": ["elser_embedding"]
}
},
"dest": {
"index": "kibana-siem-rule-migrations-integrations-backup"
},
"script": {
"source": """
def embedding = ctx._source.remove('elser_embedding');
if (embedding != null) {
if (embedding instanceof Map && embedding.containsKey('text')) {
ctx._source['elser_embedding'] = embedding['text'];
} else if (embedding instanceof String) {
ctx._source['elser_embedding'] = embedding;
}
}
""",
"lang": "painless"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't this is correct. I think elster_embedding should be re-indexed as-is. embeddings are created in a separate field called _inference_fields.

Alternatively, we can simply create an empty index. Integration index is populated automatically when a migration is started.


- ELSER ML model must be available on the cluster.
- You must have index admin privileges.
- Expect downtime or degraded search on this index during the reindex process.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure any automatic migration is not running. If it is, it should be stopped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants