Clarify Automatic Migration trouble shooting steps when no integrations shown for translated rules#7123
Conversation
logeekal
left a comment
There was a problem hiding this comment.
Great initiative angela to add these to the docs. It looks good but i have some small comments.
| - **Semantic search returning no results or wrong results** — Jina produces dense vectors; ELSER produces sparse vectors; they are incompatible vector spaces. | ||
| - **`function_score` queries with `min_score` returning zero hits** — scores computed against mismatched embeddings are meaningless and rarely cross any threshold. | ||
|
|
||
| The root cause is that when the index was created, the ELSER inference endpoint was not available, so Elasticsearch fell back to whichever inference endpoint was configured. The field name `elser_embedding` is just a label; what matters is the `inference_id` baked into the index mapping at creation time. |
There was a problem hiding this comment.
Can we show here the example of the wrong inference model ID and query to check it.
There was a problem hiding this comment.
added a reference to see the inference model ID.
https://github.com/elastic/docs-content/pull/7123/changes#diff-f411b451c4783970ac6e6c8440f549f60551e166db8420ea644a3b63f8a27fcaR79
| PUT _inference/sparse_embedding/elser-2-elasticsearch | ||
| { | ||
| "service": "elser", | ||
| "service_settings": { "num_allocations": 1, "num_threads": 1 } | ||
| } | ||
| ``` |
There was a problem hiding this comment.
Are we sure about this? Should we direct user to go to Trained Models -> Deploy Elser?
There was a problem hiding this comment.
Added a link to
https://www.elastic.co/docs/explore-analyze/machine-learning/nlp/ml-nlp-elser#download-deploy-elser
The request is coming from the page as well.
…and steps for ELSER inference endpoint configuration.
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
🔍 Preview links for changed docs |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
…clarifying the API request instructions.
…ic migration documentation
… to use ELSER as the default inference modal (#275766) (#276269) # Backport This will backport the following commits from `main` to `9.4`: - [[Automatic Migration] Set automatic migration integration index to use ELSER as the default inference modal (#275766)](#275766) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Angela Chuang","email":"6295984+angorayc@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-07-03T14:50:08Z","message":"[Automatic Migration] Set automatic migration integration index to use ELSER as the default inference modal (#275766)\n\n## Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations can be created with\nelser_embedding bound to a non-ELSER inference endpoint, causing\ntranslated rules to show no recommended integrations.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials -\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","backport:version","Team:Automatic Migrations","v9.5.0","v9.4.2","v9.4.3","v9.4.4"],"title":"[Automatic Migration] Set automatic migration integration index to use ELSER as the default inference modal","number":275766,"url":"https://github.com/elastic/kibana/pull/275766","mergeCommit":{"message":"[Automatic Migration] Set automatic migration integration index to use ELSER as the default inference modal (#275766)\n\n## Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations can be created with\nelser_embedding bound to a non-ELSER inference endpoint, causing\ntranslated rules to show no recommended integrations.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials -\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5"}},"sourceBranch":"main","suggestedTargetBranches":["9.4"],"targetPullRequestStates":[{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/275766","number":275766,"mergeCommit":{"message":"[Automatic Migration] Set automatic migration integration index to use ELSER as the default inference modal (#275766)\n\n## Summary\n\nhttps://github.com//issues/275757\n\nkibana-siem-rule-migrations-integrations can be created with\nelser_embedding bound to a non-ELSER inference endpoint, causing\ntranslated rules to show no recommended integrations.\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n\n- [x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials -\nhttps://github.com/elastic/docs-content/pull/7123/commits/1712990dae4bb3f6df9d6ad3878fa061a808afaf\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios","sha":"4355d7a332bd8a2748eb5faaa332c57b74eeb4b5"}},{"branch":"9.4","label":"v9.4.2","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Angela Chuang <6295984+angorayc@users.noreply.github.com>
| "_source": { | ||
| "excludes": ["elser_embedding"] | ||
| } | ||
| }, | ||
| "dest": { | ||
| "index": "kibana-siem-rule-migrations-integrations-backup" | ||
| }, | ||
| "script": { | ||
| "source": """ | ||
| def embedding = ctx._source.remove('elser_embedding'); | ||
| if (embedding != null) { | ||
| if (embedding instanceof Map && embedding.containsKey('text')) { | ||
| ctx._source['elser_embedding'] = embedding['text']; | ||
| } else if (embedding instanceof String) { | ||
| ctx._source['elser_embedding'] = embedding; | ||
| } | ||
| } | ||
| """, | ||
| "lang": "painless" |
There was a problem hiding this comment.
I don't this is correct. I think elster_embedding should be re-indexed as-is. embeddings are created in a separate field called _inference_fields.
Alternatively, we can simply create an empty index. Integration index is populated automatically when a migration is started.
|
|
||
| - ELSER ML model must be available on the cluster. | ||
| - You must have index admin privileges. | ||
| - Expect downtime or degraded search on this index during the reindex process. |
There was a problem hiding this comment.
Make sure any automatic migration is not running. If it is, it should be stopped.
This PR tightens wording in the Automatic Migration FAQ and closes a troubleshooting gap where translated rules show no integrations despite correct index mapping. It also makes the troubleshooting flow more actionable by directing users to the correct Kibana UI location before API checks.
FAQ wording cleanup
solutions/security/get-started/automatic-migration.mdto focus on the actionable symptom:Troubleshooting flow: operator guidance
solutions/security/get-started/automatic-migration-troubleshooting.mdStep 1, added explicit guidance on where to run APIs:Troubleshooting flow: post-Step-13 model verification