Skip to content

[docs] Fix full-text index option defaults and search query examples #9062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JingsongLi merged 1 commit into from
Aug 7, 2026
+13 −11
Merged

[docs] Fix full-text index option defaults and search query examples #9062

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/flink/procedures.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1164,7 +1164,7 @@ All available procedures are listed below.
CALL sys.full_text_search(<br/>
`table` => 'default.articles',<br/>
`column` => 'content',<br/>
query => '{"match":{"column":"content","terms":"paimon lake"}}',<br/>
query => '{"match":{"query":"paimon lake"}}',<br/>
top_k => 10,<br/>
projection => 'id,content,__paimon_search_score')
</td>
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/primary-key-table/global-index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ SELECT id, content, __paimon_search_score
FROM full_text_search(
'items',
'content',
'{"match":{"column":"content","terms":"paimon lake"}}',
'{"match":{"query":"paimon lake"}}',
10
)
ORDER BY __paimon_search_score DESC;
Expand All @@ -431,7 +431,7 @@ relevance score is required. `top_k` must be between 1 and 10,000.
CALL sys.full_text_search(
`table` => 'default.items',
`column` => 'content',
query => '{"match":{"column":"content","terms":"paimon lake"}}',
query => '{"match":{"query":"paimon lake"}}',
top_k => 10,
projection => 'id,content,__paimon_search_score'
);
Expand Down Expand Up @@ -480,7 +480,7 @@ FROM hybrid_search(
'options', map())),
array(named_struct(
'column', 'content',
'query', '{"match":{"column":"content","terms":"paimon lake"}}',
'query', '{"match":{"query":"paimon lake"}}',
'limit', 20,
'weight', 1.0f,
'options', map())),
Expand Down
16 changes: 9 additions & 7 deletions paimon-full-text/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ scores through its configured `rrf`, `weighted_score`, or `mrr` ranker.
CALL sys.full_text_search(
`table` => 'default.articles',
`column` => 'content',
query => '{"match":{"column":"content","terms":"paimon lake"}}',
query => '{"match":{"query":"paimon lake"}}',
top_k => 10,
projection => 'id,content,__paimon_search_score'
);
Expand Down Expand Up @@ -178,16 +178,18 @@ Available tokenizer options:
| Option | Default | Description |
|--------|---------|-------------|
| `full-text.tokenizer` | `default` | Tokenizer used by the full-text index. Supported values: `default`, `simple`, `whitespace`, `raw`, `ngram`, `jieba`. |
| `full-text.ngram.min-gram` | `2` | Minimum gram length for the `ngram` tokenizer. |
| `full-text.ngram.max-gram` | `2` | Maximum gram length for the `ngram` tokenizer. |
| `full-text.ngram.min-gram` | `3` | Minimum gram length for the `ngram` tokenizer. |
| `full-text.ngram.max-gram` | `3` | Maximum gram length for the `ngram` tokenizer. |
| `full-text.ngram.prefix-only` | `false` | Whether the `ngram` tokenizer only emits prefix ngrams. |
| `full-text.jieba.search-mode` | `true` | Whether the `jieba` tokenizer uses search mode. |
| `full-text.jieba.ordinal-position` | `true` | Whether the `jieba` tokenizer uses ordinal positions. |
| `full-text.lower-case` | `true` | Whether configurable tokenizers lowercase emitted tokens. |
| `full-text.max-token-length` | `40` | Maximum token length kept by configurable tokenizers. |
| `full-text.ascii-folding` | `false` | Whether to normalize non-ASCII Latin characters to ASCII. |
| `full-text.stem` | `false` | Whether to apply stemming to emitted tokens. |
| `full-text.ascii-folding` | `true` | Whether to normalize non-ASCII Latin characters to ASCII. |
| `full-text.stem` | `true` | Whether to apply stemming to emitted tokens. |
| `full-text.language` | `english` | Language used by stemming and built-in stop word filters. |
| `full-text.remove-stop-words` | `false` | Whether to remove built-in stop words for the configured language. |
| `full-text.stop-words` | ` ` | Semicolon-separated custom stop words to remove. |
| `full-text.remove-stop-words` | `true` | Whether to remove built-in stop words for the configured language. |
| `full-text.stop-words` | empty | Semicolon-separated custom stop words to remove. Requires `full-text.remove-stop-words=true`. |
| `full-text.with-position` | `true` | Whether to store term positions for phrase queries. |

Tokenizer settings are persisted in each global index file's metadata. Readers use that metadata
Expand Down
Loading