Update multimodal docs with new models

[hengtaoguo]

Description

• Update multimodal.md to include new models supported over the past few months.
• Add Qwen3-Omni announce in the qwen_moe doc, including E2E test scripts.

Tests

Build docs for readthedocs:

# Build:
cd docs && sphinx-build -b html . _build/html

# Serve:
python -m http.server -d _build/html

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[Rohan-Bierneni]

LGTM! Can we also add a link in

maxtext/tests/end_to_end/tpu/qwen/moe/run_qwen_moe.md

Line 4 in be7748b

Qwen3 is a family of open-source large language models from the Qwen team at Alibaba. This documentation covers the integration of the following Qwen Mixture-of-Experts (MoE) models into MaxText:

that has a link to this multimodal doc for qwen3.5 section

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Rohan-Bierneni]

Thank you for the change I have left some comments

[Rohan-Bierneni]

The overall script are formatted very good split between part 1 and 2. Mainly we need some changes to script 2 to make it compatible with our xlml tests.

We also need another pr in xlml once this is merged to add the tests in our DAG.

Thank you for making these changes!

[Rohan-Bierneni]

Can you also add the command to convert to scanned checkpoint below.

If scanned checkpoitn will not be used/is not supported in the checkpoint util then no worries we can skip it

[hengtaoguo]

Due to the deepstack features in the vision encoder, Omni doesn't have a full scanned support. We can skip it and use unscanned for now.

[Rohan-Bierneni]

Since this will be used by xlml for tests, the default value should output to gs://runner-maxtext-logs/$(date +%Y-%m-%d-%H-%M).

Similar to how it is set here:

maxtext/tests/end_to_end/tpu/qwen/moe/qwen3.5-35b-a3b/2_test_qwen3.5_35b_a3b.sh

Line 22 in 3190805

if [ -z "${BASE_OUTPUT_PATH}" ]; then

[hengtaoguo]

Good call, done.

[Rohan-Bierneni]

XLML will pull an already converted checkpoint from gs://maxtext-model-checkpoints.

Can you upload a converted checkpoint to this gcs bucket, and set the value here to point to that checkpoint.

[hengtaoguo]

Uploaded a new checkpoint to gs://maxtext-model-checkpoints/qwen3-omni-30b-a3b/unscanned/0/items.

[Rohan-Bierneni]

Should be able to remove this logic if using hardcoded gcs bucket gs://runner-maxtext-logs

[hengtaoguo]

Removed this redundant block.

[Rohan-Bierneni]

Before the decode tests, do you think we should have forward_pass_logit_check test as well. This would compare to hf golden logits and would need to be stored in specific gcs bucket.

Ex:

maxtext/tests/end_to_end/tpu/qwen/moe/qwen3.5-35b-a3b/2_test_qwen3.5_35b_a3b.sh

Line 43 in 3190805

GOLDEN_LOGITS_DISK_LOCATION="/deps/tests/assets/golden_logits/golden_data_${MODEL_NAME}.jsonl"

[hengtaoguo]

That would be great but vision branch may undergo some precision issue, as we translated image processing steps from Torch to JAX, and some interpolation operations cannot closely match. So now we mostly look at whether the output are making sense. I will do that as a follow up.

[Rohan-Bierneni]

The way we point to base.yml file currently in toher scripts is like this:

${MAXTEXT_CONFIGS_DIR:-${MAXTEXT_REPO_ROOT:-$PWD}/src/maxtext/configs}/base.yml

Can we change it to this for other instances in this script for consistency.

[hengtaoguo]

Thanks for the catch, all base.yml paths have been updated.