Skip to content

(Openinference Migration: Langchain): Capture multimodal image content (OpenAI image_url and Anthropic image blocks) as Blob/Uri message parts.#296

Draft
rads-1996 wants to merge 5 commits into
open-telemetry:mainfrom
rads-1996:langchain-multimodal-images
Draft

(Openinference Migration: Langchain): Capture multimodal image content (OpenAI image_url and Anthropic image blocks) as Blob/Uri message parts.#296
rads-1996 wants to merge 5 commits into
open-telemetry:mainfrom
rads-1996:langchain-multimodal-images

Conversation

@rads-1996

@rads-1996 rads-1996 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Description

Part of the langchain migration PRs - #272

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. List any relevant details for your test
configuration.

  • uv run tox -e py312-test-instrumentation-genai-langchain -- -q
  • uv run tox -e py312-test-instrumentation-genai-langchain-conformance -- -q
  • uv run --python 3.12 tox -e lint-instrumentation-genai-langchain

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

  • Followed the style guidelines of this project
  • Changelog updated if the change requires an entry
  • Unit tests added
  • Documentation updated

@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Jul 21, 2026

Copy link
Copy Markdown

Pull request dashboard status

Status last refreshed: 2026-07-23 21:42:30 UTC.

  • Waiting on: Author
  • Next step: Move out of draft to request review.

This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected.

@rads-1996
rads-1996 marked this pull request as ready for review July 21, 2026 21:18
@rads-1996
rads-1996 requested a review from a team as a code owner July 21, 2026 21:18
Copilot AI review requested due to automatic review settings July 21, 2026 21:18
@rads-1996
rads-1996 marked this pull request as draft July 21, 2026 21:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds multimodal image support to the LangChain GenAI instrumentation by converting OpenAI image_url and Anthropic image blocks into Blob / Uri message parts, and extends the unit test suite to validate the new parsing behavior.

Changes:

  • Add multimodal image parsing helpers (_media_part, _image_from_url) to convert LangChain image blocks into Blob/Uri parts.
  • Extend callback-handler tests to cover OpenAI and Anthropic image content shapes (including data URIs and base64 sources).
  • Add a changelog fragment documenting the new capability.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py Adds image block parsing and base64 decoding to emit Blob/Uri message parts.
instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py Adds unit tests for data-URI, HTTP-URI, and Anthropic image/source parsing into message parts.
instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/296.added Documents the new multimodal image capture support.

@rads-1996
rads-1996 force-pushed the langchain-multimodal-images branch 2 times, most recently from 4dab00f to aa26c4f Compare July 22, 2026 15:15
@rads-1996
rads-1996 force-pushed the langchain-multimodal-images branch from aa26c4f to ed3949d Compare July 23, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants