[RHDHBUGS- 3302] using Markitdown for AI notebooks

[JslYoon]

Hey, I just made a Pull Request!

For AI notebooks, we are using markitdown to convert documetns into markdown before vectorizing for performance improvement.

✔️ Checklist

https://redhat.atlassian.net/browse/RHDHBUGS-3302

• A changeset describing the change and affected packages. (more info)
• Added or Updated documentation
• Tests for new functionality and regression tests for bug fixes
• Screenshots attached (for UI changes)

[rhdh-gh-app]

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Unexpected Changesets

The following changeset(s) reference packages that have not been changed in this PR:

• /home/runner/work/rhdh-plugins/rhdh-plugins/workspaces/intelligent-assistant/.changeset/fuzzy-peaches-shake.md: @red-hat-developer-hub/backstage-plugin-lightspeed-backend

Note that only changes that affect the published package require changesets, for example changes to tests and storybook stories do not require changesets.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@red-hat-developer-hub/backstage-plugin-intelligent-assistant-backend	workspaces/intelligent-assistant/plugins/intelligent-assistant-backend	minor	v3.1.0

[rhdh-qodo-merge]

PR Summary by Qodo

AI Notebooks: convert uploads to Markdown with markitdown before vectorizing

✨ Enhancement ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Convert uploaded notebook documents to Markdown via markitdown before vectorization.
• Upload converted content as .md (text/markdown) to the Files API.
• Add markitdown-ts dependency and changesets for notebook-related releases.

Diagram

graph TD
  C[Client] --> R["Notebooks router"] --> M["Markdown conversion"] --> F[("Files API")] --> U["Vector store upsert"] --> V[("Vector store")]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Pass-through for .md/.txt (convert only rich formats)

• ➕ Avoids unnecessary conversion work/latency for already-text inputs
• ➕ Reduces risk of conversion altering plain text semantics
• ➖ Slightly more branching logic; must define what is considered safe pass-through

2. Reuse existing parseFileContent pipeline for size/type enforcement

• ➕ Keeps a single, consistent validation path (size limits, parsing errors)
• ➕ Leverages existing tests around file parsing
• ➖ May duplicate work if markitdown becomes the canonical converter
• ➖ Requires deciding how parseFileContent and markitdown interact for PDFs

3. Move markdown conversion to the background job

• ➕ Faster HTTP response and less chance of request timeouts for large PDFs
• ➕ Makes the request handler simpler and more resilient
• ➖ Requires persisting the original upload (or buffering) before conversion
• ➖ More complex failure/status reporting (conversion failures become async)

Recommendation: The PR’s approach (normalize to Markdown before vectorization) is a solid direction for retrieval quality/performance. Consider extending plaintext passthrough to .md/.txt (and possibly other known-text types) and/or reusing existing file-size validation to avoid accidental regression in upload constraints; moving conversion async is optional depending on observed request latency for PDFs.

Files changed (7) +618 / -21

Enhancement (3) +75 / -17

documentService.tsUpload Markdown files to Files API (text/markdown) +9/-11

Upload Markdown files to Files API (text/markdown)

• Changes document uploads to always send Markdown content as a .md file with MIME type text/markdown. Removes the previous toFile-based upload wrapper and simplifies the file payload structure.

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/documentService.ts

markitdownClient.tsAdd MarkItDown client wrapper for buffer → Markdown conversion +53/-0

Add MarkItDown client wrapper for buffer → Markdown conversion

• Adds convertToMarkdown(buffer, originalName) using markitdown-ts, with plaintext passthrough for json/yaml/yml/log. Throws an InputError when conversion produces no Markdown output.

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/markitdownClient.ts

notebooksRouters.tsConvert uploaded document to Markdown before vector-store upsert +13/-6

Convert uploaded document to Markdown before vector-store upsert

• Replaces the previous parseFileContent flow with a markitdown-based conversion step, then uploads the Markdown to the Files API. Adds an explicit guard for missing req.file and updates wording to reflect attaching a file to the vector store in the background.

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/notebooksRouters.ts

Other (4) +543 / -4

fuzzy-peaches-shake.mdAdd changeset for notebooks 404 passthrough bugfix release +5/-0

Add changeset for notebooks 404 passthrough bugfix release

• Adds a changeset entry to publish a minor release for the lightspeed backend plugin, documenting a notebooks route 404 passthrough fix.

workspaces/intelligent-assistant/.changeset/fuzzy-peaches-shake.md

six-chairs-turn.mdAdd changeset documenting Markdown cleanup before vectorization +5/-0

Add changeset documenting Markdown cleanup before vectorization

• Adds a changeset entry to publish a minor release for the intelligent-assistant backend plugin, documenting the switch to Markdown cleanup before vectorization.

workspaces/intelligent-assistant/.changeset/six-chairs-turn.md

package.jsonAdd markitdown-ts dependency for document-to-Markdown conversion +1/-0

Add markitdown-ts dependency for document-to-Markdown conversion

• Introduces markitdown-ts as a runtime dependency to support converting uploaded documents into Markdown prior to ingestion.

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/package.json

yarn.lockLockfile updates for markitdown-ts transitive dependencies +532/-4

Lockfile updates for markitdown-ts transitive dependencies

• Updates the workspace lockfile to include markitdown-ts and its transitive dependency graph (including document parsing/conversion libraries).

workspaces/intelligent-assistant/yarn.lock

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Context used

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

✅ Compliance rules (platform): 11 rules

✅ Cross-repo context

  Explored: repo: redhat-developer/rhdh-operator (sha: 095451d9)

  Explored: repo: redhat-developer/rhdh-local (sha: 2ae9e8c8)

  Explored: repo: redhat-developer/rhdh (sha: 6bcb0141)

  Not relevant to this PR: redhat-developer/rhdh-chart

1. FileType ignored in conversion 🐞 Bug ≡ Correctness

Description

The documents upload route validates req.body.fileType, but convertToMarkdown derives
file_extension only from req.file.originalname, so extensionless or mismatched filenames can be
converted with the wrong/empty type and fail or produce incorrect markdown before vectorization.
This is a regression from the previous parsing flow that used the validated fileType to drive
parsing.

Code

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/notebooksRouters.ts[R408-412]

+      const markdown = await convertToMarkdown(
+        req.file.buffer,
+        req.file.originalname,
      );
+      const fileId = await documentService.uploadFile(markdown, title);

Relevance

⭐⭐⭐ High

Likely regression: team historically emphasizes consistent notebook file-type validation and upload
correctness.

PR-#3104
PR-#2499

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The router validates fileType but does not pass it into conversion; conversion determines
file_extension solely from originalName, while the previous parsing utilities in this codebase
explicitly take fileType to drive parsing behavior.

workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/notebooksRouters.ts[374-428]
workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/markitdownClient.ts[29-45]
workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/documentHelpers.ts[88-103]
workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/fileParser.ts[169-196]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The upload endpoint validates `fileType` but conversion uses `req.file.originalname` to infer the extension, which can be empty or disagree with `fileType`. This can cause markitdown to convert using the wrong file format (or no format) and break otherwise-valid requests.

## Issue Context
Previously, parsing used `fileType` (`parseFileContent(..., fileType, ...)` → `parseFile(..., fileType)`), so behavior was consistent with the validated request field.

## Fix Focus Areas
- workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/notebooksRouters.ts[374-429]
- workspaces/intelligent-assistant/plugins/intelligent-assistant-backend/src/service/notebooks/documents/markitdownClient.ts[29-45]

### Suggested approach
1. Change `convertToMarkdown` signature to accept `fileType` (validated) and use it as the authoritative extension (e.g., `ext = '.' + normalizedFileType`).
2. Optionally compare the filename extension (if present) to `fileType` and reject with `InputError` when they disagree (or fall back to `fileType` when missing).
3. Update the router to call `convertToMarkdown(req.file.buffer, req.file.originalname, fileType)`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[codecov]

Codecov Report

❌ Patch coverage is 79.16667% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.85%. Comparing base (e03b281) to head (559cb4f).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4020      +/-   ##
==========================================
- Coverage   57.87%   57.85%   -0.03%     
==========================================
  Files        2393     2393              
  Lines       95985    95918      -67     
  Branches    26803    26792      -11     
==========================================
- Hits        55551    55492      -59     
+ Misses      38940    38932       -8     
  Partials     1494     1494              

Flag	Coverage Δ		*Carryforward flag
adoption-insights	84.54% <ø> (ø)		Carriedforward from e03b281
ai-integrations	69.06% <ø> (ø)		Carriedforward from e03b281
app-defaults	69.79% <ø> (ø)		Carriedforward from e03b281
augment	46.67% <ø> (ø)		Carriedforward from e03b281
boost	76.77% <ø> (ø)		Carriedforward from e03b281
bulk-import	72.55% <ø> (ø)		Carriedforward from e03b281
cost-management	13.55% <ø> (ø)		Carriedforward from e03b281
dcm	60.72% <ø> (ø)		Carriedforward from e03b281
extensions	56.48% <ø> (ø)		Carriedforward from e03b281
global-floating-action-button	71.18% <ø> (ø)		Carriedforward from e03b281
global-header	62.17% <ø> (ø)		Carriedforward from e03b281
homepage	47.46% <ø> (ø)		Carriedforward from e03b281
install-dynamic-plugins	56.77% <ø> (ø)		Carriedforward from e03b281
intelligent-assistant	74.08% <79.16%> (-0.17%)	⬇️
konflux	91.98% <ø> (ø)		Carriedforward from e03b281
lightspeed	69.02% <ø> (ø)		Carriedforward from e03b281
mcp-integrations	83.40% <ø> (ø)		Carriedforward from e03b281
orchestrator	66.79% <ø> (ø)		Carriedforward from e03b281
quickstart	65.04% <ø> (ø)		Carriedforward from e03b281
sandbox	79.56% <ø> (ø)		Carriedforward from e03b281
scorecard	84.94% <ø> (ø)		Carriedforward from e03b281
theme	87.90% <ø> (ø)		Carriedforward from e03b281
translations	5.12% <ø> (ø)		Carriedforward from e03b281
x2a	79.31% <ø> (ø)		Carriedforward from e03b281

*This pull request uses carry forward flags. Click here to find out more.

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e03b281...559cb4f. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[fullsend-ai-review]

Review

Reason: stale-head

The review agent reviewed commit 46460bdfa1af1070f9d71a5f8fd39fab84c43c4b but the PR HEAD is now ee493a299ad474a32f8c12eb6605205a626e38c9. This review was discarded to avoid approving unreviewed code.

[fullsend-ai-review]

/fs-review

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud