Skip to content

feat: LLM post-processing for transcriptions - #113

Open
charlesdubt wants to merge 1 commit into
karansinghgit:mainfrom
charlesdubt:feat/llm-post-processing
Open

feat: LLM post-processing for transcriptions#113
charlesdubt wants to merge 1 commit into
karansinghgit:mainfrom
charlesdubt:feat/llm-post-processing

Conversation

@charlesdubt

Copy link
Copy Markdown

Summary

  • Adds an optional LLM post-processing step that runs after transcription to fix typos, remove filler words, correct grammar, and reformat text
  • Uses any OpenAI-compatible API — defaults to Ollama (localhost:11434) for fully local, offline processing with no API key
  • New settings section with toggle, endpoint/model config, and a user-editable system prompt
  • Graceful fallback: if the LLM call fails, the original transcription is used unchanged

How it works

  1. User dictates as normal → Whisper/Parakeet transcribes
  2. If post-processing is enabled, the transcription is sent to the configured LLM with the user's system prompt
  3. The LLM-cleaned text is pasted into the active app

Setup (for local use with Ollama)

  1. Install Ollama from ollama.com
  2. Run ollama pull qwen2.5:0.5b (290 MB, fast)
  3. Enable post-processing in Settings → LLM Post-Processing

Also works with OpenAI, LM Studio, or any OpenAI-compatible provider by changing the base URL and adding an API key.

Test plan

  • Toggle post-processing on/off — dictation works in both states
  • With Ollama running locally, dictation produces cleaned-up text
  • With Ollama stopped, dictation falls back to raw transcription gracefully
  • Custom system prompt changes are persisted and applied
  • API key field works for cloud providers (OpenAI, etc.)
  • Settings UI renders correctly in light and dark mode

🤖 Generated with Claude Code

Add an optional post-processing step that sends transcribed text through
an LLM to fix typos, remove filler words, correct grammar, and reformat.
Uses any OpenAI-compatible API, defaulting to Ollama (localhost) for
fully local processing with no API key required.

- New PostProcessingService with configurable prompt, model, base URL,
  and optional API key (stored in Keychain)
- Settings UI section under "LLM Post-Processing" with toggle, endpoint
  config, model name, and editable system prompt
- Integrates into the transcription pipeline after Whisper/Parakeet
  output and before clipboard paste
- Graceful fallback: if the LLM call fails, original text is used
- Removes unused whisperkit-cli product dependency from build

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant