Skip to content

Add automatic dictionary learning and audio recovery#597

Open
altic-dev wants to merge 22 commits into
mainfrom
b/automatic-custom-dictionary
Open

Add automatic dictionary learning and audio recovery#597
altic-dev wants to merge 22 commits into
mainfrom
b/automatic-custom-dictionary

Conversation

@altic-dev

@altic-dev altic-dev commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Description

Adds automatic dictionary learning from corrected dictation, a guided Train by Voice flow with pronunciation matching, and clearer live model-download progress. Also hardens Faster Recording Start by fixing split-channel Core Audio handling and automatically switching to the compatibility audio path when malformed capture is detected.

Type of Change

  • 🐞 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 🧹 Chore
  • 📝 Documentation update

Related Issue or Discussion

Addresses #272, #545, #572, and #576.
Related to #454 and #592; those broader issues are not fully resolved by this PR.

Testing

  • Tested on Intel Mac
  • Tested on Apple Silicon Mac
  • Tested on macOS version: macOS 27.0
  • Ran linter locally: swiftlint --strict --config .swiftlint.yml Sources
  • Ran formatter locally: swiftformat --config .swiftformat Sources
  • Ran tests locally: 14 focused integration tests
  • Built, installed, and launched with sh build_with_FI_incremental.sh

Screenshots / Video

image
  • No UI/visual changes; screenshots/video are not applicable.

Notes

Malformed direct capture falls back within the active recording. Three consecutive detections disable Faster Recording Start; a healthy direct recording resets the failure streak. Intel runtime was not tested.

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

The PR Policy check is blocking this PR because required template information is missing.

Please update the PR description with:

  • Related Issue or Discussion

Visual files detected:

  • .github/screenshots/automatic-dictionary-1.6.3.jpg
  • .github/screenshots/faster-recording-start-1.6.3.jpg
  • Sources/Fluid/ContentView.swift
  • Sources/Fluid/Persistence/SettingsStore.swift
  • Sources/Fluid/Services/MenuBarManager.swift
  • Sources/Fluid/UI/CustomDictionaryView.swift
  • Sources/Fluid/UI/SettingsView.swift
  • Sources/Fluid/Views/AutomaticDictionaryCorrectionOverlay.swift

Screenshots or video are required for UI, UX, settings, onboarding, overlay, menu bar, or visual behavior changes. If this PR has no visual changes, check the no-visual-change box in the template.

If this remains incomplete for 48 hours after opening, the PR may be closed.

@github-actions github-actions Bot added needs PR template Pull request is missing required template content. needs screenshots Pull request needs screenshot or video evidence. labels Jul 13, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53994e657d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Networking/ModelDownloader.swift Outdated
Comment thread Package.resolved Outdated
@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds three major features: automatic dictionary learning from corrected dictation (with a guided Train by Voice flow and pronunciation matching), clearer live model-download progress, and hardened Faster Recording Start via per-buffer CoreAudio stride calculation and runtime fallback to AVAudioEngine on malformed captures.

  • Automatic dictionary learning (AutomaticDictionaryCorrectionTracker, AutomaticDictionaryTrainingSession, AutomaticDictionarySuggestionPolicy) observes AX text-field changes after each insertion, detects user corrections, and offers a non-activating overlay to add dictionary entries with optional voice-profile training; a cooldown/dismissal policy governs when suggestions re-appear.
  • CoreAudio split-channel fix (CoreAudioCaptureSupport.c) computes the per-buffer stride from mNumberChannels × bytesPerSample instead of the ASBD-wide mBytesPerFrame, preventing the 3× speed-up seen during route changes; three consecutive mismatches disable the experimental backend.
  • Download progress is refactored into ProgressiveFileDownloader, which uses the URLSession delegate API to deliver continuous byte-progress updates; PronunciationDictionaryStore now accumulates voice enrollments across training sessions rather than replacing them.

Confidence Score: 5/5

Safe to merge; the changes are well-scoped, previously-flagged issues have been addressed, and the new code is defensively written throughout.

The CoreAudio stride fix is load-bearing but protected by the existing fv_frame_count zero-return guard and the UINT_MAX sentinel. ProgressiveFileDownloader correctly uses the session delegate-callback parameter for finishTasksAndInvalidate. PronunciationDictionaryStore.upsert now accumulates enrollments rather than replacing them, and FluidAudioProvider.dictionaryLabels uses uniquingKeysWith preventing the crash-on-duplicate-ID path. CustomDictionaryView.addTrainedReplacement correctly looks up the entry in the already-merged self.entries before calling saveEntries. No new correctness issues were found.

No files require special attention; the new AutomaticDictionaryCorrectionTracker and ProgressiveFileDownloader are the most complex additions and are the most worth re-reading on a second pass.

Reviews (11): Last reviewed commit: "fix(dictionary): dismiss suggestions glo..." | Re-trigger Greptile

Comment thread Sources/Fluid/Services/ASRService.swift
Comment thread Sources/Fluid/Networking/ModelDownloader.swift Outdated
Comment thread Sources/Fluid/Services/PronunciationDictionaryStore.swift

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 899af853ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Services/AutomaticDictionaryCorrectionTracker.swift
Comment thread Sources/Fluid/Services/FluidAudioProvider.swift Outdated
Comment thread Sources/Fluid/UI/CustomDictionaryView.swift Outdated
@github-actions github-actions Bot removed needs PR template Pull request is missing required template content. needs screenshots Pull request needs screenshot or video evidence. labels Jul 13, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eee96fc8fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Services/ASRService.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 534439a4d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Services/ASRService.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 364d2c54b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/UI/CustomDictionaryView.swift Outdated
Comment thread Sources/Fluid/Persistence/BackupService.swift
Comment thread Sources/Fluid/Services/PronunciationDictionaryStore.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ccc683a1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Services/MenuBarManager.swift
Comment thread Sources/Fluid/UI/CustomDictionaryView.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d990dd2372

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Services/ASRService.swift
Comment thread Sources/Fluid/ContentView.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d331f8ee4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/Persistence/BackupService.swift Outdated
Comment thread Sources/Fluid/Services/ASRService.swift
@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown

Want your agent to iterate on Greptile's feedback? Try greploops.

Comment thread Sources/Fluid/Services/FluidAudioProvider.swift Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3bb6929068

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Sources/Fluid/ContentView.swift
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant