Feat/Log Compaction

[Amazing-Stardom]

Log Compaction Cron Job

Currently the review_events table is growing rapidly over time.

Problem

After analysis, 95% of the rows in review_events are verbose debug logs—LLM streaming chunks, divider lines, and hunk debug prints—that are not needed for long-term storage. These were causing the table to reach 4.89M rows / 1.57 GB.

Solution

• A cron-based background job that automatically deletes old verbose logs beyond a configurable retention period (default: 30 days).
• Before deleting, a compaction summary marker is written so we retain how many events a review originally had.
• Error logs, warnings, and important milestone messages are always preserved — only noise is removed.
• A simple UI for super admins to configure the schedule, retention period, and trigger a manual run.

[RijulTP]

Implement Event Log Compaction

Overview

This change adds an automated system for event log compaction. It addresses database size growth by summarizing and deleting old review_events. A Go Cron manager performs scheduled cleanup based on configurable settings.

Technical Highlights

• EventCompactionManager: Implements a Go cron-based service for scheduled log cleanup.
• docs/Review/Event-compression.md: Documents the two-step SQL process for summary marking and batched deletion.
• internal/api/compaction_settings.go: Defines API endpoints for configuration and manual triggers.
• internal/api/review_events_endpoints.go: Modifies review event API to return original event counts using compaction markers.
• storage/core/compaction_lock_store.go: Adds PostgreSQL advisory lock for distributed leader election.
• ui/src/pages/Settings/CompactionSettingsTab.tsx: Introduces a new UI for managing compaction settings.

Impact

• Functionality: Users can configure automated event log compaction via a new UI. Review event displays now show accurate total counts for compacted reviews.
• Risk: Incorrect retention settings could lead to unintended data deletion. Large-scale compaction runs may impact database performance.

[LinceMathew]

This time frame is for retaining what data?

[Amazing-Stardom]

For event_type = log.

For example, chunking logs, which is not important in the long term storage.