[Bug][Jira] Kanban board sub-filter silently excludes issues from collection

[jbsmith7741]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

When a Jira kanban board is configured as a DevLake scope, resolved and completed issues can silently go missing from the collected data. Specifically:

• Issue counts in DevLake are significantly lower than what Jira reports for the same board filter
• DORA and cycle time metrics are underreported for kanban teams
• Resolved issues disappear from DevLake after being assigned a released fixVersion in Jira
• No errors or warnings are produced — syncs complete successfully with incomplete data

The root cause is that Jira kanban boards support a sub-filter query (subQuery) that is separate from the board's saved filter JQL. This sub-filter is configured in the board's settings and is applied server-side by Jira's Agile API before returning results. A common sub-filter is:

fixVersion in unreleasedVersions() OR fixVersion is EMPTY

DevLake collects board issues via GET /rest/agile/1.0/board/{boardId}/issue (issue_collector.go). Jira applies both the saved filter JQL and the sub-filter server-side, so DevLake receives only the filtered subset with no way to detect that issues were excluded.

The board configuration endpoint (GET /rest/agile/1.0/board/{boardId}/configuration) returns the sub-filter in a subQuery field. DevLake already calls this endpoint in board_filter_begin_collector.go to fetch the saved filter, but the subQuery field is not parsed into the BoardConfiguration struct — it is silently dropped.

Jira board config
├── filter.id → saved filter JQL (fetched and stored in _tool_jira_boards.jql)
└── subQuery  → sub-filter query (not parsed, not stored)

Collection: GET agile/1.0/board/{boardId}/issue
  → Jira applies saved filter JQL + subQuery server-side
  → DevLake receives only the filtered subset
  → _raw_jira_api_issues / _tool_jira_board_issues

This affects only kanban boards. Scrum boards are not affected because they do not use the sub-filter mechanism in the same way.

What do you expect to happen

DevLake should collect all issues matching the board's saved filter JQL, regardless of any kanban sub-filter. At minimum, DevLake should parse and store the subQuery field so users can see when a sub-filter is active and understand why issue counts may differ from Jira.

How to reproduce

1. Create or identify a Jira kanban board that has a sub-filter configured (Board Settings > General > Sub-filter Query), such as fixVersion in unreleasedVersions() OR fixVersion is EMPTY
2. Ensure the board's saved filter JQL matches issues that have a released fixVersion
3. Configure the board as a DevLake scope and run a full sync
4. Compare the issue count in _tool_jira_board_issues for that board against the count returned by running the board's saved filter JQL directly via the Jira search API (/rest/api/2/search?jql=...)
5. The DevLake count will be lower — issues with a released fixVersion are missing

You can confirm the sub-filter is present by checking the board configuration API:

GET /rest/agile/1.0/board/{boardId}/configuration

The response will include a subQuery field if a sub-filter is configured.

Anything else

Related issues:

• [Bug][Jira] issues disappearing from dataset, progressive load issue? #7826 — "Issues disappearing from dataset" — reports the same symptoms (issues silently vanishing from DevLake while still present in Jira under the board filter). The kanban sub-filter may be an undiscovered root cause for at least some of those reports.
• [Question][Jira Connector] Is there a way to allow Apache DevLake to connect on Jira Projects instead of a Jira Boards ? #8196 — "Is there a way to allow DevLake to connect on Jira Projects instead of Jira Boards?" — requesting project-based scoping, which would also bypass this problem since the search API does not apply board-level sub-filters.

Potential solutions:

Option A: Use the search API instead of the board API. Replace the board issue endpoint with the Jira search API (api/2/search or api/3/search/jql), using the board's saved filter JQL directly. The saved filter JQL is already fetched and stored in _tool_jira_boards.jql by board_filter_begin_collector.go. The epic collector (epic_collector.go) already uses this search-based approach as a reference pattern.

Option B: Parse the sub-filter and allow override. Read the subQuery field from the board configuration response (already fetched but not parsed), store it on _tool_jira_boards (new column), and add a scope config flag (e.g., ignore_board_sub_filter) that switches to the search API when enabled. This preserves backward compatibility while giving users control.

Version

v1.0.3-beta12@1d21f19

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct