Track problems and self-identify gotchas - #83
Open
Logan-Huang wants to merge 2 commits into
Open
Conversation
Adds a fourth shared convention to skills/astrodb-directions.md: a root-level gotchas.md problem log, sibling to workflow.md. workflow.md records why a decision was made about *this dataset*. gotchas.md records what tripped the skill up and what should change in the skill — friction caused by the skills themselves, addressed to this repo's maintainers rather than to a future reader of the dataset. The convention defines what counts as significant (wrong or incomplete instructions, a deviation from the written steps, a dependency behaving differently than documented, a step that reported success while producing a wrong result), what to leave out, and a self-classification each entry carries: `gotcha` when another user or dataset would hit it again, `one-off` when it was specific to this run. Entries are formatted so they can be pasted into a GitHub issue as-is, and require a concrete "Suggested change". When a run produces gotchas, the skill names them in its completion message and points the user at the astrodb-bot issue tracker. Skills never open an issue or PR themselves and never invoke gh — reporting is the user's call. All 9 SKILL.md files get the Step 0 pointer update and one Completion Checklist item, so the step has a contract rather than relying on the Step 0 read alone. Closes astrodbtoolkit#76 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GTPUPhEeWFAfqpVpQNJQo4
kelle
reviewed
Jul 31, 2026
| # AstroDB Gotchas | ||
|
|
||
| Problems hit while running the AstroDB skills, and suggested fixes to the skills themselves. | ||
| Entries marked **gotcha** are worth reporting at https://github.com/astrodbtoolkit/astrodb-bot. |
kelle
requested changes
Jul 31, 2026
kelle
left a comment
kelle
left a comment
Contributor
There was a problem hiding this comment.
Make it so that the instructions in each skill are more generic and won't need to be updated if we change astrodb-directions.
|
|
||
| 1. Read `references/astrodb-directions.md` — it defines the `workflow.md`, artifact-folder, and | ||
| completion-checklist conventions this skill follows. | ||
| 1. Read `references/astrodb-directions.md` — it defines the `workflow.md`, artifact-folder, |
Contributor
There was a problem hiding this comment.
Suggested change
| 1. Read `references/astrodb-directions.md` — it defines the `workflow.md`, artifact-folder, | |
| 1. Read `references/astrodb-directions.md` — it defines the workflow that you should use. |
Contributor
|
Also, use the skill-creator skill to modify some evals. |
…s evals Review feedback from kelle on PR astrodbtoolkit#83: - Each skill's Step 0 now just says astrodb-directions.md "defines the workflow that you should use" instead of enumerating which conventions it covers, so the skills don't need editing when that file changes. - The per-skill gotchas checklist item points at the problem-log convention in astrodb-directions.md rather than restating the whole protocol. - The gotchas.md header template links straight to /issues/new instead of the repo root. - Added gotchas coverage to the parse-table, ingest-publications, and website evals: dataset quirks belong in workflow.md, conditions the skills already document aren't gotchas, clean runs log nothing, real deviations carry a concrete Suggested change, and the skill never runs gh. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AERgwTzT5azbQw4RfqMLyu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a shared convention to skills/astrodb-directions.md: a gotchas.md problem log, sibling to workflow.md.
gotchas.md records what tripped the skill up and what should change in the skill.
Not sure if this should be implemented in a different way?
When a run produces gotchas, the skill names them in its completion message and points the user at the astrodb-bot issue tracker. Skills never open an issue or PR themselves and never invoke gh — reporting is the user's call.
All 9 SKILL.md files get the Step 0 pointer update and one Completion Checklist item, so the step has a contract rather than relying on the Step 0 read alone.
Closes #76