Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #487 +/- ##
==========================================
+ Coverage 61.40% 61.72% +0.32%
==========================================
Files 77 77
Lines 6868 6937 +69
Branches 676 690 +14
==========================================
+ Hits 4217 4282 +65
Misses 2535 2535
- Partials 116 120 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR adds an ord_text field to one-shot extraction schemas and threads it through the one-shot parsing and finalization pipeline so the verbatim source sentence can be emitted into final CSV outputs, with a regression test ensuring the field round-trips.
Changes:
- Add required
ord_textto one-shot schemas (geothermal electricity, GHP, water rights demo) with instructions to keep it verbatim. - Include
ord_textin one-shot parser DataFrame construction and in finalized output column sets. - Extend unit test coverage to assert
ord_textis written to both quantitative and qualitative output CSVs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
compass/utilities/finalize.py |
Adds ord_text to parsed/output column definitions and qualitative output column selection. |
compass/plugin/one_shot/components.py |
Ensures one-shot parsing output includes ord_text when present in schema outputs. |
tests/python/unit/utilities/test_utilities_finalize.py |
Regression test for ord_text round-tripping into both output CSVs. |
examples/water_rights_demo/one-shot/water_rights_schema.json5 |
Adds required ord_text field, updates examples/instructions for verbatim extraction. |
compass/extraction/ghp/geothermal_heat_pump_schema.json5 |
Adds required ord_text field plus guidance/examples. |
compass/extraction/geothermal_electricity/geothermal_schema.json |
Adds required ord_text field plus guidance/examples. |
| "Extract only enacted district requirements, not proposed language or general background text.", | ||
| "Use direct excerpts/quotes in summary whenever possible.", | ||
| "Whenever a feature row is emitted, ord_text must be the first full sentence from the source document corresponding to the requirement being extracted, copied verbatim: for quantitative features the full source sentence containing the extracted value, and for qualitative features the first full source sentence relating to the requirement. Unlike summary, ord_text must be a single contiguous sentence reproduced exactly as written, with no paraphrasing, normalization, ellipses, added context, or commentary. If a feature has no requirement, omit the feature row entirely rather than emitting a row with an empty ord_text.", | ||
| "If a feature has no requirement, set value, units, section, and summary to null or omit the feature row.", |
There was a problem hiding this comment.
This might be worth adjusting?
ppinchuk
left a comment
ppinchuk
left a comment
There was a problem hiding this comment.
This is really nitpicky, but can we use ord_first_sentence instead of ord_text by chance? In my experience, a more explicit key name actually helps the model, and I am hping this would distinguish it enough from the summary output
| "units": "HH:MM (24-hour)", | ||
| "section": "Section 6.7 - Drilling Operations", | ||
| "summary": "'Routine geothermal drilling activities may occur only between 7:00 a.m. and 7:00 p.m., Monday through Saturday, and shall not occur on Sundays or legal holidays except in an emergency.'", | ||
| "ord_text": "Routine geothermal drilling activities may occur only between 7:00 a.m. and 7:00 p.m., Monday through Saturday, and shall not occur on Sundays or legal holidays except in an emergency.", |
There was a problem hiding this comment.
Can we add one or two examples where the sentence for the ordinance text extraction is completely different than the summary?
There was a problem hiding this comment.
We should remove the summary - and use value for both qualitative and quantitative features (like how we ask the interns to do)
Adds a verbatim
ord_textfield to the one-shot extraction schemas (geothermal electricity, GHP, water rights demo), capturing the source sentence an ordinance value/requirement was extracted from, unparaphrased.Threads
ord_textthroughfinalize.pyso it routes into both the qualitative and quantitative output CSVs, and through theSchemaOrdinanceParsercolumn list. Includes a regression test confirming the value round-trips into both output files.