-
Notifications
You must be signed in to change notification settings - Fork 2
update documentation for regi-python #19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
adamkorynta
wants to merge
4
commits into
main
Choose a base branch
from
feature/jpype-documentation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+426
−42
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # AGENTS.md | ||
|
|
||
| This file provides repository-specific guidance for AI coding agents working in regi-python. | ||
|
|
||
| ## Repository Overview | ||
|
|
||
| `regi-headless` now uses a **Python-first execution model**. | ||
|
|
||
| - `regi-headless/src/main/python/regi_python` is the runtime bridge. It starts the JVM with **JPype** and exposes `regi_session()` and `run_headless(calculation_callback)`. | ||
| - `district-scripts/` and the example scripts under `regi-headless/src/test/resources/...` are now Python entrypoints that call into Java through the bridge. | ||
|
|
||
| Treat this repo as a Python orchestration layer over REGI Java libraries. | ||
|
|
||
| ## Current Architecture | ||
|
|
||
| - `regi-headless/src/main/python/regi_python/regi_python.py` is the current Python entrypoint. | ||
| - `regi_session()` owns JVM startup/shutdown. | ||
| - `run_headless()` creates the REGI domain and invokes a Python callback with a Java-backed registry. | ||
| - `regi-headless/src/main/java/...` contains the Java support layer that the Python bridge calls through JPype. | ||
| - `district-scripts/` contains active district scripts written calling Java-backed calculations inside the callback. | ||
| - `regi-headless/src/test/resources/usace/rowcps/headless/examples` contains the same pattern for example scripts used by tests. | ||
|
|
||
| ## File Groups To Treat Differently | ||
|
|
||
| - `regi-headless/src/main/python/` | ||
| - python to java bridge code | ||
| - prefer modern Python style and small, testable helpers | ||
| - `district-scripts/` | ||
| - active district-owned operational scripts | ||
| - preserve behavior unless the task explicitly changes calculation results or API usage | ||
| - `regi-headless/src/test/resources/usace/rowcps/headless/examples/` | ||
| - example scripts used by tests | ||
| - keep them aligned with the district-script pattern | ||
| - `docs/` | ||
| - useful for historical context and documentation of features moving forward | ||
| - `docs/agent-guides/` is the shared source of truth for agent-facing workflow guidance | ||
|
|
||
| ## Working Rules | ||
|
|
||
| - Keep edits scoped to the requested migration target. | ||
| - When a script needs a new Java method, update the Java public API and the test harness together; the script test suite validates against the Java source-defined scriptable API. | ||
| - Avoid broad refactors outside the requested script family. | ||
|
|
||
| ## Verification | ||
|
|
||
| Common checks in this repo: | ||
|
|
||
| - `./gradlew buildPythonWheel` | ||
| - `./gradlew testPythonWheel` | ||
|
|
||
| The Python package is built from `regi-headless/src/main/python`, and the wheel test verifies the package metadata, importability, public API exposure, and bundled Java jars. | ||
|
|
||
| ## Environment Notes | ||
|
|
||
| The Python bridge expects the Java environment to be available, and the repo documentation currently references these variables: | ||
|
|
||
| - `JAVA_HOME` | ||
| - `CDA_URL` | ||
| - `CDA_API_KEY` | ||
| - `OFFICE_ID` | ||
| - `REGI_LOG_LEVEL` | ||
|
|
||
| Use the repo's current bridge code and tests as the source of truth for behavior; use the docs for orientation, not as a strict contract. `regi-headless/src/test/python/test_district_scripts.py` is the main contract for migrated script shape and allowed API calls. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,75 +1,105 @@ | ||
| # REGI-Headless | ||
| # regi-python | ||
|
|
||
| > [!IMPORTANT] | ||
| > **Notice: Project Refactor in Progress** | ||
| > This project is undergoing a large refactor for [CWMS Data API](https://github.com/USACE/cwms-data-api) support. | ||
| > It will transition from a Java project consuming Jython scripts to a **Python project** that | ||
| > utilizes **JPype** to call underlying REGI Java libraries. | ||
| `regi-python` is the Python bridge over the REGI Java libraries. | ||
|
|
||
| `REGI-Headless` is a Java-based command-line tool and library designed to run | ||
| **REGI** calculations in a headless environment. | ||
| It allows users to execute complex hydrological calculations and manage gate settings via Jython | ||
| scripts without the need for a graphical interface. | ||
| The client-facing Python API is documented in [docs/PYTHON_API.md](docs/PYTHON_API.md). | ||
| For script migration guidance, see [docs/JYTHON_TO_JPYPE_MIGRATION.md](docs/JYTHON_TO_JPYPE_MIGRATION.md). | ||
|
|
||
| ## Features | ||
| ## What Lives Where | ||
|
|
||
| - **Headless Execution**: Run REGI calculations as part of automated workflows or on servers. | ||
| - **Database Integration**: Connects to CWMS data retrieval and storage. | ||
| - **Modular Calculations**: Includes support for: | ||
| - Inflow calculations (Clone, Compute, Auto-Adjust, Balance All, etc.) | ||
| - Flow Group and gate settings calculations. | ||
| - `regi-headless/src/main/python/regi_python/` | ||
| - Python bridge code | ||
| - Public entry points: `regi_session()` and `run_headless(calculation_callback)` | ||
| - `regi-headless/src/main/java/` | ||
| - Java support layer used by the bridge calling into REGI calculation and data access libraries | ||
| - `district-scripts/` | ||
| - Copy of district-owned Python scripts used as examples for smoke tests | ||
|
|
||
| ## Project Structure | ||
| ## Requirements | ||
|
|
||
| - `regi-headless/`: Core Java implementation, including `RegiCLI`. | ||
| - `district-scripts/`: Example scripts and district-specific configurations. | ||
| - `docs/`: Additional documentation. | ||
| - Java JDK 21 or higher | ||
| - Python 3.11 or higher | ||
|
|
||
| ## Getting Started | ||
| ### Environment Variables | ||
| - `JAVA_HOME` set for JPype startup | ||
| - `CDA_URL` url for CDA instance to connect to | ||
| - `CDA_API_KEY` required for accessing and storing data in CDA | ||
| - `OFFICE_ID` session scoped office for data access | ||
| - `REGI_LOG_LEVEL` for logging verbosity | ||
| - `REGI_LOG_FORMAT` overrides the default Python log format | ||
|
|
||
| ### Prerequisites | ||
| ## Building the Wheel | ||
|
|
||
| - Java JDK 21 or higher. | ||
| Build the Python wheel with Gradle: | ||
|
|
||
| ### CWMS Data API Configuration | ||
| ```powershell | ||
| ./gradlew buildPythonWheel | ||
| ``` | ||
|
|
||
| The wheel is written to `regi-headless/build/install/regi_python/dist/`. | ||
|
|
||
| The library reads the following environment variables when establishing the CWMS Data API data source: | ||
| Release tags become wheel versions and must be PEP 440 compatible. For example, use `0.0.2a0`, `0.0.2b0`, or `0.0.2rc0` instead of `0.0.2-alpha`, `0.0.2-beta`, or `0.0.2-rc`. | ||
|
|
||
| - `CDA_URL`: Base URL for the CWMS Data API endpoint. | ||
| - `API_KEY`: API key used for CWMS Data API authentication and authorization. | ||
| - `OFFICE_ID`: CWMS office identifier used to scope the session. | ||
| Install the built wheel into a Python environment: | ||
|
|
||
| ```powershell | ||
| pip install regi_python-*.whl | ||
| ``` | ||
|
|
||
| The factory uses these values to authenticate, resolve the current user, and persist the connected office and time zone into the REGI project. | ||
| ### Notes | ||
|
|
||
| ### Building | ||
| - The Python package name is `regi_python`. | ||
| - The wheel metadata name is `regi-python`. | ||
| - The bundled Java jars are packaged inside `regi_python/lib/`. | ||
|
|
||
| ## Releases | ||
|
|
||
| Releases are published from the GitHub repository at [USACE-WaterManagement/regi-python](https://github.com/USACE-WaterManagement/regi-python). A release build attaches the Python wheel and checksum file to the GitHub Release for the matching tag. | ||
|
|
||
| To consume a published wheel, download the wheel asset from the release and install it with your package manager. Use the exact wheel filename from the release asset URL. | ||
|
|
||
| ```powershell | ||
| ./gradlew build | ||
| pip install https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl | ||
| ``` | ||
|
|
||
| Details TBD. | ||
| ```powershell | ||
| uv pip install https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl | ||
| ``` | ||
|
|
||
| ## Usage | ||
| ```powershell | ||
| poetry add https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl | ||
| ``` | ||
|
|
||
| ```powershell | ||
| pdm add https://github.com/USACE-WaterManagement/regi-python/releases/download/<tag>/regi_python-<version>-py3-none-any.whl | ||
| ``` | ||
|
|
||
| If your tool does not support direct wheel URLs, download the asset from the release page and install it from the local `.whl` file instead. | ||
|
|
||
| TBD | ||
| ## Using The Bridge | ||
|
|
||
| ### Command Line Options | ||
| See [docs/PYTHON_API.md](docs/PYTHON_API.md) for the full Python-facing contract. | ||
|
|
||
| TBD | ||
| ## Verification | ||
|
|
||
| ### Example | ||
| Run the wheel smoke tests: | ||
|
|
||
| TBD | ||
| ```powershell | ||
| ./gradlew testPythonWheel | ||
| ``` | ||
|
|
||
| Run the script/API compatibility smoke test: | ||
|
|
||
| ## Testing | ||
| ```powershell | ||
| ./gradlew smokeTestDistrictScripts | ||
| ``` | ||
|
|
||
| TBD | ||
| `./gradlew check` runs both along with Java unit tests. | ||
|
|
||
| ## Maintainers | ||
|
|
||
| See [MAINTAINERS.md](MAINTAINERS.md) for a list of project maintainers. | ||
| See [MAINTAINERS.md](MAINTAINERS.md). | ||
|
|
||
| ## License | ||
|
|
||
| See [LICENSE](LICENSE) for licensing information. | ||
| See [LICENSE](LICENSE). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| # Migrating REGI Headless District Jython Scripts to `regi_python` | ||
|
|
||
| This guide captures the script migration pattern used in the district-scripts history, | ||
| moving from old Python 2 Jython-style top-level scripts to the current JPype-based `regi_python` bridge. | ||
|
|
||
| ### Callback Mechanism | ||
| The key mental shift is that your script no longer procedurally runs from top to bottom. | ||
| Instead, you define your calculation logic inside a function – a callback – and hand that function to `run_headless()`. | ||
| The REGI framework starts the JVM, prepares the execution environment, and then calls your callback. | ||
| The top level of your script only needs to define and hand off the callback. | ||
|
|
||
| ## Core Migration Pattern | ||
|
|
||
| Old scripts typically: | ||
|
|
||
| - imported Java classes at module import time | ||
| - accessed global `registry` and calculation objects at top level | ||
| - executed the calculation immediately when the file was loaded | ||
| - relied on shell wrappers such as `RunHeadlessJython.bat` or `.sh` | ||
|
|
||
| New scripts should: | ||
|
|
||
| - import `regi_session` and `run_headless` from `regi_python` | ||
| - move Java imports inside a callback, such as `run_calculations(registry)` below, which `run_headless` invokes once the JVM has started | ||
| - wrap the calculation logic in a function such as `run_calculations(registry)` | ||
| - keep `registry.getCalculation(...)` and Java method calls unchanged | ||
| - end with `if __name__ == "__main__": with regi_session(): run_headless(run_calculations)` | ||
|
|
||
| ## Before And After | ||
|
|
||
| ### Before | ||
|
|
||
| ```python | ||
| from java.util import Calendar | ||
| from java.util import TimeZone | ||
|
|
||
| gateCalc = registry.getCalculation(1.0, "Gate Flow") | ||
| timeZone = TimeZone.getTimeZone("US/Central") | ||
| startCal = Calendar.getInstance(timeZone) | ||
| endCal = Calendar.getInstance(timeZone) | ||
| gateCalc.computeFlowGroup("SWT", "FOSS", startCal.getTime(), endCal.getTime(), "Flow.FOSS.Project_Total") | ||
| ``` | ||
|
|
||
| ### After | ||
|
|
||
| ```python | ||
| from regi_python import regi_session, run_headless | ||
|
|
||
|
|
||
| def run_calculations(registry): | ||
| from java.util import Calendar | ||
| from java.util import TimeZone | ||
|
|
||
| gateCalc = registry.getCalculation(1.0, "Gate Flow") | ||
| timeZone = TimeZone.getTimeZone("US/Central") | ||
| startCal = Calendar.getInstance(timeZone) | ||
| endCal = Calendar.getInstance(timeZone) | ||
| gateCalc.computeFlowGroup("SWT", "FOSS", startCal.getTime(), endCal.getTime(), "Flow.FOSS.Project_Total") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| with regi_session(): | ||
| run_headless(run_calculations) | ||
| ``` | ||
|
|
||
| ## Translation Rules | ||
|
|
||
| ### 1. Move Java imports inside the callback | ||
|
|
||
| JPype-backed imports should happen after the JVM starts. | ||
| Put them inside `run_calculations(registry)` or a nested helper only called from that callback. | ||
|
|
||
| This is the biggest behavioral difference from Jython scripts. | ||
|
|
||
| ### 2. Keep the registry lookup pattern | ||
|
|
||
| The migration does not change the scriptable REGI API. Calls like these stay the same: | ||
|
|
||
| - `registry.getNames(1.0)` | ||
| - `registry.getCalculation(1.0, "Inflow")` | ||
| - `registry.getCalculation(1.0, "Gate Flow")` | ||
|
|
||
| The change is only the Python wrapper around those calls. | ||
|
|
||
| ### 3. Convert top-level execution into a callback | ||
|
|
||
| Scripts must be converted from immediate execution to a `run_calculations()` function. | ||
| That keeps import side effects out of the module. | ||
|
|
||
| ### 4. Replace Python 2 `print` statements | ||
|
|
||
| Old scripts often contain lines like: | ||
|
|
||
| ```python | ||
| print "Error Computing Flow Group" | ||
| ``` | ||
|
|
||
| Update these to Python 3 syntax: | ||
|
|
||
| ```python | ||
| print("Error Computing Flow Group") | ||
| ``` | ||
|
|
||
| ### 5. Keep optional Java-side logging calls | ||
|
|
||
| Calls such as `LoggingOptions.setDbMessageLevel(2)` and `LoggingOptions.setMetricsEnabled(True)` still belong in the script if the district workflow depends on them. They do not move to `regi_python`; they simply live inside the callback now. | ||
|
|
||
| ### 6. Remove the shell wrapper | ||
|
|
||
| The old `RunHeadlessJython` launchers are no longer needed. The Python file itself becomes the entry point: | ||
|
|
||
| ```python | ||
| if __name__ == "__main__": | ||
| with regi_session(): | ||
| run_headless(run_calculations) | ||
| ``` | ||
|
|
||
| ## Project-Specific Examples | ||
|
|
||
| The current district scripts show the same migration shape across several script families: | ||
|
|
||
| - `SWF/InflowCalcComputedInflow.py` | ||
| - `SWF/InflowCalcComputeEvapAsFlow.py` | ||
| - `SWF/GateSettings.py` | ||
| - `SWT/GateFlowGroup1.py` | ||
| - `SWL/Big3-GateFlow.py` | ||
|
|
||
| The differences between them are the calculation names, location lists, and optional logging or flow-group loops. The migration pattern itself is the same. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.