diff --git a/.claude/lint-rules/orphaned_elements.star b/.claude/lint-rules/orphaned_elements.star index e3d16cdf5..1aa790cb2 100644 --- a/.claude/lint-rules/orphaned_elements.star +++ b/.claude/lint-rules/orphaned_elements.star @@ -54,10 +54,13 @@ def check(): # Get references to this microflow refs = refs_to(mf.qualified_name) - # Check if any reference is a call + # A scheduled event is an entry point: it runs the microflow without + # anything "calling" it, so a 'schedule' edge counts as a caller. Without + # this, a microflow that runs nightly in production was reported as + # orphaned — with the suggestion "Remove if unused". has_callers = False for ref in refs: - if ref.ref_kind == "call": + if ref.ref_kind == "call" or ref.ref_kind == "schedule": has_callers = True break diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index 4bc455552..dcf2816cc 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -468,8 +468,25 @@ extracting `OffsetExpression`/`LimitExpression`. | A microflow's `StableId` is a different GUID after every mxcli write, so a microflow always differs from itself and no amount of no-op elision can settle it. Downstream: every `callMicroflow` entry in `deployment/model/operations.json` gets a new `operationId` on each build | `microflow_write.go` registered `StableId` as a `codec` `FreshGUIDField`, minting a fresh GUID per write. It is the one field on a microflow whose *stated purpose* is not to change: Mendix declares it `ModelPropertyAttribute("StableId", RetentionType.DesignTime)` with `IsIdentifier = true`, seeds it once via the one-time `MicroflowStableIdConversion`, and transplants it across a marketplace module update in `PackageUtils.RescueStableIDs` | `modelsdk/canon/identity.go` (`identityFields`, `CarryIdentity`) — the stored value is carried onto the rebuilt document before the comparison, so an otherwise-unchanged microflow compares equal and is elided | **Establish that a property is an identity before adding a row to `identityFields`** — do not assume it from the name. The method that settled this one: strict-boundary `strings` scan of every runtime jar (a substring match gives false positives — `newPersistableIds` contains `stableId`); `monodis` the modeler assembly and read the `ModelPropertyAttribute` blob; look for an `IOneTimeConversion` named after the field; search the packaging assembly for a get→set pair; then build with `mxbuild --target=deploy` and reproduce the derivation against `deployment/`. That last step is what proved the value escapes the model: `operationId == base64(uuid5(projectId, StableId).bytes_le)`, 10 of 10 exact. **Carry only a key both documents already have** — never invent one, and dispatch on the stored `$Type` (CLAUDE.md, overlay writes). **Identity preservation is not gated by `MXCLI_ALWAYS_WRITE`**: that flag turns off eliding a write, not preserving what the document is. Tests `modelsdk/canon/identity_test.go`, `sdk/mpr/writer_elision_test.go`. ADR-0008 | | `call workflow`, `get workflow data/workflows/activity records`, `open`/`lock`/`unlock workflow` and `workflow operation` all DESCRIBE as a placeholder, so a describe→edit→exec cycle deletes them — while authoring and building them works fine | The modelsdk reader (the DEFAULT engine) had no case for any of the eight; each read back with a nil Action. The formatters and grammar were already there, so only `actionFromGen` was missing | New `mdl/backend/modelsdk/microflow_workflow_read.go` (+ dispatch in `microflow_read_actions.go`), `mdl/grammar/domains/MDLMicroflow.g4` + `mdl/visitor/visitor_microflow_workflow.go` (positional `call workflow` form), `mdl/executor/cmd_microflows_builder_workflow.go` (abort reason) | **Read against the WRITER's keys, not gen's** — gen binds every "get" action's result as `VariableName` while the model stores `OutputVariableName`, so an accessor-based reader silently drops the variable; round-trip tests (not reader-only tests on hand-written BSON) are what catch it. **`WorkflowSelection` has two variants and the object form uses `WorkflowDefinitionVariable`**, not `WorkflowVariable`; its ABSENCE is meaningful (the all-workflows case), so do not synthesise one. **Dispatch the operation on `$Type` before reading fields** — only Abort carries a Reason. **Finishing the reader exposes what could not be seen while nothing rendered**: `call workflow` described to a positional form the grammar rejected (the model never stores the parameter NAME, so DESCRIBE cannot emit the named form), and the abort reason stored the expression *including its quotes* into a StringTemplate Text, so Mendix rendered the quotes at runtime and every round trip doubled them. **Still open**: `lock/unlock workflow all` writes an activity mxbuild rejects with CE1825 — a lock always needs a specific definition, and supplying an empty selection does not satisfy it. Tests `microflow_workflow_read_test.go`, example `mdl-examples/bug-tests/workflow-actions-describe.mdl` | | `transform $In with Module.Transformer` reports "Created microflow" and the build then fails `[CE0008] "No action defined."` — the activity is in the model with no action inside it | The modelsdk writer (the DEFAULT engine) had no `TransformJsonAction` case, so it fell through to `default: return nil`. The grammar, builder, DESCRIBE formatter and the LEGACY writer all handled it, which is why it looked supported. The reader was missing too, so even a correctly written action described as a placeholder | `mdl/backend/modelsdk/microflow_write.go` (writer case), `mdl/backend/modelsdk/microflow_read_actions.go` (reader cases for `TransformJsonAction`, `CallExternalAction`, `RestOperationCallAction`) | **A missing WRITER case and a missing READER case present identically in a coverage audit** — grepping for reader cases found `transform` "authorable but unreadable" when it was actually unwritable, a strictly worse bug. Check both directions before sizing the work. **Mirror the legacy serializer for keys** (`InputVariableName`/`OutputVariableName`/`Transformation`); for the other two, note `CallExternalAction` stores its result under `VariableName` (not `ResultVariableName`) and REST's two mapping lists are NOT symmetric — the query list keys its name as `QueryParameter`. **Do not reconstruct `CallExternalAction.ResultDataType`**: it is resolved from the consumed service's cached `$metadata` at write time, so reading it back would let a stale value round-trip as if authored. **CE0008 turning into CE1613 is progress, not a new bug** — the action now exists and the build has moved on to validating its reference. Tests `microflow_integration_actions_test.go`, example `mdl-examples/bug-tests/transform-json-write-and-describe.mdl` | +| `describe Module.Name` reports `no describable document named "..."` for a document that plainly exists and that `describe building block Module.Name` / `describe icon collection Module.Name` describes fine when the type is named explicitly | Two lists in different packages drifted apart. Bare DESCRIBE resolves the type by looking the qualified name up in the catalog `objects` view and mapping its `ObjectType` through `objectTypeToDescribeKind`. Building blocks were built into `building_blocks_data` but never joined into the `objects` view union, so the lookup returned no row; icon collections had **no catalog table at all**, leaving the `ICON_COLLECTION` entry in the map as dead code that nothing could ever emit. Measured on a stock 7-module marketplace project: 43 of 251 documents (40 building blocks + 3 icon collections) were unreachable this way — auto-detect coverage 81%, explicit-type coverage 98% | `mdl/executor/describe_auto.go` (`objectTypeToDescribeKind`), `mdl/catalog/tables.go` (the `objects` view union + the `*_data` table), `mdl/catalog/builder.go` (`buildSimpleNamedDocs` call), `mdl/catalog/catalog.go` (the `CATALOG.*` table list) | Add the type in **all four** places — a map entry alone does nothing if the view never emits the `ObjectType`, and a view row alone does nothing if the map has no kind. For a document with just name/folder/documentation, the builder is one `buildSimpleNamedDocs("", "", "
' [PAGE Module.Page | MICROFLOW Module.Flow] [ICON Module.Collection.name];\n" + + " MENU '' [ICON Module.Collection.name] ( );\n" + + ");\n" + + "DESCRIBE MENU Module.Name;\n" + + "DROP MENU Module.Name;", + Example: "CREATE OR MODIFY MENU MyModule.Main_Menu (\n" + + " menu item 'Home' page MyModule.Home_Web icon Atlas_Core.Atlas.home;\n" + + " menu item 'Run' microflow MyModule.DoThing;\n" + + " menu 'Admin' (\n" + + " menu item 'Accounts' page Administration.Account_Overview;\n" + + " );\n" + + " menu item 'Plain';\n" + + ");\n\n" + + "-- Notes:\n" + + "-- * A menu document is the reusable menu a menu widget points at. It is\n" + + "-- NOT the menu inside a navigation profile — for that use\n" + + "-- SHOW NAVIGATION MENU and ALTER NAVIGATION. Both use these same items.\n" + + "-- * OR MODIFY replaces the item list wholesale; an omitted item is removed.\n" + + "-- The document's identity and export level are preserved.\n" + + "-- * ICON names an icon collection entry. A glyph or image icon cannot be\n" + + "-- expressed in MDL; DESCRIBE flags those rather than dropping them silently.\n" + + "-- * A page with required parameters cannot be opened from a menu item\n" + + "-- without an argument — Mendix reports CE1571.", + SeeAlso: []string{"navigation.create", "navigation.show", "page.show"}, + }) + // ── Fragment ────────────────────────────────────────────────────────── Register(SyntaxFeature{ diff --git a/cmd/mxcli/tui/icons.go b/cmd/mxcli/tui/icons.go index 6e0b0dd0c..4106d39ee 100644 --- a/cmd/mxcli/tui/icons.go +++ b/cmd/mxcli/tui/icons.go @@ -35,6 +35,7 @@ var typeIconMap = map[string]string{ // Constants & events "constant": "π", "scheduledevent": "⏰", + "queue": "🧵", // Actions "javaaction": "☕", diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index 0ea07706e..dc2e98729 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -111,6 +111,7 @@ - [Business Events](language/business-events.md) - [Event Services](language/event-services.md) - [Publishing and Consuming Events](language/pub-sub-events.md) +- [Scheduled Events and Task Queues](language/scheduled-events.md) - [Image Collections](language/image-collections.md) --- @@ -256,6 +257,7 @@ - [Navigation Statements](reference/navigation/README.md) - [ALTER NAVIGATION](reference/navigation/alter-navigation.md) - [SHOW NAVIGATION](reference/navigation/show-navigation.md) + - [CREATE MENU](reference/navigation/menu.md) - [Workflow Statements](reference/workflow/README.md) - [CREATE WORKFLOW](reference/workflow/create-workflow.md) - [DROP WORKFLOW](reference/workflow/drop-workflow.md) diff --git a/docs-site/src/guides/marketplace.md b/docs-site/src/guides/marketplace.md index fc679d467..fb5628ace 100644 --- a/docs-site/src/guides/marketplace.md +++ b/docs-site/src/guides/marketplace.md @@ -61,10 +61,52 @@ mxcli marketplace install 2888 --version 7.0.3 -p app.mpr # a module | Content type | What `install` does | |---|---| | **Widget** | Copies the `.mpk` into the project's `widgets/` folder (overwrites on update). Reload in Studio Pro or run `mx update-widgets` to pick it up. | -| **Module** (new) | Imports it via `mx module-import` (requires a matching mxbuild — run `mxcli setup mxbuild -p app.mpr` if missing). | +| **Module** (new) | Copies the module in with mxcli's own writer, preserving the project's MPR format, plus everything the package ships (widgets, themesource, ...). Requires a matching mxbuild — run `mxcli setup mxbuild -p app.mpr` if missing. | | **Module** (already present) | **Reported, not modified** — see below. | | Theme / Starter App / Sample | Downloaded to disk with import instructions (import via Studio Pro). | +### The latest version is often not installable + +New releases are published against the newest Studio Pro patch within days of it shipping, and `install` with no `--version` resolves to the latest — so on a project that is not on the very newest patch, the default is routinely the one version that cannot be imported. Measured on an 11.12.1 project: the latest release of all six agent-editor stack modules required 11.12.2, published five days earlier. + +`install` and `update` check the version's published minimum before downloading anything, and name the version to use instead: + +```text +Agent Commons 4.2.0 requires Mendix 11.12.2, and the project is 11.12.1 + hint: install --version 4.1.0 (the newest release built for 11.12.1 or older) +``` + +`mxcli marketplace versions ` shows the same information as a `MIN MENDIX` column. + +### Dependencies are not resolved + +`install` installs exactly the content you name. Its dependencies are neither fetched nor named — read the check errors after each install, which identify what is missing by qualified name. The error count is **not monotonic**, because each new module brings its own unmet dependencies: installing the agent-editor stack into a vanilla 11.12.1 app went 0 → 15 → 0 → 18 → 1 → 22 → 1 → 1. Dependencies include widget content as well as modules (`CE0462 "Could not find widget ..."`). + +## Why installs do not use `mx module-import` + +`mx module-import` rewrites an MPR v2 project as v1. Measured on a blank Mendix 11.12.1 app, a single import turned a 69 KB `.mpr` plus 341 `.mxunit` files into one 14 MB SQLite blob with no `mprcontents/` — and the same was observed independently on 11.13.0, so it is not version-specific: + +```text +before .mpr 69,632 bytes + 341 .mxunit tables: Unit, _MetaData, _Transaction +after .mpr 14,295,040 bytes + 0 .mxunit tables: Unit, _MetaData +``` + +That is not cosmetic. The v2 layout is what makes the model diffable and mergeable per document: it is what [`mxcli diff-local`](../tools/diff.md) reads, and what makes an idempotent re-run observable as "no files changed". The conversion is **one-way** — `mx convert` targets Mendix *versions*, not storage formats. + +So `install` copies the module's units directly instead, which keeps the project in whatever format it already uses and also works for theme modules (`module-import` refuses those outright). Measured: CommunityCommons 11.5.1 into a vanilla 11.12.1 app — 128 units and 126 bundled files, `mprcontents/` grew from 369 to 497 `.mxunit` files, and `mx check` reports 0 errors. + +`--allow-format-change` selects the legacy `module-import` path, which still refuses to run silently: + +```text +refusing to import: app.mpr uses the MPR v2 storage format, and 'mx module-import' +would rewrite it as v1. +... + - Import the module in Studio Pro, which preserves the format; or + - pass --allow-format-change to accept the conversion to MPR v1. +``` + +If you take that route the command states plainly that the project is now v1. + ## Updating an existing module Updating a module that is **already in the project is not done automatically**. `install` detects it, reports the installed and target versions, and stops: @@ -76,9 +118,195 @@ In-place module updates are not applied automatically (they can discard local edits and change persistent-entity IDs, which loses data). Update via Studio Pro. ``` +Before you update in Studio Pro, the question worth answering is **whether anyone has edited the module since it was installed** — because the update will not ask. That is what `marketplace diff` is for; see below. + Two reasons make automatic in-place module updates unsafe: 1. **Local edits.** Teams sometimes modify a marketplace module after importing it; a blind re-import would discard those changes. 2. **Persistent-entity IDs.** A fresh import assigns new entity `$ID`s. The runtime database keys data by entity ID, so re-importing a module with persistent entities would make the runtime treat them as *different* entities — **losing data**. Studio Pro's Marketplace **Update** performs an ID-preserving merge that the `mx` CLI does not expose, so module updates are left to Studio Pro for now. + +## Updating a module (`marketplace update`) + +`update` replaces an installed module with another published version, preserving the two things a plain replace destroys. + +```bash +# Refuses if you have edited the module, naming what it would discard +mxcli marketplace update 23513 -p app.mpr --to 4.5.0 + +# Park those edits as re-executable MDL, then update over them +mxcli marketplace update 23513 -p app.mpr --to 4.5.0 --save-edits ./local-edits +mxcli marketplace update 23513 -p app.mpr --to 4.5.0 --force +mxcli exec ./local-edits/entity-Account.mdl -p app.mpr +``` + +```text +Administration updated 4.3.2 → 4.5.0 + 28 units copied, 9 element identities preserved, 2 role grant(s) restored. + + Removed in 4.5.0 (1) — their database columns or tables will go on the next deploy: + Account/MyLocalEdit +``` + +### What it preserves, and why + +- **Element identity.** The runtime keys entities and their attributes on the model's `GUID`, so a module whose documents are replaced without carrying the old GUIDs is a *different* module to the database and its tables are dropped on the next deploy. Studio Pro transplants them; so does this. +- **Access.** A user role's grant of a module role lives in the project's security document, not the module, so removing the module takes the grants with it and putting it back does not return them. +- **Everything else the package ships.** A module is not only its model: the `.mpk` carries widget binaries under `widgets/`, styling and design-property declarations under `themesource/`, and whatever else it needs. All of it is replaced — only `project.mpr` and `package.xml` are manifest rather than payload. DataWidgets 3.11.3 replaces 49 such files, and skipping them leaves the app running old widget code and reporting `CE6083` for design properties the module itself declares. + +It does not use `mx module-import`, which would rewrite an MPR v2 project as v1 and refuses theme modules. Units are copied with mxcli's own writer, so the project keeps its format. + + +### Local edits + +Local edits are **not** preserved. `update` refuses when it finds any, `--save-edits` writes them out first, and `--force` proceeds. Two limits on the saved files: + +- They are the element's **resulting state, not a diff**, so replaying restores additions and changes but not removals. +- An element that could not be described has nothing to save, and is reported rather than skipped. + +### Afterwards + +```bash +mxcli fix widgets -p app.mpr +mxcli fix design-properties -p app.mpr +mxcli docker check -p app.mpr +mxcli diff-local -p app.mpr +``` + +A headless install or update leaves two repairs for Mendix's own tools: **CE0463** (the project's stored widget instances are older than the widget packages beside them) and **CE6087** (a module references design properties an older Atlas spelled differently). Both are expected, not faults in the install. See [`mxcli fix`](#repairing-the-model-mxcli-fix) below. + +Measured: Administration 4.3.2 → 4.5.0 and DataWidgets 3.5.0 → 3.11.3 both reach **0 errors** afterwards. + +## Repairing the model (`mxcli fix`) + +`mx update-widgets` and `mx rename-design-properties` each fix something only Mendix can fix, and each rewrites an MPR v2 project into the single-file v1 format while doing it. Measured on 11.12.1: `update-widgets` took 369 `.mxunit` files to 0 and a 69,632-byte index to 14,405,632 bytes; `rename-design-properties` took 1,865 files to 0 and a 249,856-byte index to 39,895,040 bytes, having renamed 149 design properties across 41 documents. The conversion is one-way. + +```bash +mxcli fix widgets -p app.mpr # CE0463 +mxcli fix design-properties -p app.mpr # CE6087 +``` + +```text +Updated design properties: 42 unit(s) changed. + Storage: 1868 .mxunit file(s), unchanged from 1868 before (MPR v2 preserved). +``` + +Each runs the same Mendix tool, reads every unit back out of the converted file, restores the v2 storage, and writes the changed units into it through mxcli's own writer. The storage count is printed before and after because that is where the failure this exists to prevent would show up — as a zero. + +Measured end to end on a vanilla 11.12.1 app carrying the agent-editor stack: `mx check` reported **203 errors** (202 × CE0463 + 1 × CE6087), and **0** after the two commands, with the project still MPR v2 (1,868 `.mxunit` files) — reproduced from a restored pre-fix snapshot. + +Re-running is free: a second run reports 0 units changed, because [idempotent writes](../internals/idempotent-writes.md) elide a unit whose content did not really change. An MPR v1 project is passed straight through, since these tools write v1 natively. + +| Command | Persists? | Use | +|---|---|---| +| `mxcli fix widgets` | **yes** | the fix — after any headless install | +| `mxcli fix design-properties` | **yes** | the fix — after any headless install | +| `mxcli docker check` | no | runs the widget resync under a snapshot so the *check* is not tripped by CE0463; the stored model stays stale | +| `mxcli widget sync` | yes, partial | reconciles widget schemas in mxcli's own code; clears 7 of 40 on the reference fixture | + +CE6087 is distinct from `CE6083`, which is a *missing* design-property declaration and is fixed by installing everything the package ships — something `install` and `update` already do. + +`update` does **not** roll back. Work on a copy or have the project in version control. + +## Has this module been edited? (`marketplace diff`) + +Studio Pro's Marketplace **Update** replaces the module and discards local edits without asking. `marketplace diff` answers the question that decides whether that is safe: + +```bash +# What have I changed in this module since installing it? +mxcli marketplace diff 23513 -p app.mpr +``` + +```text +Administration — installed 4.3.2 (Mendix 11.12.1) + + Locally modified (1 of 21 elements): + changed ENTITY Account +``` + +An untouched module reports how much was actually checked, not just a verdict: + +```text + No local modifications: 21 of 21 elements verified unchanged. +``` + +### What it does + +1. Reads which marketplace version each module in the project records. The project stores the marketplace **version UUID** per module, so the module and the exact release it came from are both identified without guessing — the listing name does not help here (content 23513 is listed as "Administration module" and installs a module called `Administration`; "Data Widgets" installs `DataWidgets`). +2. Downloads that version's `.mpk` and imports it into a throwaway reference project built **at the project's own Mendix version**, so the package goes through the same conversion the installed copy did. +3. Describes every element of the module on both sides and compares the descriptions. + +Comparison is on `DESCRIBE` output rather than raw storage: an *untouched* module differs from its own published package in thousands of BSON paths, because the installed copy carries subtrees the package does not. + +Requires the mxbuild toolchain for the project's Mendix version — `mxcli setup mxbuild -p app.mpr`. Building the reference at a *different* version is refused rather than warned about, because Mendix's own conversions would then show up as your edits. + +### Theme modules + +`mx module-import` refuses a theme module outright ("Importing theme module is not supported"), which would take Atlas_Core, Atlas_Web_Content and Conversational UI off the table. The refusal is gated on a single flag on the module document inside the package, so `diff` clears it on **its own throwaway copy** before importing — the published package and your project are untouched. + +Atlas modules are among the most-edited in real projects, so this matters more than the module count suggests: + +```text +Atlas_Web_Content — installed 4.1.0 (Mendix 11.12.1) + + No local modifications found, but 46 of 89 elements could not be read — + this is not a clean bill of health. + + Not comparable (46) — reported as unknown, never as unchanged: + unknown PAGE_TEMPLATE Blank (no DESCRIBE support for PAGE_TEMPLATE) + ... +``` + +The 46 are page templates, which have no `DESCRIBE` handler yet. They are reported rather than quietly counted as unchanged — see [`CATALOG.PAGE_TEMPLATES`](../tools/catalog-tables.md). + +### What an upgrade would touch + +`--to` adds the other half of the question: what the module's author changed, and whether it collides with what you changed. + +```bash +mxcli marketplace diff 23513 -p app.mpr --to 4.5.0 +``` + +```text + Upgrading to 4.5.0 would touch 5 element(s), 1 of which you have modified: + CONFLICT ENTITY Account + + Studio Pro's update would discard those local edits without asking. +``` + +### In CI + +`--json` emits the machine-readable form, for a build gate that fails when a marketplace module has been edited: + +```bash +mxcli marketplace diff 23513 -p app.mpr --json +``` + +```json +{ + "module": "Administration", + "installedVersion": "4.3.2", + "mendixVersion": "11.12.1", + "locallyModified": true, + "verified": true, + "modified": ["ENTITY Account"], + "unchangedCount": 20 +} +``` + +Read **both** `locallyModified` and `verified`. `verified: false` means at least one element could not be described, so "no modifications found" is not a conclusion you can act on — an element that cannot be read is reported as `unknown`, never as unchanged: + +```text + No local modifications found, but 1 of 21 elements could not be read — + this is not a clean bill of health. +``` + +### Flags + +| Flag | Purpose | +|---|---| +| `-p, --project` | The project holding the installed module (required). | +| `--to ` | Also report what upgrading to this version would touch, and which of those you have modified. | +| `--module ` | Name the module explicitly, when the project records no marketplace version for it (a hand-imported copy) or several modules match. | +| `--json` | Emit JSON instead of text. | diff --git a/docs-site/src/language/scheduled-events.md b/docs-site/src/language/scheduled-events.md new file mode 100644 index 000000000..f6da79561 --- /dev/null +++ b/docs-site/src/language/scheduled-events.md @@ -0,0 +1,185 @@ +# Scheduled Events and Task Queues + +Two Mendix features for work that runs outside a user request. They are unrelated +and easy to confuse: + +- A **scheduled event** is Mendix's cron — it runs a microflow on a repeating + schedule. +- A **task queue** bounds how many queued microflow calls run at once. + +A scheduled event does **not** go through a task queue. Its own concurrency +control is `OnOverlap`, which decides what happens when a run is still going when +the next one is due. + +## Scheduled Events + +### Inspecting + +```sql +-- All scheduled events, or one module's +LIST SCHEDULED EVENTS; +LIST SCHEDULED EVENTS IN Ops; + +-- Re-executable MDL for one event +DESCRIBE SCHEDULED EVENT Ops.NightlyCleanup; +``` + +`SHOW` is accepted as a synonym for `LIST`. + +### CREATE SCHEDULED EVENT + +```sql +CREATE [OR MODIFY] SCHEDULED EVENT . ( + Microflow: ., + Repeat: , + , + +); +``` + +`Microflow` and `Repeat` are always required. + +#### Repeat variants + +The repeat rule is stored as one of eight Mendix schedule types, and they differ +in **which fields they carry** — not just in their values. MDL mirrors that: each +repeat takes only its own fields, and a field belonging to another repeat is +refused by both `mxcli check` (rule `MDL-SCHED01`) and `mxcli exec` rather than +silently dropped. + +| Repeat | Fields | Reads as | +|--------|--------|----------| +| `Minutely` | `Multiplier` | every N minutes | +| `Hourly` | `Multiplier`, `MinuteOffset` | every N hours, at :MM | +| `Daily` | `HourOfDay`, `MinuteOfHour` | every day at HH:MM | +| `Weekly` | `Weekdays`, `HourOfDay`, `MinuteOfHour` | on the named days at HH:MM | +| `MonthlyByDate` | `Multiplier`, `MonthOffset`, `DayOfMonth`, `HourOfDay`, `MinuteOfHour` | the Dth of every N months | +| `MonthlyByWeekday` | `Multiplier`, `MonthOffset`, `DaySelector`, `Weekday`, `HourOfDay`, `MinuteOfHour` | the last Friday of every N months | +| `YearlyByDate` | `Month`, `DayOfMonth`, `HourOfDay`, `MinuteOfHour` | every 2 January | +| `YearlyByWeekday` | `Month`, `DaySelector`, `Weekday`, `HourOfDay`, `MinuteOfHour` | the first Monday of March | + +Field values: + +| Field | Value | +|-------|-------| +| `Multiplier` | how many units between runs (1 or more; defaults to 1) | +| `MinuteOffset` | 0–59, the minute past the hour | +| `MonthOffset` | 0-based, which month of a multi-month cycle fires | +| `HourOfDay` / `MinuteOfHour` | 0–23 / 0–59 | +| `DayOfMonth` / `Month` | 1–31 / 1–12 | +| `Weekdays` | a quoted list, e.g. `'Monday, Friday'` (case-insensitive) | +| `DaySelector` | `First`, `Second`, `Third`, `Fourth`, `Last` | +| `Weekday` | `Sunday` … `Saturday` | + +A value outside its range is an error, not a truncation: a schedule that is +stored but can never fire is worse than a refusal, because nothing downstream +reports it. + +#### Optional properties + +| Property | Values | Default | +|----------|--------|---------| +| `Enabled` | `true` / `false` | `false` | +| `OnOverlap` | `DelayNext` / `SkipNext` | `DelayNext` | +| `TimeZone` | `UTC` / `Server` | `UTC` | +| `StartDateTime` | an RFC 3339 timestamp; the event does not run before it | none | +| `Documentation` | free text | none | + +`SkipNext` drops a run that would overlap the previous one; `DelayNext` queues it +until the previous one finishes. + +### Examples + +```sql +-- Every night at 04:00 in the server's timezone +CREATE SCHEDULED EVENT Ops.NightlyCleanup ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + HourOfDay: 4, + MinuteOfHour: 0, + TimeZone: Server, + Enabled: true +); + +-- Every two hours, 23 minutes past +CREATE SCHEDULED EVENT Ops.HourlyPing ( + Microflow: Ops.SE_Ping, + Repeat: Hourly, + Multiplier: 2, + MinuteOffset: 23 +); + +-- Mondays and Fridays at 09:30 +CREATE SCHEDULED EVENT Ops.WeeklyReport ( + Microflow: Ops.SE_Report, + Repeat: Weekly, + Weekdays: 'Monday, Friday', + HourOfDay: 9, + MinuteOfHour: 30 +); + +-- The last Friday of every third month, at 18:00 +CREATE SCHEDULED EVENT Ops.QuarterEnd ( + Microflow: Ops.SE_Close, + Repeat: MonthlyByWeekday, + Multiplier: 3, + MonthOffset: 2, + DaySelector: Last, + Weekday: Friday, + HourOfDay: 18 +); + +DROP SCHEDULED EVENT Ops.HourlyPing; +``` + +### A note on `Interval` / `IntervalType` + +Stored events also carry an `Interval` and `IntervalType` pair. These predate the +`Schedule` child and Studio Pro does **not** keep them in sync with it — a real +Mendix module ships an event storing `0` / `Minute` next to a daily schedule of +01:00. MDL has no syntax for them: a new event gets the pair that matches its +repeat, and `CREATE OR MODIFY` carries whatever is stored through untouched. +`DESCRIBE` reports them as a comment so the output stays re-executable. + +## Task Queues + +A task queue bounds how many instances of a queued microflow call run at once. + +```sql +CREATE [OR MODIFY] QUEUE . [( + Parallelism: , + ClusterWide: true|false, + Documentation: '' +)]; + +LIST QUEUES [IN ]; +DESCRIBE QUEUE .; +DROP QUEUE .; +``` + +| Property | Meaning | Default | +|----------|---------|---------| +| `Parallelism` | how many tasks run at once — an **expression**, not a number | `1` | +| `ClusterWide` | `true` applies the limit across the cluster; `false` per runtime instance | `false` | + +Mendix stores parallelism as an expression string, so a bare integer and a quoted +one mean the same thing and an arbitrary expression is legal: + +```sql +CREATE QUEUE Ops.OrderProcessing ( Parallelism: 3, ClusterWide: true ); +CREATE QUEUE Ops.Mail; -- defaults: 1, per-instance +CREATE OR MODIFY QUEUE Ops.OrderProcessing ( Parallelism: '$MyModule.Workers' ); +``` + +### Binding a call to a queue is not yet expressible + +MDL cannot yet author a *queued call* — the binding lives on the call activity +inside a microflow, not on the queue. Because rebuilding a microflow would drop +an existing binding, `CREATE OR REPLACE|MODIFY MICROFLOW` is **refused** when the +stored microflow has a queued call, naming the queues that would be lost. Change +those microflows in Studio Pro. + +Without that refusal the binding was written back as null and the project then +looked *healthier* than before — `mx check` stopped reporting +`CE1613 "The selected task queue no longer exists"`, because the configuration +the error was about had been deleted. diff --git a/docs-site/src/reference/navigation/README.md b/docs-site/src/reference/navigation/README.md index e1004b088..180a281be 100644 --- a/docs-site/src/reference/navigation/README.md +++ b/docs-site/src/reference/navigation/README.md @@ -10,6 +10,7 @@ Mendix applications can have multiple navigation profiles (Responsive, Tablet, P |-----------|-------------| | [ALTER NAVIGATION](alter-navigation.md) | Create or replace a navigation profile with home pages, login page, and menus | | [SHOW NAVIGATION](show-navigation.md) | Display navigation profiles, menus, and home page assignments | +| [CREATE MENU](menu.md) | Create, describe and drop standalone menu documents (`Menus$MenuDocument`) | ## Related Statements diff --git a/docs-site/src/reference/navigation/menu.md b/docs-site/src/reference/navigation/menu.md new file mode 100644 index 000000000..8e3fecec8 --- /dev/null +++ b/docs-site/src/reference/navigation/menu.md @@ -0,0 +1,131 @@ +# CREATE MENU + +## Synopsis + +```sql +CREATE [ OR MODIFY ] MENU module.name ( menu_item [ menu_item ... ] ) +DESCRIBE MENU module.name +DROP MENU module.name +``` + +Where each `menu_item` is one of: + +```sql +MENU ITEM 'caption' [ PAGE module.page | MICROFLOW module.microflow ] [ ICON module.collection.icon ] ; +MENU 'caption' [ ICON module.collection.icon ] ( nested_items ) ; +``` + +## Description + +Manages standalone **menu documents** (`Menus$MenuDocument`) — reusable menus that a +menu widget on a page points at. Atlas_Core ships two of them, `Phone_Menu` and +`Tablet_Menu`. + +A menu document is **not** the menu inside a navigation profile, although the two +are easy to confuse: both are built from the same menu items, which is why the item +syntax here is identical to the `MENU (...)` block of +[ALTER NAVIGATION](alter-navigation.md). They differ in where they live and how they +are read: + +| | Profile menu | Menu document | +|---|---|---| +| Lives in | a navigation profile | its own document | +| Used by | the app's navigation | a menu widget you place on a page | +| Read with | `SHOW NAVIGATION MENU` | `DESCRIBE MENU module.name` | +| Written with | `CREATE OR REPLACE NAVIGATION` | `CREATE OR MODIFY MENU` | + +`OR MODIFY` replaces the item list **wholesale**, exactly as `CREATE OR REPLACE +NAVIGATION` does: the list you give is the document's complete contents, so an +omitted item is a removed item. The document's identity, container and export level +are preserved, so menu widgets pointing at it keep working. + +`DESCRIBE MENU` emits a re-executable `CREATE OR MODIFY MENU` statement, so +describe → edit → exec is the normal editing loop, and describe → exec → describe is +a fixed point. + +## Parameters + +`module.name` +: Qualified name of the menu document. + +`'caption'` +: The item's label, in single quotes. + +`PAGE` / `MICROFLOW` +: Optional target opened when the item is clicked. An item with neither is inert + (stored as `Forms$NoAction`), which is normal for an item that only groups + sub-items. + +`ICON` +: Optional icon, given as a qualified name into an icon collection — not a string. + Hyphenated segments are double-quoted: `Atlas_Core.Atlas."layout-2"`. + +## Examples + +Create a menu with a nested sub-menu: + +```sql +CREATE MENU MyModule.Main_Menu ( + menu item 'Home' page MyModule.Home_Web icon Atlas_Core.Atlas.home; + menu item 'Run' microflow MyModule.DoThing; + menu 'Admin' ( + menu item 'Accounts' page Administration.Account_Overview; + ); + menu item 'Plain'; +); +``` + +Replace its contents (the two omitted items are removed): + +```sql +CREATE OR MODIFY MENU MyModule.Main_Menu ( + menu item 'Home' page MyModule.Home_Web icon Atlas_Core.Atlas.home; + menu item 'Run' microflow MyModule.DoThing; +); +``` + +Read one back — including Atlas's own: + +```sql +DESCRIBE MENU Atlas_Core.Phone_Menu; +``` + +The type is auto-detected too, so the bare form works: + +```sql +DESCRIBE Atlas_Core.Phone_Menu; +``` + +From the command line: + +```bash +mxcli describe menu Atlas_Core.Phone_Menu -p app.mpr +``` + +## Notes + +**A menu item cannot open a page that takes a required parameter.** There is nowhere +to supply the argument, and Mendix rejects the model with **CE1571** ("No argument +has been selected for parameter …") reported against `Menu item`. Point the item at a +parameterless page, or call a microflow that opens the page with its argument. + +**Only icon-collection icons round-trip.** `ICON` writes a +`Forms$IconCollectionIcon`. A glyph icon (which carries a numeric code) or an image +icon cannot be expressed in MDL, so `DESCRIBE` reports those on their own comment +line rather than dropping them silently: + +``` +-- icon a numeric glyph code (Forms$GlyphIcon) is not reproducible by CREATE MENU; +-- set it in Studio Pro +``` + +Re-running such output therefore loses that icon — visibly, not silently. + +**Authoring requires the default engine.** Under `MXCLI_ENGINE=legacy`, +create/modify/drop refuse rather than writing a differently-shaped document. Reading +(`DESCRIBE MENU`) works on both engines. + +## See Also + +- [ALTER NAVIGATION](alter-navigation.md) — the menu inside a navigation profile +- [SHOW NAVIGATION](show-navigation.md) — read profile menus and home pages diff --git a/docs-site/src/tools/catalog-tables.md b/docs-site/src/tools/catalog-tables.md index d6bdbfe39..ea141f2c0 100644 --- a/docs-site/src/tools/catalog-tables.md +++ b/docs-site/src/tools/catalog-tables.md @@ -111,6 +111,28 @@ WHERE ModuleName = 'Sales' ORDER BY Name; ``` +Page **templates** are not pages and are not in this table — see +`CATALOG.PAGE_TEMPLATES`. + +### CATALOG.PAGE_TEMPLATES + +The starting points Studio Pro's "new page" dialog offers (`Forms$PageTemplate`). +A separate document type from pages: Atlas_Web_Content ships 46 templates and no +pages at all. + +| Column | Description | +|--------|-------------| +| `Id` | Unique identifier | +| `Name` | Template name | +| `ModuleName` | Module containing the template | +| `QualifiedName` | Full qualified name | +| `Folder` | Folder path within the module | +| `Description` | Documentation | + +There is no `DESCRIBE PAGE TEMPLATE`, so a template is indexed but not +describable — tools that walk a module report it as *unknown*, never as +unchanged. + ### CATALOG.ACCESS_RULES Information about entity access rules (available after full refresh). @@ -130,6 +152,62 @@ JOIN CATALOG.ENTITIES e ON ar.EntityId = e.Id WHERE e.ModuleName = 'Sales'; ``` +### CATALOG.SCHEDULED_EVENTS + +Scheduled events — Mendix's cron. + +| Column | Description | +|--------|-------------| +| `Name`, `QualifiedName`, `ModuleName`, `Folder` | Identity | +| `Microflow` | Qualified name of the microflow the event runs | +| `Repeat` | Schedule variant: `Minute`, `Hour`, `Day`, `Week`, `MonthDate`, `MonthWeekday`, `YearDate`, `YearWeekday` | +| `RepeatDescription` | The schedule as a phrase, e.g. `weekly Mon/Fri at 09:30` | +| `IntervalSeconds` | Gap between runs, derived from the schedule | +| `Enabled` | 1 if the event runs | +| `TimeZone` | `UTC` or `Server` | +| `OnOverlap` | `DelayNext` or `SkipNext` | + +`IntervalSeconds` comes from the schedule, **not** from the stored +`Interval`/`IntervalType` pair. Those are a legacy sibling that Studio Pro writes +and does not keep in sync — a shipped Mendix module stores `0`/`Minute` beside a +daily schedule — so a query keyed on them would read a nightly job as firing +every 0 seconds. Month and year figures are averages (30 and 365 days): the +column is for thresholds and ordering, not calendar arithmetic. + +```sql +-- Anything that fires more often than once a minute +select QualifiedName, RepeatDescription, Microflow +from CATALOG.SCHEDULED_EVENTS +where Enabled = 1 and IntervalSeconds < 60; + +-- Scheduled events whose microflow no longer exists +select se.QualifiedName, se.Microflow +from CATALOG.SCHEDULED_EVENTS se +left join CATALOG.MICROFLOWS m on m.QualifiedName = se.Microflow +where m.Id is null; +``` + +A scheduled event also produces a `schedule` row in `CATALOG.REFS`, so +`show callers of ` and the dead-asset analysis both see it. Without +that edge a microflow run only by a scheduled event looked unreferenced. + +### CATALOG.QUEUES + +Task queues. + +| Column | Description | +|--------|-------------| +| `Name`, `QualifiedName`, `ModuleName`, `Folder` | Identity | +| `Parallelism` | How many tasks run at once — an **expression string**, not a number | +| `ClusterWide` | 1 if the limit applies across the cluster | + +`Parallelism` is stored as text because Mendix stores an expression: a query must +not assume it parses as an integer. + +```sql +select QualifiedName, Parallelism, ClusterWide from CATALOG.QUEUES; +``` + ## Graph-Analysis Tables The dependency graph (`CATALOG.REFS`, full refresh) is analysed by a family of diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index d4cb8d818..fa8ab3e1b 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -138,6 +138,86 @@ create constant MyModule.MaxRetries type integer default 3; create constant MyModule.EnableLogging type boolean default true; ``` +## Task Queues + +| Statement | Syntax | Notes | +|-----------|--------|-------| +| Show queues | `show queues [in module];` (`list queues` too) | Parallelism + cluster-wide flag | +| Describe queue | `describe queue Module.Name;` | Re-executable MDL | +| Create queue | `create [or modify] queue Module.Name ( Parallelism: 3, ClusterWide: true );` | Body optional; defaults `1` / `false` | +| Drop queue | `drop queue Module.Name;` | | + +`Parallelism` is an **expression**, not a number — Mendix stores it as a string +(`Queues$BasicQueueConfig.ParallelismExpression`). A bare integer is the common +case; quote anything else. + +Binding a microflow **call** to a queue is not yet expressible in MDL. Because a +rebuild would drop an existing binding, `create or replace|modify microflow` is +**refused** when the stored microflow has a queued call — change those in Studio +Pro. (Without the refusal the binding was written back as null and `mx check` +stopped reporting CE1613, so the project looked healthy while the configuration +was gone.) + +**Example:** +```sql +create queue Ops.OrderProcessing ( Parallelism: 3, ClusterWide: true ); +create queue Ops.Mail; +create or modify queue Ops.OrderProcessing ( Parallelism: '$MyModule.Workers' ); +drop queue Ops.Mail; +``` + +## Scheduled Events + +Mendix's cron: run a microflow on a repeating schedule. + +| Statement | Syntax | Notes | +|-----------|--------|-------| +| Show scheduled events | `show scheduled events [in module];` (`list` too) | Repeat, microflow, enabled | +| Describe scheduled event | `describe scheduled event Module.Name;` | Re-executable MDL | +| Create scheduled event | `create [or modify] scheduled event Module.Name ( Microflow: ..., Repeat: ..., ... );` | | +| Drop scheduled event | `drop scheduled event Module.Name;` | | + +`Microflow` and `Repeat` are always required. Each repeat takes **only** its own +fields — anything else is refused by `mxcli check` (MDL-SCHED01) and by `exec`: + +| Repeat | Fields | +|--------|--------| +| `Minutely` | `Multiplier` | +| `Hourly` | `Multiplier`, `MinuteOffset` | +| `Daily` | `HourOfDay`, `MinuteOfHour` | +| `Weekly` | `Weekdays`, `HourOfDay`, `MinuteOfHour` | +| `MonthlyByDate` | `Multiplier`, `MonthOffset`, `DayOfMonth`, `HourOfDay`, `MinuteOfHour` | +| `MonthlyByWeekday` | `Multiplier`, `MonthOffset`, `DaySelector`, `Weekday`, `HourOfDay`, `MinuteOfHour` | +| `YearlyByDate` | `Month`, `DayOfMonth`, `HourOfDay`, `MinuteOfHour` | +| `YearlyByWeekday` | `Month`, `DaySelector`, `Weekday`, `HourOfDay`, `MinuteOfHour` | + +Optional on any repeat: `Enabled` (default false), `OnOverlap` +(`DelayNext` default / `SkipNext`), `TimeZone` (`UTC` default / `Server`), +`StartDateTime` (RFC 3339), `Documentation`. + +`OnOverlap` is a scheduled event's own concurrency control — scheduled events do +**not** go through a task queue. + +**Example:** +```sql +create scheduled event Ops.NightlyCleanup ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + HourOfDay: 4, + MinuteOfHour: 0, + TimeZone: Server, + Enabled: true +); + +create scheduled event Ops.WeeklyReport ( + Microflow: Ops.SE_Report, + Repeat: Weekly, + Weekdays: 'Monday, Friday', + HourOfDay: 9, + MinuteOfHour: 30 +); +``` + ## OData Clients, Services & External Entities | Statement | Syntax | Notes | @@ -964,6 +1044,12 @@ MDL uses explicit property declarations for pages: | Describe snippet | `describe snippet Module.Name;` | Round-trippable MDL output | | List building blocks | `show building blocks [in module];` | Read-only; cannot be authored via MDL | | Describe building block | `describe building block Module.Name;` | Informational (header comment + widget tree), not a `create` statement | +| Create menu | `create [or modify] menu Module.Name ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | +| Describe menu | `describe menu Module.Name;` | Round-trippable MDL. Not the navigation-profile menu — see `show navigation menu` | +| Drop menu | `drop menu Module.Name;` | | +| Create menu | `create [or modify] menu Module.Name ( );` | Standalone `Menus$MenuDocument`. Full replacement: the item list is the document's complete contents | +| Describe menu | `describe menu Module.Name;` | Round-trippable MDL. Not the navigation-profile menu — see `show navigation menu` | +| Drop menu | `drop menu Module.Name;` | | **DataGrid Column Properties:** @@ -1165,7 +1251,7 @@ CLI subcommand: `mxcli sql --driver postgres --dsn '...' "select 1"` (see `mxcli | Refresh with refs | `refresh catalog full;` | Include cross-references and source | | Show catalog tables | `show catalog tables;` | List available queryable tables | | Query catalog | `select ... from CATALOG. [where ...];` | SQL against project metadata | -| Show callers | `show callers of Module.Name;` | What calls this element | +| Show callers | `show callers of Module.Name;` | What INVOKES this element: microflow call activities, page action buttons and other widget actions, calculated attributes, and navigation entries. A page counts as a caller of the microflow its button runs, and of the page that button opens | | Show callees | `show callees of Module.Name;` | What this element calls | | Show references | `show references of Module.Name;` | All references to/from | | Show impact | `show impact of Module.Name;` | Impact analysis | @@ -1174,6 +1260,8 @@ CLI subcommand: `mxcli sql --driver postgres --dsn '...' "select 1"` (see `mxcli Cross-reference commands require `refresh catalog full` to populate reference data. +`show callers` covers invocation only. A document that merely *uses a type* — an entity as a page datasource, a microflow parameter, an entity's generalization — is not a caller of it; `show references to` lists those. + ## Connection & Session | Statement | Syntax | Notes | diff --git a/docs/11-proposals/PROPOSAL_marketplace_module_upgrade.md b/docs/11-proposals/PROPOSAL_marketplace_module_upgrade.md index c80c8b060..76e18098d 100644 --- a/docs/11-proposals/PROPOSAL_marketplace_module_upgrade.md +++ b/docs/11-proposals/PROPOSAL_marketplace_module_upgrade.md @@ -7,7 +7,8 @@ date: 2026-08-04 # Proposal: `mxcli marketplace diff` — detect local modification of an installed module **Status:** Draft -**Date:** 2026-08-04 (initial), revised 2026-08-10 (Studio Pro update measured) +**Date:** 2026-08-04 (initial), revised 2026-08-10 (Studio Pro update measured), +2026-08-11 (DESCRIBE coverage measured — §7; `GUID` = database identity measured — §8) ## Revision 2026-08-10 — what Studio Pro actually does, measured @@ -323,6 +324,107 @@ siblings). This is the one piece of work both this proposal and script idempotence need, and neither can be finished well without it. +> **Superseded in part (2026-08-11).** Script idempotence shipped *without* a stable +> key for unnamed elements: `modelsdk/canon` sidesteps element matching entirely by +> normalising each `$ID` to its index in a containment walk, so it never has to pair +> two elements up (ADR-0008). The claim that neither could be finished without this +> key held for the BSON-structural diff §4 was reasoning about; it does **not** bind +> the design below, which compares DESCRIBE output rather than BSON structure and so +> never keys an unnamed widget at all. The key problem is real but is now scoped to +> anything wanting an element-level *structural* diff — not to `marketplace diff`. + +### 7. DESCRIBE coverage is sufficient, and was measured (2026-08-11) + +The design below is bounded by DESCRIBE coverage, and its honesty rule — an +un-describable element must be reported **unknown**, never clean — is only affordable +if the unknown bucket is small. That was unmeasured, so the whole proposal rested on +an assumption. It has now been measured against `testdata/expr-checker/minimal.mpr`, +which carries seven real marketplace modules with recorded `AppStoreVersion`s +(Administration 4.3.2, Atlas_Core 4.1.3, Atlas_Web_Content 4.1.0, DataWidgets 3.5.0, +FeedbackModule 4.0.2, NanoflowCommons 6.0.0, WebActions 2.11.0). + +**Method.** The denominator is the set of *named units read from the MPR itself* — +369 units, 251 named documents after excluding 80 folders and the unnamed +module-level units. It deliberately does **not** come from the catalog: the `objects` +view indexes only describable types and `show modules` only has columns for types +mxcli models, so either source reports 100% coverage by construction. Each document +was then actually described, and the outcome recorded. + +**Result — 247 of 251 (98%) describable.** + +| | Docs | | +|---|---|---| +| Auto-detected by bare `describe Module.Name` | 204 → **247** | 81% → **98%** | +| Reachable only by naming the type explicitly | 43 → **0** | | +| Not describable at all | 4 | 2% | + +The 43 in the middle row were building blocks (40) and icon collections (3): both had +working explicit handlers, but building blocks were never joined into the catalog's +`objects` view and icon collections had no catalog table at all, so bare DESCRIBE +reported them as not found. Fixed here — the second number in each row is post-fix, +verified end-to-end on the same 251 documents, with no new name ambiguity. A drift +test (`TestDescribeAutoCoversCatalogObjectTypes`) now fails when a type reaches the +view without a describe kind. + +The remaining 4 are two genuine defects, both out of scope for this proposal: + +- **Import/export mapping describe is broken.** The catalog indexes both, but + `describe import mapping FeedbackModule.IMM_PostResponse` errors `not found`, and + naming the type explicitly does not help. +- **Menu documents have no DESCRIBE at all** (2 in Atlas_Core). No grammar, no handler. + +**Two gaps this measurement also exposed, which the differ must handle.** + +- **Page templates were conflated with pages** (closed 2026-08-11). All 46 + `Forms$PageTemplate` units reported `ObjectType = PAGE` and described as + `create or modify page`, so `show modules` reported Atlas_Web_Content as having + 46 pages when it has zero. + + This was recorded here as "harmless for describe-to-describe comparison, since + both sides conflate identically". That was wrong, and the error was in the same + family as the security one above: the conflation was checked, the *content* was + not. A page template describes with an **empty body** — its widgets hang off + `LayoutCall`, and the page describe path reads `FormCall` — so those 46 + elements compared on nothing but name, folder and CSS class. The differ was + reporting them unchanged without having looked inside them, which is precisely + the false negative the honesty rule exists to prevent, and it was hiding behind + a bug filed as cosmetic. + + Cause: `listUnitsByType` matched on a type **prefix**, and `Forms$Page` is a + prefix of `Forms$PageTemplate`. Both engines now match exactly, templates are + indexed as their own `PAGE_TEMPLATE` catalog type, and — having no DESCRIBE + handler — they are reported **unknown**, which is the truthful version of what + the differ was already doing. +- **Module roles were invisible** (closed 2026-08-11). This was originally recorded + here as "module security is invisible", which was wrong and briefly made the + security hole look like the largest risk in the proposal. Re-measured: three of the + four parts of module security were **already** in the describe surface — + entity access rules in `DESCRIBE ENTITY` (`grant on (...) where + ''`), page access in `DESCRIBE PAGE` (`grant view on page ...`), and + microflow access in `DESCRIBE MICROFLOW` (`grant execute on microflow ...`). Only + the module's **role list** was missing, because it lives in the module's own + `Security$ModuleSecurity` unit and belongs to no document. `DESCRIBE MODULE` now + emits it, sorted for stable comparison. + + The original error came from grepping describe output for `role|access|allowed` — + a pattern that cannot match `grant view on page P to Administration.Administrator;`. + Absence of evidence from a search is not evidence of absence: check the emitter, or + grep for the statement you expect to see rather than for words describing it. + +Folders need no separate coverage: folder membership is captured inside each +document's describe (`Folder: 'Phone/PageTemplates/Form'`), so a document moved +between folders by an upgrade is visible. + +**What this does not establish.** It measures *invocation success and non-trivial +output* — not round-trip fidelity. A describe can succeed and silently drop a +property, which is exactly what #812, #111 and #57/#58 were. So 98% is an **upper +bound on what a differ can compare**, not evidence that the comparison is faithful; +establishing that needs describe → execute → re-describe round-tripping, which is a +separate and much larger exercise. Also: one project, and all seven modules are +Mendix-authored — a third-party module may use document types absent here. Building +blocks and icon collections describe read-only ("cannot be created via MDL"), which +is fine for diff but blocks a Phase 2 *replace* of those two types. + ## Design ### Compare semantically, not structurally @@ -402,20 +504,135 @@ Administration — installed 4.3.2, latest 4.5.0 No MDL syntax is added. This is a CLI-only, read-only command. +### 8. What `GUID` is for: the database keys on it (measured 2026-08-11) + +§4 established that Studio Pro's update renumbers every `$ID` and preserves every +`GUID`. That made `GUID` the *only* candidate carrier of database identity, but +the proposal was careful to call it inference. It is now measured. + +**Method.** A blank Mendix 11.12.1 app with `Administration` 4.3.2, booted with +`mxcli run --local` against a local PostgreSQL, so the runtime creates and then +re-synchronises a real schema. The lever is one that Studio Pro does not expose +and mxcli does: rewrite a single BSON value in the stored model and boot again. +Nothing points *at* a `GUID` — it is not a pointer target — so changing it is a +safe one-value edit, unlike renumbering an `$ID` (ADR-0008). + +**The runtime writes its own identity map.** `mendixsystem$entity` and +`mendixsystem$attribute` record, per element, the id the database knows it by: + +``` +mendixsystem$entity b16e49ea-91df-4caa-aed8-6ba4c4e133c5 Administration.Account administration$account +mendixsystem$attribute aac00d66-7cc1-4def-a8d6-8b81fa1f5477 FullName +``` + +Those are the model's own `GUID`s. `Account` stores +`ea 49 6e b1 df 91 aa 4c ae d8 6b a4 c4 e1 33 c5`, which is +`b16e49ea-91df-4caa-aed8-6ba4c4e133c5` once the .NET field order is undone — and +`FullName`'s decodes to `aac00d66-…` likewise. The mapping is byte-identical, not +merely correlated. It is also the same `b16e49ea…` recorded in §4 on a different +project at a different Mendix version, because the `GUID` is a property of the +*published module*, stable across every project that installs it. + +**Changing only the `GUID` destroys the data.** + +| Run | Model change | `mendixsystem$entity.id` | `administration$account` | +|---|---|---|---| +| 1 | — (baseline) | `b16e49ea-…` | 1 row inserted | +| 2 | `GUID` → `a0a1a2…` | `a3a2a1a0-…` | **0 rows** | +| 3 | `GUID` restored | `b16e49ea-…` | table recreated, empty | +| 4 | none (control) | `b16e49ea-…` | **row survives** | + +Run 2 changed nothing else — same entity name, same table name, same attributes — +and the runtime treated it as a different entity. Run 4 is the control that makes +run 2 readable: an unchanged reboot preserves the row, so the loss was caused by +the identity change and not by restarting. + +**What this settles, and what it does not.** Studio Pro's update preserves exactly +the identity the database keys on, so `$ID` renumbering — all 94 of them — is +irrelevant to data safety. "A `GUID`-preserving replace is data-safe" is now a +measured claim at the level of entity and attribute identity. + +It does **not** say the upgrade is harmless. An element the new version deletes +still loses its column or table, which is a schema decision rather than an +identity failure, and §4's destroyed local edit is untouched by any of this. Nor +was the generated DDL read statement-by-statement: the runtime logs the count +(596 commands cold, 38 on the identity change) but not the text at INFO. The +outcome was measured instead, which is the stronger evidence for the question +asked. + ## Implementation Plan Phase 1 is the whole of this proposal; phase 2 is named only to show where it leads. ### Phase 1 — `marketplace diff` (read-only) +§7 measured the risk this phase actually carries — DESCRIBE coverage — at 98% of the +documents in a seven-module marketplace project, so the design below is viable as +written. Three prerequisites fall out of that measurement, in priority order: + +1. ~~**Report module security as unknown** (or close the gap).~~ **Closed.** The gap + was narrower than first recorded — only the module role list, not module security + as a whole; see the correction above. `DESCRIBE MODULE` now emits roles. +2. ~~**Fix import/export mapping describe.**~~ **Closed** — the cause was + `moduleNameFor` reading a unit's direct container instead of walking to the + enclosing module, so every foldered document missed. +3. ~~**Distinguish page templates from pages**, or accept and document the + conflation.~~ **Closed 2026-08-11**, and it was not "the least severe" as + recorded — see the corrected finding above. `Forms$Page` being a prefix of + `Forms$PageTemplate` fed 46 templates into a prefix-matched page query; they + then described as pages with an empty body, so the differ judged them + unchanged without reading them. Templates are now their own catalog type and + report as unknown. + +Also closed since: the bare-DESCRIBE auto-detect gap (43 documents), and menu +documents, which had no DESCRIBE at all and now have full CRUD. Bare-DESCRIBE +coverage over the fixture is 251/251. + +**Phase 1 is therefore unblocked.** + | File | Change | |------|--------| -| `cmd/mxcli/cmd_marketplace.go` | New `diff` subcommand: flags `--to`, `--format`, module-name resolution | -| `cmd/mxcli/marketplace/compare.go` *(new)* | Orchestration: resolve installed version → download → convert → describe both sides → report | -| `cmd/mxcli/marketplace/scratch.go` *(new)* | Build the scratch project at the consuming project's version; wraps `mx convert` on the `.mpk` | -| `mdl/executor/` (describe paths) | Expose a programmatic "describe this element" entry point; today DESCRIBE is reachable only as a statement | -| `mdl/backend/` | Interface method to enumerate a module's elements with name + `$Type` (the catalog has this; it needs a backend-level accessor) | -| `docs-site/src/` | User-facing page for the command | +| `cmd/mxcli/cmd_marketplace_diff.go` *(new)* | The `diff` subcommand: flags `--to`, `--module`, `--json`; module + version resolution | +| `cmd/mxcli/marketplace/snapshot.go` *(new)* | Enumerate a module's elements from the catalog and capture DESCRIBE output for each | +| `cmd/mxcli/marketplace/compare.go` *(new)* | Match two snapshots by name+type and classify each element | +| `cmd/mxcli/marketplace/scratch.go` *(new)* | Build the reference project at the consuming project's version and import the `.mpk` into it | +| `cmd/mxcli/marketplace/report.go` *(new)* | Human and JSON rendering, including the honesty rule | +| `internal/marketplace/client.go` | Paginate `Versions` (see below) | +| `docs-site/src/guides/marketplace.md` | User-facing documentation for the command | + +Two things the plan above got wrong, both found by running it: + +- **The scratch project is built with `mx create-project` + `mx module-import`, + not `mx convert` on the `.mpk`.** A blank project is not empty — it already + ships Administration, Atlas_Core, DataWidgets and friends — so the template's + copy is dropped through mxcli's own `DROP MODULE` before the package is + imported, or `module-import` refuses the name (exit 47). +- **No new backend or executor entry point was needed.** The catalog's `objects` + view already enumerates a module's elements with name + type, and DESCRIBE is + reachable programmatically by executing an `ast.DescribeStmt` against an + executor writing to a buffer. Adding an interface method would have been a + second way to do the same thing. + +**How the module and its version are identified.** Each installed module records +`AppStoreGuid`, and that GUID is the marketplace **version** UUID — a blank +11.12.1 project carries `2059615c-…` for Administration, which is exactly +content 23513's version 4.3.2, and `225ac9cf-…` for DataWidgets, content +116540's version 3.5.0. Matching on it identifies both the module and the exact +release with no network call and no guessing at the listing name (content 23513 +is listed as "Administration module" and installs `Administration`). + +Matching on the version *number* instead looks equivalent and is not: that same +blank project has Atlas_Web_Content at 4.1.0 and Administration's content has +also published a 4.1.0, so a number match selects two modules and cannot tell +them apart. This was not reasoned out — it was the first real run of the +command, which refused rather than guessing. + +**A latent bug this surfaced.** `/v1/content/{id}/versions` pages: it returns 10 +versions unpaged and caps `limit` at 20. `Client.Versions` asked once, so +`marketplace versions` showed exactly ten of everything and an older installed +version looked unpublished — Data Widgets has 131 releases and mxcli could see +10. Fixed by walking pages until one comes back short; `marketplace +versions`/`download`/`install` all benefit. ### Phase 2 — `marketplace update` (deferred, not proposed here) @@ -473,6 +690,34 @@ The control from §3 is the primary test and it is fully reproducible: integration test under `-tags integration` because it shells out to `mx convert` and the marketplace API. +### What has run (2026-08-11) + +Both controls pass, against real marketplace content rather than a fixture. + +- **Negative control.** `marketplace diff 23513 -p `: + *"No local modifications: 21 of 21 elements verified unchanged."* The + reference is downloaded, imported and described from scratch each run, so this + also demonstrates the build is reproducible — `TestPackageProject_ReferenceIsReproducibleAndDiffable` + asserts the same thing on two independently built references. +- **Positive control.** One added attribute + (`alter entity Administration.Account add attribute LocalNote: String(100)`) → + *"Locally modified (1 of 21 elements): changed ENTITY Account"*, and nothing + else. +- **Upgrade impact.** `--to 4.5.0` reports five elements touched by the author, + one of which (`ENTITY Account`) collides with the local edit. The control for + *that* is `--to 4.3.2` — upgrading to the version already installed — which + reports nothing touched, so the five are real author changes and not noise + from the reference-building path. +- **Coverage honesty** is unit-tested rather than measured, because the fixture + module has no un-describable element: `TestDiffResult_UnknownIsNeverACleanBillOfHealth` + asserts an unknown element never renders as a clean verification, and fails + with exactly the dangerous output ("No local modifications: 1 of 2 elements + verified unchanged") when the branch that distinguishes the two is removed. + +Not yet run: the recorded `01-before` → `02-after` Studio Pro update pair. The +renumbering assertion it exists to make is already covered in principle — the +comparison never reads an `$ID` — but the fixture remains the end-to-end proof. + ## Open Questions 1. **Scratch-project conversion cost.** Each diff runs `mx convert` on a package @@ -499,13 +744,10 @@ The control from §3 is the primary test and it is fully reproducible: tracked the update correctly (`v4.3.2` → `v4.5.0`). It is a read-only oracle and does not remove the need for a writer, but `diff` can use it to cross-check the version a project claims. -5. **What is `GUID` actually for?** *(new)* It is the one identity Studio Pro - preserves across a full renumber, which is strong circumstantial evidence it - carries the database mapping — but that is inference. The decisive check is to - build against a populated database before and after an update and compare the - generated DDL: additive means identity survived where it counts, a DROP/CREATE - on a module table means it did not. Until then, "the upgrade is data-safe" is a - model-level argument, not a measured one. +5. ~~**What is `GUID` actually for?**~~ **Answered by measurement 2026-08-11 — see + §8.** It is the database's identity for an entity and for each of its + attributes. Changing only an entity's `GUID`, with its name, table name and + attributes untouched, destroys its data. 6. **Does Studio Pro warn before discarding a local edit?** *(new)* §4 shows the edit is gone from the stored model, but a snapshot cannot see a dialog. This changes how the proposal should describe the status quo: "Studio Pro silently diff --git a/docs/11-proposals/data/marketplace-upgrade/GUID_IDENTITY.md b/docs/11-proposals/data/marketplace-upgrade/GUID_IDENTITY.md new file mode 100644 index 000000000..951e12274 --- /dev/null +++ b/docs/11-proposals/data/marketplace-upgrade/GUID_IDENTITY.md @@ -0,0 +1,83 @@ +# `GUID` is the database's identity — method and results + +Measurement for +[`PROPOSAL_marketplace_module_upgrade.md` §8](../../PROPOSAL_marketplace_module_upgrade.md), +run 2026-08-11 on Mendix **11.12.1**. + +The question: §4 showed Studio Pro's module update renumbers every `$ID` and +preserves every `GUID`. That makes `GUID` the only candidate carrier of database +identity — but it is inference until something is measured against a real +database. + +## Why this is measurable here and not in Studio Pro + +The lever is a single BSON value in the stored model, and mxcli can write one +where Studio Pro offers no such control. `GUID` is safe to edit in isolation +because **nothing points at it** — unlike an `$ID`, which is a pointer target and +cannot be rewritten without rewriting every reference in the same pass +(ADR-0008). + +## Setup + +```bash +mxcli run --local --setup --ensure-db -p E2E.mpr # local PostgreSQL + runtime +mxcli run --local -p E2E.mpr # boot; runtime creates the schema +``` + +Subject: a blank 11.12.1 app, `Administration` 4.3.2, entity `Account` +(persistent, so it gets a table). + +## The runtime writes its own identity map + +```sql +select id, entity_name, table_name from mendixsystem$entity + where entity_name = 'Administration.Account'; +-- b16e49ea-91df-4caa-aed8-6ba4c4e133c5 | Administration.Account | administration$account + +select a.id, a.attribute_name from mendixsystem$attribute a + join mendixsystem$entity e on a.entity_id = e.id + where e.entity_name = 'Administration.Account'; +-- aac00d66-7cc1-4def-a8d6-8b81fa1f5477 | FullName +-- f9c5f2aa-6ab9-4e62-9cbc-950405335377 | Email +``` + +Those are the model's own `GUID`s. `Account` stores the bytes +`ea 49 6e b1 df 91 aa 4c ae d8 6b a4 c4 e1 33 c5`; undoing the .NET field order +(first three groups little-endian) gives `b16e49ea-91df-4caa-aed8-6ba4c4e133c5`. +`FullName` decodes to `aac00d66-…` the same way. **Byte-identical, not +correlated.** + +Note `b16e49ea…` is also the `GUID` recorded in §4 — a different project, a +different Mendix version. The `GUID` belongs to the *published module*. + +## Runs + +| Run | Model change | `mendixsystem$entity.id` | `administration$account` | +|---|---|---|---| +| 1 | — (baseline) | `b16e49ea-…` | 1 row inserted | +| 2 | `GUID` → `a0a1a2a3…` | `a3a2a1a0-…` | **0 rows** | +| 3 | `GUID` restored | `b16e49ea-…` | table recreated, empty | +| 4 | none (**control**) | `b16e49ea-…` | **row survives** | + +Run 2 changed nothing else: same entity name, same table name, same attributes. +Run 4 is what makes run 2 readable — an unchanged reboot preserves the row, so +the loss came from the identity change and not from restarting. + +## Conclusion + +The database keys entities *and their attributes* on the model's `GUID`. Studio +Pro's update preserves exactly that, so `$ID` renumbering is irrelevant to data +safety, and a `GUID`-preserving replace is data-safe at the level of element +identity. + +## What this does not establish + +- **Not that the upgrade is harmless.** An element the new version deletes still + loses its column or table. That is a schema decision, not an identity failure — + and §4's silently destroyed local edit is untouched by any of this. +- **Not the DDL text.** The runtime logs the command count (596 cold, 38 on the + identity change) but not the statements at INFO level. The *outcome* was + measured instead, which answers the question asked more directly than the + statements would. +- **Not associations.** Association `GUID`s were observed in the model but their + join-table identity was not exercised. diff --git a/internal/marketplace/client.go b/internal/marketplace/client.go index 28ec62a97..5544c8eff 100644 --- a/internal/marketplace/client.go +++ b/internal/marketplace/client.go @@ -26,6 +26,14 @@ const ( // (End-of-catalog normally terminates the loop first: an offset past the // end returns a short/empty page.) maxSearchPages = 50 + // versionPageSize is the server-side cap on the /v1/content/{id}/versions + // `limit` parameter: asking for more than 20 still returns 20, and asking + // for nothing at all returns 10. + versionPageSize = 20 + // maxVersionPages bounds the version walk. The longest histories in the + // catalog today are well under a hundred releases, so this is a runaway + // guard rather than a real limit. + maxVersionPages = 25 // searchConcurrency bounds how many catalog pages are fetched in parallel // during a deep keyword scan (the first page is always fetched alone so a // common early match stays a single request). @@ -211,12 +219,28 @@ func (c *Client) Get(ctx context.Context, contentID int) (*Content, error) { // Versions returns all published versions for a content item, ordered // newest first (per the API). +// +// The endpoint pages: it returns 10 versions when asked for none in particular +// and caps `limit` at 20, so the versions a long-lived module was actually +// installed from fall off the end. Data Widgets has 40+ published versions and +// an unpaged call returns the newest 10 — which is why `marketplace versions` +// used to show exactly ten of everything, and why looking up an older installed +// version reported it as not published. Pages are walked until one comes back +// short. func (c *Client) Versions(ctx context.Context, contentID int) (*VersionList, error) { - var out VersionList - if err := c.get(ctx, fmt.Sprintf("/v1/content/%d/versions", contentID), &out); err != nil { - return nil, err + var all VersionList + for offset := 0; offset < versionPageSize*maxVersionPages; offset += versionPageSize { + var page VersionList + path := fmt.Sprintf("/v1/content/%d/versions?limit=%d&offset=%d", contentID, versionPageSize, offset) + if err := c.get(ctx, path, &page); err != nil { + return nil, err + } + all.Items = append(all.Items, page.Items...) + if len(page.Items) < versionPageSize { + break + } } - return &out, nil + return &all, nil } // Download streams the .mpk for the given version to dst and returns the diff --git a/internal/marketplace/client_test.go b/internal/marketplace/client_test.go index 2a2cee91f..037fc41ab 100644 --- a/internal/marketplace/client_test.go +++ b/internal/marketplace/client_test.go @@ -413,6 +413,68 @@ func TestVersions_ParsesList(t *testing.T) { } } +// TestVersions_WalksEveryPage guards the pagination fix. +// +// The live endpoint returns 10 versions unpaged and caps `limit` at 20, so a +// module with a long release history loses its older versions — including the +// one a project was actually installed from, which then looks unpublished. +// The mock reproduces that cap exactly: any limit above the page size is +// clamped, so a client that asks once and trusts the answer fails here. +func TestVersions_WalksEveryPage(t *testing.T) { + const total = 47 + var requests int + + client, _ := newMockServer(t, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/v1/content/170/versions" { + w.WriteHeader(http.StatusNotFound) + return + } + requests++ + offset, _ := strconv.Atoi(r.URL.Query().Get("offset")) + limit, _ := strconv.Atoi(r.URL.Query().Get("limit")) + if limit <= 0 || limit > versionPageSize { + limit = versionPageSize // the server's own cap + } + + var items []string + for i := offset; i < offset+limit && i < total; i++ { + items = append(items, fmt.Sprintf(`{"versionNumber":"1.0.%d","versionId":"id-%d"}`, i, i)) + } + _, _ = fmt.Fprintf(w, `{"items":[%s]}`, strings.Join(items, ",")) + }) + + got, err := client.Versions(context.Background(), 170) + if err != nil { + t.Fatal(err) + } + if len(got.Items) != total { + t.Fatalf("got %d versions over %d request(s), want all %d — older versions are being dropped", + len(got.Items), requests, total) + } + // Order must survive the walk: callers take the newest as "latest". + if got.Items[0].VersionNumber != "1.0.0" || got.Items[total-1].VersionNumber != fmt.Sprintf("1.0.%d", total-1) { + t.Errorf("pages were not concatenated in order: first=%q last=%q", + got.Items[0].VersionNumber, got.Items[total-1].VersionNumber) + } +} + +// TestVersions_StopsOnAShortPage checks the walk terminates on the common case +// of a content item with fewer versions than one page. +func TestVersions_StopsOnAShortPage(t *testing.T) { + var requests int + client, _ := newMockServer(t, func(w http.ResponseWriter, _ *http.Request) { + requests++ + _, _ = w.Write([]byte(sampleVersions)) + }) + + if _, err := client.Versions(context.Background(), 170); err != nil { + t.Fatal(err) + } + if requests != 1 { + t.Errorf("a short first page should end the walk; made %d requests", requests) + } +} + func TestGet_HTTPErrorIsReported(t *testing.T) { client, _ := newMockServer(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusNotFound) diff --git a/mdl-examples/bug-tests/queue-authoring-and-rewrite-guard.mdl b/mdl-examples/bug-tests/queue-authoring-and-rewrite-guard.mdl new file mode 100644 index 000000000..5887a3354 --- /dev/null +++ b/mdl-examples/bug-tests/queue-authoring-and-rewrite-guard.mdl @@ -0,0 +1,61 @@ +-- ============================================================================ +-- Task queues: authorable in MDL, and never silently dropped by a rewrite +-- ============================================================================ +-- +-- Symptom (measured on Mendix 11.13): +-- A microflow call bound to a task queue in Studio Pro stores a +-- Queues$QueueSettings node on the call. `create or replace microflow` rebuilt +-- the microflow and both engines wrote `QueueSettings: null` — the binding was +-- gone. Nothing reported it. Worse, `mx check` went from +-- [CE1613] "The selected task queue no longer exists" +-- to 0 errors, because mxcli had deleted the configuration that the error was +-- about. `describe microflow` never showed the binding either, so the loss was +-- invisible from every angle. +-- +-- Two root causes, two fixes: +-- 1. MDL could not author a queue at all, so a script could not restate one. +-- CREATE/DROP/SHOW/DESCRIBE QUEUE now exist. The BSON is pinned against the +-- four Studio Pro-authored queues in Mendix Business Events 3.12.1 +-- (Consumer_Queue, Consumer_Processor_Queue, Producer_Queue, +-- Outbox_Cleanup_Queue), which agree exactly: Config is a +-- Queues$BasicQueueConfig carrying ParallelismExpression as a STRING, with +-- the int32 `Parallelism` absent in all four. +-- 2. MDL still cannot author a *queued call*. Until it can, rewriting a +-- microflow that has one is refused rather than performed lossily +-- (guard-don't-drop, ADR-0005). +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/queue-authoring-and-rewrite-guard.mdl -p app.mpr +-- mx check app.mpr # 0 errors +-- mxcli -p app.mpr -c "list queues" +-- mxcli -p app.mpr -c "describe queue Ops.OrderProcessing" # re-executable MDL +-- ============================================================================ + +create module Ops; +/ +-- Parallelism is an EXPRESSION, not a number. A bare integer is the common case; +-- quote anything else. +create queue Ops.OrderProcessing ( + Parallelism: 3, + ClusterWide: true +); +/ +-- No properties: the defaults ("1", not cluster-wide) match a queue created in +-- Studio Pro with nothing changed. +create queue Ops.Mail; +/ +-- Re-running is idempotent with OR MODIFY, and the stored $ID is reused so any +-- call already bound to the queue keeps pointing at it. +create or modify queue Ops.OrderProcessing ( + Parallelism: 8, + ClusterWide: true +); +/ +-- "queue" is a plausible attribute name; adding the keyword must not take it +-- away (it is listed in the `keyword` rule, so it still parses as an identifier). +create entity Ops.Job ( + queue: String(100), + Status: String(50) +); +/ +drop queue Ops.Mail; diff --git a/mdl-examples/bug-tests/scheduled-event-repeat-fields.fail.mdl b/mdl-examples/bug-tests/scheduled-event-repeat-fields.fail.mdl new file mode 100644 index 000000000..d1c869094 --- /dev/null +++ b/mdl-examples/bug-tests/scheduled-event-repeat-fields.fail.mdl @@ -0,0 +1,25 @@ +-- Scheduled event: a field from the wrong Repeat must be rejected by `check`. +-- +-- NEGATIVE TEST (.fail.mdl) — EXPECTED to fail `mxcli check`. +-- `make check-mdl` inverts the exit code for .fail.mdl files: an unexpected +-- pass would be reported as a regression of the MDL-SCHED01 rule. +-- +-- The eight ScheduledEvents$*Schedule variants differ in WHICH fields they +-- carry, not just in their values. A DaySchedule has no Multiplier, so writing +-- one would put a property on the node that the type does not declare — the +-- shape that mxbuild accepts and Studio Pro cannot open +-- (System.InvalidOperationException at MprProperty). +-- +-- The rule runs in the no-project pass, so a plain `mxcli check` catches it — +-- it is decidable from the statement alone, and exec calls the same function, +-- so check and exec cannot drift. +-- +-- Correct form: drop Multiplier (Daily takes HourOfDay and MinuteOfHour), or +-- switch to a Repeat that has one. + +create scheduled event Ops.Bad ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + Multiplier: 3, + HourOfDay: 4 +); diff --git a/mdl-examples/doctype-tests/26-menu-examples.mdl b/mdl-examples/doctype-tests/26-menu-examples.mdl new file mode 100644 index 000000000..32e839450 --- /dev/null +++ b/mdl-examples/doctype-tests/26-menu-examples.mdl @@ -0,0 +1,114 @@ +-- ============================================================================ +-- Menu Examples - MDL Syntax +-- ============================================================================ +-- +-- A menu document (Menus$MenuDocument) is a standalone, reusable menu that a +-- menu widget points at. Atlas_Core ships two of them, Phone_Menu and +-- Tablet_Menu, and Studio Pro creates them via Add > Menu. +-- +-- It is NOT the menu inside a navigation profile. The two are easy to confuse +-- because both are built from the same Menus$MenuItem elements — which is why +-- the item syntax below is identical to the MENU (...) block of +-- CREATE NAVIGATION: +-- +-- create or modify menu MenuTest.Main_Menu ( ... ); -- this file +-- show navigation menu; -- profile menu +-- +-- This script is self-contained: it creates the module, and the page and +-- microflow the menu points at, so it runs against a fresh project. +-- ============================================================================ + +create module MenuTest; + +-- Targets for the menu items below. The page deliberately takes no parameters: +-- a menu item has nowhere to supply an argument, so opening a parameterised +-- page from one is rejected by Mendix with CE1571 (see Gotchas). +create page MenuTest.Landing +( + title: 'Landing', + layout: Atlas_Core.Atlas_Default, + url: 'menutest_landing' +) +{ + DYNAMICTEXT welcome (Content: 'Welcome', RenderMode: H2) +} + +create page MenuTest.Reports +( + title: 'Reports', + layout: Atlas_Core.Atlas_Default, + url: 'menutest_reports' +) +{ + DYNAMICTEXT heading (Content: 'Reports', RenderMode: H2) +} + +create microflow MenuTest.Rebuild () +begin +end; + +-- ---------------------------------------------------------------------------- +-- Create +-- ---------------------------------------------------------------------------- +-- Item forms: +-- * page target -> menu item '
' page +-- * microflow target -> menu item '' microflow +-- * no action -> menu item '' +-- * sub-menu -> menu '' ( ...nested items... ) +-- +-- ICON names an entry in an icon collection. It is optional on every form. +create menu MenuTest.Main_Menu ( + menu item 'Home' page MenuTest.Landing icon Atlas_Core.Atlas_Filled.home; + menu item 'Rebuild' microflow MenuTest.Rebuild; + menu 'Admin' ( + menu item 'Reports' page MenuTest.Reports; + ); + menu item 'Plain'; +); + +-- OR MODIFY replaces the item list wholesale, exactly like CREATE NAVIGATION: +-- the list given is the document's complete contents, so an omitted item is a +-- removed item. The document's identity and its export level are preserved, so +-- menu widgets pointing at it keep working. +create or modify menu MenuTest.Main_Menu ( + menu item 'Home' page MenuTest.Landing icon Atlas_Core.Atlas_Filled.home; + menu item 'Rebuild' microflow MenuTest.Rebuild; +); + +-- ---------------------------------------------------------------------------- +-- Describe +-- ---------------------------------------------------------------------------- +-- DESCRIBE emits a re-executable CREATE OR MODIFY statement, so +-- describe -> exec -> describe is a fixed point. +describe menu MenuTest.Main_Menu; + +-- The type is auto-detected too, as for any other document type. +describe MenuTest.Main_Menu; + +-- ---------------------------------------------------------------------------- +-- Drop +-- ---------------------------------------------------------------------------- +drop menu MenuTest.Main_Menu; + +-- ---------------------------------------------------------------------------- +-- Gotchas +-- ---------------------------------------------------------------------------- +-- +-- 1. A page with required parameters cannot be opened from a menu item without +-- supplying an argument. Mendix rejects it with CE1571 ("No argument has been +-- selected for parameter ..."), which `mx check` reports at "Menu item". +-- Point the item at a parameterless page, or open the page from a microflow. +-- +-- 2. Only Forms$IconCollectionIcon can be expressed by the ICON clause. A glyph +-- icon (numeric code) or an image icon is reported by DESCRIBE on its own +-- comment line rather than dropped silently: +-- +-- -- icon a numeric glyph code (Forms$GlyphIcon) is not reproducible by +-- -- CREATE MENU; set it in Studio Pro +-- +-- Re-running such output therefore loses that icon — visibly, not silently. +-- +-- 3. Authoring requires the default (modelsdk) engine. Under +-- MXCLI_ENGINE=legacy, create/modify/drop refuse rather than write a +-- differently-shaped document — which is why this script is skipped for the +-- legacy engine in the doctype gate. diff --git a/mdl-examples/doctype-tests/scheduled-events.mdl b/mdl-examples/doctype-tests/scheduled-events.mdl new file mode 100644 index 000000000..b4d3ed555 --- /dev/null +++ b/mdl-examples/doctype-tests/scheduled-events.mdl @@ -0,0 +1,133 @@ +-- ============================================================================ +-- Scheduled events — all eight repeat variants +-- ============================================================================ +-- +-- Mendix's cron. A scheduled event runs a microflow on a repeating schedule; +-- the repeat rule is a ScheduledEvents$Schedule child with eight variants that +-- differ in WHICH fields they carry, so MDL names the variant (Repeat) and then +-- takes only that variant's fields. A field from another variant is refused — +-- writing a merged field set produces a document mxbuild accepts and Studio Pro +-- cannot open. +-- +-- Scheduled events do NOT go through a task queue: OnOverlap +-- (DelayNext | SkipNext) is their own concurrency control. +-- +-- The document shape is pinned against four Studio Pro-authored events (Workflow +-- Commons 4.11.0, OIDC SSO 4.6.0, SAML 4.2.1 ×2). Those cover Daily and Hourly; +-- the other six variants are metamodel-derived and verified to load with 0 +-- errors from `mx check`. +-- +-- Verify: +-- mxcli exec mdl-examples/doctype-tests/scheduled-events.mdl -p app.mpr +-- mx check app.mpr # 0 errors +-- mxcli -p app.mpr -c "list scheduled events" +-- mxcli -p app.mpr -c "describe scheduled event Ops.NightlyCleanup" +-- ============================================================================ + +create module Ops; +/ +create microflow Ops.SE_Cleanup() +begin + return; +end; +/ +-- Daily: a fixed time of day, no multiplier (the variant has none). +create scheduled event Ops.NightlyCleanup ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + HourOfDay: 4, + MinuteOfHour: 0, + TimeZone: Server, + Enabled: true, + StartDateTime: '2026-01-01T04:00:00Z' +); +/ +-- Hourly: every N hours, at a fixed minute past the hour. +create scheduled event Ops.HourlyPing ( + Microflow: Ops.SE_Cleanup, + Repeat: Hourly, + Multiplier: 2, + MinuteOffset: 23 +); +/ +-- Minutely: the only field is the multiplier. +create scheduled event Ops.EveryFiveMinutes ( + Microflow: Ops.SE_Cleanup, + Repeat: Minutely, + Multiplier: 5 +); +/ +-- Weekly: a day list plus a time. Weekday names are case-insensitive. +create scheduled event Ops.WeeklyReport ( + Microflow: Ops.SE_Cleanup, + Repeat: Weekly, + Weekdays: 'Monday, Friday', + HourOfDay: 9, + MinuteOfHour: 30 +); +/ +-- MonthlyByDate: the Nth day of every M months. +create scheduled event Ops.MidMonth ( + Microflow: Ops.SE_Cleanup, + Repeat: MonthlyByDate, + Multiplier: 1, + DayOfMonth: 15, + HourOfDay: 2, + MinuteOfHour: 0 +); +/ +-- MonthlyByWeekday: "the last Friday of every third month". MonthOffset picks +-- which month of the three-month cycle fires (0-based). +create scheduled event Ops.QuarterEnd ( + Microflow: Ops.SE_Cleanup, + Repeat: MonthlyByWeekday, + Multiplier: 3, + MonthOffset: 2, + DaySelector: Last, + Weekday: Friday, + HourOfDay: 18, + MinuteOfHour: 0 +); +/ +-- YearlyByDate: a calendar date. Month is 1-12. +create scheduled event Ops.YearOpen ( + Microflow: Ops.SE_Cleanup, + Repeat: YearlyByDate, + Month: 1, + DayOfMonth: 2, + HourOfDay: 8, + MinuteOfHour: 0 +); +/ +-- YearlyByWeekday: "the first Monday of March". +create scheduled event Ops.YearWeek ( + Microflow: Ops.SE_Cleanup, + Repeat: YearlyByWeekday, + Month: 3, + DaySelector: First, + Weekday: Monday, + HourOfDay: 7, + MinuteOfHour: 15 +); +/ +-- Re-running with OR MODIFY reuses the stored $ID and preserves the legacy +-- Interval/IntervalType pair, which Studio Pro writes but does not keep in sync +-- with Schedule and MDL cannot author. +create or modify scheduled event Ops.NightlyCleanup ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + HourOfDay: 5, + MinuteOfHour: 0, + TimeZone: Server, + Enabled: true, + OnOverlap: SkipNext +); +/ +-- "scheduled" and "event" stay usable as ordinary names: both are in the +-- grammar's `keyword` rule, so adding the tokens did not take them away. +create entity Ops.Job ( + scheduled: Boolean, + event: String(50) +); +/ +drop scheduled event Ops.EveryFiveMinutes; diff --git a/mdl/ast/ast_navigation.go b/mdl/ast/ast_navigation.go index 7ce9c6715..20232428d 100644 --- a/mdl/ast/ast_navigation.go +++ b/mdl/ast/ast_navigation.go @@ -31,3 +31,25 @@ type NavMenuItemDef struct { Icon string // ICON 'Module.Collection.name', empty for none Items []NavMenuItemDef // Sub-items (for MENU 'caption' (...)) } + +// CreateMenuStmt is `create [or modify] menu Module.Name ( )` — a +// standalone Menus$MenuDocument, not the menu inside a navigation profile. +// Items reuse NavMenuItemDef so both constructs share one item syntax. +// +// Like CREATE NAVIGATION, this is a full replacement: the item list given is the +// document's complete contents, so an omitted item is a removed item. +type CreateMenuStmt struct { + Name QualifiedName + Items []NavMenuItemDef + CreateOrModify bool // CREATE OR MODIFY / OR REPLACE + Documentation string +} + +func (s *CreateMenuStmt) isStatement() {} + +// DropMenuStmt is `drop menu Module.Name`. +type DropMenuStmt struct { + Name QualifiedName +} + +func (s *DropMenuStmt) isStatement() {} diff --git a/mdl/ast/ast_query.go b/mdl/ast/ast_query.go index d2b80be1f..cb8300007 100644 --- a/mdl/ast/ast_query.go +++ b/mdl/ast/ast_query.go @@ -330,6 +330,9 @@ const ( DescribeConsumedMCPService // DESCRIBE CONSUMED MCP SERVICE Module.Name (agent-editor MCP document) DescribeJarDependency // DESCRIBE JAR DEPENDENCY ModuleName 'group:artifact' DescribeBuildingBlock // DESCRIBE BUILDING BLOCK Module.Name + DescribeMenu // DESCRIBE MENU Module.Name (standalone Menus$MenuDocument) + DescribeQueue // DESCRIBE QUEUE Module.Name + DescribeScheduledEvent // DESCRIBE SCHEDULED EVENT Module.Name DescribeAuto // DESCRIBE Module.Name — type auto-detected at execution time ) @@ -418,6 +421,8 @@ func (t DescribeObjectType) String() string { return "JAR DEPENDENCY" case DescribeBuildingBlock: return "BUILDING BLOCK" + case DescribeMenu: + return "MENU" case DescribeAuto: return "AUTO" default: diff --git a/mdl/ast/ast_queue.go b/mdl/ast/ast_queue.go new file mode 100644 index 000000000..1ad13ee0b --- /dev/null +++ b/mdl/ast/ast_queue.go @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: Apache-2.0 + +package ast + +// CreateQueueStmt represents: +// +// CREATE [OR REPLACE|MODIFY] QUEUE Module.Name ( Parallelism: 3, ClusterWide: true ); +type CreateQueueStmt struct { + Name QualifiedName + Documentation string + // Parallelism is kept as written. Mendix stores it as an expression string + // (Queues$BasicQueueConfig.ParallelismExpression), so `3` and `'3'` are the + // same thing and an arbitrary expression is legal. + Parallelism string + ClusterWide bool + ExportLevel string + // CreateOrModify is set by the shared CREATE OR REPLACE/MODIFY prefix. + CreateOrModify bool +} + +func (s *CreateQueueStmt) isStatement() {} + +// DropQueueStmt represents: DROP QUEUE Module.Name; +type DropQueueStmt struct { + Name QualifiedName +} + +func (s *DropQueueStmt) isStatement() {} + +// ShowQueuesStmt represents: SHOW|LIST QUEUES [IN Module]; +type ShowQueuesStmt struct { + Module string +} + +func (s *ShowQueuesStmt) isStatement() {} + +// DescribeQueueStmt represents: DESCRIBE QUEUE Module.Name; +type DescribeQueueStmt struct { + Name QualifiedName +} + +func (s *DescribeQueueStmt) isStatement() {} diff --git a/mdl/ast/ast_scheduledevent.go b/mdl/ast/ast_scheduledevent.go new file mode 100644 index 000000000..7e97bf0ca --- /dev/null +++ b/mdl/ast/ast_scheduledevent.go @@ -0,0 +1,75 @@ +// SPDX-License-Identifier: Apache-2.0 + +package ast + +// CreateScheduledEventStmt represents: +// +// CREATE [OR REPLACE|MODIFY] SCHEDULED EVENT Module.Name ( +// Microflow: Module.MF, Repeat: Daily, HourOfDay: 4, ... +// ); +// +// Properties are kept as written and validated by the executor, which knows +// which fields the chosen Repeat actually has. Numeric fields are pointers so +// "not mentioned" stays distinguishable from "mentioned as 0" — 0 is a real +// hour, minute and month offset. +type CreateScheduledEventStmt struct { + Name QualifiedName + Documentation string + // Microflow is the qualified name of the microflow to run. + Microflow string + // Repeat names the schedule variant: Minutely, Hourly, Daily, Weekly, + // MonthlyByDate, MonthlyByWeekday, YearlyByDate, YearlyByWeekday. + Repeat string + + Multiplier *int + MinuteOffset *int + MonthOffset *int + HourOfDay *int + MinuteOfHour *int + DayOfMonth *int + Month *int + + // Weekdays is the day list of a Weekly repeat, as written + // ("Monday, Friday"); DaySelector/Weekday are the two fields of the + // ByWeekday repeats. + Weekdays string + DaySelector string + Weekday string + + // StartDateTime is an RFC 3339 timestamp; TimeZone is UTC or Server. + StartDateTime string + TimeZone string + // OnOverlap is DelayNext or SkipNext — what happens when a run is still + // going when the next one is due. + OnOverlap string + + Enabled *bool + Excluded *bool + ExportLevel string + + // CreateOrModify is set by the shared CREATE OR REPLACE/MODIFY prefix. + CreateOrModify bool +} + +func (s *CreateScheduledEventStmt) isStatement() {} + +// DropScheduledEventStmt represents: DROP SCHEDULED EVENT Module.Name; +type DropScheduledEventStmt struct { + Name QualifiedName +} + +func (s *DropScheduledEventStmt) isStatement() {} + +// ShowScheduledEventsStmt represents: SHOW|LIST SCHEDULED EVENTS [IN Module]; +type ShowScheduledEventsStmt struct { + Module string +} + +func (s *ShowScheduledEventsStmt) isStatement() {} + +// DescribeScheduledEventStmt represents: DESCRIBE SCHEDULED EVENT Module.Name; +type DescribeScheduledEventStmt struct { + Name QualifiedName +} + +func (s *DescribeScheduledEventStmt) isStatement() {} diff --git a/mdl/backend/backend.go b/mdl/backend/backend.go index 69bf47719..c5ebf95d2 100644 --- a/mdl/backend/backend.go +++ b/mdl/backend/backend.go @@ -26,6 +26,7 @@ type FullBackend interface { WorkflowBackend SettingsBackend ImageBackend + QueueBackend ScheduledEventBackend RenameBackend RawUnitBackend diff --git a/mdl/backend/infrastructure.go b/mdl/backend/infrastructure.go index 254b6366e..ff88a0ddb 100644 --- a/mdl/backend/infrastructure.go +++ b/mdl/backend/infrastructure.go @@ -84,8 +84,19 @@ type ImageBackend interface { ListIconCollections() ([]*types.IconCollection, error) } +// QueueBackend provides task queue (Queues$Queue) operations. +type QueueBackend interface { + ListQueues() ([]*types.Queue, error) + CreateQueue(q *types.Queue) error + UpdateQueue(q *types.Queue) error + DeleteQueue(id string) error +} + // ScheduledEventBackend provides scheduled event operations. type ScheduledEventBackend interface { ListScheduledEvents() ([]*model.ScheduledEvent, error) GetScheduledEvent(id model.ID) (*model.ScheduledEvent, error) + CreateScheduledEvent(ev *model.ScheduledEvent) error + UpdateScheduledEvent(ev *model.ScheduledEvent) error + DeleteScheduledEvent(id string) error } diff --git a/mdl/backend/mcp/unsupported_gen.go b/mdl/backend/mcp/unsupported_gen.go index 637e5e5d6..26ea73661 100644 --- a/mdl/backend/mcp/unsupported_gen.go +++ b/mdl/backend/mcp/unsupported_gen.go @@ -184,6 +184,11 @@ func (unsupportedBackend) CreateLayout(_ *pages.Layout) (err0 error) { return } +func (unsupportedBackend) CreateMenuDocument(_ *types.MenuDocument) (err0 error) { + err0 = errUnsupported("CreateMenuDocument") + return +} + func (unsupportedBackend) CreateMicroflow(_ *microflows.Microflow) (err0 error) { err0 = errUnsupported("CreateMicroflow") return @@ -214,6 +219,16 @@ func (unsupportedBackend) CreatePublishedRestService(_ *model.PublishedRestServi return } +func (unsupportedBackend) CreateQueue(_ *types.Queue) (err0 error) { + err0 = errUnsupported("CreateQueue") + return +} + +func (unsupportedBackend) CreateScheduledEvent(_ *model.ScheduledEvent) (err0 error) { + err0 = errUnsupported("CreateScheduledEvent") + return +} + func (unsupportedBackend) CreateSnippet(_ *pages.Snippet) (err0 error) { err0 = errUnsupported("CreateSnippet") return @@ -354,6 +369,11 @@ func (unsupportedBackend) DeleteLayout(_ model.ID) (err0 error) { return } +func (unsupportedBackend) DeleteMenuDocument(_ model.ID) (err0 error) { + err0 = errUnsupported("DeleteMenuDocument") + return +} + func (unsupportedBackend) DeleteMicroflow(_ model.ID) (err0 error) { err0 = errUnsupported("DeleteMicroflow") return @@ -389,6 +409,16 @@ func (unsupportedBackend) DeletePublishedRestService(_ model.ID) (err0 error) { return } +func (unsupportedBackend) DeleteQueue(_ string) (err0 error) { + err0 = errUnsupported("DeleteQueue") + return +} + +func (unsupportedBackend) DeleteScheduledEvent(_ string) (err0 error) { + err0 = errUnsupported("DeleteScheduledEvent") + return +} + func (unsupportedBackend) DeleteSnippet(_ model.ID) (err0 error) { err0 = errUnsupported("DeleteSnippet") return @@ -484,6 +514,11 @@ func (unsupportedBackend) GetMendixVersion() (r0 string, err1 error) { return } +func (unsupportedBackend) GetMenuDocumentByQualifiedName(_ string, _ string) (r0 *types.MenuDocument, err1 error) { + err1 = errUnsupported("GetMenuDocumentByQualifiedName") + return +} + func (unsupportedBackend) GetMicroflow(_ model.ID) (r0 *microflows.Microflow, err1 error) { err1 = errUnsupported("GetMicroflow") return @@ -707,6 +742,11 @@ func (unsupportedBackend) ListLayouts() (r0 []*pages.Layout, err1 error) { return } +func (unsupportedBackend) ListMenuDocuments() (r0 []*types.MenuDocument, err1 error) { + err1 = errUnsupported("ListMenuDocuments") + return +} + func (unsupportedBackend) ListMicroflows() (r0 []*microflows.Microflow, err1 error) { err1 = errUnsupported("ListMicroflows") return @@ -757,6 +797,11 @@ func (unsupportedBackend) ListPublishedRestServices() (r0 []*model.PublishedRest return } +func (unsupportedBackend) ListQueues() (r0 []*types.Queue, err1 error) { + err1 = errUnsupported("ListQueues") + return +} + func (unsupportedBackend) ListRawUnits(_ string) (r0 []*types.RawUnitInfo, err1 error) { err1 = errUnsupported("ListRawUnits") return @@ -1122,6 +1167,11 @@ func (unsupportedBackend) UpdateLayout(_ *pages.Layout) (err0 error) { return } +func (unsupportedBackend) UpdateMenuDocument(_ *types.MenuDocument) (err0 error) { + err0 = errUnsupported("UpdateMenuDocument") + return +} + func (unsupportedBackend) UpdateMicroflow(_ *microflows.Microflow) (err0 error) { err0 = errUnsupported("UpdateMicroflow") return @@ -1182,11 +1232,21 @@ func (unsupportedBackend) UpdateQualifiedNameInAllUnits(_ string, _ string) (r0 return } +func (unsupportedBackend) UpdateQueue(_ *types.Queue) (err0 error) { + err0 = errUnsupported("UpdateQueue") + return +} + func (unsupportedBackend) UpdateRawUnit(_ string, _ []uint8) (err0 error) { err0 = errUnsupported("UpdateRawUnit") return } +func (unsupportedBackend) UpdateScheduledEvent(_ *model.ScheduledEvent) (err0 error) { + err0 = errUnsupported("UpdateScheduledEvent") + return +} + func (unsupportedBackend) UpdateSnippet(_ *pages.Snippet) (err0 error) { err0 = errUnsupported("UpdateSnippet") return diff --git a/mdl/backend/mock/backend.go b/mdl/backend/mock/backend.go index b2d9e8c62..754016e14 100644 --- a/mdl/backend/mock/backend.go +++ b/mdl/backend/mock/backend.go @@ -244,15 +244,30 @@ type MockBackend struct { UpdateProjectSettingsFunc func(ps *model.ProjectSettings) error // ImageBackend - ListImageCollectionsFunc func() ([]*types.ImageCollection, error) - ListIconCollectionsFunc func() ([]*types.IconCollection, error) - CreateImageCollectionFunc func(ic *types.ImageCollection) error - UpdateImageCollectionFunc func(ic *types.ImageCollection) error - DeleteImageCollectionFunc func(id string) error + ListImageCollectionsFunc func() ([]*types.ImageCollection, error) + ListIconCollectionsFunc func() ([]*types.IconCollection, error) + + ListMenuDocumentsFunc func() ([]*types.MenuDocument, error) + GetMenuDocumentByQualifiedNameFunc func(moduleName, name string) (*types.MenuDocument, error) + CreateMenuDocumentFunc func(md *types.MenuDocument) error + UpdateMenuDocumentFunc func(md *types.MenuDocument) error + DeleteMenuDocumentFunc func(id model.ID) error + CreateImageCollectionFunc func(ic *types.ImageCollection) error + UpdateImageCollectionFunc func(ic *types.ImageCollection) error + DeleteImageCollectionFunc func(id string) error + + // QueueBackend + ListQueuesFunc func() ([]*types.Queue, error) + CreateQueueFunc func(q *types.Queue) error + UpdateQueueFunc func(q *types.Queue) error + DeleteQueueFunc func(id string) error // ScheduledEventBackend - ListScheduledEventsFunc func() ([]*model.ScheduledEvent, error) - GetScheduledEventFunc func(id model.ID) (*model.ScheduledEvent, error) + ListScheduledEventsFunc func() ([]*model.ScheduledEvent, error) + GetScheduledEventFunc func(id model.ID) (*model.ScheduledEvent, error) + CreateScheduledEventFunc func(ev *model.ScheduledEvent) error + UpdateScheduledEventFunc func(ev *model.ScheduledEvent) error + DeleteScheduledEventFunc func(id string) error // RenameBackend UpdateQualifiedNameInAllUnitsFunc func(oldName, newName string) (int, error) diff --git a/mdl/backend/mock/mock_navigation.go b/mdl/backend/mock/mock_navigation.go index b0940d9a1..60962ee66 100644 --- a/mdl/backend/mock/mock_navigation.go +++ b/mdl/backend/mock/mock_navigation.go @@ -3,6 +3,8 @@ package mock import ( + "fmt" + "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" ) @@ -27,3 +29,38 @@ func (m *MockBackend) UpdateNavigationProfile(navDocID model.ID, profileName str } return nil } + +func (m *MockBackend) ListMenuDocuments() ([]*types.MenuDocument, error) { + if m.ListMenuDocumentsFunc != nil { + return m.ListMenuDocumentsFunc() + } + return nil, fmt.Errorf("MockBackend.ListMenuDocuments not configured") +} + +func (m *MockBackend) GetMenuDocumentByQualifiedName(moduleName, name string) (*types.MenuDocument, error) { + if m.GetMenuDocumentByQualifiedNameFunc != nil { + return m.GetMenuDocumentByQualifiedNameFunc(moduleName, name) + } + return nil, fmt.Errorf("MockBackend.GetMenuDocumentByQualifiedName not configured") +} + +func (m *MockBackend) CreateMenuDocument(md *types.MenuDocument) error { + if m.CreateMenuDocumentFunc != nil { + return m.CreateMenuDocumentFunc(md) + } + return fmt.Errorf("MockBackend.CreateMenuDocument not configured") +} + +func (m *MockBackend) UpdateMenuDocument(md *types.MenuDocument) error { + if m.UpdateMenuDocumentFunc != nil { + return m.UpdateMenuDocumentFunc(md) + } + return fmt.Errorf("MockBackend.UpdateMenuDocument not configured") +} + +func (m *MockBackend) DeleteMenuDocument(id model.ID) error { + if m.DeleteMenuDocumentFunc != nil { + return m.DeleteMenuDocumentFunc(id) + } + return fmt.Errorf("MockBackend.DeleteMenuDocument not configured") +} diff --git a/mdl/backend/mock/mock_queue.go b/mdl/backend/mock/mock_queue.go new file mode 100644 index 000000000..e57de049f --- /dev/null +++ b/mdl/backend/mock/mock_queue.go @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mock + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" +) + +// QueueBackend — task queues (Queues$Queue). +// +// The write methods default to a descriptive error rather than nil so a test +// that forgets to configure one fails on the missing stub instead of silently +// reporting a write that never happened. + +func (m *MockBackend) ListQueues() ([]*types.Queue, error) { + if m.ListQueuesFunc != nil { + return m.ListQueuesFunc() + } + return nil, nil +} + +func (m *MockBackend) CreateQueue(q *types.Queue) error { + if m.CreateQueueFunc != nil { + return m.CreateQueueFunc(q) + } + return fmt.Errorf("MockBackend.CreateQueue not configured") +} + +func (m *MockBackend) UpdateQueue(q *types.Queue) error { + if m.UpdateQueueFunc != nil { + return m.UpdateQueueFunc(q) + } + return fmt.Errorf("MockBackend.UpdateQueue not configured") +} + +func (m *MockBackend) DeleteQueue(id string) error { + if m.DeleteQueueFunc != nil { + return m.DeleteQueueFunc(id) + } + return fmt.Errorf("MockBackend.DeleteQueue not configured") +} diff --git a/mdl/backend/mock/mock_workflow.go b/mdl/backend/mock/mock_workflow.go index 84cf8464b..4d792482f 100644 --- a/mdl/backend/mock/mock_workflow.go +++ b/mdl/backend/mock/mock_workflow.go @@ -123,3 +123,28 @@ func (m *MockBackend) GetScheduledEvent(id model.ID) (*model.ScheduledEvent, err } return nil, nil } + +// The write methods default to a descriptive error rather than nil so a test +// that forgets to configure one fails on the missing stub instead of silently +// reporting a write that never happened. + +func (m *MockBackend) CreateScheduledEvent(ev *model.ScheduledEvent) error { + if m.CreateScheduledEventFunc != nil { + return m.CreateScheduledEventFunc(ev) + } + return fmt.Errorf("MockBackend.CreateScheduledEvent not configured") +} + +func (m *MockBackend) UpdateScheduledEvent(ev *model.ScheduledEvent) error { + if m.UpdateScheduledEventFunc != nil { + return m.UpdateScheduledEventFunc(ev) + } + return fmt.Errorf("MockBackend.UpdateScheduledEvent not configured") +} + +func (m *MockBackend) DeleteScheduledEvent(id string) error { + if m.DeleteScheduledEventFunc != nil { + return m.DeleteScheduledEventFunc(id) + } + return fmt.Errorf("MockBackend.DeleteScheduledEvent not configured") +} diff --git a/mdl/backend/modelsdk/constant.go b/mdl/backend/modelsdk/constant.go index b2cc74358..aa11a877a 100644 --- a/mdl/backend/modelsdk/constant.go +++ b/mdl/backend/modelsdk/constant.go @@ -3,9 +3,9 @@ package modelsdkbackend import ( + "github.com/mendixlabs/mxcli/modelsdk/element" genConst "github.com/mendixlabs/mxcli/modelsdk/gen/constants" genDT "github.com/mendixlabs/mxcli/modelsdk/gen/datatypes" - "github.com/mendixlabs/mxcli/modelsdk/element" "github.com/mendixlabs/mxcli/modelsdk/mprread" "github.com/mendixlabs/mxcli/model" diff --git a/mdl/backend/modelsdk/domainmodel_write.go b/mdl/backend/modelsdk/domainmodel_write.go index 23631d6be..51831ca6d 100644 --- a/mdl/backend/modelsdk/domainmodel_write.go +++ b/mdl/backend/modelsdk/domainmodel_write.go @@ -459,11 +459,22 @@ func (b *Backend) moduleNameFor(unitID model.ID) string { if err != nil { return "" } + parentOf := make(map[string]string, len(units)) for _, u := range units { - if u.ID == string(unitID) { - if mi, _ := b.reader.GetModule(u.ContainerID); mi != nil { - return mi.Name - } + parentOf[u.ID] = u.ContainerID + } + + // A document is not necessarily a direct child of its module — Studio Pro + // nests documents in folders, and folders in folders, which is the norm in + // marketplace modules. So walk the containment chain up to the first + // enclosing module rather than reading only the immediate container. + // The project root is its own container, so stop on a self-reference as + // well as on a missing entry. + seen := make(map[string]bool, 8) + for id := parentOf[string(unitID)]; id != "" && !seen[id]; id = parentOf[id] { + seen[id] = true + if mi, _ := b.reader.GetModule(id); mi != nil { + return mi.Name } } return "" diff --git a/mdl/backend/modelsdk/menu_read.go b/mdl/backend/modelsdk/menu_read.go new file mode 100644 index 000000000..768db8ebb --- /dev/null +++ b/mdl/backend/modelsdk/menu_read.go @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + genMenus "github.com/mendixlabs/mxcli/modelsdk/gen/menus" + "github.com/mendixlabs/mxcli/modelsdk/mprread" +) + +// ListMenuDocuments reads every standalone Menus$MenuDocument unit. +// +// A menu document's entries are the same Menus$MenuItem elements a navigation +// profile holds, so the recursive conversion reuses navMenuItemFromGen — which +// already handles the three icon variants and the client-action dispatch. The +// only structural difference is that a menu document wraps its entries in a +// Menus$MenuItemCollection rather than holding them directly. +func (b *Backend) ListMenuDocuments() ([]*types.MenuDocument, error) { + if b.reader == nil { + return nil, fmt.Errorf("ListMenuDocuments: not connected") + } + units, err := mprread.ListUnitsWithContainer[*genMenus.MenuDocument](b.reader) + if err != nil { + return nil, err + } + + out := make([]*types.MenuDocument, 0, len(units)) + for _, u := range units { + g := u.Element + md := &types.MenuDocument{ + ID: model.ID(g.ID()), + ContainerID: model.ID(u.ContainerID), + Name: g.Name(), + Documentation: g.Documentation(), + ExportLevel: g.ExportLevel(), + Excluded: g.Excluded(), + } + if coll, ok := g.ItemCollection().(*genMenus.MenuItemCollection); ok && coll != nil { + for _, el := range coll.ItemsItems() { + if item := navMenuItemFromGen(el); item != nil { + md.Items = append(md.Items, item) + } + } + } + out = append(out, md) + } + return out, nil +} + +// GetMenuDocumentByQualifiedName finds a menu document by module + name. +func (b *Backend) GetMenuDocumentByQualifiedName(moduleName, name string) (*types.MenuDocument, error) { + all, err := b.ListMenuDocuments() + if err != nil { + return nil, err + } + for _, md := range all { + if md.Name == name && b.moduleNameFor(md.ID) == moduleName { + return md, nil + } + } + return nil, fmt.Errorf("menu not found: %s.%s", moduleName, name) +} diff --git a/mdl/backend/modelsdk/menu_write.go b/mdl/backend/modelsdk/menu_write.go new file mode 100644 index 000000000..cc321887f --- /dev/null +++ b/mdl/backend/modelsdk/menu_write.go @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + "github.com/mendixlabs/mxcli/modelsdk/element" + genMenus "github.com/mendixlabs/mxcli/modelsdk/gen/menus" + genPages "github.com/mendixlabs/mxcli/modelsdk/gen/pages" + genTexts "github.com/mendixlabs/mxcli/modelsdk/gen/texts" + mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr" +) + +// menuDocumentType is the BSON storage name of a standalone menu document. +const menuDocumentType = "Menus$MenuDocument" + +// CreateMenuDocument writes a new Menus$MenuDocument unit. +// +// This goes through gen + codec rather than hand-built BSON, which is what makes +// the typed-array markers come out right: the codec emits the default marker 3 +// for a PartList, and Studio Pro's own menu documents carry 3 on both the +// collection's Items and each item's sub-Items (verified against Atlas_Core's +// Phone_Menu / Tablet_Menu). The navigation writers build their menu items by +// hand and use 1 — that difference is deliberate here and not copied. +func (b *Backend) CreateMenuDocument(md *types.MenuDocument) error { + if md == nil { + return fmt.Errorf("CreateMenuDocument: nil menu") + } + if b.writer == nil { + return fmt.Errorf("CreateMenuDocument: not connected for writing") + } + if md.ID == "" { + md.ID = model.ID(mmpr.GenerateID()) + } + contents, err := encodeMenuDocument(md) + if err != nil { + return fmt.Errorf("CreateMenuDocument: encode: %w", err) + } + return b.writer.InsertUnit(string(md.ID), string(md.ContainerID), "Documents", menuDocumentType, contents) +} + +// UpdateMenuDocument rebuilds a menu document and rewrites its unit. The document +// is small and rebuilt wholesale, matching how enumerations are updated. +func (b *Backend) UpdateMenuDocument(md *types.MenuDocument) error { + if md == nil { + return fmt.Errorf("UpdateMenuDocument: nil menu") + } + if b.writer == nil { + return fmt.Errorf("UpdateMenuDocument: not connected for writing") + } + if md.ID == "" { + return fmt.Errorf("UpdateMenuDocument: menu %q has no ID", md.Name) + } + contents, err := encodeMenuDocument(md) + if err != nil { + return fmt.Errorf("UpdateMenuDocument: encode: %w", err) + } + return b.writer.UpdateRawUnit(string(md.ID), contents) +} + +// DeleteMenuDocument removes a menu document unit by ID. +func (b *Backend) DeleteMenuDocument(id model.ID) error { + if b.writer == nil { + return fmt.Errorf("DeleteMenuDocument: not connected for writing") + } + return b.writer.DeleteUnit(string(id)) +} + +func encodeMenuDocument(md *types.MenuDocument) ([]byte, error) { + g := genMenus.NewMenuDocument() + g.SetID(element.ID(md.ID)) + g.SetName(md.Name) + g.SetDocumentation(md.Documentation) + g.SetExcluded(md.Excluded) + exportLevel := md.ExportLevel + if exportLevel == "" { + exportLevel = "Hidden" + } + g.SetExportLevel(exportLevel) + + coll := genMenus.NewMenuItemCollection() + coll.SetID(element.ID(mmpr.GenerateID())) + for _, item := range md.Items { + coll.AddItems(menuItemToGen(item)) + } + g.SetItemCollection(coll) + + return (&codec.Encoder{}).Encode(g) +} + +// menuItemToGen converts one semantic menu item (and its sub-items) to gen. It is +// the inverse of navMenuItemFromGen, and deliberately mirrors the same three +// concerns: caption, icon, action. +func menuItemToGen(item *types.NavMenuItem) element.Element { + g := genMenus.NewMenuItem() + g.SetID(element.ID(mmpr.GenerateID())) + g.SetCaption(menuCaptionToGen(item.Caption)) + g.SetIcon(menuIconToGen(item)) + g.SetAction(menuActionToGen(item)) + for _, sub := range item.Items { + g.AddItems(menuItemToGen(sub)) + } + return g +} + +// menuCaptionToGen builds the Texts$Text / Texts$Translation pair a caption is +// stored as. en_US matches what the navigation writers emit. +func menuCaptionToGen(caption string) element.Element { + t := genTexts.NewText() + t.SetID(element.ID(mmpr.GenerateID())) + tr := genTexts.NewTranslation() + tr.SetID(element.ID(mmpr.GenerateID())) + tr.SetLanguageCode("en_US") + tr.SetText(caption) + t.AddTranslations(tr) + return t +} + +// menuIconToGen emits only Forms$IconCollectionIcon, the one variant MDL can +// name. A glyph icon carries a numeric code and an image icon points into an +// image collection; neither is expressible in the ICON clause, so an item that +// had one keeps no icon rather than getting a wrong one. DESCRIBE flags those on +// the way out, so the loss is visible rather than silent. +func menuIconToGen(item *types.NavMenuItem) element.Element { + if item.Icon == "" { + return nil + } + icon := genPages.NewIconCollectionIcon() + icon.SetID(element.ID(mmpr.GenerateID())) + icon.SetImageQualifiedName(item.Icon) + return icon +} + +// menuActionToGen builds the item's client action. The gen type names are the SDK +// names; their storage names are what Mendix writes — PageClientAction is +// Forms$FormAction, NoClientAction is Forms$NoAction. +func menuActionToGen(item *types.NavMenuItem) element.Element { + switch { + case item.Page != "": + a := genPages.NewPageClientAction() + a.SetID(element.ID(mmpr.GenerateID())) + ps := genPages.NewPageSettings() + ps.SetID(element.ID(mmpr.GenerateID())) + ps.SetPageQualifiedName(item.Page) + a.SetPageSettings(ps) + return a + case item.Microflow != "": + a := genPages.NewMicroflowClientAction() + a.SetID(element.ID(mmpr.GenerateID())) + ms := genPages.NewMicroflowSettings() + ms.SetID(element.ID(mmpr.GenerateID())) + ms.SetMicroflowQualifiedName(item.Microflow) + a.SetMicroflowSettings(ms) + return a + default: + a := genPages.NewNoClientAction() + a.SetID(element.ID(mmpr.GenerateID())) + return a + } +} diff --git a/mdl/backend/modelsdk/module_resolution_test.go b/mdl/backend/modelsdk/module_resolution_test.go new file mode 100644 index 000000000..78b82cf87 --- /dev/null +++ b/mdl/backend/modelsdk/module_resolution_test.go @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import "testing" + +// Documents in the fixture that live inside a folder rather than directly under +// their module. Studio Pro puts most documents in folders, and marketplace +// modules almost always do (the FeedbackModule ships _Docs/, Pages/, Private/…), +// so folder nesting is the normal case rather than an edge case. +// +// moduleNameFor resolved a unit's module by reading its *direct* container, so +// any document one or more folders deep resolved to "" and every by-qualified- +// name lookup built on it reported "not found". `describe import mapping +// FeedbackModule.IMM_PostResponse` was the reported symptom; the same helper +// backs the CREATE OR MODIFY existence check and DROP for these types, so the +// failure mode there is worse — an existence check that answers "no" turns a +// modify into an attempted create. +func TestByQualifiedNameFindsDocumentsInsideFolders(t *testing.T) { + b := New() + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect(%s): %v", fixture, err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + t.Run("import mapping", func(t *testing.T) { + im, err := b.GetImportMappingByQualifiedName("FeedbackModule", "IMM_PostResponse") + if err != nil { + t.Fatalf("GetImportMappingByQualifiedName: %v", err) + } + if im.Name != "IMM_PostResponse" { + t.Errorf("Name = %q, want IMM_PostResponse", im.Name) + } + }) + + t.Run("export mapping", func(t *testing.T) { + em, err := b.GetExportMappingByQualifiedName("FeedbackModule", "EXM_PostFeedback") + if err != nil { + t.Fatalf("GetExportMappingByQualifiedName: %v", err) + } + if em.Name != "EXM_PostFeedback" { + t.Errorf("Name = %q, want EXM_PostFeedback", em.Name) + } + }) + + t.Run("json structure", func(t *testing.T) { + js, err := b.GetJsonStructureByQualifiedName("FeedbackModule", "JSON_AppInsightsRequest") + if err != nil { + t.Fatalf("GetJsonStructureByQualifiedName: %v", err) + } + if js.Name != "JSON_AppInsightsRequest" { + t.Errorf("Name = %q, want JSON_AppInsightsRequest", js.Name) + } + }) +} + +// TestByQualifiedNameRejectsWrongModule guards the other direction: walking up +// the container chain must stop at the first enclosing module, not resolve a +// name against any module that happens to contain a like-named document. +func TestByQualifiedNameRejectsWrongModule(t *testing.T) { + b := New() + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect(%s): %v", fixture, err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + if _, err := b.GetImportMappingByQualifiedName("Administration", "IMM_PostResponse"); err == nil { + t.Error("expected an error for a mapping that lives in a different module, got nil") + } +} + +// TestListMenuDocuments reads the standalone Menus$MenuDocument units from the +// vendored fixture. Atlas_Core ships Phone_Menu and Tablet_Menu, both foldered, +// so this also exercises the module-resolution walk above. +func TestListMenuDocuments(t *testing.T) { + b := New() + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect(%s): %v", fixture, err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + all, err := b.ListMenuDocuments() + if err != nil { + t.Fatalf("ListMenuDocuments: %v", err) + } + if len(all) != 2 { + t.Fatalf("got %d menu documents, want 2 (Phone_Menu, Tablet_Menu)", len(all)) + } + + md, err := b.GetMenuDocumentByQualifiedName("Atlas_Core", "Phone_Menu") + if err != nil { + t.Fatalf("GetMenuDocumentByQualifiedName: %v", err) + } + if len(md.Items) != 4 { + t.Fatalf("got %d top-level items, want 4", len(md.Items)) + } + // Items must carry their caption and icon, not just exist — an item parsed + // into an empty struct would still satisfy a count assertion. + first := md.Items[0] + if first.Caption != "Home" { + t.Errorf("first item caption = %q, want Home", first.Caption) + } + if first.Icon != "Atlas_Core.Atlas_Filled.home" { + t.Errorf("first item icon = %q, want Atlas_Core.Atlas_Filled.home", first.Icon) + } +} diff --git a/mdl/backend/modelsdk/page.go b/mdl/backend/modelsdk/page.go index d81f21b9c..f9e4d7f31 100644 --- a/mdl/backend/modelsdk/page.go +++ b/mdl/backend/modelsdk/page.go @@ -17,6 +17,11 @@ import ( // excluded, title, URL, parameter count — not the widget tree, so this stays // shallow. Widget-tree conversion (DESCRIBE PAGE / ALTER) is a later phase. +// ListPages returns pages only. Page templates are a separate document type with +// separate content (their widgets hang off LayoutCall, not FormCall) and are read +// by ListPageTemplates — this used to include them, to match legacy's +// prefix-matched `Forms$Page` query, which made `show modules` report 46 pages +// for a module with none. func (b *Backend) ListPages() ([]*pages.Page, error) { units, err := mprread.ListUnitsWithContainer[*genPg.Page](b.reader) if err != nil { @@ -26,26 +31,34 @@ func (b *Backend) ListPages() ([]*pages.Page, error) { for _, u := range units { out = append(out, pageFromGen(u.Element, u.ContainerID)) } - // Legacy ListPages calls listUnitsByType("Forms$Page"), which is - // prefix-matched and therefore also sweeps in Forms$PageTemplate units. - // Replicate that here so SHOW PAGES matches legacy. (The modelsdk reader - // is strict-typed, so we add templates explicitly.) - tmpls, err := mprread.ListUnitsWithContainer[*genPg.PageTemplate](b.reader) + return out, nil +} + +// ListPageTemplates reads page template units. Like ListPages this is a header +// read — name, module, documentation — matching what the legacy parser captures. +// The template's content is deliberately not converted: it hangs off LayoutCall +// rather than FormCall, nothing consumes it yet, and a half-converted tree would +// compare equal to a different one. +func (b *Backend) ListPageTemplates() ([]*pages.PageTemplate, error) { + units, err := mprread.ListUnitsWithContainer[*genPg.PageTemplate](b.reader) if err != nil { return nil, err } - for _, u := range tmpls { - out = append(out, pageTemplateAsPage(u.Element, u.ContainerID)) + out := make([]*pages.PageTemplate, 0, len(units)) + for _, u := range units { + out = append(out, pageTemplateFromGen(u.Element, u.ContainerID)) } return out, nil } -// pageTemplateAsPage adapts a page template into the Page shape SHOW PAGES -// expects. Templates carry only name + excluded; title/url/params stay zero, -// matching legacy's prefix-match behaviour. -func pageTemplateAsPage(pt *genPg.PageTemplate, containerID model.ID) *pages.Page { - out := &pages.Page{ContainerID: containerID, Name: pt.Name(), Excluded: pt.Excluded()} +func pageTemplateFromGen(pt *genPg.PageTemplate, containerID model.ID) *pages.PageTemplate { + out := &pages.PageTemplate{ + ContainerID: containerID, + Name: pt.Name(), + Documentation: pt.Documentation(), + } out.ID = model.ID(pt.ID()) + out.TypeName = "Forms$PageTemplate" return out } diff --git a/mdl/backend/modelsdk/page_test.go b/mdl/backend/modelsdk/page_test.go index 6ab5eef45..e29490d38 100644 --- a/mdl/backend/modelsdk/page_test.go +++ b/mdl/backend/modelsdk/page_test.go @@ -5,9 +5,14 @@ package modelsdkbackend import "testing" // TestReadSlice_Pages checks the page adapter: real pages carry a decoded title -// (requires the Texts$Text gen package to be registered) and the list includes -// page templates, matching legacy's prefix-matched ListPages. SHOW PAGES is -// cross-checked byte-for-byte against the legacy engine in the plan validation. +// (requires the Texts$Text gen package to be registered), and page templates are +// NOT in the list. +// +// The exclusion is the point. Both engines used to return pages + templates +// together, because legacy asked the reader for "Forms$Page" and that query was +// prefix-matched — `Forms$PageTemplate` starts with `Forms$Page`. The modelsdk +// backend then bolted templates on deliberately to match. The fixture's +// Atlas_Web_Content has 46 templates and no pages, and reported 46 pages. func TestReadSlice_Pages(t *testing.T) { b := New() if err := b.Connect(fixture); err != nil { @@ -19,24 +24,53 @@ func TestReadSlice_Pages(t *testing.T) { if err != nil { t.Fatalf("ListPages: %v", err) } - // 16 Forms$Page + 46 Forms$PageTemplate in the fixture. - if len(pgs) != 62 { - t.Fatalf("ListPages count = %d, want 62 (pages + templates)", len(pgs)) + // 16 Forms$Page in the fixture; the 46 Forms$PageTemplate units are not pages. + if len(pgs) != 16 { + t.Fatalf("ListPages count = %d, want 16 (pages only, no templates)", len(pgs)) } - var titled, template bool + var titled bool for _, p := range pgs { if p.Name == "Account_Edit" && p.Title != nil && p.Title.GetTranslation("en_US") == "Edit Account" { titled = true } if p.Name == "Blank" { // an Atlas page template - template = true + t.Errorf("page template %q is being returned as a page", p.Name) } } if !titled { t.Error("Account_Edit title not decoded as 'Edit Account' (Texts$Text registration?)") } - if !template { - t.Error("page templates not included in ListPages") +} + +// TestReadSlice_PageTemplates is the other half: templates are still readable, +// under their own type. Removing them from ListPages must not make them +// invisible — a document nothing can enumerate is worse than one filed wrong, +// because nothing reports its absence. +func TestReadSlice_PageTemplates(t *testing.T) { + b := New() + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + tmpls, err := b.ListPageTemplates() + if err != nil { + t.Fatalf("ListPageTemplates: %v", err) + } + if len(tmpls) != 46 { + t.Fatalf("ListPageTemplates count = %d, want 46", len(tmpls)) + } + var found bool + for _, pt := range tmpls { + if pt.Name == "Blank" { + found = true + if pt.ContainerID == "" { + t.Error("template has no container, so its module cannot be resolved") + } + } + } + if !found { + t.Error("the Blank page template was not read") } } diff --git a/mdl/backend/modelsdk/queue_write.go b/mdl/backend/modelsdk/queue_write.go new file mode 100644 index 000000000..9f31b5b93 --- /dev/null +++ b/mdl/backend/modelsdk/queue_write.go @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/bsonutil" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr" +) + +// queueTypeName is the BSON storage name for a task queue document. +const queueTypeName = "Queues$Queue" + +// ListQueues reads every Queues$Queue unit. +func (b *Backend) ListQueues() ([]*types.Queue, error) { + units, err := b.reader.ListRawUnitsByType(queueTypeName) + if err != nil { + return nil, err + } + out := make([]*types.Queue, 0, len(units)) + for _, u := range units { + var doc bson.M + if err := bson.Unmarshal(u.Contents, &doc); err != nil { + return nil, fmt.Errorf("unmarshal queue %s: %w", u.ID, err) + } + q := &types.Queue{ContainerID: model.ID(u.ContainerID)} + q.ID = model.ID(u.ID) + q.TypeName = queueTypeName + q.Name, _ = doc["Name"].(string) + q.Documentation, _ = doc["Documentation"].(string) + q.Excluded, _ = doc["Excluded"].(bool) + q.ExportLevel, _ = doc["ExportLevel"].(string) + // Parallelism and cluster scope live on the nested Config node. + if cfg, ok := doc["Config"].(bson.M); ok { + q.Parallelism, _ = cfg["ParallelismExpression"].(string) + q.ClusterWide, _ = cfg["ClusterWide"].(bool) + } + out = append(out, q) + } + return out, nil +} + +// CreateQueue inserts a new Queues$Queue document. +func (b *Backend) CreateQueue(q *types.Queue) error { + if q == nil { + return fmt.Errorf("CreateQueue: nil queue") + } + if b.writer == nil { + return fmt.Errorf("CreateQueue: not connected for writing") + } + if q.ID == "" { + q.ID = model.ID(mmpr.GenerateID()) + } + return b.writer.InsertUnit(string(q.ID), string(q.ContainerID), "Documents", queueTypeName, serializeQueue(q)) +} + +// UpdateQueue rewrites an existing queue in place. +func (b *Backend) UpdateQueue(q *types.Queue) error { + if q == nil { + return fmt.Errorf("UpdateQueue: nil queue") + } + if b.writer == nil { + return fmt.Errorf("UpdateQueue: not connected for writing") + } + return b.writer.UpdateRawUnit(string(q.ID), serializeQueue(q)) +} + +// DeleteQueue removes a queue unit by ID. +func (b *Backend) DeleteQueue(id string) error { + if b.writer == nil { + return fmt.Errorf("DeleteQueue: not connected for writing") + } + return b.writer.DeleteUnit(id) +} + +// serializeQueue writes the document in the shape Studio Pro produces. +// +// Keys are alphabetical, matching every other writer here and the observed +// documents. Two things are deliberate: +// +// - ParallelismExpression is a STRING. Queues$BasicQueueConfig also declares an +// int32 Parallelism, but Studio Pro wrote it in none of the four reference +// documents, so writing it would be inventing a property. +// - An empty Parallelism becomes "1", which is what every observed queue has +// and what Mendix treats as the default; an empty expression is not a +// meaningful queue configuration. +func serializeQueue(q *types.Queue) []byte { + parallelism := q.Parallelism + if parallelism == "" { + parallelism = "1" + } + exportLevel := q.ExportLevel + if exportLevel == "" { + exportLevel = "Hidden" + } + config := bson.D{ + {Key: "$ID", Value: bsonutil.IDToBsonBinary(mmpr.GenerateID())}, + {Key: "$Type", Value: "Queues$BasicQueueConfig"}, + {Key: "ClusterWide", Value: q.ClusterWide}, + {Key: "ParallelismExpression", Value: parallelism}, + } + doc := bson.D{ + {Key: "$ID", Value: bsonutil.IDToBsonBinary(string(q.ID))}, + {Key: "$Type", Value: queueTypeName}, + {Key: "Config", Value: config}, + {Key: "Documentation", Value: q.Documentation}, + {Key: "Excluded", Value: q.Excluded}, + {Key: "ExportLevel", Value: exportLevel}, + {Key: "Name", Value: q.Name}, + } + out, err := bson.Marshal(doc) + if err != nil { + // bson.Marshal on a fixed bson.D of primitives cannot fail; a nil return + // would be written as an empty unit, so surface it loudly instead. + panic(fmt.Sprintf("serializeQueue: marshal %q: %v", q.Name, err)) + } + return out +} diff --git a/mdl/backend/modelsdk/queue_write_test.go b/mdl/backend/modelsdk/queue_write_test.go new file mode 100644 index 000000000..29064256f --- /dev/null +++ b/mdl/backend/modelsdk/queue_write_test.go @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/types" +) + +// TestSerializeQueue_MatchesStudioProShape pins the document against four real +// Studio Pro-authored queues from the Mendix Business Events module +// (Consumer_Queue, Consumer_Processor_Queue, Producer_Queue, +// Outbox_Cleanup_Queue), which agree exactly on this shape. +// +// Two assertions are the whole point of the test: +// +// - ParallelismExpression is a STRING. Queues$BasicQueueConfig also declares an +// int32 `Parallelism`, and writing that instead (or as well) would be +// inventing a property Mendix does not write. +// - `Parallelism` must be ABSENT. It appeared in none of the four references. +func TestSerializeQueue_MatchesStudioProShape(t *testing.T) { + q := &types.Queue{Name: "OrderProcessing", Parallelism: "3", ClusterWide: true} + q.ID = "11111111-1111-1111-1111-111111111111" + + var doc bson.M + if err := bson.Unmarshal(serializeQueue(q), &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + + if got := doc["$Type"]; got != "Queues$Queue" { + t.Errorf("$Type = %v, want Queues$Queue", got) + } + if got := doc["Name"]; got != "OrderProcessing" { + t.Errorf("Name = %v", got) + } + if got := doc["ExportLevel"]; got != "Hidden" { + t.Errorf("ExportLevel = %v, want Hidden (every reference document uses it)", got) + } + if _, ok := doc["Excluded"].(bool); !ok { + t.Errorf("Excluded missing or not a bool: %v", doc["Excluded"]) + } + + cfg, ok := doc["Config"].(bson.M) + if !ok { + t.Fatalf("Config is not a document: %T", doc["Config"]) + } + if got := cfg["$Type"]; got != "Queues$BasicQueueConfig" { + t.Errorf("Config.$Type = %v", got) + } + if got, ok := cfg["ParallelismExpression"].(string); !ok || got != "3" { + t.Errorf("ParallelismExpression = %#v, want the string \"3\" — Mendix stores an expression, not a number", cfg["ParallelismExpression"]) + } + if _, present := cfg["Parallelism"]; present { + t.Error("Parallelism must not be written: Studio Pro wrote it in none of the four reference queues") + } + if got := cfg["ClusterWide"]; got != true { + t.Errorf("ClusterWide = %v, want true", got) + } +} + +// TestSerializeQueue_Defaults covers the empty case: every observed queue has a +// parallelism, so an unspecified one becomes "1" rather than an empty +// expression, which is not a meaningful configuration. +func TestSerializeQueue_Defaults(t *testing.T) { + q := &types.Queue{Name: "Plain"} + q.ID = "22222222-2222-2222-2222-222222222222" + + var doc bson.M + if err := bson.Unmarshal(serializeQueue(q), &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + cfg := doc["Config"].(bson.M) + if got := cfg["ParallelismExpression"]; got != "1" { + t.Errorf("default ParallelismExpression = %v, want \"1\"", got) + } + if got := cfg["ClusterWide"]; got != false { + t.Errorf("default ClusterWide = %v, want false", got) + } + if got := doc["ExportLevel"]; got != "Hidden" { + t.Errorf("default ExportLevel = %v, want Hidden", got) + } +} + +// TestQueueRoundTrip writes a queue and reads it back through the real backend. +func TestQueueRoundTrip(t *testing.T) { + proj := copyFixture(t) + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + mod, err := b.GetModuleByName("MyFirstModule") + if err != nil || mod == nil { + t.Fatalf("GetModuleByName: %v", err) + } + if err := b.CreateQueue(&types.Queue{ + ContainerID: mod.ID, Name: "ZzQueue", Parallelism: "5", ClusterWide: true, + }); err != nil { + t.Fatalf("CreateQueue: %v", err) + } + + b2 := New() + if err := b2.Connect(proj); err != nil { + t.Fatalf("reconnect: %v", err) + } + t.Cleanup(func() { _ = b2.Disconnect() }) + + queues, err := b2.ListQueues() + if err != nil { + t.Fatalf("ListQueues: %v", err) + } + for _, q := range queues { + if q.Name != "ZzQueue" { + continue + } + if q.Parallelism != "5" { + t.Errorf("Parallelism = %q, want 5", q.Parallelism) + } + if !q.ClusterWide { + t.Error("ClusterWide did not round-trip") + } + return + } + t.Fatalf("ZzQueue not found after create (got %d queues)", len(queues)) +} diff --git a/mdl/backend/modelsdk/scheduledevent_read.go b/mdl/backend/modelsdk/scheduledevent_read.go index 8bf928674..9d4e8727a 100644 --- a/mdl/backend/modelsdk/scheduledevent_read.go +++ b/mdl/backend/modelsdk/scheduledevent_read.go @@ -5,60 +5,88 @@ package modelsdkbackend import ( "fmt" - genSched "github.com/mendixlabs/mxcli/modelsdk/gen/scheduledevents" - "github.com/mendixlabs/mxcli/modelsdk/mprread" + "go.mongodb.org/mongo-driver/bson" + sched "github.com/mendixlabs/mxcli/mdl/scheduledevents" "github.com/mendixlabs/mxcli/model" + mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr" ) -// Codec-native scheduled-event read. Used by SHOW STRUCTURE (per-module counts) -// and the project tree; the modelsdk engine previously left these unimplemented, -// so scheduled events were silently undercounted (the callers swallow the -// not-implemented error). Unlike JavaScript actions, the gen codec decodes -// ScheduledEvent under the same storage keys the legacy parser uses (Name / -// Documentation / Microflow / Enabled / Interval / IntervalType), so gen -// accessors are correct here — no raw-key reads needed. +// Scheduled events. Used by SHOW STRUCTURE (per-module counts), the project +// tree, and SHOW/DESCRIBE/CREATE/DROP SCHEDULED EVENT. +// +// The document is read and written as raw BSON through mdl/scheduledevents +// rather than through modelsdk/gen, because gen's generated types disagree with +// what Studio Pro actually writes on two properties: every integer is stored as +// int64 while gen declares int32 (the mismatch behind issue #585), and +// StartDateTime is a BSON UTC datetime while gen declares a string. func (b *Backend) ListScheduledEvents() ([]*model.ScheduledEvent, error) { - units, err := mprread.ListUnitsWithContainer[*genSched.ScheduledEvent](b.reader) + units, err := b.reader.ListRawUnitsByType(sched.TypeName) if err != nil { return nil, err } out := make([]*model.ScheduledEvent, 0, len(units)) for _, u := range units { - out = append(out, scheduledEventFromGen(u.Element, u.ContainerID)) + var doc bson.M + if err := bson.Unmarshal(u.Contents, &doc); err != nil { + return nil, fmt.Errorf("unmarshal scheduled event %s: %w", u.ID, err) + } + out = append(out, sched.Parse(doc, model.ID(u.ID), model.ID(u.ContainerID))) } return out, nil } func (b *Backend) GetScheduledEvent(id model.ID) (*model.ScheduledEvent, error) { - units, err := mprread.ListUnitsWithContainer[*genSched.ScheduledEvent](b.reader) + events, err := b.ListScheduledEvents() if err != nil { return nil, err } - for _, u := range units { - if model.ID(u.Element.ID()) == id { - return scheduledEventFromGen(u.Element, u.ContainerID), nil + for _, ev := range events { + if ev.ID == id { + return ev, nil } } return nil, fmt.Errorf("scheduled event not found: %s", id) } -// scheduledEventFromGen converts a gen ScheduledEvent to the semantic type. Mirrors -// the legacy parseScheduledEvent field set: MicroflowID holds the by-name microflow -// reference (BSON "Microflow"), and Interval comes through the int32 decoder (which -// accepts the int64 Studio Pro actually writes — issue #585). -func scheduledEventFromGen(g *genSched.ScheduledEvent, containerID model.ID) *model.ScheduledEvent { - ev := &model.ScheduledEvent{ - ContainerID: containerID, - Name: g.Name(), - Documentation: g.Documentation(), - MicroflowID: model.ID(g.MicroflowQualifiedName()), - Interval: int(g.Interval()), - IntervalType: g.IntervalType(), - Enabled: g.Enabled(), +// CreateScheduledEvent inserts a new scheduled event document. +func (b *Backend) CreateScheduledEvent(ev *model.ScheduledEvent) error { + if ev == nil { + return fmt.Errorf("CreateScheduledEvent: nil event") + } + if b.writer == nil { + return fmt.Errorf("CreateScheduledEvent: not connected for writing") + } + if ev.ID == "" { + ev.ID = model.ID(mmpr.GenerateID()) + } + contents, err := sched.Serialize(ev) + if err != nil { + return err + } + return b.writer.InsertUnit(string(ev.ID), string(ev.ContainerID), "Documents", sched.TypeName, contents) +} + +// UpdateScheduledEvent rewrites an existing scheduled event in place. +func (b *Backend) UpdateScheduledEvent(ev *model.ScheduledEvent) error { + if ev == nil { + return fmt.Errorf("UpdateScheduledEvent: nil event") + } + if b.writer == nil { + return fmt.Errorf("UpdateScheduledEvent: not connected for writing") + } + contents, err := sched.Serialize(ev) + if err != nil { + return err + } + return b.writer.UpdateRawUnit(string(ev.ID), contents) +} + +// DeleteScheduledEvent removes a scheduled event unit by ID. +func (b *Backend) DeleteScheduledEvent(id string) error { + if b.writer == nil { + return fmt.Errorf("DeleteScheduledEvent: not connected for writing") } - ev.ID = model.ID(g.ID()) - ev.TypeName = "ScheduledEvents$ScheduledEvent" - return ev + return b.writer.DeleteUnit(id) } diff --git a/mdl/backend/modelsdk/scheduledevent_read_test.go b/mdl/backend/modelsdk/scheduledevent_read_test.go index bb97048f6..01ef70971 100644 --- a/mdl/backend/modelsdk/scheduledevent_read_test.go +++ b/mdl/backend/modelsdk/scheduledevent_read_test.go @@ -4,58 +4,11 @@ package modelsdkbackend import ( "testing" + "time" "github.com/mendixlabs/mxcli/model" - genSched "github.com/mendixlabs/mxcli/modelsdk/gen/scheduledevents" ) -// TestScheduledEventFromGen guards the gen→semantic mapping the modelsdk read -// relies on. (No committed fixture contains scheduled events, and modelsdk has no -// scheduled-event write path, so the converter is exercised directly; the -// List/Get plumbing reuses the ListUnitsWithContainer pattern covered by the -// page/java reads.) The key mappings: MicroflowID carries the by-name microflow -// reference (BSON "Microflow"), Interval narrows the int32 accessor to int. -func TestScheduledEventFromGen(t *testing.T) { - g := genSched.NewScheduledEvent() - g.SetID("evt-1") - g.SetName("SE_Cleanup") - g.SetDocumentation("nightly cleanup") - g.SetMicroflowQualifiedName("MyModule.DoCleanup") - g.SetInterval(86400) - g.SetIntervalType("Day") - g.SetEnabled(true) - - ev := scheduledEventFromGen(g, model.ID("mod-1")) - - if ev.ID != "evt-1" { - t.Errorf("ID = %q, want evt-1", ev.ID) - } - if ev.ContainerID != "mod-1" { - t.Errorf("ContainerID = %q, want mod-1", ev.ContainerID) - } - if ev.TypeName != "ScheduledEvents$ScheduledEvent" { - t.Errorf("TypeName = %q", ev.TypeName) - } - if ev.Name != "SE_Cleanup" { - t.Errorf("Name = %q, want SE_Cleanup", ev.Name) - } - if ev.Documentation != "nightly cleanup" { - t.Errorf("Documentation = %q", ev.Documentation) - } - if ev.MicroflowID != "MyModule.DoCleanup" { - t.Errorf("MicroflowID = %q, want MyModule.DoCleanup", ev.MicroflowID) - } - if ev.Interval != 86400 { - t.Errorf("Interval = %d, want 86400", ev.Interval) - } - if ev.IntervalType != "Day" { - t.Errorf("IntervalType = %q, want Day", ev.IntervalType) - } - if !ev.Enabled { - t.Error("Enabled = false, want true") - } -} - // TestListScheduledEvents_Empty confirms the read returns an empty (not error) // result on a project with no scheduled events — the minimal fixture — so SHOW // STRUCTURE no longer swallows a not-implemented error. @@ -74,3 +27,83 @@ func TestListScheduledEvents_Empty(t *testing.T) { t.Errorf("got %d scheduled events, want 0 (minimal fixture has none)", len(events)) } } + +// TestScheduledEventRoundTrip writes an event and reads it back through the real +// backend. A round trip (rather than a reader-only test on hand-written BSON) is +// what catches a reader keyed on different property names than the writer +// produces — the failure mode recorded for the workflow-activity reads. +func TestScheduledEventRoundTrip(t *testing.T) { + proj := copyFixture(t) + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + mod, err := b.GetModuleByName("MyFirstModule") + if err != nil || mod == nil { + t.Fatalf("GetModuleByName: %v", err) + } + start := time.Date(2026, 1, 1, 4, 0, 0, 0, time.UTC) + want := &model.ScheduledEvent{ + ContainerID: mod.ID, + Name: "ZzNightly", + Documentation: "nightly cleanup", + MicroflowID: "MyFirstModule.DoCleanup", + StartDateTime: &start, + TimeZone: "Server", + OnOverlap: "SkipNext", + Interval: 1, + IntervalType: "Day", + Enabled: true, + Schedule: &model.Schedule{ + Kind: model.ScheduleMonthWeekday, Multiplier: 3, MonthOffset: 2, + DaySelector: "Last", Weekday: "Friday", HourOfDay: 18, MinuteOfHour: 30, + }, + } + if err := b.CreateScheduledEvent(want); err != nil { + t.Fatalf("CreateScheduledEvent: %v", err) + } + + b2 := New() + if err := b2.Connect(proj); err != nil { + t.Fatalf("reconnect: %v", err) + } + t.Cleanup(func() { _ = b2.Disconnect() }) + + events, err := b2.ListScheduledEvents() + if err != nil { + t.Fatalf("ListScheduledEvents: %v", err) + } + for _, got := range events { + if got.Name != "ZzNightly" { + continue + } + if got.MicroflowID != want.MicroflowID { + t.Errorf("MicroflowID = %q", got.MicroflowID) + } + if got.Documentation != want.Documentation { + t.Errorf("Documentation = %q", got.Documentation) + } + if got.TimeZone != "Server" || got.OnOverlap != "SkipNext" { + t.Errorf("TimeZone/OnOverlap = %q/%q", got.TimeZone, got.OnOverlap) + } + if got.Interval != 1 || got.IntervalType != "Day" { + t.Errorf("legacy pair = %d/%q", got.Interval, got.IntervalType) + } + if !got.Enabled { + t.Error("Enabled did not round-trip") + } + if got.StartDateTime == nil || !got.StartDateTime.Equal(start) { + t.Errorf("StartDateTime = %v, want %v", got.StartDateTime, start) + } + if got.Schedule == nil { + t.Fatal("Schedule did not round-trip") + } + if *got.Schedule != *want.Schedule { + t.Errorf("Schedule = %+v, want %+v", *got.Schedule, *want.Schedule) + } + return + } + t.Fatalf("ZzNightly not found after create (got %d events)", len(events)) +} diff --git a/mdl/backend/modelsdk/unimplemented_gen.go b/mdl/backend/modelsdk/unimplemented_gen.go index ce24b4208..f3ceed68d 100644 --- a/mdl/backend/modelsdk/unimplemented_gen.go +++ b/mdl/backend/modelsdk/unimplemented_gen.go @@ -180,6 +180,14 @@ func (unimplemented) CreatePublishedRestService(_ *model.PublishedRestService) e return errUnimplemented("CreatePublishedRestService") } +func (unimplemented) CreateQueue(_ *types.Queue) error { + return errUnimplemented("CreateQueue") +} + +func (unimplemented) CreateScheduledEvent(_ *model.ScheduledEvent) error { + return errUnimplemented("CreateScheduledEvent") +} + func (unimplemented) CreateSnippet(_ *pages.Snippet) error { return errUnimplemented("CreateSnippet") } @@ -321,6 +329,14 @@ func (unimplemented) DeletePublishedRestService(_ model.ID) error { return errUnimplemented("DeletePublishedRestService") } +func (unimplemented) DeleteQueue(_ string) error { + return errUnimplemented("DeleteQueue") +} + +func (unimplemented) DeleteScheduledEvent(_ string) error { + return errUnimplemented("DeleteScheduledEvent") +} + func (unimplemented) DeleteSnippet(_ model.ID) error { return errUnimplemented("DeleteSnippet") } @@ -685,6 +701,11 @@ func (unimplemented) ListPublishedRestServices() ([]*model.PublishedRestService, return r0, errUnimplemented("ListPublishedRestServices") } +func (unimplemented) ListQueues() ([]*types.Queue, error) { + var r0 []*types.Queue + return r0, errUnimplemented("ListQueues") +} + func (unimplemented) ListRawUnits(_ string) ([]*types.RawUnitInfo, error) { var r0 []*types.RawUnitInfo return r0, errUnimplemented("ListRawUnits") @@ -1061,10 +1082,18 @@ func (unimplemented) UpdateQualifiedNameInAllUnits(_ string, _ string) (int, err return r0, errUnimplemented("UpdateQualifiedNameInAllUnits") } +func (unimplemented) UpdateQueue(_ *types.Queue) error { + return errUnimplemented("UpdateQueue") +} + func (unimplemented) UpdateRawUnit(_ string, _ []uint8) error { return errUnimplemented("UpdateRawUnit") } +func (unimplemented) UpdateScheduledEvent(_ *model.ScheduledEvent) error { + return errUnimplemented("UpdateScheduledEvent") +} + func (unimplemented) UpdateSnippet(_ *pages.Snippet) error { return errUnimplemented("UpdateSnippet") } diff --git a/mdl/backend/modelsdk/widget_write_navlist_test.go b/mdl/backend/modelsdk/widget_write_navlist_test.go index d75e8acb2..c74769c0e 100644 --- a/mdl/backend/modelsdk/widget_write_navlist_test.go +++ b/mdl/backend/modelsdk/widget_write_navlist_test.go @@ -14,7 +14,7 @@ import ( // TestNavListItemToGen_WritesNames guards ledger finding #24: the modelsdk // writer must emit the navigation item's Name and give the caption's generated // DynamicText a name — otherwise Studio Pro rejects the project with CE7247 -// "name cannot be empty" / CE0495 "duplicate name ''". +// "name cannot be empty" / CE0495 "duplicate name ”". func TestNavListItemToGen_WritesNames(t *testing.T) { item := &pages.NavigationListItem{ Name: "itemTransactions", diff --git a/mdl/backend/mpr/backend.go b/mdl/backend/mpr/backend.go index 4daeaa705..17ee83992 100644 --- a/mdl/backend/mpr/backend.go +++ b/mdl/backend/mpr/backend.go @@ -702,6 +702,23 @@ func (b *MprBackend) ListIconCollections() ([]*types.IconCollection, error) { return b.reader.ListIconCollections() } +// --------------------------------------------------------------------------- +// QueueBackend +// --------------------------------------------------------------------------- + +func (b *MprBackend) ListQueues() ([]*types.Queue, error) { + return b.reader.ListQueues() +} +func (b *MprBackend) CreateQueue(q *types.Queue) error { + return b.writer.CreateQueue(q) +} +func (b *MprBackend) UpdateQueue(q *types.Queue) error { + return b.writer.UpdateQueue(q) +} +func (b *MprBackend) DeleteQueue(id string) error { + return b.writer.DeleteQueue(id) +} + // --------------------------------------------------------------------------- // ScheduledEventBackend // --------------------------------------------------------------------------- @@ -712,6 +729,15 @@ func (b *MprBackend) ListScheduledEvents() ([]*model.ScheduledEvent, error) { func (b *MprBackend) GetScheduledEvent(id model.ID) (*model.ScheduledEvent, error) { return b.reader.GetScheduledEvent(id) } +func (b *MprBackend) CreateScheduledEvent(ev *model.ScheduledEvent) error { + return b.writer.CreateScheduledEvent(ev) +} +func (b *MprBackend) UpdateScheduledEvent(ev *model.ScheduledEvent) error { + return b.writer.UpdateScheduledEvent(ev) +} +func (b *MprBackend) DeleteScheduledEvent(id string) error { + return b.writer.DeleteScheduledEvent(id) +} // --------------------------------------------------------------------------- // RenameBackend @@ -871,3 +897,24 @@ func (b *MprBackend) useCallMicroflowActivityName() bool { func (b *MprBackend) SerializeWorkflowActivity(a workflows.WorkflowActivity) (any, error) { return mpr.SerializeWorkflowActivity(a, b.useCallMicroflowActivityName()), nil } + +func (b *MprBackend) ListMenuDocuments() ([]*types.MenuDocument, error) { + return b.reader.ListMenuDocuments() +} +func (b *MprBackend) GetMenuDocumentByQualifiedName(moduleName, name string) (*types.MenuDocument, error) { + return b.reader.GetMenuDocumentByQualifiedName(moduleName, name) +} + +// Menu-document writes are implemented on the modelsdk engine only. The legacy +// writer builds menu items by hand with typed-array marker 1, which does not +// match what Studio Pro stores in a menu document (3) — rather than ship a +// second, differently-shaped writer, this refuses so the caller is told plainly. +func (b *MprBackend) CreateMenuDocument(md *types.MenuDocument) error { + return errors.New("creating a menu requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} +func (b *MprBackend) UpdateMenuDocument(md *types.MenuDocument) error { + return errors.New("modifying a menu requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} +func (b *MprBackend) DeleteMenuDocument(id model.ID) error { + return errors.New("dropping a menu requires the modelsdk engine — rerun without MXCLI_ENGINE=legacy") +} diff --git a/mdl/backend/navigation.go b/mdl/backend/navigation.go index 3c47cc0de..0318e5651 100644 --- a/mdl/backend/navigation.go +++ b/mdl/backend/navigation.go @@ -12,4 +12,12 @@ type NavigationBackend interface { ListNavigationDocuments() ([]*types.NavigationDocument, error) GetNavigation() (*types.NavigationDocument, error) UpdateNavigationProfile(navDocID model.ID, profileName string, spec types.NavigationProfileSpec) error + + // Menu documents are standalone reusable menus (Menus$MenuDocument), not + // the menu embedded in a navigation profile. + ListMenuDocuments() ([]*types.MenuDocument, error) + GetMenuDocumentByQualifiedName(moduleName, name string) (*types.MenuDocument, error) + CreateMenuDocument(md *types.MenuDocument) error + UpdateMenuDocument(md *types.MenuDocument) error + DeleteMenuDocument(id model.ID) error } diff --git a/mdl/catalog/builder.go b/mdl/catalog/builder.go index d3aeafc87..ad27c1119 100644 --- a/mdl/catalog/builder.go +++ b/mdl/catalog/builder.go @@ -91,6 +91,11 @@ type Builder struct { resolution float64 // Leiden resolution for the graph-analysis pass describeFunc DescribeFunc + // Scheduled event → microflow edges, collected while cataloguing the events + // and emitted by buildReferences (a later pass). Carried on the Builder + // rather than re-queried because CatalogTx has no Query. + scheduledEventRefs []scheduledEventRef + // Built-in widget definitions supplied by the caller — used to populate // the widget_definitions catalog table alongside project widgets/. builtinWidgetMetas []WidgetDefinitionMeta @@ -398,10 +403,34 @@ func (b *Builder) Build(progress ProgressFunc) error { return fmt.Errorf("failed to build image collections: %w", err) } + if err := b.buildSimpleNamedDocs("CustomIcons$CustomIconCollection", "icon_collections", "Icon Collections"); err != nil { + return fmt.Errorf("failed to build icon collections: %w", err) + } + + if err := b.buildSimpleNamedDocs("Menus$MenuDocument", "menus", "Menus"); err != nil { + return fmt.Errorf("failed to build menus: %w", err) + } + + // Page templates are indexed in their own right rather than as pages. They + // were previously swept into pages_data by a prefix-matched Forms$Page query; + // dropping them from the catalog entirely instead would make 46 documents in + // a stock Atlas project invisible to anything that enumerates a module. + if err := b.buildSimpleNamedDocs("Forms$PageTemplate", "page_templates", "Page Templates"); err != nil { + return fmt.Errorf("failed to build page templates: %w", err) + } + if err := b.buildSimpleNamedDocs("DataTransformers$DataTransformer", "data_transformers", "Data Transformers"); err != nil { return fmt.Errorf("failed to build data transformers: %w", err) } + if err := b.buildScheduledEvents(); err != nil { + return fmt.Errorf("failed to build scheduled events: %w", err) + } + + if err := b.buildQueues(); err != nil { + return fmt.Errorf("failed to build queues: %w", err) + } + if err := b.buildAgentEditorDocs(); err != nil { return fmt.Errorf("failed to build agent-editor documents: %w", err) } diff --git a/mdl/catalog/builder_graph.go b/mdl/catalog/builder_graph.go index 12cc298c0..eb926f4ea 100644 --- a/mdl/catalog/builder_graph.go +++ b/mdl/catalog/builder_graph.go @@ -15,6 +15,10 @@ import ( var graphRefKinds = []string{ "call", "retrieve", "create", "change", "delete", "associate", "generalize", "parameter", "return", + // A scheduled event is an entry point: the microflow it runs is reachable + // even though nothing calls it. Without this kind, GRAPH_DEAD_ASSETS reports + // every scheduled microflow as dead. + "schedule", } // betweennessNodeCap bounds the O(V*E) betweenness computation. Above it, diff --git a/mdl/catalog/builder_pages.go b/mdl/catalog/builder_pages.go index 7cc9d8758..99f8f78a9 100644 --- a/mdl/catalog/builder_pages.go +++ b/mdl/catalog/builder_pages.go @@ -34,9 +34,9 @@ func (b *Builder) buildPages() error { if b.fullMode { widgetStmt, err = b.tx.Prepare(` INSERT INTO widgets_data (Id, Name, WidgetType, ContainerId, ContainerQualifiedName, ContainerType, - ModuleName, Folder, EntityRef, AttributeRef, MicroflowRef, NanoflowRef, Description, + ModuleName, Folder, EntityRef, AttributeRef, MicroflowRef, NanoflowRef, PageRef, Description, ProjectId, SnapshotId) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) `) if err != nil { return err @@ -114,6 +114,7 @@ func (b *Builder) buildPages() error { w.AttributeRef, w.MicroflowRef, w.NanoflowRef, + w.PageRef, "", projectID, snapshotID, ); err != nil { @@ -156,9 +157,9 @@ func (b *Builder) buildSnippets() error { if b.fullMode { widgetStmt, err = b.tx.Prepare(` INSERT INTO widgets_data (Id, Name, WidgetType, ContainerId, ContainerQualifiedName, ContainerType, - ModuleName, Folder, EntityRef, AttributeRef, MicroflowRef, NanoflowRef, Description, + ModuleName, Folder, EntityRef, AttributeRef, MicroflowRef, NanoflowRef, PageRef, Description, ProjectId, SnapshotId) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) `) if err != nil { return err @@ -205,7 +206,7 @@ func (b *Builder) buildSnippets() error { w.ID, w.Name, w.WidgetType, string(sn.ID), qualifiedName, "SNIPPET", moduleName, folder, - w.EntityRef, w.AttributeRef, w.MicroflowRef, w.NanoflowRef, "", + w.EntityRef, w.AttributeRef, w.MicroflowRef, w.NanoflowRef, w.PageRef, "", projectID, snapshotID, ); err != nil { return fmt.Errorf("insert widget %s for snippet %s: %w", w.Name, qualifiedName, err) @@ -292,6 +293,7 @@ type rawWidgetInfo struct { AttributeRef string MicroflowRef string // action/datasource microflow (Forms$MicroflowSettings.Microflow, …) NanoflowRef string // action/datasource nanoflow + PageRef string // action page (Forms$PageSettings.Form) — see scanWidgetOwnRefs } // widgetChildKeys are the keys under which a widget nests *other* widgets. The @@ -306,8 +308,8 @@ var widgetChildKeys = map[string]bool{ // Returns the lexicographically smallest match of each kind so the result is // deterministic regardless of BSON map iteration order; page-level dedup in the // refs projection makes the per-widget choice immaterial to the final graph. -func scanWidgetOwnRefs(w map[string]any) (entity, microflow, nanoflow string) { - var ents, mfs, nfs []string +func scanWidgetOwnRefs(w map[string]any) (entity, microflow, nanoflow, page string) { + var ents, mfs, nfs, forms []string var walk func(v any) walk = func(v any) { switch x := v.(type) { @@ -321,6 +323,14 @@ func scanWidgetOwnRefs(w map[string]any) (entity, microflow, nanoflow string) { if s, ok := x["Nanoflow"].(string); ok && s != "" { nfs = append(nfs, s) } + // A page reference is stored under "Form" — Mendix's original word for + // a page, the same rename behind ShowFormAction/CloseFormAction. Not + // collecting it is why an action button that opens a page created no + // reference, so the page reported "(no callers found)" however many + // buttons pointed at it (issue #773). + if s, ok := x["Form"].(string); ok && s != "" { + forms = append(forms, s) + } for k, val := range x { if widgetChildKeys[k] { continue @@ -346,7 +356,7 @@ func scanWidgetOwnRefs(w map[string]any) (entity, microflow, nanoflow string) { } return m } - return min(ents), min(mfs), min(nfs) + return min(ents), min(mfs), min(nfs), min(forms) } // extractLayoutRef extracts the layout reference from raw page BSON. Regular @@ -461,7 +471,7 @@ func extractWidgetsRecursive(w map[string]any) []rawWidgetInfo { // Extract datasource entity + action microflow/nanoflow references from this // widget's own content (not its child widgets). - widget.EntityRef, widget.MicroflowRef, widget.NanoflowRef = scanWidgetOwnRefs(w) + widget.EntityRef, widget.MicroflowRef, widget.NanoflowRef, widget.PageRef = scanWidgetOwnRefs(w) // Index user-authored containers, but skip the synthetic // "conditionalVisibilityWidget*" wrapper that mxcli / Studio Pro insert as a diff --git a/mdl/catalog/builder_pages_test.go b/mdl/catalog/builder_pages_test.go index 838c5063d..b565d2fbc 100644 --- a/mdl/catalog/builder_pages_test.go +++ b/mdl/catalog/builder_pages_test.go @@ -24,7 +24,7 @@ func TestScanWidgetOwnRefs(t *testing.T) { }, }, } - ent, mf, nf := scanWidgetOwnRefs(dataView) + ent, mf, nf, _ := scanWidgetOwnRefs(dataView) if ent != "Sales.Order" { t.Errorf("entity = %q, want Sales.Order", ent) } @@ -41,7 +41,7 @@ func TestScanWidgetOwnRefs(t *testing.T) { "Action": map[string]any{"Settings": map[string]any{"Microflow": "M.DoThing"}}, "Extra": map[string]any{"Nanoflow": "M.DoNano"}, } - if _, mf, nf := scanWidgetOwnRefs(button); mf != "M.DoThing" || nf != "M.DoNano" { + if _, mf, nf, _ := scanWidgetOwnRefs(button); mf != "M.DoThing" || nf != "M.DoNano" { t.Errorf("button refs = (%q,%q), want (M.DoThing, M.DoNano)", mf, nf) } @@ -52,13 +52,13 @@ func TestScanWidgetOwnRefs(t *testing.T) { "B": map[string]any{"Microflow": "M.Alpha"}, } for range 5 { - if _, mf, _ := scanWidgetOwnRefs(multi); mf != "M.Alpha" { + if _, mf, _, _ := scanWidgetOwnRefs(multi); mf != "M.Alpha" { t.Fatalf("non-deterministic microflow pick: got %q, want M.Alpha", mf) } } // No refs. - if e, m, n := scanWidgetOwnRefs(map[string]any{"$Type": "Forms$Label"}); e != "" || m != "" || n != "" { + if e, m, n, p := scanWidgetOwnRefs(map[string]any{"$Type": "Forms$Label"}); e != "" || m != "" || n != "" || p != "" { t.Errorf("expected no refs, got (%q,%q,%q)", e, m, n) } } @@ -245,9 +245,9 @@ func TestExtractWidgetsRecursive(t *testing.T) { t.Run("indexes a styled conditionalVisibilityWidget (not transparent)", func(t *testing.T) { // If a wrapper-named container carries styling it is a real user widget. w := map[string]any{ - "$ID": "div3", - "Name": "conditionalVisibilityWidget9", - "$Type": "Forms$DivContainer", + "$ID": "div3", + "Name": "conditionalVisibilityWidget9", + "$Type": "Forms$DivContainer", "Appearance": map[string]any{"Class": "card"}, } got := extractWidgetsRecursive(w) @@ -557,3 +557,65 @@ func TestBytesToHex(t *testing.T) { }) } } + +// upstream #773: a widget action that OPENS a page created no reference at all, +// so a page reachable only from a button reported "(no callers found)" — a false +// negative that reads as "safe to delete". +// +// The cause was one missing key. scanWidgetOwnRefs collected Entity, Microflow +// and Nanoflow but not `Form`, which is where a page reference lives — "Form" +// being Mendix's original word for a page, the same rename behind +// ShowFormAction/CloseFormAction. +func TestScanWidgetOwnRefs_PageReference(t *testing.T) { + // An action button that opens a page: `show_page Module.Target`. + showPage := map[string]any{ + "$Type": "Forms$ActionButton", + "Action": map[string]any{ + "$Type": "Forms$ShowPageClientAction", + "PageSettings": map[string]any{"$Type": "Forms$PageSettings", "Form": "Sales.OrderDetail"}, + }, + } + if _, _, _, page := scanWidgetOwnRefs(showPage); page != "Sales.OrderDetail" { + t.Errorf("page = %q, want Sales.OrderDetail", page) + } + + // The compound the issue reported — "create object … then open page" — is ONE + // action carrying BOTH an entity and a page. Collecting only the entity (which + // is what happened) still leaves the page with no inbound reference. + createThenShow := map[string]any{ + "$Type": "Forms$ActionButton", + "Action": map[string]any{ + "$Type": "Forms$CreateObjectClientAction", + "Entity": "Sales.Order", + "PageSettings": map[string]any{"$Type": "Forms$PageSettings", "Form": "Sales.OrderDetail"}, + }, + } + ent, _, _, page := scanWidgetOwnRefs(createThenShow) + if ent != "Sales.Order" { + t.Errorf("entity = %q, want Sales.Order", ent) + } + if page != "Sales.OrderDetail" { + t.Errorf("page = %q, want Sales.OrderDetail — the page half of "+ + "'create object … then open page' is the reference the issue reported missing", page) + } + + // A child widget's page must not be attributed to its parent, same as the + // existing microflow rule. + container := map[string]any{ + "$Type": "Forms$DivContainer", + "Widgets": []any{ + map[string]any{ + "$Type": "Forms$ActionButton", + "Action": map[string]any{"PageSettings": map[string]any{"Form": "Sales.ChildPage"}}, + }, + }, + } + if _, _, _, page := scanWidgetOwnRefs(container); page != "" { + t.Errorf("page = %q, want empty — a child button's page must not leak to the container", page) + } + + // A widget with no action references no page. + if _, _, _, page := scanWidgetOwnRefs(map[string]any{"$Type": "Forms$Label"}); page != "" { + t.Errorf("page = %q, want empty", page) + } +} diff --git a/mdl/catalog/builder_references.go b/mdl/catalog/builder_references.go index 26519d0be..4b840db91 100644 --- a/mdl/catalog/builder_references.go +++ b/mdl/catalog/builder_references.go @@ -32,6 +32,7 @@ const ( RefKindDelete = "delete" // Microflow deletes an entity object RefKindCalculate = "calculate" // Calculated attribute uses a microflow RefKindReturn = "return" // Microflow/nanoflow returns an entity type + RefKindSchedule = "schedule" // Scheduled event runs a microflow ) // collectActionActivities returns all ActionActivity objects from an ObjectCollection, @@ -366,6 +367,10 @@ func (b *Builder) buildReferences() error { {"EntityRef", "ENTITY", RefKindDatasource}, {"MicroflowRef", "MICROFLOW", RefKindAction}, {"NanoflowRef", "NANOFLOW", RefKindAction}, + // A widget action that opens a page. Without this row, a page reachable + // only from a button had no inbound reference and `show callers` / + // `show references` reported it as unused (issue #773). + {"PageRef", "PAGE", RefKindShowPage}, } for _, p := range widgetProjections { res, perr := b.tx.Exec( @@ -477,10 +482,35 @@ func (b *Builder) buildReferences() error { } } + // Scheduled events run a microflow. Without this edge the microflow looks + // unreferenced: `show callers` reported none, GRAPH_DEAD_ASSETS listed it, + // and QUAL004 said "is not called from anywhere" with the suggestion + // "Remove if unused" — on a microflow that runs nightly in production. + refCount += b.extractScheduledEventRefs(stmt, projectID, snapshotID) + b.report("References", refCount) return nil } +// extractScheduledEventRefs emits one `schedule` edge per scheduled event, from +// the event to the microflow it runs. +// +// The edges are collected by buildScheduledEvents, which runs earlier in the +// same transaction. +func (b *Builder) extractScheduledEventRefs(stmt *sql.Stmt, projectID, snapshotID string) int { + count := 0 + for _, r := range b.scheduledEventRefs { + if _, err := stmt.Exec( + "SCHEDULED_EVENT", "", r.qualifiedName, + "MICROFLOW", "", r.microflow, + RefKindSchedule, r.moduleName, projectID, snapshotID, + ); err == nil { + count++ + } + } + return count +} + // extractMenuItemRefs extracts page and microflow references from menu items recursively. func (b *Builder) extractMenuItemRefs(stmt *sql.Stmt, items []*types.NavMenuItem, sourceName, projectID, snapshotID string) int { refCount := 0 diff --git a/mdl/catalog/builder_scheduling.go b/mdl/catalog/builder_scheduling.go new file mode 100644 index 000000000..c7a7cbd59 --- /dev/null +++ b/mdl/catalog/builder_scheduling.go @@ -0,0 +1,227 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import ( + "fmt" + "strings" + + "go.mongodb.org/mongo-driver/bson" + + sched "github.com/mendixlabs/mxcli/mdl/scheduledevents" + "github.com/mendixlabs/mxcli/model" +) + +// Catalog rows for the two scheduling document types. +// +// Both read through the raw-unit surface and decode with the same codec the +// writers use, so the catalog cannot drift from what is stored. Scheduled events +// get their own builder rather than going through buildSimpleNamedDocs because +// the interesting columns — which microflow runs, how often, whether it is +// enabled — are the whole reason to query them. + +// scheduledEventRef is one event → microflow edge, held until buildReferences. +type scheduledEventRef struct{ qualifiedName, moduleName, microflow string } + +// buildScheduledEvents catalogs ScheduledEvents$ScheduledEvent units. +func (b *Builder) buildScheduledEvents() error { + units, err := b.reader.ListRawUnitsByType(sched.TypeName) + if err != nil { + return err + } + + stmt, err := b.tx.Prepare( + `INSERT INTO scheduled_events_data + (Id, Name, QualifiedName, ModuleName, Folder, Description, Microflow, + Repeat, RepeatDescription, IntervalSeconds, Enabled, TimeZone, OnOverlap, + ProjectId, SnapshotId) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ) + if err != nil { + return err + } + defer stmt.Close() + + projectID, snapshotID := b.snapshotMeta() + + count := 0 + for _, u := range units { + var doc bson.M + if err := bson.Unmarshal(u.Contents, &doc); err != nil { + continue + } + ev := sched.Parse(doc, model.ID(u.ID), model.ID(u.ContainerID)) + if ev.Name == "" { + continue + } + moduleID := b.hierarchy.findModuleID(u.ContainerID) + moduleName := b.hierarchy.getModuleName(moduleID) + + repeat, desc := "", "" + if ev.Schedule != nil { + repeat = string(ev.Schedule.Kind) + desc = describeSchedule(ev.Schedule) + } + if _, err := stmt.Exec( + string(u.ID), ev.Name, moduleName+"."+ev.Name, moduleName, + b.hierarchy.buildFolderPath(u.ContainerID), ev.Documentation, + string(ev.MicroflowID), repeat, desc, + scheduleIntervalSeconds(ev.Schedule), boolToInt(ev.Enabled), + ev.TimeZone, ev.OnOverlap, projectID, snapshotID, + ); err != nil { + return err + } + if ev.MicroflowID != "" { + b.scheduledEventRefs = append(b.scheduledEventRefs, scheduledEventRef{ + qualifiedName: moduleName + "." + ev.Name, + moduleName: moduleName, + microflow: string(ev.MicroflowID), + }) + } + count++ + } + + b.report("Scheduled Events", count) + return nil +} + +// buildQueues catalogs Queues$Queue units. +func (b *Builder) buildQueues() error { + units, err := b.reader.ListRawUnitsByType("Queues$Queue") + if err != nil { + return err + } + + stmt, err := b.tx.Prepare( + `INSERT INTO queues_data + (Id, Name, QualifiedName, ModuleName, Folder, Description, Parallelism, + ClusterWide, ProjectId, SnapshotId) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ) + if err != nil { + return err + } + defer stmt.Close() + + projectID, snapshotID := b.snapshotMeta() + + count := 0 + for _, u := range units { + var doc bson.M + if err := bson.Unmarshal(u.Contents, &doc); err != nil { + continue + } + name, _ := doc["Name"].(string) + if name == "" { + continue + } + documentation, _ := doc["Documentation"].(string) + // Parallelism lives on the nested Config node and is an EXPRESSION + // string, so it is stored as text rather than an integer column. + parallelism := "" + clusterWide := false + if cfg, ok := doc["Config"].(bson.M); ok { + parallelism, _ = cfg["ParallelismExpression"].(string) + clusterWide, _ = cfg["ClusterWide"].(bool) + } + moduleID := b.hierarchy.findModuleID(u.ContainerID) + moduleName := b.hierarchy.getModuleName(moduleID) + + if _, err := stmt.Exec( + string(u.ID), name, moduleName+"."+name, moduleName, + b.hierarchy.buildFolderPath(u.ContainerID), documentation, + parallelism, boolToInt(clusterWide), projectID, snapshotID, + ); err != nil { + return err + } + count++ + } + + b.report("Task Queues", count) + return nil +} + +// scheduleIntervalSeconds is how often the event fires, derived from the +// Schedule child. +// +// It deliberately does NOT use the stored Interval/IntervalType pair, which is a +// legacy sibling of Schedule that Studio Pro writes and does not keep in sync — +// Workflow Commons ships an event storing 0/"Minute" beside a DaySchedule of +// 01:00, which would catalog as "fires every 0 seconds". +// +// The month and year figures are averages (30 and 365 days); the column is for +// ordering and thresholds ("anything under a minute"), not for arithmetic on +// calendar dates. +func scheduleIntervalSeconds(s *model.Schedule) int64 { + if s == nil { + return 0 + } + mult := int64(s.Multiplier) + if mult < 1 { + mult = 1 + } + const day = int64(86400) + switch s.Kind { + case model.ScheduleMinute: + return mult * 60 + case model.ScheduleHour: + return mult * 3600 + case model.ScheduleDay: + return day + case model.ScheduleWeek: + // A weekly schedule can name several days, so the gap between runs is + // the week divided by how many are selected. + n := int64(0) + for _, on := range s.Weekdays { + if on { + n++ + } + } + if n == 0 { + return 7 * day + } + return (7 * day) / n + case model.ScheduleMonthDate, model.ScheduleMonthWeekday: + return mult * 30 * day + case model.ScheduleYearDate, model.ScheduleYearWeekday: + return 365 * day + } + return 0 +} + +// describeSchedule renders the repeat rule as a short human phrase, so a catalog +// query is readable without decoding the variant's fields. +func describeSchedule(s *model.Schedule) string { + if s == nil { + return "" + } + at := fmt.Sprintf("%02d:%02d", s.HourOfDay, s.MinuteOfHour) + switch s.Kind { + case model.ScheduleMinute: + return fmt.Sprintf("every %d min", s.Multiplier) + case model.ScheduleHour: + return fmt.Sprintf("every %dh at :%02d", s.Multiplier, s.MinuteOffset) + case model.ScheduleDay: + return "daily at " + at + case model.ScheduleWeek: + var days []string + for i, on := range s.Weekdays { + if on { + days = append(days, sched.WeekdayNames[i][:3]) + } + } + if days == nil { + days = []string{"(no days)"} + } + return "weekly " + strings.Join(days, "/") + " at " + at + case model.ScheduleMonthDate: + return fmt.Sprintf("every %d month(s) on day %d at %s", s.Multiplier, s.DayOfMonth, at) + case model.ScheduleMonthWeekday: + return fmt.Sprintf("every %d month(s) on the %s %s at %s", s.Multiplier, s.DaySelector, s.Weekday, at) + case model.ScheduleYearDate: + return fmt.Sprintf("yearly on %d/%d at %s", s.Month, s.DayOfMonth, at) + case model.ScheduleYearWeekday: + return fmt.Sprintf("yearly on the %s %s of month %d at %s", s.DaySelector, s.Weekday, s.Month, at) + } + return string(s.Kind) +} diff --git a/mdl/catalog/builder_scheduling_test.go b/mdl/catalog/builder_scheduling_test.go new file mode 100644 index 000000000..ad22ec327 --- /dev/null +++ b/mdl/catalog/builder_scheduling_test.go @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/model" +) + +// TestScheduleIntervalSeconds pins the interval a catalog query sees. +// +// It is derived from the Schedule child on purpose. The stored +// Interval/IntervalType pair is a legacy sibling that Studio Pro writes and does +// NOT keep in sync — Workflow Commons 4.11.0 ships an event storing 0/"Minute" +// beside a DaySchedule of 01:00 — so a threshold query keyed on the pair would +// read a nightly job as firing every 0 seconds. +func TestScheduleIntervalSeconds(t *testing.T) { + const day = int64(86400) + tests := []struct { + name string + sched *model.Schedule + want int64 + }{ + {"nil", nil, 0}, + {"every 2 minutes", &model.Schedule{Kind: model.ScheduleMinute, Multiplier: 2}, 120}, + {"every 3 hours", &model.Schedule{Kind: model.ScheduleHour, Multiplier: 3}, 10800}, + {"daily", &model.Schedule{Kind: model.ScheduleDay}, day}, + {"weekly, one day", &model.Schedule{Kind: model.ScheduleWeek, + Weekdays: [7]bool{false, true, false, false, false, false, false}}, 7 * day}, + // Two selected days means it fires twice a week, not once. + {"weekly, two days", &model.Schedule{Kind: model.ScheduleWeek, + Weekdays: [7]bool{false, true, false, false, false, true, false}}, (7 * day) / 2}, + {"weekly, no days", &model.Schedule{Kind: model.ScheduleWeek}, 7 * day}, + {"every 3 months", &model.Schedule{Kind: model.ScheduleMonthDate, Multiplier: 3}, 3 * 30 * day}, + {"yearly", &model.Schedule{Kind: model.ScheduleYearWeekday}, 365 * day}, + // An unstated multiplier is 1, not 0 — 0 would read as "never fires". + {"multiplier defaults to 1", &model.Schedule{Kind: model.ScheduleHour}, 3600}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := scheduleIntervalSeconds(tt.sched); got != tt.want { + t.Errorf("scheduleIntervalSeconds = %d, want %d", got, tt.want) + } + }) + } +} + +// TestDescribeSchedule checks the human phrase each variant produces, and that +// only that variant's fields reach it — a MonthWeekday must not print a day of +// the month, or the catalog would describe a schedule the model does not have. +func TestDescribeSchedule(t *testing.T) { + full := model.Schedule{ + Multiplier: 3, MinuteOffset: 23, MonthOffset: 2, + HourOfDay: 18, MinuteOfHour: 30, DayOfMonth: 15, Month: 3, + DaySelector: "Last", Weekday: "Friday", + Weekdays: [7]bool{false, true, false, false, false, true, false}, + } + tests := []struct { + kind model.ScheduleKind + want string + mustNot []string + }{ + {model.ScheduleMinute, "every 3 min", []string{"18:30", "Friday"}}, + {model.ScheduleHour, "every 3h at :23", []string{"18:30", "Friday"}}, + {model.ScheduleDay, "daily at 18:30", []string{"Friday", "3 month"}}, + {model.ScheduleWeek, "weekly Mon/Fri at 18:30", []string{"Last"}}, + {model.ScheduleMonthDate, "every 3 month(s) on day 15 at 18:30", []string{"Friday"}}, + {model.ScheduleMonthWeekday, "every 3 month(s) on the Last Friday at 18:30", []string{"day 15"}}, + {model.ScheduleYearDate, "yearly on 3/15 at 18:30", []string{"Friday"}}, + {model.ScheduleYearWeekday, "yearly on the Last Friday of month 3 at 18:30", []string{"day 15"}}, + } + for _, tt := range tests { + t.Run(string(tt.kind), func(t *testing.T) { + s := full + s.Kind = tt.kind + got := describeSchedule(&s) + if got != tt.want { + t.Errorf("describeSchedule = %q, want %q", got, tt.want) + } + for _, bad := range tt.mustNot { + if strings.Contains(got, bad) { + t.Errorf("describeSchedule = %q — %q belongs to a different variant", got, bad) + } + } + }) + } + if got := describeSchedule(nil); got != "" { + t.Errorf("describeSchedule(nil) = %q, want empty", got) + } +} diff --git a/mdl/catalog/catalog.go b/mdl/catalog/catalog.go index 07383a3d7..6f592b0c7 100644 --- a/mdl/catalog/catalog.go +++ b/mdl/catalog/catalog.go @@ -107,6 +107,7 @@ func (c *Catalog) Tables() []string { "CATALOG.MICROFLOWS", "CATALOG.NANOFLOWS", "CATALOG.PAGES", + "CATALOG.PAGE_TEMPLATES", "CATALOG.SNIPPETS", "CATALOG.BUILDING_BLOCKS", "CATALOG.LAYOUTS", @@ -115,6 +116,10 @@ func (c *Catalog) Tables() []string { "CATALOG.JAVA_ACTION_PARAMETERS", "CATALOG.JAVASCRIPT_ACTIONS", "CATALOG.IMAGE_COLLECTIONS", + "CATALOG.ICON_COLLECTIONS", + "CATALOG.MENUS", + "CATALOG.SCHEDULED_EVENTS", + "CATALOG.QUEUES", "CATALOG.DATA_TRANSFORMERS", "CATALOG.AGENTS", "CATALOG.AI_MODELS", diff --git a/mdl/catalog/catalog_test.go b/mdl/catalog/catalog_test.go index 4dddb5488..6b9a7cee7 100644 --- a/mdl/catalog/catalog_test.go +++ b/mdl/catalog/catalog_test.go @@ -231,6 +231,12 @@ func TestObjectsView_IncludesNewDocumentTypes(t *testing.T) { }{ {"javascript_actions", "JAVASCRIPT_ACTION"}, {"image_collections", "IMAGE_COLLECTION"}, + // Both were built into their own tables but never joined to the objects + // view, so bare `DESCRIBE Module.Name` could not resolve them. + {"building_blocks", "BUILDING_BLOCK"}, + {"icon_collections", "ICON_COLLECTION"}, + {"menus", "MENU"}, + {"page_templates", "PAGE_TEMPLATE"}, {"data_transformers", "DATA_TRANSFORMER"}, {"agents", "AGENT"}, {"ai_models", "AI_MODEL"}, diff --git a/mdl/catalog/tables.go b/mdl/catalog/tables.go index b39426c6b..635796242 100644 --- a/mdl/catalog/tables.go +++ b/mdl/catalog/tables.go @@ -312,6 +312,87 @@ func (c *Catalog) createTables() error { )`, viewWithFullSnapshot("image_collections"), + // icon_collections (custom icon sets; read-only, referenced by widgets) + `CREATE TABLE IF NOT EXISTS icon_collections_data ( + Id TEXT PRIMARY KEY, + Name TEXT, + QualifiedName TEXT, + ModuleName TEXT, + Folder TEXT, + Description TEXT, + ProjectId TEXT, + SnapshotId TEXT + )`, + viewWithFullSnapshot("icon_collections"), + + // page_templates (Forms$PageTemplate; the starting points Studio Pro's + // "new page" dialog offers). A separate table from pages: templates are a + // different document type whose content hangs off LayoutCall, and folding + // them into pages made every module that ships templates report pages it + // does not have. + `CREATE TABLE IF NOT EXISTS page_templates_data ( + Id TEXT PRIMARY KEY, + Name TEXT, + QualifiedName TEXT, + ModuleName TEXT, + Folder TEXT, + Description TEXT, + ProjectId TEXT, + SnapshotId TEXT + )`, + viewWithFullSnapshot("page_templates"), + + // menus (standalone Menus$MenuDocument; read-only reusable menus) + `CREATE TABLE IF NOT EXISTS menus_data ( + Id TEXT PRIMARY KEY, + Name TEXT, + QualifiedName TEXT, + ModuleName TEXT, + Folder TEXT, + Description TEXT, + ProjectId TEXT, + SnapshotId TEXT + )`, + viewWithFullSnapshot("menus"), + + // scheduled_events — Mendix's cron. Repeat/RepeatDescription come from the + // Schedule child; IntervalSeconds is derived from it, NOT from the legacy + // Interval/IntervalType pair (Studio Pro does not keep those in sync). + `CREATE TABLE IF NOT EXISTS scheduled_events_data ( + Id TEXT PRIMARY KEY, + Name TEXT, + QualifiedName TEXT, + ModuleName TEXT, + Folder TEXT, + Description TEXT, + Microflow TEXT, + Repeat TEXT, + RepeatDescription TEXT, + IntervalSeconds INTEGER, + Enabled INTEGER, + TimeZone TEXT, + OnOverlap TEXT, + ProjectId TEXT, + SnapshotId TEXT + )`, + viewWithFullSnapshot("scheduled_events"), + + // queues — task queues (Queues$Queue). Parallelism is an expression + // string, not a number. + `CREATE TABLE IF NOT EXISTS queues_data ( + Id TEXT PRIMARY KEY, + Name TEXT, + QualifiedName TEXT, + ModuleName TEXT, + Folder TEXT, + Description TEXT, + Parallelism TEXT, + ClusterWide INTEGER, + ProjectId TEXT, + SnapshotId TEXT + )`, + viewWithFullSnapshot("queues"), + // data_transformers `CREATE TABLE IF NOT EXISTS data_transformers_data ( Id TEXT PRIMARY KEY, @@ -416,6 +497,11 @@ func (c *Catalog) createTables() error { AttributeRef TEXT, MicroflowRef TEXT, NanoflowRef TEXT, + -- The page a widget's action opens (show_page, and the page half of + -- "create object … then open page"). Without it a page reachable only + -- from a button had no inbound reference at all and read as dead code + -- (issue #773). + PageRef TEXT, Description TEXT, ProjectId TEXT, SnapshotId TEXT @@ -896,6 +982,10 @@ func (c *Catalog) createTables() error { ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM snippets UNION ALL + SELECT Id, 'BUILDING_BLOCK' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM building_blocks + UNION ALL SELECT Id, 'LAYOUT' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM layouts @@ -920,6 +1010,26 @@ func (c *Catalog) createTables() error { ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM image_collections UNION ALL + SELECT Id, 'ICON_COLLECTION' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM icon_collections + UNION ALL + SELECT Id, 'MENU' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM menus + UNION ALL + SELECT Id, 'PAGE_TEMPLATE' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM page_templates + UNION ALL + SELECT Id, 'SCHEDULED_EVENT' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM scheduled_events + UNION ALL + SELECT Id, 'QUEUE' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, + ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource + FROM queues + UNION ALL SELECT Id, 'DATA_TRANSFORMER' as ObjectType, Name, QualifiedName, ModuleName, Folder, Description, ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM data_transformers diff --git a/mdl/enginecompare/bsoncompare.go b/mdl/enginecompare/bsoncompare.go index 2cfdcb3ae..8de761e1a 100644 --- a/mdl/enginecompare/bsoncompare.go +++ b/mdl/enginecompare/bsoncompare.go @@ -10,13 +10,13 @@ import ( "go.mongodb.org/mongo-driver/v2/bson" "github.com/mendixlabs/mxcli/modelsdk/codec" - genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" genConst "github.com/mendixlabs/mxcli/modelsdk/gen/constants" _ "github.com/mendixlabs/mxcli/modelsdk/gen/datatypes" // register DataTypes$* for constant decode + genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" genEnum "github.com/mendixlabs/mxcli/modelsdk/gen/enumerations" genMf "github.com/mendixlabs/mxcli/modelsdk/gen/microflows" - "github.com/mendixlabs/mxcli/modelsdk/mprread" mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr" + "github.com/mendixlabs/mxcli/modelsdk/mprread" ) // MicroflowCanonBSON returns the canonicalized raw BSON of a named microflow unit diff --git a/mdl/enginecompare/write_gen_test.go b/mdl/enginecompare/write_gen_test.go index 38d502165..8c5ae8344 100644 --- a/mdl/enginecompare/write_gen_test.go +++ b/mdl/enginecompare/write_gen_test.go @@ -1,10 +1,26 @@ package enginecompare + import "testing" + func TestWriteParity_Generalization(t *testing.T) { const s = "CREATE PERSISTENT ENTITY MyFirstModule.GenParent;CREATE PERSISTENT ENTITY MyFirstModule.GenChild EXTENDS MyFirstModule.GenParent;" - lp := copyProject(t); if _,err:=Run(Legacy,lp,s);err!=nil{t.Fatalf("legacy: %v",err)} - mp := copyProject(t); if _,err:=Run(ModelSDK,mp,s);err!=nil{t.Fatalf("modelsdk: %v",err)} - leg,err:=EntityCanonBSON(lp,"MyFirstModule","GenChild"); if err!=nil{t.Fatalf("leg: %v",err)} - msd,err:=EntityCanonBSON(mp,"MyFirstModule","GenChild"); if err!=nil{t.Fatalf("msd: %v",err)} - if leg!=msd { t.Errorf("Generalization divergence:\nlegacy: %s\nmodelsdk: %s", leg, msd) } + lp := copyProject(t) + if _, err := Run(Legacy, lp, s); err != nil { + t.Fatalf("legacy: %v", err) + } + mp := copyProject(t) + if _, err := Run(ModelSDK, mp, s); err != nil { + t.Fatalf("modelsdk: %v", err) + } + leg, err := EntityCanonBSON(lp, "MyFirstModule", "GenChild") + if err != nil { + t.Fatalf("leg: %v", err) + } + msd, err := EntityCanonBSON(mp, "MyFirstModule", "GenChild") + if err != nil { + t.Fatalf("msd: %v", err) + } + if leg != msd { + t.Errorf("Generalization divergence:\nlegacy: %s\nmodelsdk: %s", leg, msd) + } } diff --git a/mdl/enginecompare/write_valid_test.go b/mdl/enginecompare/write_valid_test.go index 15222bb6c..9a77b51ff 100644 --- a/mdl/enginecompare/write_valid_test.go +++ b/mdl/enginecompare/write_valid_test.go @@ -1,11 +1,27 @@ package enginecompare + import "testing" + func TestWriteParity_ValidationRules(t *testing.T) { const s = "CREATE PERSISTENT ENTITY MyFirstModule.ValTest " + "( Name: string(100) not null error 'Name is required', Code: string(20) unique error 'Code must be unique' )" - lp := copyProject(t); if _,e:=Run(Legacy,lp,s);e!=nil{t.Fatalf("legacy: %v",e)} - mp := copyProject(t); if _,e:=Run(ModelSDK,mp,s);e!=nil{t.Fatalf("modelsdk: %v",e)} - leg,e:=EntityCanonBSON(lp,"MyFirstModule","ValTest"); if e!=nil{t.Fatalf("leg: %v",e)} - msd,e:=EntityCanonBSON(mp,"MyFirstModule","ValTest"); if e!=nil{t.Fatalf("msd: %v",e)} - if leg!=msd { t.Errorf("ValidationRules divergence:\nlegacy: %s\nmodelsdk: %s", leg, msd) } + lp := copyProject(t) + if _, e := Run(Legacy, lp, s); e != nil { + t.Fatalf("legacy: %v", e) + } + mp := copyProject(t) + if _, e := Run(ModelSDK, mp, s); e != nil { + t.Fatalf("modelsdk: %v", e) + } + leg, e := EntityCanonBSON(lp, "MyFirstModule", "ValTest") + if e != nil { + t.Fatalf("leg: %v", e) + } + msd, e := EntityCanonBSON(mp, "MyFirstModule", "ValTest") + if e != nil { + t.Fatalf("msd: %v", e) + } + if leg != msd { + t.Errorf("ValidationRules divergence:\nlegacy: %s\nmodelsdk: %s", leg, msd) + } } diff --git a/mdl/executor/cmd_agenteditor_models.go b/mdl/executor/cmd_agenteditor_models.go index 72e0486c2..46ce245af 100644 --- a/mdl/executor/cmd_agenteditor_models.go +++ b/mdl/executor/cmd_agenteditor_models.go @@ -136,6 +136,16 @@ func execCreateAgentEditorModel(ctx *ExecContext, s *ast.CreateModelStmt) error return mdlerrors.NewNotConnected() } + // Agent Editor documents need Studio Pro 11.9+ and the AgentEditorCommons + // module. Nothing downstream catches an older project: the documents are + // custom blobs, so mxbuild does not validate them and the build stays green + // while Studio Pro cannot open the result. + if err := checkFeature(ctx, "agent_documents", "agent_model", + "create model", + "upgrade your project to Mendix 11.9+ and install the AgentEditorCommons module"); err != nil { + return err + } + existing := findAgentEditorModel(ctx, s.Name.Module, s.Name.Name) if existing != nil && !s.CreateOrModify { return mdlerrors.NewAlreadyExists("model", s.Name.String()) diff --git a/mdl/executor/cmd_agenteditor_write.go b/mdl/executor/cmd_agenteditor_write.go index ffb9b5b12..42f3454b2 100644 --- a/mdl/executor/cmd_agenteditor_write.go +++ b/mdl/executor/cmd_agenteditor_write.go @@ -21,6 +21,16 @@ func execCreateConsumedMCPService(ctx *ExecContext, s *ast.CreateConsumedMCPServ return mdlerrors.NewNotConnected() } + // Agent Editor documents need Studio Pro 11.9+ and the AgentEditorCommons + // module. Nothing downstream catches an older project: the documents are + // custom blobs, so mxbuild does not validate them and the build stays green + // while Studio Pro cannot open the result. + if err := checkFeature(ctx, "agent_documents", "agent_consumed_mcp_service", + "create consumed mcp service", + "upgrade your project to Mendix 11.9+ and install the AgentEditorCommons module"); err != nil { + return err + } + existing := findAgentEditorConsumedMCPService(ctx, s.Name.Module, s.Name.Name) if existing != nil && !s.CreateOrModify { return mdlerrors.NewAlreadyExists("consumed mcp service", s.Name.String()) @@ -83,6 +93,16 @@ func execCreateKnowledgeBase(ctx *ExecContext, s *ast.CreateKnowledgeBaseStmt) e return mdlerrors.NewNotConnected() } + // Agent Editor documents need Studio Pro 11.9+ and the AgentEditorCommons + // module. Nothing downstream catches an older project: the documents are + // custom blobs, so mxbuild does not validate them and the build stays green + // while Studio Pro cannot open the result. + if err := checkFeature(ctx, "agent_documents", "agent_knowledge_base", + "create knowledge base", + "upgrade your project to Mendix 11.9+ and install the AgentEditorCommons module"); err != nil { + return err + } + existing := findAgentEditorKnowledgeBase(ctx, s.Name.Module, s.Name.Name) if existing != nil && !s.CreateOrModify { return mdlerrors.NewAlreadyExists("knowledge base", s.Name.String()) @@ -162,6 +182,16 @@ func execCreateAgent(ctx *ExecContext, s *ast.CreateAgentStmt) error { return mdlerrors.NewNotConnected() } + // Agent Editor documents need Studio Pro 11.9+ and the AgentEditorCommons + // module. Nothing downstream catches an older project: the documents are + // custom blobs, so mxbuild does not validate them and the build stays green + // while Studio Pro cannot open the result. + if err := checkFeature(ctx, "agent_documents", "agent", + "create agent", + "upgrade your project to Mendix 11.9+ and install the AgentEditorCommons module"); err != nil { + return err + } + existingAgent := findAgentEditorAgent(ctx, s.Name.Module, s.Name.Name) if existingAgent != nil && !s.CreateOrModify { return mdlerrors.NewAlreadyExists("agent", s.Name.String()) diff --git a/mdl/executor/cmd_features.go b/mdl/executor/cmd_features.go index 20e73e545..ab57832ca 100644 --- a/mdl/executor/cmd_features.go +++ b/mdl/executor/cmd_features.go @@ -24,6 +24,15 @@ func checkFeature(ctx *ExecContext, area, name, statement, hint string) error { return nil // Registry unavailable; don't block execution } rpv := ctx.Backend.ProjectVersion() + if rpv == nil { + // A backend that cannot report a version cannot be version-checked. Skip, + // matching what this function already does when the project is not + // connected or the registry will not load: a version gate exists to give + // an actionable error, never to block work it cannot evaluate. (The mock + // backend returns nil unless a test configures it, so without this guard + // every gated handler panics under test rather than under a user.) + return nil + } pv := versions.SemVer{Major: rpv.MajorVersion, Minor: rpv.MinorVersion, Patch: rpv.PatchVersion} if reg.IsAvailable(area, name, pv) { return nil diff --git a/mdl/executor/cmd_features_mock_test.go b/mdl/executor/cmd_features_mock_test.go index afc11bc0a..30778207b 100644 --- a/mdl/executor/cmd_features_mock_test.go +++ b/mdl/executor/cmd_features_mock_test.go @@ -105,3 +105,46 @@ func TestShowFeatures_InArea_ForVersion(t *testing.T) { // Area filter narrows output; assert header contains area name. assertContainsStr(t, buf.String(), "domain_model") } + +// TestCheckFeature_AgentDocumentsRefusedBelow119 exercises the gate that +// mxcli-formula1 FINDINGS §53 reported missing, at the layer a user meets it. +// +// The agent doctypes are the one version gate with no downstream safety net: +// their documents are custom blobs, so mxbuild validates nothing and a project +// below 11.9 builds green while Studio Pro cannot open the result. If this gate +// is absent the only signal the user gets is silence. +func TestCheckFeature_AgentDocumentsRefusedBelow119(t *testing.T) { + atVersion := func(major, minor int) *mock.MockBackend { + return &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ProjectVersionFunc: func() *types.ProjectVersion { + return &types.ProjectVersion{MajorVersion: major, MinorVersion: minor} + }, + } + } + + ctx, _ := newMockCtx(t, withBackend(atVersion(11, 8))) + err := checkFeature(ctx, "agent_documents", "agent", "create agent", "upgrade") + assertError(t, err) + // The message must name the version and the missing module, or the user + // cannot act: installing AgentEditorCommons is half the requirement. + assertContainsStr(t, err.Error(), "11.9") + + ctx, _ = newMockCtx(t, withBackend(atVersion(11, 12))) + if err := checkFeature(ctx, "agent_documents", "agent", "create agent", "upgrade"); err != nil { + t.Errorf("11.12 satisfies the 11.9 minimum; got: %v", err) + } +} + +// TestCheckFeature_SkipsWhenTheBackendHasNoVersion guards the fail-open path. +// A backend that cannot report a version cannot be version-checked, and a gate +// exists to give an actionable error rather than to block work it cannot +// evaluate. Without the guard every gated handler panics on a nil version. +func TestCheckFeature_SkipsWhenTheBackendHasNoVersion(t *testing.T) { + mb := &mock.MockBackend{IsConnectedFunc: func() bool { return true }} // ProjectVersion() → nil + ctx, _ := newMockCtx(t, withBackend(mb)) + + if err := checkFeature(ctx, "agent_documents", "agent", "create agent", "upgrade"); err != nil { + t.Errorf("an unknown project version must not block execution; got: %v", err) + } +} diff --git a/mdl/executor/cmd_menus.go b/mdl/executor/cmd_menus.go new file mode 100644 index 000000000..d4ddecbb8 --- /dev/null +++ b/mdl/executor/cmd_menus.go @@ -0,0 +1,142 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" +) + +// describeMenu renders a standalone Menus$MenuDocument — the reusable menu a +// menu widget points at, as opposed to the menu embedded in a navigation +// profile. Atlas_Core ships Phone_Menu and Tablet_Menu. +// +// The entries are ordinary Menus$MenuItem elements, so the tree is rendered by +// printMenuMDL, the same renderer DESCRIBE NAVIGATION uses — and the same syntax +// CREATE MENU accepts, so the output round-trips. +func describeMenu(ctx *ExecContext, name ast.QualifiedName) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + + md, err := ctx.Backend.GetMenuDocumentByQualifiedName(name.Module, name.Name) + if err != nil { + if strings.Contains(err.Error(), "not found") { + return mdlerrors.NewNotFound("menu", name.String()) + } + return mdlerrors.NewBackend("get menu", err) + } + + if md.Documentation != "" { + fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", + strings.ReplaceAll(md.Documentation, "\n", "\n * ")) + } + + if md.Excluded { + fmt.Fprintln(ctx.Output, "-- Excluded from the project") + } + + // Output is re-executable: the item syntax is the same one CREATE MENU + // accepts, so describe → exec → describe is a fixed point. + fmt.Fprintf(ctx.Output, "create or modify menu %s.%s (\n", name.Module, md.Name) + printMenuMDL(ctx.Output, md.Items, 1, "CREATE MENU") + fmt.Fprintln(ctx.Output, ");") + return nil +} + +// execCreateMenu handles CREATE [OR MODIFY] MENU Module.Name ( items ). +// +// Like CREATE NAVIGATION, the item list is the document's complete contents, so +// a modify replaces the items wholesale rather than merging. The existing +// document's $ID is reused on modify, so references to the menu survive and the +// unit is rewritten in place rather than replaced. +func execCreateMenu(ctx *ExecContext, s *ast.CreateMenuStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + + mod, err := ctx.Backend.GetModuleByName(s.Name.Module) + if err != nil || mod == nil { + return mdlerrors.NewNotFound("module", s.Name.Module) + } + + existing, _ := ctx.Backend.GetMenuDocumentByQualifiedName(s.Name.Module, s.Name.Name) + if existing != nil && !s.CreateOrModify { + return mdlerrors.NewAlreadyExists("menu", s.Name.String()) + } + + md := &types.MenuDocument{ + Name: s.Name.Name, + ContainerID: mod.ID, + Documentation: s.Documentation, + Items: menuItemsFromAST(s.Items), + } + + if existing != nil { + // Preserve the document's identity and the properties MDL does not + // author, so a modify does not silently reset them. + md.ID = existing.ID + md.ContainerID = existing.ContainerID + md.ExportLevel = existing.ExportLevel + md.Excluded = existing.Excluded + if md.Documentation == "" { + md.Documentation = existing.Documentation + } + if err := ctx.Backend.UpdateMenuDocument(md); err != nil { + return mdlerrors.NewBackend("update menu", err) + } + fmt.Fprintf(ctx.Output, "Modified menu %s\n", s.Name.String()) + return nil + } + + if err := ctx.Backend.CreateMenuDocument(md); err != nil { + return mdlerrors.NewBackend("create menu", err) + } + fmt.Fprintf(ctx.Output, "Created menu %s\n", s.Name.String()) + return nil +} + +// execDropMenu handles DROP MENU Module.Name. +func execDropMenu(ctx *ExecContext, s *ast.DropMenuStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + md, err := ctx.Backend.GetMenuDocumentByQualifiedName(s.Name.Module, s.Name.Name) + if err != nil || md == nil { + return mdlerrors.NewNotFound("menu", s.Name.String()) + } + if err := ctx.Backend.DeleteMenuDocument(md.ID); err != nil { + return mdlerrors.NewBackend("drop menu", err) + } + fmt.Fprintf(ctx.Output, "Dropped menu %s\n", s.Name.String()) + return nil +} + +// menuItemsFromAST converts parsed menu items to the semantic model. The AST and +// semantic shapes differ only in how the target is held (pointer vs string), so +// this stays a direct mapping rather than acquiring behaviour. +func menuItemsFromAST(defs []ast.NavMenuItemDef) []*types.NavMenuItem { + var out []*types.NavMenuItem + for _, d := range defs { + item := &types.NavMenuItem{Caption: d.Caption, Icon: d.Icon} + if d.Icon != "" { + item.IconType = "Forms$IconCollectionIcon" + } + if d.Page != nil { + item.Page = d.Page.String() + item.ActionType = "PageAction" + } else if d.Microflow != nil { + item.Microflow = d.Microflow.String() + item.ActionType = "MicroflowAction" + } else { + item.ActionType = "NoAction" + } + item.Items = menuItemsFromAST(d.Items) + out = append(out, item) + } + return out +} diff --git a/mdl/executor/cmd_menus_mock_test.go b/mdl/executor/cmd_menus_mock_test.go new file mode 100644 index 000000000..95dd5c0e5 --- /dev/null +++ b/mdl/executor/cmd_menus_mock_test.go @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// menuBackend returns a MockBackend serving one menu document. +func menuBackend(md *types.MenuDocument) *mock.MockBackend { + return &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + GetMenuDocumentByQualifiedNameFunc: func(moduleName, name string) (*types.MenuDocument, error) { + if md != nil && md.Name == name { + return md, nil + } + return nil, fmt.Errorf("menu not found: %s.%s", moduleName, name) + }, + } +} + +// TestDescribeMenu_Nested is the renderer's real exercise. The vendored fixture's +// Atlas menus are flat — every MenuItem's Items array holds only the list marker +// — so recursion, page/microflow targets and the non-round-trippable icon note +// have no coverage there. This builds a menu that has all of them. +func TestDescribeMenu_Nested(t *testing.T) { + md := &types.MenuDocument{ + Name: "Main_Menu", + ExportLevel: "Hidden", + Items: []*types.NavMenuItem{ + { + Caption: "Home", + Page: "MyModule.Home_Web", + IconType: "Forms$IconCollectionIcon", + Icon: "Atlas_Core.Atlas.home", + }, + { + Caption: "Admin", + Items: []*types.NavMenuItem{ + {Caption: "Accounts", Page: "Administration.Account_Overview"}, + {Caption: "Rebuild", Microflow: "Administration.Rebuild"}, + }, + }, + { + // A glyph icon carries a numeric Code and no name, so it cannot be + // expressed in MDL. It must be flagged, not silently dropped. + Caption: "Settings", + IconType: "Forms$GlyphIcon", + }, + }, + } + + ctx, buf := newMockCtx(t, withBackend(menuBackend(md))) + assertNoError(t, describeMenu(ctx, ast.QualifiedName{Module: "Atlas_Core", Name: "Main_Menu"})) + out := buf.String() + + // The output is re-executable, so it opens with the statement that recreates it. + assertContainsStr(t, out, "create or modify menu Atlas_Core.Main_Menu (") + assertContainsStr(t, out, "menu item 'Home' page MyModule.Home_Web icon Atlas_Core.Atlas.home;") + + // A sub-menu opens a nested block and its children are indented one level in. + assertContainsStr(t, out, "menu 'Admin' (") + assertContainsStr(t, out, " menu item 'Accounts' page Administration.Account_Overview;") + assertContainsStr(t, out, " menu item 'Rebuild' microflow Administration.Rebuild;") + + // The glyph icon is reported rather than dropped, and points at the statement + // that would have to reproduce it — CREATE MENU, not CREATE NAVIGATION. + assertContainsStr(t, out, "is not reproducible by CREATE MENU") + if strings.Contains(out, "CREATE NAVIGATION") { + t.Errorf("menu output should not point at CREATE NAVIGATION, which authors a profile menu, not a menu document:\n%s", out) + } +} + +func TestDescribeMenu_Empty(t *testing.T) { + md := &types.MenuDocument{Name: "Empty_Menu"} + ctx, buf := newMockCtx(t, withBackend(menuBackend(md))) + assertNoError(t, describeMenu(ctx, ast.QualifiedName{Module: "Atlas_Core", Name: "Empty_Menu"})) + // An empty menu still describes to a statement that recreates it. + assertContainsStr(t, buf.String(), "create or modify menu Atlas_Core.Empty_Menu (") + assertContainsStr(t, buf.String(), ");") +} + +func TestDescribeMenu_NotFound(t *testing.T) { + ctx, _ := newMockCtx(t, withBackend(menuBackend(nil))) + err := describeMenu(ctx, ast.QualifiedName{Module: "Atlas_Core", Name: "Nope"}) + if err == nil { + t.Fatal("expected an error for a menu that does not exist") + } + if !strings.Contains(err.Error(), "not found") { + t.Errorf("error = %q, want it to report the menu was not found", err) + } +} + +// TestDescribeMenu_Documentation checks the doc comment is emitted, since a +// menu document carries Documentation like any other document. +func TestDescribeMenu_Documentation(t *testing.T) { + md := &types.MenuDocument{Name: "Doc_Menu", Documentation: "Phone navigation."} + ctx, buf := newMockCtx(t, withBackend(menuBackend(md))) + assertNoError(t, describeMenu(ctx, ast.QualifiedName{Module: "Atlas_Core", Name: "Doc_Menu"})) + assertContainsStr(t, buf.String(), "Phone navigation.") +} + +// TestCreateMenu_RejectsDuplicateWithoutOrModify pins the create/modify split: +// a plain CREATE against an existing menu must fail rather than silently +// replacing a document the author did not mean to touch. +func TestCreateMenu_RejectsDuplicateWithoutOrModify(t *testing.T) { + existing := &types.MenuDocument{ID: "menu-1", Name: "Main_Menu"} + mb := menuBackend(existing) + mb.GetModuleByNameFunc = func(name string) (*model.Module, error) { + return &model.Module{BaseElement: model.BaseElement{ID: "mod-1"}, Name: name}, nil + } + + ctx, _ := newMockCtx(t, withBackend(mb)) + err := execCreateMenu(ctx, &ast.CreateMenuStmt{ + Name: ast.QualifiedName{Module: "MyModule", Name: "Main_Menu"}, + }) + if err == nil { + t.Fatal("expected CREATE MENU on an existing menu to fail without OR MODIFY") + } +} + +// TestCreateMenu_OrModifyPreservesIdentity checks that a modify rewrites the +// stored document rather than minting a new one, and does not reset the +// properties MDL does not author. Losing the ID would break every menu widget +// pointing at it; resetting ExportLevel would silently change the module's API. +func TestCreateMenu_OrModifyPreservesIdentity(t *testing.T) { + existing := &types.MenuDocument{ + ID: "menu-1", ContainerID: "folder-9", Name: "Main_Menu", + ExportLevel: "Public", Documentation: "kept", + } + var updated *types.MenuDocument + mb := menuBackend(existing) + mb.GetModuleByNameFunc = func(name string) (*model.Module, error) { + return &model.Module{BaseElement: model.BaseElement{ID: "mod-1"}, Name: name}, nil + } + mb.UpdateMenuDocumentFunc = func(md *types.MenuDocument) error { updated = md; return nil } + mb.CreateMenuDocumentFunc = func(md *types.MenuDocument) error { + t.Error("OR MODIFY must update the existing menu, not create a second one") + return nil + } + + ctx, _ := newMockCtx(t, withBackend(mb)) + assertNoError(t, execCreateMenu(ctx, &ast.CreateMenuStmt{ + Name: ast.QualifiedName{Module: "MyModule", Name: "Main_Menu"}, + CreateOrModify: true, + Items: []ast.NavMenuItemDef{{Caption: "Home"}}, + })) + + if updated == nil { + t.Fatal("UpdateMenuDocument was not called") + } + if updated.ID != "menu-1" { + t.Errorf("ID = %q, want the stored menu-1 — a fresh ID orphans every reference", updated.ID) + } + if updated.ContainerID != "folder-9" { + t.Errorf("ContainerID = %q, want folder-9 — a modify must not move the document", updated.ContainerID) + } + if updated.ExportLevel != "Public" { + t.Errorf("ExportLevel = %q, want Public preserved", updated.ExportLevel) + } + if updated.Documentation != "kept" { + t.Errorf("Documentation = %q, want the stored text preserved", updated.Documentation) + } + if len(updated.Items) != 1 || updated.Items[0].Caption != "Home" { + t.Errorf("items were not replaced by the statement's list: %+v", updated.Items) + } +} + +// TestMenuItemsFromAST_Nested covers the AST→model mapping the executor owns, +// including the recursion the fixture's flat Atlas menus cannot exercise. +func TestMenuItemsFromAST_Nested(t *testing.T) { + page := ast.QualifiedName{Module: "M", Name: "P"} + mf := ast.QualifiedName{Module: "M", Name: "F"} + items := menuItemsFromAST([]ast.NavMenuItemDef{{ + Caption: "Top", + Items: []ast.NavMenuItemDef{ + {Caption: "Pg", Page: &page, Icon: "M.C.i"}, + {Caption: "Mf", Microflow: &mf}, + {Caption: "Plain"}, + }, + }}) + + if len(items) != 1 || len(items[0].Items) != 3 { + t.Fatalf("expected 1 top item with 3 children, got %+v", items) + } + kids := items[0].Items + if kids[0].Page != "M.P" || kids[0].ActionType != "PageAction" { + t.Errorf("page child = %+v", kids[0]) + } + if kids[0].IconType != "Forms$IconCollectionIcon" { + t.Errorf("an ICON clause must record the icon type it round-trips as, got %q", kids[0].IconType) + } + if kids[1].Microflow != "M.F" || kids[1].ActionType != "MicroflowAction" { + t.Errorf("microflow child = %+v", kids[1]) + } + if kids[2].ActionType != "NoAction" { + t.Errorf("a target-less item must be NoAction, got %q", kids[2].ActionType) + } +} diff --git a/mdl/executor/cmd_microflows_create.go b/mdl/executor/cmd_microflows_create.go index d48a134bc..81aa0238d 100644 --- a/mdl/executor/cmd_microflows_create.go +++ b/mdl/executor/cmd_microflows_create.go @@ -91,6 +91,14 @@ func execCreateMicroflow(ctx *ExecContext, s *ast.CreateMicroflowStmt) error { // For CREATE OR REPLACE/MODIFY, reuse the existing ID to preserve references qualifiedName := s.Name.Module + "." + s.Name.Name + + // Refuse before writing if the stored microflow has a call bound to a task + // queue: the rebuild would null it out and nothing downstream would notice. + if existingID != "" { + if err := checkNoQueuedCalls(ctx, existingID, qualifiedName); err != nil { + return err + } + } microflowID := model.ID(types.GenerateID()) if existingID != "" { microflowID = existingID diff --git a/mdl/executor/cmd_modules.go b/mdl/executor/cmd_modules.go index e4e102339..99cb3e454 100644 --- a/mdl/executor/cmd_modules.go +++ b/mdl/executor/cmd_modules.go @@ -15,6 +15,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/security" ) // execCreateModule handles CREATE MODULE statements. @@ -601,6 +602,12 @@ func describeModule(ctx *ExecContext, moduleName string, withAll bool) error { // Output basic CREATE MODULE statement fmt.Fprintf(ctx.Output, "create module %s;\n", targetModule.Name) + // Module roles live in the module's own Security$ModuleSecurity unit rather + // than in any document, so a sweep that describes every document in a module + // still never reports them. Emitting them here is what lets a describe-based + // comparison see a marketplace update that adds, removes or renames a role. + describeModuleRoles(ctx, targetModule) + if !withAll { fmt.Fprintln(ctx.Output, "/") return nil @@ -1203,3 +1210,30 @@ func jarDepIdxByCoord(deps []*types.JarDependency, coordinate string) int { } // Executor method wrappers for callers in unmigrated files. + +// describeModuleRoles emits the module's roles as re-executable statements. +// +// Failures are deliberately silent: DESCRIBE MODULE is useful without the roles, +// and a backend that cannot read module security (the MCP backend, for one) +// should not turn a working describe into an error. +func describeModuleRoles(ctx *ExecContext, mod *model.Module) { + ms, err := ctx.Backend.GetModuleSecurity(mod.ID) + if err != nil || ms == nil || len(ms.ModuleRoles) == 0 { + return + } + // Sort so the output is stable: a differ comparing two describes must not see + // a change because the reader returned roles in a different order. + roles := append([]*security.ModuleRole(nil), ms.ModuleRoles...) + sort.Slice(roles, func(i, j int) bool { return roles[i].Name < roles[j].Name }) + + for _, r := range roles { + // Same shape DESCRIBE MODULE ROLE emits, so the two agree. + fmt.Fprintf(ctx.Output, "create module role %s.%s", mod.Name, r.Name) + if r.Description != "" { + // Double embedded quotes; a description containing one would + // otherwise terminate the literal and produce unparseable output. + fmt.Fprintf(ctx.Output, " description '%s'", strings.ReplaceAll(r.Description, "'", "''")) + } + fmt.Fprintln(ctx.Output, ";") + } +} diff --git a/mdl/executor/cmd_modules_security_test.go b/mdl/executor/cmd_modules_security_test.go new file mode 100644 index 000000000..ac3c85f81 --- /dev/null +++ b/mdl/executor/cmd_modules_security_test.go @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/security" +) + +// TestDescribeModule_EmitsModuleRoles covers the one part of module security that +// no other describe reaches. +// +// Entity access rules already surface in DESCRIBE ENTITY as `grant ... on ...`, +// page access in DESCRIBE PAGE as `grant view on page ...`, and microflow access +// in DESCRIBE MICROFLOW as `grant execute on microflow ...`. Module *roles* live +// in the module's own Security$ModuleSecurity unit and belong to no document, so +// before this they were invisible to any describe-based comparison — a +// marketplace update that added or renamed a role would read as no change. +func TestDescribeModule_EmitsModuleRoles(t *testing.T) { + mod := mkModule("Administration") + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + GetModuleSecurityFunc: func(id model.ID) (*security.ModuleSecurity, error) { + // Deliberately out of alphabetical order: the emitter must sort, or a + // differ sees a phantom change when the reader's order shifts. + return &security.ModuleSecurity{ + ModuleRoles: []*security.ModuleRole{ + {Name: "User"}, + {Name: "Administrator", Description: "Full access"}, + }, + }, nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb)) + assertNoError(t, describeModule(ctx, "Administration", false)) + out := buf.String() + + assertContainsStr(t, out, "create module role Administration.Administrator description 'Full access';") + assertContainsStr(t, out, "create module role Administration.User;") + + if strings.Index(out, "Administration.Administrator") > strings.Index(out, "Administration.User") { + t.Errorf("module roles must be emitted in a stable sorted order, got:\n%s", out) + } +} + +// TestDescribeModule_RoleDescriptionQuoting guards the literal: an apostrophe in a +// description would otherwise close the string and make the output unparseable. +func TestDescribeModule_RoleDescriptionQuoting(t *testing.T) { + mod := mkModule("Sales") + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + GetModuleSecurityFunc: func(id model.ID) (*security.ModuleSecurity, error) { + return &security.ModuleSecurity{ + ModuleRoles: []*security.ModuleRole{{Name: "Rep", Description: "it's theirs"}}, + }, nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb)) + assertNoError(t, describeModule(ctx, "Sales", false)) + assertContainsStr(t, buf.String(), "description 'it''s theirs'") +} + +// TestDescribeModule_SurvivesUnreadableSecurity keeps DESCRIBE MODULE working on a +// backend that cannot read module security (the MCP backend, for one) rather than +// turning a useful describe into an error. +func TestDescribeModule_SurvivesUnreadableSecurity(t *testing.T) { + mod := mkModule("Administration") + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + // GetModuleSecurityFunc left nil: the mock's default returns an error. + } + + ctx, buf := newMockCtx(t, withBackend(mb)) + assertNoError(t, describeModule(ctx, "Administration", false)) + assertContainsStr(t, buf.String(), "create module Administration;") +} diff --git a/mdl/executor/cmd_navigation.go b/mdl/executor/cmd_navigation.go index c01caccb1..32a040c4d 100644 --- a/mdl/executor/cmd_navigation.go +++ b/mdl/executor/cmd_navigation.go @@ -292,7 +292,7 @@ func outputNavigationProfile(ctx *ExecContext, p *types.NavigationProfile) { // Menu items if len(p.MenuItems) > 0 { fmt.Fprintln(ctx.Output, " menu (") - printMenuMDL(ctx.Output, p.MenuItems, 2) + printMenuMDL(ctx.Output, p.MenuItems, 2, "CREATE NAVIGATION") fmt.Fprintln(ctx.Output, " )") } @@ -344,15 +344,17 @@ func menuItemTarget(item *types.NavMenuItem) string { return "" } -// printMenuMDL prints menu items in MDL-style format. -func printMenuMDL(w io.Writer, items []*types.NavMenuItem, depth int) { +// printMenuMDL prints menu items in MDL-style format. reproducer names the +// construct an icon note should point at — navigation menus are authored by +// CREATE NAVIGATION, while a standalone menu document cannot be authored at all. +func printMenuMDL(w io.Writer, items []*types.NavMenuItem, depth int, reproducer string) { indent := strings.Repeat(" ", depth) for _, item := range items { icon := menuItemIconMDL(item) if len(item.Items) > 0 { // Sub-menu container fmt.Fprintf(w, "%smenu '%s'%s (\n", indent, item.Caption, icon) - printMenuMDL(w, item.Items, depth+1) + printMenuMDL(w, item.Items, depth+1, reproducer) fmt.Fprintf(w, "%s);\n", indent) } else if item.Page != "" { fmt.Fprintf(w, "%smenu item '%s' page %s%s;\n", indent, item.Caption, item.Page, icon) @@ -361,7 +363,7 @@ func printMenuMDL(w io.Writer, items []*types.NavMenuItem, depth int) { } else { fmt.Fprintf(w, "%smenu item '%s'%s;\n", indent, item.Caption, icon) } - if note := menuItemIconNote(item); note != "" { + if note := menuItemIconNote(item, reproducer); note != "" { fmt.Fprintf(w, "%s%s\n", indent, note) } } @@ -381,7 +383,7 @@ func menuItemIconMDL(item *types.NavMenuItem) string { // Forms$IconCollectionIcon; a glyph icon (numeric Code) or an image icon // (pointing into an image collection, not an icon collection) is a different // element and would have to be guessed at. -func menuItemIconNote(item *types.NavMenuItem) string { +func menuItemIconNote(item *types.NavMenuItem, reproducer string) string { if item.IconType == "" || strings.HasSuffix(item.IconType, "IconCollectionIcon") { return "" } @@ -389,6 +391,6 @@ func menuItemIconNote(item *types.NavMenuItem) string { if target == "" { target = "a numeric glyph code" } - return fmt.Sprintf("-- icon %s (%s) is not reproducible by CREATE NAVIGATION; set it in Studio Pro", - target, item.IconType) + return fmt.Sprintf("-- icon %s (%s) is not reproducible by %s; set it in Studio Pro", + target, item.IconType, reproducer) } diff --git a/mdl/executor/cmd_navigation_icon_test.go b/mdl/executor/cmd_navigation_icon_test.go index fbb9986d8..8f6cc5189 100644 --- a/mdl/executor/cmd_navigation_icon_test.go +++ b/mdl/executor/cmd_navigation_icon_test.go @@ -15,7 +15,7 @@ import ( // menuMDL renders items through the DESCRIBE emitter. func menuMDL(items []*types.NavMenuItem) string { var b bytes.Buffer - printMenuMDL(&b, items, 0) + printMenuMDL(&b, items, 0, "CREATE NAVIGATION") return b.String() } diff --git a/mdl/executor/cmd_queues.go b/mdl/executor/cmd_queues.go new file mode 100644 index 000000000..76600c65b --- /dev/null +++ b/mdl/executor/cmd_queues.go @@ -0,0 +1,183 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package executor — task queue commands (CREATE/DROP/SHOW/DESCRIBE QUEUE). +// +// A Mendix task queue (Queues$Queue) governs how many instances of a queued +// microflow call run at once, and whether that limit is per runtime instance or +// cluster-wide. +package executor + +import ( + "fmt" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" +) + +// findQueue returns the queue with the given module-qualified name, or nil. +func findQueue(ctx *ExecContext, moduleName, name string) *types.Queue { + queues, err := ctx.Backend.ListQueues() + if err != nil { + return nil + } + h, err := getHierarchy(ctx) + if err != nil { + return nil + } + for _, q := range queues { + if !strings.EqualFold(q.Name, name) { + continue + } + mod := h.GetModuleName(h.FindModuleID(q.ContainerID)) + if strings.EqualFold(mod, moduleName) { + return q + } + } + return nil +} + +// execCreateQueue handles CREATE [OR REPLACE|MODIFY] QUEUE Module.Name (...). +func execCreateQueue(ctx *ExecContext, s *ast.CreateQueueStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + + module, err := findOrCreateModule(ctx, s.Name.Module) + if err != nil { + return err + } + + existing := findQueue(ctx, s.Name.Module, s.Name.Name) + if existing != nil && !s.CreateOrModify { + return mdlerrors.NewAlreadyExists("queue", s.Name.String()) + } + + containerID := module.ID + if existing != nil { + containerID = existing.ContainerID + } + + q := &types.Queue{ + ContainerID: containerID, + Name: s.Name.Name, + Documentation: s.Documentation, + Parallelism: s.Parallelism, + ClusterWide: s.ClusterWide, + ExportLevel: s.ExportLevel, + } + + if existing != nil { + q.ID = existing.ID + if err := ctx.Backend.UpdateQueue(q); err != nil { + return mdlerrors.NewBackend("update queue", err) + } + fmt.Fprintf(ctx.Output, "Modified queue: %s\n", s.Name.String()) + return nil + } + if err := ctx.Backend.CreateQueue(q); err != nil { + return mdlerrors.NewBackend("create queue", err) + } + fmt.Fprintf(ctx.Output, "Created queue: %s\n", s.Name.String()) + return nil +} + +// execDropQueue handles DROP QUEUE Module.Name. +func execDropQueue(ctx *ExecContext, s *ast.DropQueueStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + existing := findQueue(ctx, s.Name.Module, s.Name.Name) + if existing == nil { + return mdlerrors.NewNotFound("queue", s.Name.String()) + } + if err := ctx.Backend.DeleteQueue(string(existing.ID)); err != nil { + return mdlerrors.NewBackend("drop queue", err) + } + fmt.Fprintf(ctx.Output, "Dropped queue: %s\n", s.Name.String()) + return nil +} + +// execShowQueues handles SHOW|LIST QUEUES [IN Module]. +func execShowQueues(ctx *ExecContext, s *ast.ShowQueuesStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + queues, err := ctx.Backend.ListQueues() + if err != nil { + return mdlerrors.NewBackend("list queues", err) + } + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + type row struct{ qualified, parallelism, clusterWide string } + var rows []row + for _, q := range queues { + mod := h.GetModuleName(h.FindModuleID(q.ContainerID)) + if s.Module != "" && !strings.EqualFold(mod, s.Module) { + continue + } + rows = append(rows, row{ + qualified: mod + "." + q.Name, + parallelism: q.Parallelism, + clusterWide: fmt.Sprintf("%t", q.ClusterWide), + }) + } + sort.Slice(rows, func(i, j int) bool { return rows[i].qualified < rows[j].qualified }) + + result := &TableResult{ + Columns: []string{"Queue", "Parallelism", "Cluster Wide"}, + Summary: fmt.Sprintf("(%d queue(s))", len(rows)), + } + for _, r := range rows { + result.Rows = append(result.Rows, []any{r.qualified, r.parallelism, r.clusterWide}) + } + return writeResult(ctx, result) +} + +// execDescribeQueue handles DESCRIBE QUEUE Module.Name, emitting re-executable +// MDL so describe → exec round-trips. +func execDescribeQueue(ctx *ExecContext, s *ast.DescribeQueueStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + q := findQueue(ctx, s.Name.Module, s.Name.Name) + if q == nil { + return mdlerrors.NewNotFound("queue", s.Name.String()) + } + + if q.Documentation != "" { + fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", q.Documentation) + } + fmt.Fprintf(ctx.Output, "create or modify queue %s (\n", s.Name.String()) + // Parallelism is an expression string; quote it unless it is a plain integer, + // so an expression survives the round-trip. + fmt.Fprintf(ctx.Output, " Parallelism: %s,\n", formatParallelism(q.Parallelism)) + fmt.Fprintf(ctx.Output, " ClusterWide: %t,\n", q.ClusterWide) + fmt.Fprint(ctx.Output, ");\n") + return nil +} + +// formatParallelism renders the expression bare when it is a plain integer and +// quoted otherwise. +func formatParallelism(expr string) string { + if expr == "" { + return "1" + } + for _, r := range expr { + if r < '0' || r > '9' { + return "'" + strings.ReplaceAll(expr, "'", "''") + "'" + } + } + return expr +} diff --git a/mdl/executor/cmd_queues_mock_test.go b/mdl/executor/cmd_queues_mock_test.go new file mode 100644 index 000000000..c004b8df8 --- /dev/null +++ b/mdl/executor/cmd_queues_mock_test.go @@ -0,0 +1,247 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +func mkQueue(containerID model.ID, name, parallelism string, clusterWide bool) *types.Queue { + q := &types.Queue{ + ContainerID: containerID, + Name: name, + Parallelism: parallelism, + ClusterWide: clusterWide, + } + q.ID = nextID("queue") + return q +} + +func TestShowQueues_Mock(t *testing.T) { + mod := mkModule("Ops") + q1 := mkQueue(mod.ID, "OrderProcessing", "3", true) + q2 := mkQueue(mod.ID, "Mail", "1", false) + + h := mkHierarchy(mod) + withContainer(h, q1.ContainerID, mod.ID) + withContainer(h, q2.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{q1, q2}, nil }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execShowQueues(ctx, &ast.ShowQueuesStmt{})) + + out := buf.String() + assertContainsStr(t, out, "Ops.OrderProcessing") + assertContainsStr(t, out, "Ops.Mail") + assertContainsStr(t, out, "(2 queue(s))") +} + +func TestShowQueues_Mock_FilterByModule(t *testing.T) { + alpha := mkModule("Alpha") + beta := mkModule("Beta") + q1 := mkQueue(alpha.ID, "One", "1", false) + q2 := mkQueue(beta.ID, "Two", "2", false) + + h := mkHierarchy(alpha, beta) + withContainer(h, q1.ContainerID, alpha.ID) + withContainer(h, q2.ContainerID, beta.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{q1, q2}, nil }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execShowQueues(ctx, &ast.ShowQueuesStmt{Module: "Beta"})) + + out := buf.String() + assertNotContainsStr(t, out, "Alpha.One") + assertContainsStr(t, out, "Beta.Two") +} + +// TestCreateQueue_Mock_PassesParallelismThrough checks that the expression +// reaches the backend as written. Parallelism is an expression, so it must not +// be parsed into a number anywhere on the way down. +func TestCreateQueue_Mock_PassesParallelismThrough(t *testing.T) { + mod := mkModule("Ops") + h := mkHierarchy(mod) + + var created *types.Queue + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return nil, nil }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + CreateQueueFunc: func(q *types.Queue) error { + created = q + return nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + stmt := &ast.CreateQueueStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "OrderProcessing"}, + Parallelism: "$Config/Workers", + ClusterWide: true, + } + assertNoError(t, execCreateQueue(ctx, stmt)) + + if created == nil { + t.Fatal("CreateQueue was not called") + } + if created.Parallelism != "$Config/Workers" { + t.Errorf("Parallelism = %q, want the expression verbatim", created.Parallelism) + } + if !created.ClusterWide { + t.Error("ClusterWide did not reach the backend") + } + assertContainsStr(t, buf.String(), "Created queue: Ops.OrderProcessing") +} + +func TestCreateQueue_Mock_DuplicateWithoutOrModify(t *testing.T) { + mod := mkModule("Ops") + existing := mkQueue(mod.ID, "OrderProcessing", "1", false) + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{existing}, nil }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + CreateQueueFunc: func(q *types.Queue) error { + t.Error("CreateQueue must not be called for a duplicate") + return nil + }, + } + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + stmt := &ast.CreateQueueStmt{Name: ast.QualifiedName{Module: "Ops", Name: "OrderProcessing"}} + if err := execCreateQueue(ctx, stmt); err == nil { + t.Fatal("expected an already-exists error") + } +} + +func TestCreateQueue_Mock_OrModifyUpdates(t *testing.T) { + mod := mkModule("Ops") + existing := mkQueue(mod.ID, "OrderProcessing", "1", false) + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + var updated *types.Queue + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{existing}, nil }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + UpdateQueueFunc: func(q *types.Queue) error { + updated = q + return nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + stmt := &ast.CreateQueueStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "OrderProcessing"}, + Parallelism: "8", + CreateOrModify: true, + } + assertNoError(t, execCreateQueue(ctx, stmt)) + + if updated == nil { + t.Fatal("UpdateQueue was not called") + } + // The stored ID must be reused, or the update becomes a second queue. + if updated.ID != existing.ID { + t.Errorf("ID = %q, want the existing %q", updated.ID, existing.ID) + } + if updated.Parallelism != "8" { + t.Errorf("Parallelism = %q, want 8", updated.Parallelism) + } + assertContainsStr(t, buf.String(), "Modified queue: Ops.OrderProcessing") +} + +func TestDropQueue_Mock(t *testing.T) { + mod := mkModule("Ops") + existing := mkQueue(mod.ID, "OrderProcessing", "1", false) + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + var deleted string + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{existing}, nil }, + DeleteQueueFunc: func(id string) error { + deleted = id + return nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + stmt := &ast.DropQueueStmt{Name: ast.QualifiedName{Module: "Ops", Name: "OrderProcessing"}} + assertNoError(t, execDropQueue(ctx, stmt)) + + if deleted != string(existing.ID) { + t.Errorf("deleted %q, want %q", deleted, existing.ID) + } + assertContainsStr(t, buf.String(), "Dropped queue: Ops.OrderProcessing") +} + +func TestDropQueue_Mock_NotFound(t *testing.T) { + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return nil, nil }, + DeleteQueueFunc: func(id string) error { + t.Error("DeleteQueue must not be called when the queue does not exist") + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(mkHierarchy())) + stmt := &ast.DropQueueStmt{Name: ast.QualifiedName{Module: "Ops", Name: "Missing"}} + if err := execDropQueue(ctx, stmt); err == nil { + t.Fatal("expected a not-found error") + } +} + +// TestDescribeQueue_Mock_RoundTrips checks that DESCRIBE emits MDL that can be +// fed straight back in, including quoting a non-numeric parallelism expression. +func TestDescribeQueue_Mock_RoundTrips(t *testing.T) { + mod := mkModule("Ops") + q := mkQueue(mod.ID, "OrderProcessing", "$Config/Workers", true) + h := mkHierarchy(mod) + withContainer(h, q.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListQueuesFunc: func() ([]*types.Queue, error) { return []*types.Queue{q}, nil }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + stmt := &ast.DescribeQueueStmt{Name: ast.QualifiedName{Module: "Ops", Name: "OrderProcessing"}} + assertNoError(t, execDescribeQueue(ctx, stmt)) + + out := buf.String() + assertContainsStr(t, out, "create or modify queue Ops.OrderProcessing (") + assertContainsStr(t, out, "Parallelism: '$Config/Workers',") + assertContainsStr(t, out, "ClusterWide: true,") +} + +func TestFormatParallelism(t *testing.T) { + tests := []struct{ in, want string }{ + {"", "1"}, + {"3", "3"}, + {"$Config/Workers", "'$Config/Workers'"}, + {"it's", "'it''s'"}, // Mendix escapes a quote by doubling it, never with a backslash + } + for _, tt := range tests { + if got := formatParallelism(tt.in); got != tt.want { + t.Errorf("formatParallelism(%q) = %q, want %q", tt.in, got, tt.want) + } + } +} diff --git a/mdl/executor/cmd_scheduledevents.go b/mdl/executor/cmd_scheduledevents.go new file mode 100644 index 000000000..a89b2c4ac --- /dev/null +++ b/mdl/executor/cmd_scheduledevents.go @@ -0,0 +1,586 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package executor — scheduled event commands (CREATE/DROP/SHOW/DESCRIBE +// SCHEDULED EVENT). +// +// A Mendix scheduled event runs a microflow on a repeating schedule. It is +// Mendix's cron: the repeat rule is a ScheduledEvents$Schedule child with eight +// variants, and OnOverlap (DelayNext|SkipNext) is its own concurrency control — +// scheduled events do not go through a task queue. +package executor + +import ( + "fmt" + "sort" + "strings" + "time" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// repeatKinds maps the MDL Repeat value to the storage's schedule variant. +// The MDL spellings are the calendar words a reader expects; the storage names +// are Mendix's. +var repeatKinds = map[string]model.ScheduleKind{ + "minutely": model.ScheduleMinute, + "hourly": model.ScheduleHour, + "daily": model.ScheduleDay, + "weekly": model.ScheduleWeek, + "monthlybydate": model.ScheduleMonthDate, + "monthlybyweekday": model.ScheduleMonthWeekday, + "yearlybydate": model.ScheduleYearDate, + "yearlybyweekday": model.ScheduleYearWeekday, +} + +// repeatNames is the reverse map, for DESCRIBE. +var repeatNames = map[model.ScheduleKind]string{ + model.ScheduleMinute: "Minutely", + model.ScheduleHour: "Hourly", + model.ScheduleDay: "Daily", + model.ScheduleWeek: "Weekly", + model.ScheduleMonthDate: "MonthlyByDate", + model.ScheduleMonthWeekday: "MonthlyByWeekday", + model.ScheduleYearDate: "YearlyByDate", + model.ScheduleYearWeekday: "YearlyByWeekday", +} + +// repeatFields lists the properties each repeat actually uses. A property that +// belongs to a different variant is REFUSED rather than ignored: the variants +// differ in which fields they carry, so silently dropping one would write a +// schedule that does not do what the script says. +var repeatFields = map[model.ScheduleKind][]string{ + model.ScheduleMinute: {"Multiplier"}, + model.ScheduleHour: {"Multiplier", "MinuteOffset"}, + model.ScheduleDay: {"HourOfDay", "MinuteOfHour"}, + model.ScheduleWeek: {"Weekdays", "HourOfDay", "MinuteOfHour"}, + model.ScheduleMonthDate: {"Multiplier", "MonthOffset", "DayOfMonth", "HourOfDay", "MinuteOfHour"}, + model.ScheduleMonthWeekday: {"Multiplier", "MonthOffset", "DaySelector", "Weekday", "HourOfDay", "MinuteOfHour"}, + model.ScheduleYearDate: {"Month", "DayOfMonth", "HourOfDay", "MinuteOfHour"}, + model.ScheduleYearWeekday: {"Month", "DaySelector", "Weekday", "HourOfDay", "MinuteOfHour"}, +} + +var daySelectors = []string{"First", "Second", "Third", "Fourth", "Last"} + +var weekdayNames = []string{"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"} + +// findScheduledEvent returns the event with the given module-qualified name, or nil. +func findScheduledEvent(ctx *ExecContext, moduleName, name string) *model.ScheduledEvent { + events, err := ctx.Backend.ListScheduledEvents() + if err != nil { + return nil + } + h, err := getHierarchy(ctx) + if err != nil { + return nil + } + for _, ev := range events { + if !strings.EqualFold(ev.Name, name) { + continue + } + mod := h.GetModuleName(h.FindModuleID(ev.ContainerID)) + if strings.EqualFold(mod, moduleName) { + return ev + } + } + return nil +} + +// execCreateScheduledEvent handles CREATE [OR REPLACE|MODIFY] SCHEDULED EVENT. +func execCreateScheduledEvent(ctx *ExecContext, s *ast.CreateScheduledEventStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + + module, err := findOrCreateModule(ctx, s.Name.Module) + if err != nil { + return err + } + + existing := findScheduledEvent(ctx, s.Name.Module, s.Name.Name) + if existing != nil && !s.CreateOrModify { + return mdlerrors.NewAlreadyExists("scheduled event", s.Name.String()) + } + + ev, err := scheduledEventFromStmt(s) + if err != nil { + return err + } + ev.ContainerID = module.ID + if existing != nil { + ev.ID = existing.ID + ev.ContainerID = existing.ContainerID + // Interval/IntervalType are legacy siblings of Schedule that Studio Pro + // writes but does not keep in sync, and MDL has no syntax for them. + // Carry the stored values so a modify does not invent new ones. + ev.Interval = existing.Interval + ev.IntervalType = existing.IntervalType + if err := ctx.Backend.UpdateScheduledEvent(ev); err != nil { + return mdlerrors.NewBackend("update scheduled event", err) + } + fmt.Fprintf(ctx.Output, "Modified scheduled event: %s\n", s.Name.String()) + return nil + } + if err := ctx.Backend.CreateScheduledEvent(ev); err != nil { + return mdlerrors.NewBackend("create scheduled event", err) + } + fmt.Fprintf(ctx.Output, "Created scheduled event: %s\n", s.Name.String()) + return nil +} + +// scheduledEventFromStmt validates the statement and builds the semantic event. +func scheduledEventFromStmt(s *ast.CreateScheduledEventStmt) (*model.ScheduledEvent, error) { + if strings.TrimSpace(s.Microflow) == "" { + return nil, mdlerrors.NewValidation( + "scheduled event " + s.Name.String() + " has no Microflow — a scheduled event runs a microflow, " + + "so add `Microflow: Module.MyMicroflow` to the property list") + } + ev := &model.ScheduledEvent{ + Name: s.Name.Name, + Documentation: s.Documentation, + MicroflowID: model.ID(s.Microflow), + TimeZone: s.TimeZone, + OnOverlap: s.OnOverlap, + ExportLevel: s.ExportLevel, + } + if s.Enabled != nil { + ev.Enabled = *s.Enabled + } + if s.Excluded != nil { + ev.Excluded = *s.Excluded + } + if err := validateEnumProperty("TimeZone", ev.TimeZone, []string{"UTC", "Server"}); err != nil { + return nil, err + } + if err := validateEnumProperty("OnOverlap", ev.OnOverlap, []string{"DelayNext", "SkipNext"}); err != nil { + return nil, err + } + if s.StartDateTime != "" { + t, err := time.Parse(time.RFC3339, s.StartDateTime) + if err != nil { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "StartDateTime %q is not an RFC 3339 timestamp (e.g. '2026-01-01T04:00:00Z')", s.StartDateTime)) + } + utc := t.UTC() + ev.StartDateTime = &utc + } + + sched, err := scheduleFromStmt(s) + if err != nil { + return nil, err + } + ev.Schedule = sched + ev.Interval, ev.IntervalType = legacyIntervalFor(sched) + return ev, nil +} + +// legacyIntervalFor derives the Interval/IntervalType pair a new event gets. +// +// These are legacy siblings of Schedule with no MDL syntax, but they are not +// optional: IntervalType is a Mendix enumeration, and writing an empty string +// puts a value in the model that the enumeration does not have. Every Studio +// Pro-authored event carries a real one. +// +// The mapping is what the two self-consistent references show (SAML: Day/1 +// beside a DaySchedule; OIDC: Hour/1 beside an HourSchedule with Multiplier 1). +// It is only used on CREATE — a modify carries the stored pair through +// untouched, because Studio Pro does NOT keep these in sync with Schedule (the +// Workflow Commons event stores Minute/0 next to a DaySchedule) and re-deriving +// them would overwrite whatever the developer's Studio Pro left behind. +func legacyIntervalFor(s *model.Schedule) (int, string) { + if s == nil { + return 1, "Day" + } + switch s.Kind { + case model.ScheduleMinute: + return s.Multiplier, "Minute" + case model.ScheduleHour: + return s.Multiplier, "Hour" + case model.ScheduleDay: + return 1, "Day" + case model.ScheduleWeek: + return 1, "Week" + case model.ScheduleMonthDate, model.ScheduleMonthWeekday: + return s.Multiplier, "Month" + case model.ScheduleYearDate, model.ScheduleYearWeekday: + return 1, "Year" + } + return 1, "Day" +} + +// scheduleFromStmt builds the repeat rule, refusing any field that does not +// belong to the chosen Repeat. +func scheduleFromStmt(s *ast.CreateScheduledEventStmt) (*model.Schedule, error) { + if strings.TrimSpace(s.Repeat) == "" { + return nil, mdlerrors.NewValidation( + "scheduled event " + s.Name.String() + " has no Repeat — add one of " + + strings.Join(sortedRepeatNames(), ", ")) + } + kind, ok := repeatKinds[strings.ToLower(s.Repeat)] + if !ok { + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "unknown Repeat %q — expected one of %s", s.Repeat, strings.Join(sortedRepeatNames(), ", "))) + } + + // Which properties the statement actually set, so a field belonging to a + // different variant is reported instead of silently dropped. + set := map[string]bool{ + "Multiplier": s.Multiplier != nil, + "MinuteOffset": s.MinuteOffset != nil, + "MonthOffset": s.MonthOffset != nil, + "HourOfDay": s.HourOfDay != nil, + "MinuteOfHour": s.MinuteOfHour != nil, + "DayOfMonth": s.DayOfMonth != nil, + "Month": s.Month != nil, + "Weekdays": s.Weekdays != "", + "DaySelector": s.DaySelector != "", + "Weekday": s.Weekday != "", + } + allowed := map[string]bool{} + for _, f := range repeatFields[kind] { + allowed[f] = true + } + var stray []string + for f, isSet := range set { + if isSet && !allowed[f] { + stray = append(stray, f) + } + } + if len(stray) > 0 { + sort.Strings(stray) + return nil, mdlerrors.NewValidation(fmt.Sprintf( + "Repeat %s does not have %s — it takes %s", + s.Repeat, strings.Join(stray, ", "), strings.Join(repeatFields[kind], ", "))) + } + + sched := &model.Schedule{Kind: kind} + if s.Multiplier != nil { + sched.Multiplier = *s.Multiplier + } else if allowed["Multiplier"] { + // "every 1 " — an unstated multiplier of 0 would never fire. + sched.Multiplier = 1 + } + if s.MinuteOffset != nil { + sched.MinuteOffset = *s.MinuteOffset + } + if s.MonthOffset != nil { + sched.MonthOffset = *s.MonthOffset + } + if s.HourOfDay != nil { + sched.HourOfDay = *s.HourOfDay + } + if s.MinuteOfHour != nil { + sched.MinuteOfHour = *s.MinuteOfHour + } + if s.DayOfMonth != nil { + sched.DayOfMonth = *s.DayOfMonth + } else if allowed["DayOfMonth"] { + sched.DayOfMonth = 1 + } + if s.Month != nil { + sched.Month = *s.Month + } else if allowed["Month"] { + sched.Month = 1 + } + sched.DaySelector = s.DaySelector + sched.Weekday = s.Weekday + + if err := validateScheduleRanges(sched, allowed); err != nil { + return nil, err + } + if allowed["Weekdays"] { + days, err := parseWeekdays(s.Weekdays) + if err != nil { + return nil, err + } + sched.Weekdays = days + } + if allowed["DaySelector"] { + if err := validateEnumProperty("DaySelector", sched.DaySelector, daySelectors); err != nil { + return nil, err + } + if sched.DaySelector == "" { + sched.DaySelector = "First" + } + if err := validateEnumProperty("Weekday", sched.Weekday, weekdayNames); err != nil { + return nil, err + } + if sched.Weekday == "" { + sched.Weekday = "Monday" + } + } + return sched, nil +} + +// validateScheduleRanges rejects values Mendix's editor would not let you enter. +// A schedule that is stored but can never fire is worse than a refusal, because +// nothing downstream reports it. +func validateScheduleRanges(s *model.Schedule, allowed map[string]bool) error { + checks := []struct { + name string + value int + min, max int + }{ + {"Multiplier", s.Multiplier, 1, 1_000_000}, + {"MinuteOffset", s.MinuteOffset, 0, 59}, + {"MonthOffset", s.MonthOffset, 0, 11}, + {"HourOfDay", s.HourOfDay, 0, 23}, + {"MinuteOfHour", s.MinuteOfHour, 0, 59}, + {"DayOfMonth", s.DayOfMonth, 1, 31}, + {"Month", s.Month, 1, 12}, + } + for _, c := range checks { + if !allowed[c.name] { + continue + } + if c.value < c.min || c.value > c.max { + return mdlerrors.NewValidation(fmt.Sprintf( + "%s is %d — it must be between %d and %d", c.name, c.value, c.min, c.max)) + } + } + return nil +} + +// parseWeekdays turns "Monday, Friday" into the seven flags of a WeekSchedule. +func parseWeekdays(spec string) ([7]bool, error) { + var days [7]bool + if strings.TrimSpace(spec) == "" { + return days, mdlerrors.NewValidation( + "Repeat Weekly needs Weekdays — e.g. Weekdays: 'Monday, Friday'") + } + for _, part := range strings.Split(spec, ",") { + name := strings.TrimSpace(part) + if name == "" { + continue + } + idx := -1 + for i, w := range weekdayNames { + if strings.EqualFold(w, name) { + idx = i + break + } + } + if idx < 0 { + return days, mdlerrors.NewValidation(fmt.Sprintf( + "unknown weekday %q in Weekdays — expected one of %s", name, strings.Join(weekdayNames, ", "))) + } + days[idx] = true + } + return days, nil +} + +// validateEnumProperty accepts an empty value (the writer supplies the default) +// and otherwise requires an exact match, normalising nothing — an enum value +// Mendix does not know produces a document that loads and misbehaves. +func validateEnumProperty(name, value string, allowed []string) error { + if value == "" { + return nil + } + for _, a := range allowed { + if a == value { + return nil + } + } + for _, a := range allowed { + if strings.EqualFold(a, value) { + return mdlerrors.NewValidation(fmt.Sprintf( + "%s %q has the wrong casing — Mendix stores it as %q", name, value, a)) + } + } + return mdlerrors.NewValidation(fmt.Sprintf( + "unknown %s %q — expected one of %s", name, value, strings.Join(allowed, ", "))) +} + +func sortedRepeatNames() []string { + out := make([]string, 0, len(repeatNames)) + for _, n := range repeatNames { + out = append(out, n) + } + sort.Strings(out) + return out +} + +// execDropScheduledEvent handles DROP SCHEDULED EVENT Module.Name. +func execDropScheduledEvent(ctx *ExecContext, s *ast.DropScheduledEventStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + if !ctx.ConnectedForWrite() { + return mdlerrors.NewNotConnectedWrite() + } + existing := findScheduledEvent(ctx, s.Name.Module, s.Name.Name) + if existing == nil { + return mdlerrors.NewNotFound("scheduled event", s.Name.String()) + } + if err := ctx.Backend.DeleteScheduledEvent(string(existing.ID)); err != nil { + return mdlerrors.NewBackend("drop scheduled event", err) + } + fmt.Fprintf(ctx.Output, "Dropped scheduled event: %s\n", s.Name.String()) + return nil +} + +// execShowScheduledEvents handles SHOW|LIST SCHEDULED EVENTS [IN Module]. +func execShowScheduledEvents(ctx *ExecContext, s *ast.ShowScheduledEventsStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + events, err := ctx.Backend.ListScheduledEvents() + if err != nil { + return mdlerrors.NewBackend("list scheduled events", err) + } + h, err := getHierarchy(ctx) + if err != nil { + return mdlerrors.NewBackend("build hierarchy", err) + } + + type row struct{ qualified, repeat, microflow, enabled string } + var rows []row + for _, ev := range events { + mod := h.GetModuleName(h.FindModuleID(ev.ContainerID)) + if s.Module != "" && !strings.EqualFold(mod, s.Module) { + continue + } + rows = append(rows, row{ + qualified: mod + "." + ev.Name, + repeat: describeRepeat(ev.Schedule), + microflow: string(ev.MicroflowID), + enabled: fmt.Sprintf("%t", ev.Enabled), + }) + } + sort.Slice(rows, func(i, j int) bool { return rows[i].qualified < rows[j].qualified }) + + result := &TableResult{ + Columns: []string{"Scheduled Event", "Repeat", "Microflow", "Enabled"}, + Summary: fmt.Sprintf("(%d scheduled event(s))", len(rows)), + } + for _, r := range rows { + result.Rows = append(result.Rows, []any{r.qualified, r.repeat, r.microflow, r.enabled}) + } + return writeResult(ctx, result) +} + +// describeRepeat renders the schedule as a short human phrase for the listing. +func describeRepeat(s *model.Schedule) string { + if s == nil { + return "(none)" + } + at := fmt.Sprintf("%02d:%02d", s.HourOfDay, s.MinuteOfHour) + switch s.Kind { + case model.ScheduleMinute: + return fmt.Sprintf("every %d min", s.Multiplier) + case model.ScheduleHour: + return fmt.Sprintf("every %dh at :%02d", s.Multiplier, s.MinuteOffset) + case model.ScheduleDay: + return "daily at " + at + case model.ScheduleWeek: + return "weekly " + strings.Join(setWeekdayNames(s.Weekdays), "/") + " at " + at + case model.ScheduleMonthDate: + return fmt.Sprintf("every %d month(s) on day %d at %s", s.Multiplier, s.DayOfMonth, at) + case model.ScheduleMonthWeekday: + return fmt.Sprintf("every %d month(s) on the %s %s at %s", s.Multiplier, s.DaySelector, s.Weekday, at) + case model.ScheduleYearDate: + return fmt.Sprintf("yearly on %d/%d at %s", s.Month, s.DayOfMonth, at) + case model.ScheduleYearWeekday: + return fmt.Sprintf("yearly on the %s %s of month %d at %s", s.DaySelector, s.Weekday, s.Month, at) + } + return string(s.Kind) +} + +func setWeekdayNames(days [7]bool) []string { + var out []string + for i, on := range days { + if on { + out = append(out, weekdayNames[i][:3]) + } + } + if out == nil { + return []string{"(no days)"} + } + return out +} + +// execDescribeScheduledEvent handles DESCRIBE SCHEDULED EVENT Module.Name, +// emitting re-executable MDL so describe → exec round-trips. +func execDescribeScheduledEvent(ctx *ExecContext, s *ast.DescribeScheduledEventStmt) error { + if !ctx.Connected() { + return mdlerrors.NewNotConnected() + } + ev := findScheduledEvent(ctx, s.Name.Module, s.Name.Name) + if ev == nil { + return mdlerrors.NewNotFound("scheduled event", s.Name.String()) + } + + if ev.Documentation != "" { + fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", ev.Documentation) + } + fmt.Fprintf(ctx.Output, "create or modify scheduled event %s (\n", s.Name.String()) + fmt.Fprintf(ctx.Output, " Microflow: %s,\n", ev.MicroflowID) + for _, line := range describeScheduleProperties(ev.Schedule) { + fmt.Fprintf(ctx.Output, " %s,\n", line) + } + fmt.Fprintf(ctx.Output, " Enabled: %t,\n", ev.Enabled) + if ev.OnOverlap != "" { + fmt.Fprintf(ctx.Output, " OnOverlap: %s,\n", ev.OnOverlap) + } + if ev.TimeZone != "" { + fmt.Fprintf(ctx.Output, " TimeZone: %s,\n", ev.TimeZone) + } + // A zero StartDateTime is how "no start restriction" is stored (Mendix's own + // DateTime.MinValue), not a date anyone chose — emitting it would put + // '0001-01-01T00:00:00Z' into every describe. + if ev.StartDateTime != nil && !ev.StartDateTime.IsZero() { + fmt.Fprintf(ctx.Output, " StartDateTime: '%s',\n", ev.StartDateTime.Format(time.RFC3339)) + } + fmt.Fprint(ctx.Output, ");\n") + // Interval/IntervalType have no MDL syntax: they are legacy siblings of + // Schedule that Studio Pro writes without keeping in sync. Reporting them as + // a comment keeps the output honest without making it non-re-executable. + if ev.IntervalType != "" { + fmt.Fprintf(ctx.Output, "-- legacy (preserved, not authorable): Interval %d %s\n", ev.Interval, ev.IntervalType) + } + return nil +} + +// describeScheduleProperties emits the Repeat plus exactly the fields that +// repeat uses, in the order repeatFields lists them. +func describeScheduleProperties(s *model.Schedule) []string { + if s == nil { + return nil + } + out := []string{"Repeat: " + repeatNames[s.Kind]} + for _, f := range repeatFields[s.Kind] { + switch f { + case "Multiplier": + out = append(out, fmt.Sprintf("Multiplier: %d", s.Multiplier)) + case "MinuteOffset": + out = append(out, fmt.Sprintf("MinuteOffset: %d", s.MinuteOffset)) + case "MonthOffset": + out = append(out, fmt.Sprintf("MonthOffset: %d", s.MonthOffset)) + case "HourOfDay": + out = append(out, fmt.Sprintf("HourOfDay: %d", s.HourOfDay)) + case "MinuteOfHour": + out = append(out, fmt.Sprintf("MinuteOfHour: %d", s.MinuteOfHour)) + case "DayOfMonth": + out = append(out, fmt.Sprintf("DayOfMonth: %d", s.DayOfMonth)) + case "Month": + out = append(out, fmt.Sprintf("Month: %d", s.Month)) + case "DaySelector": + out = append(out, "DaySelector: "+s.DaySelector) + case "Weekday": + out = append(out, "Weekday: "+s.Weekday) + case "Weekdays": + var names []string + for i, on := range s.Weekdays { + if on { + names = append(names, weekdayNames[i]) + } + } + out = append(out, "Weekdays: '"+strings.Join(names, ", ")+"'") + } + } + return out +} diff --git a/mdl/executor/cmd_scheduledevents_test.go b/mdl/executor/cmd_scheduledevents_test.go new file mode 100644 index 000000000..bae0dbc0b --- /dev/null +++ b/mdl/executor/cmd_scheduledevents_test.go @@ -0,0 +1,456 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" +) + +func mkScheduledEvent(containerID model.ID, name string, s *model.Schedule) *model.ScheduledEvent { + ev := &model.ScheduledEvent{ + ContainerID: containerID, + Name: name, + MicroflowID: "Ops.SE_Cleanup", + Schedule: s, + OnOverlap: "DelayNext", + TimeZone: "UTC", + } + ev.ID = nextID("sched") + return ev +} + +func intPtr(v int) *int { return &v } + +// TestScheduleFromStmt_EachRepeatTakesOnlyItsOwnFields is the load-bearing test: +// the eight ScheduledEvents$*Schedule variants differ in WHICH fields they +// carry, and a merged field set is the shape mxbuild accepts and Studio Pro +// refuses to open. +func TestScheduleFromStmt_EachRepeatTakesOnlyItsOwnFields(t *testing.T) { + tests := []struct { + repeat string + stray string // a property that belongs to a different repeat + apply func(*ast.CreateScheduledEventStmt) + }{ + {"Daily", "Multiplier", func(s *ast.CreateScheduledEventStmt) { s.Multiplier = intPtr(3) }}, + {"Minutely", "HourOfDay", func(s *ast.CreateScheduledEventStmt) { s.HourOfDay = intPtr(4) }}, + {"Hourly", "DayOfMonth", func(s *ast.CreateScheduledEventStmt) { s.DayOfMonth = intPtr(15) }}, + {"YearlyByDate", "MonthOffset", func(s *ast.CreateScheduledEventStmt) { s.MonthOffset = intPtr(1) }}, + {"MonthlyByDate", "DaySelector", func(s *ast.CreateScheduledEventStmt) { s.DaySelector = "Last" }}, + {"Daily", "Weekdays", func(s *ast.CreateScheduledEventStmt) { s.Weekdays = "Monday" }}, + } + for _, tt := range tests { + t.Run(tt.repeat+"/"+tt.stray, func(t *testing.T) { + s := &ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, + Microflow: "Ops.MF", + Repeat: tt.repeat, + } + tt.apply(s) + _, err := scheduleFromStmt(s) + if err == nil { + t.Fatalf("%s accepted %s, which belongs to a different repeat", tt.repeat, tt.stray) + } + if !strings.Contains(err.Error(), tt.stray) { + t.Errorf("error should name the stray property:\n%s", err) + } + }) + } +} + +func TestScheduleFromStmt_RangesAreEnforced(t *testing.T) { + tests := []struct { + name string + repeat string + apply func(*ast.CreateScheduledEventStmt) + }{ + {"HourOfDay 24", "Daily", func(s *ast.CreateScheduledEventStmt) { s.HourOfDay = intPtr(24) }}, + {"MinuteOfHour 60", "Daily", func(s *ast.CreateScheduledEventStmt) { s.MinuteOfHour = intPtr(60) }}, + {"DayOfMonth 0", "MonthlyByDate", func(s *ast.CreateScheduledEventStmt) { s.DayOfMonth = intPtr(0) }}, + {"Month 13", "YearlyByDate", func(s *ast.CreateScheduledEventStmt) { s.Month = intPtr(13) }}, + {"Multiplier 0", "Minutely", func(s *ast.CreateScheduledEventStmt) { s.Multiplier = intPtr(0) }}, + {"MinuteOffset 60", "Hourly", func(s *ast.CreateScheduledEventStmt) { s.MinuteOffset = intPtr(60) }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + s := &ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, + Microflow: "Ops.MF", + Repeat: tt.repeat, + } + tt.apply(s) + if _, err := scheduleFromStmt(s); err == nil { + t.Fatalf("%s was accepted — a schedule that can never fire is worse than a refusal", tt.name) + } + }) + } +} + +func TestScheduleFromStmt_Defaults(t *testing.T) { + // An unstated multiplier of 0 would never fire, so it defaults to 1. + got, err := scheduleFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", Repeat: "Hourly", + }) + if err != nil { + t.Fatalf("scheduleFromStmt: %v", err) + } + if got.Multiplier != 1 { + t.Errorf("Multiplier = %d, want 1", got.Multiplier) + } + if got.Kind != model.ScheduleHour { + t.Errorf("Kind = %v", got.Kind) + } +} + +func TestScheduleFromStmt_Weekdays(t *testing.T) { + got, err := scheduleFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + Repeat: "Weekly", Weekdays: "monday, FRIDAY", HourOfDay: intPtr(9), + }) + if err != nil { + t.Fatalf("scheduleFromStmt: %v", err) + } + want := [7]bool{false, true, false, false, false, true, false} + if got.Weekdays != want { + t.Errorf("Weekdays = %v, want %v (case-insensitive names)", got.Weekdays, want) + } +} + +func TestScheduleFromStmt_UnknownWeekday(t *testing.T) { + _, err := scheduleFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + Repeat: "Weekly", Weekdays: "Moonday", + }) + if err == nil || !strings.Contains(err.Error(), "Moonday") { + t.Fatalf("expected an unknown-weekday error, got %v", err) + } +} + +// TestScheduledEventFromStmt_EnumCasing checks that a near-miss enum value is +// reported rather than normalised: Mendix stores the exact spelling, and a value +// the enumeration does not have produces a document that loads and misbehaves. +func TestScheduledEventFromStmt_EnumCasing(t *testing.T) { + _, err := scheduledEventFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + Repeat: "Daily", TimeZone: "server", + }) + if err == nil || !strings.Contains(err.Error(), "Server") { + t.Fatalf("expected a casing error naming the stored spelling, got %v", err) + } +} + +func TestScheduledEventFromStmt_RequiresMicroflowAndRepeat(t *testing.T) { + if _, err := scheduledEventFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Repeat: "Daily", + }); err == nil { + t.Error("a scheduled event with no Microflow was accepted") + } + if _, err := scheduledEventFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + }); err == nil { + t.Error("a scheduled event with no Repeat was accepted") + } +} + +func TestScheduledEventFromStmt_StartDateTime(t *testing.T) { + ev, err := scheduledEventFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + Repeat: "Daily", StartDateTime: "2026-01-01T04:00:00Z", + }) + if err != nil { + t.Fatalf("scheduledEventFromStmt: %v", err) + } + if ev.StartDateTime == nil || ev.StartDateTime.Year() != 2026 { + t.Errorf("StartDateTime = %v", ev.StartDateTime) + } + if _, err := scheduledEventFromStmt(&ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, Microflow: "Ops.MF", + Repeat: "Daily", StartDateTime: "1 January 2026", + }); err == nil { + t.Error("a non-RFC-3339 StartDateTime was accepted") + } +} + +// TestLegacyIntervalFor pins the Interval/IntervalType pair a new event gets. +// IntervalType is a Mendix enumeration, so an empty string is not a valid +// "unset" — every Studio Pro-authored event carries a real value. +func TestLegacyIntervalFor(t *testing.T) { + tests := []struct { + kind model.ScheduleKind + mult int + wantN int + wantType string + }{ + {model.ScheduleMinute, 5, 5, "Minute"}, + {model.ScheduleHour, 2, 2, "Hour"}, + {model.ScheduleDay, 0, 1, "Day"}, + {model.ScheduleWeek, 0, 1, "Week"}, + {model.ScheduleMonthDate, 3, 3, "Month"}, + {model.ScheduleYearWeekday, 0, 1, "Year"}, + } + for _, tt := range tests { + n, typ := legacyIntervalFor(&model.Schedule{Kind: tt.kind, Multiplier: tt.mult}) + if n != tt.wantN || typ != tt.wantType { + t.Errorf("%s -> %d/%q, want %d/%q", tt.kind, n, typ, tt.wantN, tt.wantType) + } + if typ == "" { + t.Errorf("%s produced an empty IntervalType — not a value the enumeration has", tt.kind) + } + } +} + +func TestShowScheduledEvents_Mock(t *testing.T) { + mod := mkModule("Ops") + e1 := mkScheduledEvent(mod.ID, "Nightly", &model.Schedule{Kind: model.ScheduleDay, HourOfDay: 4}) + e1.Enabled = true + e2 := mkScheduledEvent(mod.ID, "Hourly", &model.Schedule{Kind: model.ScheduleHour, Multiplier: 2, MinuteOffset: 23}) + + h := mkHierarchy(mod) + withContainer(h, e1.ContainerID, mod.ID) + withContainer(h, e2.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{e1, e2}, nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execShowScheduledEvents(ctx, &ast.ShowScheduledEventsStmt{})) + + out := buf.String() + assertContainsStr(t, out, "Ops.Nightly") + assertContainsStr(t, out, "daily at 04:00") + assertContainsStr(t, out, "every 2h at :23") + assertContainsStr(t, out, "(2 scheduled event(s))") +} + +func TestCreateScheduledEvent_Mock(t *testing.T) { + mod := mkModule("Ops") + h := mkHierarchy(mod) + + var created *model.ScheduledEvent + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { return nil, nil }, + CreateScheduledEventFunc: func(ev *model.ScheduledEvent) error { + created = ev + return nil + }, + } + + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execCreateScheduledEvent(ctx, &ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "Nightly"}, + Microflow: "Ops.SE_Cleanup", + Repeat: "Daily", + HourOfDay: intPtr(4), + MinuteOfHour: intPtr(0), + Enabled: boolPtr(true), + })) + + if created == nil { + t.Fatal("CreateScheduledEvent was not called") + } + if created.Schedule == nil || created.Schedule.Kind != model.ScheduleDay { + t.Fatalf("Schedule = %+v", created.Schedule) + } + if created.Schedule.HourOfDay != 4 { + t.Errorf("HourOfDay = %d", created.Schedule.HourOfDay) + } + if created.IntervalType == "" { + t.Error("IntervalType is empty — it is an enumeration, not an optional string") + } + assertContainsStr(t, buf.String(), "Created scheduled event: Ops.Nightly") +} + +// TestModifyScheduledEvent_Mock_PreservesLegacyInterval: Studio Pro does not +// keep Interval/IntervalType in sync with Schedule and MDL cannot author them, +// so a modify must carry the stored pair through rather than re-derive it. +func TestModifyScheduledEvent_Mock_PreservesLegacyInterval(t *testing.T) { + mod := mkModule("Ops") + existing := mkScheduledEvent(mod.ID, "Nightly", &model.Schedule{Kind: model.ScheduleDay, HourOfDay: 1}) + existing.Interval = 0 + existing.IntervalType = "Minute" // the Workflow Commons shape: stale, but real + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + var updated *model.ScheduledEvent + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{existing}, nil + }, + UpdateScheduledEventFunc: func(ev *model.ScheduledEvent) error { + updated = ev + return nil + }, + } + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execCreateScheduledEvent(ctx, &ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "Nightly"}, + Microflow: "Ops.SE_Cleanup", + Repeat: "Daily", + HourOfDay: intPtr(6), + CreateOrModify: true, + })) + + if updated == nil { + t.Fatal("UpdateScheduledEvent was not called") + } + if updated.ID != existing.ID { + t.Errorf("ID = %q, want the existing %q", updated.ID, existing.ID) + } + if updated.Interval != 0 || updated.IntervalType != "Minute" { + t.Errorf("legacy pair = %d/%q, want the stored 0/\"Minute\"", updated.Interval, updated.IntervalType) + } + if updated.Schedule.HourOfDay != 6 { + t.Errorf("HourOfDay = %d, want the new 6", updated.Schedule.HourOfDay) + } +} + +func TestCreateScheduledEvent_Mock_DuplicateWithoutOrModify(t *testing.T) { + mod := mkModule("Ops") + existing := mkScheduledEvent(mod.ID, "Nightly", &model.Schedule{Kind: model.ScheduleDay}) + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{existing}, nil + }, + CreateScheduledEventFunc: func(ev *model.ScheduledEvent) error { + t.Error("CreateScheduledEvent must not be called for a duplicate") + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + if err := execCreateScheduledEvent(ctx, &ast.CreateScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "Nightly"}, Microflow: "Ops.MF", Repeat: "Daily", + }); err == nil { + t.Fatal("expected an already-exists error") + } +} + +func TestDropScheduledEvent_Mock(t *testing.T) { + mod := mkModule("Ops") + existing := mkScheduledEvent(mod.ID, "Nightly", &model.Schedule{Kind: model.ScheduleDay}) + h := mkHierarchy(mod) + withContainer(h, existing.ContainerID, mod.ID) + + var deleted string + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{existing}, nil + }, + DeleteScheduledEventFunc: func(id string) error { + deleted = id + return nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execDropScheduledEvent(ctx, &ast.DropScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "Nightly"}, + })) + if deleted != string(existing.ID) { + t.Errorf("deleted %q, want %q", deleted, existing.ID) + } + assertContainsStr(t, buf.String(), "Dropped scheduled event: Ops.Nightly") +} + +// TestDescribeScheduledEvent_Mock_RoundTrips checks that DESCRIBE emits exactly +// the fields the repeat has, and nothing from another variant. +func TestDescribeScheduledEvent_Mock_RoundTrips(t *testing.T) { + mod := mkModule("Ops") + ev := mkScheduledEvent(mod.ID, "QuarterEnd", &model.Schedule{ + Kind: model.ScheduleMonthWeekday, Multiplier: 3, MonthOffset: 2, + DaySelector: "Last", Weekday: "Friday", HourOfDay: 18, + }) + h := mkHierarchy(mod) + withContainer(h, ev.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{ev}, nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execDescribeScheduledEvent(ctx, &ast.DescribeScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "QuarterEnd"}, + })) + + out := buf.String() + assertContainsStr(t, out, "create or modify scheduled event Ops.QuarterEnd (") + assertContainsStr(t, out, "Repeat: MonthlyByWeekday,") + assertContainsStr(t, out, "DaySelector: Last,") + assertContainsStr(t, out, "Weekday: Friday,") + // Fields of other variants must not appear, or the output would not + // re-execute: the executor refuses a field the repeat does not have. + assertNotContainsStr(t, out, "DayOfMonth") + assertNotContainsStr(t, out, "Weekdays:") + assertNotContainsStr(t, out, "MinuteOffset") +} + +// TestDescribeScheduledEvent_Mock_OutputParses is the real round-trip proof: +// asserting on strings passes against a formatter emitting something nothing +// can read. +func TestDescribeScheduledEvent_Mock_OutputParses(t *testing.T) { + mod := mkModule("Ops") + for _, sched := range []*model.Schedule{ + {Kind: model.ScheduleMinute, Multiplier: 5}, + {Kind: model.ScheduleHour, Multiplier: 2, MinuteOffset: 23}, + {Kind: model.ScheduleDay, HourOfDay: 4}, + {Kind: model.ScheduleWeek, Weekdays: [7]bool{false, true, false, false, false, true, false}, HourOfDay: 9}, + {Kind: model.ScheduleMonthDate, Multiplier: 1, DayOfMonth: 15}, + {Kind: model.ScheduleMonthWeekday, Multiplier: 3, DaySelector: "Last", Weekday: "Friday"}, + {Kind: model.ScheduleYearDate, Month: 1, DayOfMonth: 2}, + {Kind: model.ScheduleYearWeekday, Month: 3, DaySelector: "First", Weekday: "Monday"}, + } { + t.Run(string(sched.Kind), func(t *testing.T) { + ev := mkScheduledEvent(mod.ID, "E", sched) + h := mkHierarchy(mod) + withContainer(h, ev.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListScheduledEventsFunc: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{ev}, nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, execDescribeScheduledEvent(ctx, &ast.DescribeScheduledEventStmt{ + Name: ast.QualifiedName{Module: "Ops", Name: "E"}, + })) + + prog, errs := visitor.Build(buf.String()) + if len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, buf.String()) + } + stmt, ok := prog.Statements[0].(*ast.CreateScheduledEventStmt) + if !ok { + t.Fatalf("statement 0 = %T", prog.Statements[0]) + } + // And the parsed statement must be accepted by the same validation + // that guards a hand-written one. + back, err := scheduleFromStmt(stmt) + if err != nil { + t.Fatalf("describe output is refused on re-execution: %v\n%s", err, buf.String()) + } + if back.Kind != sched.Kind { + t.Errorf("round-tripped kind = %v, want %v", back.Kind, sched.Kind) + } + }) + } +} diff --git a/mdl/executor/cmd_search.go b/mdl/executor/cmd_search.go index 497ec70c6..dec48316f 100644 --- a/mdl/executor/cmd_search.go +++ b/mdl/executor/cmd_search.go @@ -11,6 +11,57 @@ import ( ) // execShowCallers handles SHOW CALLERS OF Module.Microflow [TRANSITIVE]. +// callerRefKinds are the reference kinds that mean "this thing invokes the +// target". `show callers` filtered on 'call' alone, which is the kind a MICROFLOW +// activity produces — so a microflow called from a page action button (kind +// 'action') reported "(no callers found)" even though the reference was sitting +// in the refs table, and a page opened by a button or a menu was equally +// invisible (issue #773). +// +// A false negative here reads as "nothing uses this", which is the answer +// somebody acts on before deleting a document — so the set errs toward +// including a kind rather than omitting it. 'schedule' is the same shape: a +// microflow run only by a scheduled event reported "(no callers found)", was +// listed in GRAPH_DEAD_ASSETS, and drew QUAL004 "is not called from anywhere" +// with the suggestion "Remove if unused" — on a microflow that runs nightly. +// +// Deliberately excluded: 'datasource', 'parameter', 'return', 'retrieve', +// 'create', 'change', 'delete', 'associate', 'generalize' and 'layout'. Those +// are uses of a TYPE or a LAYOUT, not invocations, and folding them in would +// make `show callers of ` a synonym for `show references to`. +var callerRefKinds = []string{ + RefKindCallerCall, // microflow/nanoflow call activity + RefKindCallerAction, // widget action: button, on-change, on-click + RefKindCallerShowPage, // microflow show-page activity, or a widget action opening a page + RefKindCallerCalculate, // calculated attribute + RefKindCallerHomePage, // navigation + RefKindCallerLoginPage, + RefKindCallerMenuItem, + RefKindCallerSchedule, // scheduled event: the microflow it runs +} + +// Kind literals, kept next to the set that uses them so the SQL below cannot +// drift from mdl/catalog's constants unnoticed. +const ( + RefKindCallerCall = "call" + RefKindCallerAction = "action" + RefKindCallerShowPage = "show_page" + RefKindCallerCalculate = "calculate" + RefKindCallerHomePage = "home_page" + RefKindCallerLoginPage = "login_page" + RefKindCallerMenuItem = "menu_item" + RefKindCallerSchedule = "schedule" +) + +// callerRefKindsSQL renders callerRefKinds as a SQL IN list. +func callerRefKindsSQL() string { + quoted := make([]string, len(callerRefKinds)) + for i, k := range callerRefKinds { + quoted[i] = "'" + k + "'" + } + return "(" + strings.Join(quoted, ", ") + ")" +} + func execShowCallers(ctx *ExecContext, s *ast.ShowStmt) error { if s.Name == nil { return mdlerrors.NewValidation("target name required for show callers") @@ -36,12 +87,12 @@ func execShowCallers(ctx *ExecContext, s *ast.ShowStmt) error { with RECURSIVE callers_cte as ( select SourceName as Caller, 1 as Depth from refs - where TargetName = ? and RefKind = 'call' + where TargetName = ? and RefKind in ` + callerRefKindsSQL() + ` union all select r.SourceName, c.Depth + 1 from refs r join callers_cte c on r.TargetName = c.Caller - where r.RefKind = 'call' and c.Depth < 10 + where r.RefKind in ` + callerRefKindsSQL() + ` and c.Depth < 10 ) select distinct Caller, min(Depth) as Depth from callers_cte @@ -53,7 +104,7 @@ func execShowCallers(ctx *ExecContext, s *ast.ShowStmt) error { query = ` select distinct SourceName as Caller, 1 as Depth from refs - where TargetName = ? and RefKind = 'call' + where TargetName = ? and RefKind in ` + callerRefKindsSQL() + ` ORDER by Caller ` } diff --git a/mdl/executor/cmd_search_callers_test.go b/mdl/executor/cmd_search_callers_test.go new file mode 100644 index 000000000..a9998eb77 --- /dev/null +++ b/mdl/executor/cmd_search_callers_test.go @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// upstream #773, second half. `show callers` filtered on RefKind = 'call' — the +// kind a MICROFLOW call activity produces. A microflow invoked from a page action +// button is recorded as 'action', and that row was already sitting in the refs +// table: the reference existed, the query hid it. A page opened by a button or a +// navigation menu was invisible for the same reason. +// +// A false negative here reads as "nothing uses this", which is the answer +// somebody acts on before deleting a document — so the set errs toward including +// a kind rather than omitting it. +func TestCallerRefKinds(t *testing.T) { + in := map[string]bool{} + for _, k := range callerRefKinds { + in[k] = true + } + + // Every way one document can INVOKE another. + for _, k := range []string{ + "call", // microflow/nanoflow call activity — the only one that used to count + "action", // widget action: the reported button case + "show_page", // a microflow, or a button, opening a page + "calculate", // calculated attribute running a microflow + "home_page", // navigation + "login_page", // + "menu_item", // + } { + if !in[k] { + t.Errorf("%q means one document invokes another and must count as a caller — "+ + "omitting it reports the target as unused, which is what #773 was", k) + } + } + + // Uses of a TYPE or a LAYOUT are NOT invocations. Folding these in would make + // `show callers of ` a synonym for `show references to` and destroy + // the distinction between the two commands. + for _, k := range []string{ + "datasource", "parameter", "return", "retrieve", + "create", "change", "delete", "associate", "generalize", "layout", + } { + if in[k] { + t.Errorf("%q is a use of a type or layout, not an invocation — including it "+ + "collapses `show callers` into `show references to`", k) + } + } +} + +// The kind list reaches SQLite as an IN list, so a malformed render silently +// matches nothing — the same "(no callers found)" the issue reported, from a +// different cause. +func TestCallerRefKindsSQL(t *testing.T) { + got := callerRefKindsSQL() + if !strings.HasPrefix(got, "(") || !strings.HasSuffix(got, ")") { + t.Fatalf("not a SQL IN list: %q", got) + } + for _, k := range callerRefKinds { + if !strings.Contains(got, "'"+k+"'") { + t.Errorf("%q is not quoted in the IN list %q", k, got) + } + } + if n := strings.Count(got, ","); n != len(callerRefKinds)-1 { + t.Errorf("%d separators for %d kinds: %q", n, len(callerRefKinds), got) + } +} diff --git a/mdl/executor/cmd_structure.go b/mdl/executor/cmd_structure.go index 2c177c205..09fba87db 100644 --- a/mdl/executor/cmd_structure.go +++ b/mdl/executor/cmd_structure.go @@ -9,6 +9,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/ast" mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/domainmodel" "github.com/mendixlabs/mxcli/sdk/javaactions" @@ -77,12 +78,13 @@ func structureDepth1JSON(ctx *ExecContext, modules []structureModule) error { beServiceCounts := queryCountByModule(ctx, "business_event_services") constantCounts := countByModuleFromBackend(ctx, "constants") scheduledEventCounts := countByModuleFromBackend(ctx, "scheduled_events") + queueCounts := countByModuleFromBackend(ctx, "queues") tr := &TableResult{ Columns: []string{ "Module", "Entities", "Enumerations", "Microflows", "Nanoflows", "Workflows", "Pages", "Snippets", "JavaActions", "Constants", - "ScheduledEvents", "ODataClients", "ODataServices", "BusinessEventServices", + "ScheduledEvents", "Queues", "ODataClients", "ODataServices", "BusinessEventServices", }, } for _, m := range modules { @@ -98,6 +100,7 @@ func structureDepth1JSON(ctx *ExecContext, modules []structureModule) error { jaCounts[m.Name], constantCounts[m.Name], scheduledEventCounts[m.Name], + queueCounts[m.Name], odataClientCounts[m.Name], odataServiceCounts[m.Name], beServiceCounts[m.Name], @@ -198,6 +201,7 @@ func structureDepth1(ctx *ExecContext, modules []structureModule) error { // Get constants and scheduled events from backend (no catalog tables) constantCounts := countByModuleFromBackend(ctx, "constants") scheduledEventCounts := countByModuleFromBackend(ctx, "scheduled_events") + queueCounts := countByModuleFromBackend(ctx, "queues") // Calculate name column width for alignment nameWidth := 0 @@ -240,6 +244,9 @@ func structureDepth1(ctx *ExecContext, modules []structureModule) error { if c := scheduledEventCounts[m.Name]; c > 0 { parts = append(parts, pluralize(c, "scheduled event", "scheduled events")) } + if c := queueCounts[m.Name]; c > 0 { + parts = append(parts, pluralize(c, "queue", "queues")) + } if c := odataClientCounts[m.Name]; c > 0 { parts = append(parts, pluralize(c, "odata client", "odata clients")) } @@ -297,6 +304,14 @@ func countByModuleFromBackend(ctx *ExecContext, kind string) map[string]int { counts[modName]++ } } + case "queues": + if queues, err := ctx.Backend.ListQueues(); err == nil { + for _, q := range queues { + modID := h.FindModuleID(q.ContainerID) + modName := h.GetModuleName(modID) + counts[modName]++ + } + } } return counts } @@ -374,6 +389,14 @@ func structureDepth2(ctx *ExecContext, modules []structureModule) error { eventsByModule[modName] = append(eventsByModule[modName], ev) } + allQueues, _ := ctx.Backend.ListQueues() + queuesByModule := make(map[string][]*types.Queue) + for _, q := range allQueues { + modID := h.FindModuleID(q.ContainerID) + modName := h.GetModuleName(modID) + queuesByModule[modName] = append(queuesByModule[modName], q) + } + // Load java actions for parameter types allJavaActions, _ := ctx.Backend.ListJavaActionsFull() jaByModule := make(map[string][]*javaactions.JavaAction) @@ -457,6 +480,14 @@ func structureDepth2(ctx *ExecContext, modules []structureModule) error { } } + // Task Queues + if queues, ok := queuesByModule[m.Name]; ok { + sort.Slice(queues, func(i, j int) bool { return queues[i].Name < queues[j].Name }) + for _, q := range queues { + fmt.Fprintf(ctx.Output, " Queue %s.%s\n", m.Name, q.Name) + } + } + // OData Clients structureODataClients(ctx, m.Name) @@ -529,6 +560,14 @@ func structureDepth3(ctx *ExecContext, modules []structureModule) error { eventsByModule[modName] = append(eventsByModule[modName], ev) } + allQueues, _ := ctx.Backend.ListQueues() + queuesByModule := make(map[string][]*types.Queue) + for _, q := range allQueues { + modID := h.FindModuleID(q.ContainerID) + modName := h.GetModuleName(modID) + queuesByModule[modName] = append(queuesByModule[modName], q) + } + allJavaActions, _ := ctx.Backend.ListJavaActionsFull() jaByModule := make(map[string][]*javaactions.JavaAction) for _, ja := range allJavaActions { @@ -615,6 +654,14 @@ func structureDepth3(ctx *ExecContext, modules []structureModule) error { } } + // Task Queues + if queues, ok := queuesByModule[m.Name]; ok { + sort.Slice(queues, func(i, j int) bool { return queues[i].Name < queues[j].Name }) + for _, q := range queues { + fmt.Fprintf(ctx.Output, " Queue %s.%s\n", m.Name, q.Name) + } + } + // OData structureODataClients(ctx, m.Name) structureODataServices(ctx, m.Name) diff --git a/mdl/executor/describe_auto.go b/mdl/executor/describe_auto.go index 7de9c51ae..fe47105a7 100644 --- a/mdl/executor/describe_auto.go +++ b/mdl/executor/describe_auto.go @@ -25,6 +25,10 @@ var objectTypeToDescribeKind = map[string]ast.DescribeObjectType{ "NANOFLOW": ast.DescribeNanoflow, "PAGE": ast.DescribePage, "SNIPPET": ast.DescribeSnippet, + "BUILDING_BLOCK": ast.DescribeBuildingBlock, + "MENU": ast.DescribeMenu, + "QUEUE": ast.DescribeQueue, + "SCHEDULED_EVENT": ast.DescribeScheduledEvent, "LAYOUT": ast.DescribeLayout, "ENUMERATION": ast.DescribeEnumeration, "CONSTANT": ast.DescribeConstant, @@ -49,6 +53,19 @@ var objectTypeToDescribeKind = map[string]ast.DescribeObjectType{ "CONSUMED_MCP_SERVICE": ast.DescribeConsumedMCPService, } +// DescribeKindFor maps a catalog `objects` view ObjectType to the DESCRIBE kind +// that renders it, reporting false for a type with no describe handler. +// +// Exported so callers outside the executor (the marketplace differ, which walks +// a whole module) resolve types through the same table bare DESCRIBE uses, +// rather than keeping a fourth copy of this mapping. Three copies already exist +// — this map, the catalog's objects view, and cmd/mxcli's own dispatch — and +// they have drifted apart once each. +func DescribeKindFor(objectType string) (ast.DescribeObjectType, bool) { + kind, ok := objectTypeToDescribeKind[objectType] + return kind, ok +} + // resolveDescribeAuto auto-detects the type of a bare `DESCRIBE Module.Name` // against the connected project. It builds (fast mode) the catalog if needed and // looks the qualified name up in the `objects` index — a complete index for every diff --git a/mdl/executor/describe_auto_test.go b/mdl/executor/describe_auto_test.go new file mode 100644 index 000000000..b7019231a --- /dev/null +++ b/mdl/executor/describe_auto_test.go @@ -0,0 +1,122 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "regexp" + "testing" + + "github.com/mendixlabs/mxcli/mdl/catalog" +) + +// notAutoDescribable lists the catalog `objects` view ObjectTypes that bare +// `DESCRIBE Module.Name` deliberately does not resolve, each with the reason. +// Everything else in the view must have an entry in objectTypeToDescribeKind. +// +// The two lists are maintained in different packages and drifted apart once: +// BUILDING_BLOCK and ICON_COLLECTION had working explicit describe handlers +// (`describe building block M.N`) while bare `describe M.N` reported "no +// describable document named ..." — 43 of the 251 documents in a 7-module +// marketplace project were unreachable that way. This test is the guard. +var notAutoDescribable = map[string]string{ + // Contract-derived rows, read from a cached $metadata / AsyncAPI document + // rather than from the project — there is no unit in the .mpr to describe. + "CONTRACT_ENTITY": "parsed from cached $metadata, not a project document", + "CONTRACT_ACTION": "parsed from cached $metadata, not a project document", + "CONTRACT_MESSAGE": "parsed from cached AsyncAPI, not a project document", + "EXTERNAL_ACTION": "exposed by a consumed service, not a project document", + "BUSINESS_EVENT": "exposed by a consumed service, not a project document", + + // Project-level, not module-scoped documents. + "JAR_DEPENDENCY": "a build-time dependency coordinate, not a document", + + // Indexed so a module's documents can be enumerated in full, but there is no + // DESCRIBE PAGE TEMPLATE handler and describing one as a page is what this + // type was just untangled from. Reporting it as un-describable is the honest + // answer; the previous behaviour emitted `create or modify page` with an + // empty body, which re-executed would create a page and which compared equal + // to every other template. + "PAGE_TEMPLATE": "no DESCRIBE handler; a template's content hangs off LayoutCall, which the page path does not read", +} + +// objectTypesInView extracts the ObjectType literals the `objects` view can +// emit, straight from the view's own SQL, so the test cannot go stale against a +// hand-maintained copy of the list. +func objectTypesInView(t *testing.T) []string { + t.Helper() + + cat, err := catalog.New() + if err != nil { + t.Fatalf("create catalog: %v", err) + } + defer cat.Close() + + row := cat.CatalogDB().QueryRow( + "SELECT sql FROM sqlite_master WHERE type = 'view' AND name = 'objects'") + var sql string + if err := row.Scan(&sql); err != nil { + t.Fatalf("read objects view SQL: %v", err) + } + + re := regexp.MustCompile(`'([A-Z_]+)' as ObjectType`) + matches := re.FindAllStringSubmatch(sql, -1) + if len(matches) == 0 { + t.Fatal("no ObjectType literals found in the objects view SQL") + } + + seen := map[string]bool{} + var out []string + for _, m := range matches { + if !seen[m[1]] { + seen[m[1]] = true + out = append(out, m[1]) + } + } + return out +} + +// TestDescribeAutoCoversCatalogObjectTypes asserts that every ObjectType the +// catalog's objects view can emit is either auto-describable or explicitly +// exempted. A new document type added to the view without a describe kind fails +// here rather than silently becoming unreachable via bare DESCRIBE. +func TestDescribeAutoCoversCatalogObjectTypes(t *testing.T) { + for _, ot := range objectTypesInView(t) { + if _, ok := objectTypeToDescribeKind[ot]; ok { + continue + } + if _, exempt := notAutoDescribable[ot]; exempt { + continue + } + t.Errorf("ObjectType %q is emitted by the catalog objects view but has no "+ + "entry in objectTypeToDescribeKind and is not listed in notAutoDescribable; "+ + "bare `DESCRIBE Module.Name` will report it as not found", ot) + } +} + +// TestDescribeAutoExemptionsAreReal guards the other direction: an exemption for +// an ObjectType the view no longer emits is dead weight that hides the next gap. +func TestDescribeAutoExemptionsAreReal(t *testing.T) { + emitted := map[string]bool{} + for _, ot := range objectTypesInView(t) { + emitted[ot] = true + } + for ot := range notAutoDescribable { + if !emitted[ot] { + t.Errorf("notAutoDescribable lists %q, which the objects view no longer emits; "+ + "remove the exemption", ot) + } + } +} + +// TestDescribeAutoResolvesReusableUIDocuments pins the specific regression: +// building blocks and icon collections are real, named, module-scoped documents +// (40 and 3 of them respectively in a stock 7-module marketplace project) and +// must resolve without the caller naming the type. +func TestDescribeAutoResolvesReusableUIDocuments(t *testing.T) { + for _, ot := range []string{"BUILDING_BLOCK", "ICON_COLLECTION"} { + if _, ok := objectTypeToDescribeKind[ot]; !ok { + t.Errorf("objectTypeToDescribeKind is missing %q, so bare `DESCRIBE Module.Name` "+ + "cannot resolve it even though an explicit describe handler exists", ot) + } + } +} diff --git a/mdl/executor/executor_query.go b/mdl/executor/executor_query.go index 2ba2519b6..3b6f9dc66 100644 --- a/mdl/executor/executor_query.go +++ b/mdl/executor/executor_query.go @@ -199,6 +199,16 @@ func execDescribe(ctx *ExecContext, s *ast.DescribeStmt) error { return describeSnippet(ctx, s.Name) case ast.DescribeBuildingBlock: return describeBuildingBlock(ctx, s.Name) + case ast.DescribeQueue: + // Queues and scheduled events carry their own statement types rather + // than a DescribeStmt kind, so bare `DESCRIBE Module.Name` reaches them + // by synthesizing one. Without this they are indexed in the catalog's + // objects view but unreachable without naming the type — the state + // BUILDING_BLOCK and ICON_COLLECTION were in when 43 of 251 documents + // in a marketplace project could not be described (see describe_auto.go). + return execDescribeQueue(ctx, &ast.DescribeQueueStmt{Name: s.Name}) + case ast.DescribeScheduledEvent: + return execDescribeScheduledEvent(ctx, &ast.DescribeScheduledEventStmt{Name: s.Name}) case ast.DescribeLayout: return describeLayout(ctx, s.Name) case ast.DescribeConstant: @@ -261,6 +271,8 @@ func execDescribe(ctx *ExecContext, s *ast.DescribeStmt) error { return describeImportMapping(ctx, s.Name) case ast.DescribeExportMapping: return describeExportMapping(ctx, s.Name) + case ast.DescribeMenu: + return describeMenu(ctx, s.Name) case ast.DescribeJarDependency: return execDescribeJarDependency(ctx, s.Name.String(), s.Qualifier) default: @@ -352,6 +364,12 @@ func describeObjectTypeLabel(t ast.DescribeObjectType) string { return "importmapping" case ast.DescribeExportMapping: return "exportmapping" + case ast.DescribeMenu: + return "menu" + case ast.DescribeQueue: + return "queue" + case ast.DescribeScheduledEvent: + return "scheduled event" default: return "unknown" } diff --git a/mdl/executor/register_stubs.go b/mdl/executor/register_stubs.go index bae5ee4fb..933d72409 100644 --- a/mdl/executor/register_stubs.go +++ b/mdl/executor/register_stubs.go @@ -205,6 +205,42 @@ func registerNavigationHandlers(r *Registry) { r.Register(&ast.AlterNavigationStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { return execAlterNavigation(ctx, stmt.(*ast.AlterNavigationStmt)) }) + r.Register(&ast.CreateMenuStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execCreateMenu(ctx, stmt.(*ast.CreateMenuStmt)) + }) + r.Register(&ast.DropMenuStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDropMenu(ctx, stmt.(*ast.DropMenuStmt)) + }) +} + +func registerQueueHandlers(r *Registry) { + r.Register(&ast.CreateQueueStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execCreateQueue(ctx, stmt.(*ast.CreateQueueStmt)) + }) + r.Register(&ast.DropQueueStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDropQueue(ctx, stmt.(*ast.DropQueueStmt)) + }) + r.Register(&ast.ShowQueuesStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execShowQueues(ctx, stmt.(*ast.ShowQueuesStmt)) + }) + r.Register(&ast.DescribeQueueStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDescribeQueue(ctx, stmt.(*ast.DescribeQueueStmt)) + }) +} + +func registerScheduledEventHandlers(r *Registry) { + r.Register(&ast.CreateScheduledEventStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execCreateScheduledEvent(ctx, stmt.(*ast.CreateScheduledEventStmt)) + }) + r.Register(&ast.DropScheduledEventStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDropScheduledEvent(ctx, stmt.(*ast.DropScheduledEventStmt)) + }) + r.Register(&ast.ShowScheduledEventsStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execShowScheduledEvents(ctx, stmt.(*ast.ShowScheduledEventsStmt)) + }) + r.Register(&ast.DescribeScheduledEventStmt{}, func(ctx *ExecContext, stmt ast.Statement) error { + return execDescribeScheduledEvent(ctx, stmt.(*ast.DescribeScheduledEventStmt)) + }) } func registerImageHandlers(r *Registry) { diff --git a/mdl/executor/registry.go b/mdl/executor/registry.go index 8756717f8..7055b6184 100644 --- a/mdl/executor/registry.go +++ b/mdl/executor/registry.go @@ -38,6 +38,8 @@ func NewRegistry() *Registry { registerSecurityHandlers(r) registerNavigationHandlers(r) registerImageHandlers(r) + registerQueueHandlers(r) + registerScheduledEventHandlers(r) registerWorkflowHandlers(r) registerBusinessEventHandlers(r) registerSettingsHandlers(r) diff --git a/mdl/executor/registry_test.go b/mdl/executor/registry_test.go index 538505250..60c83a8e2 100644 --- a/mdl/executor/registry_test.go +++ b/mdl/executor/registry_test.go @@ -167,6 +167,8 @@ func allKnownStatements() []ast.Statement { &ast.AlterModelStmt{}, &ast.AlterModuleJarDepStmt{}, &ast.AlterNavigationStmt{}, + &ast.CreateMenuStmt{}, + &ast.DropMenuStmt{}, &ast.AlterODataClientStmt{}, &ast.AlterODataServiceStmt{}, &ast.AlterPageStmt{}, @@ -206,6 +208,8 @@ func allKnownStatements() []ast.Statement { &ast.CreateODataServiceStmt{}, &ast.CreatePageStmtV3{}, &ast.CreatePublishedRestServiceStmt{}, + &ast.CreateQueueStmt{}, + &ast.CreateScheduledEventStmt{}, &ast.CreateRestClientStmt{}, &ast.CreateSnippetStmtV3{}, &ast.CreateUserRoleStmt{}, @@ -215,6 +219,8 @@ func allKnownStatements() []ast.Statement { &ast.DescribeCatalogTableStmt{}, &ast.DescribeContractFromOpenAPIStmt{}, &ast.DescribeFragmentFromStmt{}, + &ast.DescribeQueueStmt{}, + &ast.DescribeScheduledEventStmt{}, &ast.DescribeStmt{}, &ast.DescribeStylingStmt{}, &ast.DisconnectStmt{}, @@ -245,6 +251,8 @@ func allKnownStatements() []ast.Statement { &ast.DropODataServiceStmt{}, &ast.DropPageStmt{}, &ast.DropPublishedRestServiceStmt{}, + &ast.DropQueueStmt{}, + &ast.DropScheduledEventStmt{}, &ast.DropRestClientStmt{}, &ast.DropSnippetStmt{}, &ast.DropUserRoleStmt{}, @@ -278,6 +286,8 @@ func allKnownStatements() []ast.Statement { &ast.SetStmt{}, &ast.ShowDesignPropertiesStmt{}, &ast.ShowFeaturesStmt{}, + &ast.ShowQueuesStmt{}, + &ast.ShowScheduledEventsStmt{}, &ast.ShowStmt{}, &ast.ShowWidgetsStmt{}, &ast.SQLConnectionsStmt{}, diff --git a/mdl/executor/roundtrip_doctype_test.go b/mdl/executor/roundtrip_doctype_test.go index e613d8832..89f3ffbc8 100644 --- a/mdl/executor/roundtrip_doctype_test.go +++ b/mdl/executor/roundtrip_doctype_test.go @@ -50,6 +50,13 @@ var engineScriptSkip = map[string]string{ // The legacy widget builder has no `barchart` pluggable-widget template, so // page build fails ("template not found: barchart"). Passes on modelsdk. "legacy/34-chart-widget-examples.mdl": "legacy widget builder lacks the barchart template (works on modelsdk); tracked", + // Menu-document authoring is modelsdk-only *by design*, not a gap: Studio Pro + // stores a menu document's item lists with typed-array marker 3, which the + // codec emits by default, while the legacy navigation writer hand-builds menu + // items with marker 1. Rather than ship a second writer of unverified shape, + // the legacy backend refuses create/modify/drop — so the script cannot pass + // there and the refusal is the intended behaviour. + "legacy/26-menu-examples.mdl": "menu authoring is modelsdk-only by design; the legacy backend refuses it", // The legacy widget builder has no `linechart` template either, so the OL08 // LineChart object-list example (added in 6b837ad7) fails page build // ("template not found: linechart"). Passes on modelsdk. Same class as the diff --git a/mdl/executor/validate_queued_calls.go b/mdl/executor/validate_queued_calls.go new file mode 100644 index 000000000..fbbf2ea3f --- /dev/null +++ b/mdl/executor/validate_queued_calls.go @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "sort" + "strings" + + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// checkNoQueuedCalls refuses to rewrite a microflow that has a call bound to a +// task queue, because the rewrite would silently drop that binding. +// +// A CREATE OR REPLACE/MODIFY rebuilds the microflow from the statement, and both +// engines hardcode QueueSettings to null on a call — correct for a newly +// authored call, wrong for one that was already queued: +// +// codec.RegisterTypeDefaults("Microflows$MicroflowCall", codec.TypeDefaults{ +// NullFields: []string{"QueueSettings"}, ... +// +// Nothing signals the loss afterwards. Measured on Mendix 11.13: with the +// binding present `mx check` reports CE1613 ("The selected task queue … no +// longer exists") on the call activity; after the rewrite it reports 0 errors. +// So mxcli "fixes" the build by deleting the user's configuration, and the +// project then looks healthy. +// +// MDL cannot yet author a queued call, so the binding cannot be restated in the +// script either — refusing is the only option that does not lose data +// (guard-don't-drop, ADR-0005). Remove this guard when `in queue` exists and the +// rebuild carries the binding through. +func checkNoQueuedCalls(ctx *ExecContext, microflowID model.ID, qualifiedName string) error { + raw, err := ctx.Backend.GetRawUnit(microflowID) + if err != nil { + // Unreadable stored unit is not this guard's business; the rewrite path + // reports its own errors. + return nil + } + queues := queuedCallTargets(raw) + if len(queues) == 0 { + return nil + } + sort.Strings(queues) + return mdlerrors.NewUnsupported(fmt.Sprintf( + "microflow %s has %d call(s) bound to a task queue (%s), and rewriting it would silently "+ + "drop that binding — MDL cannot express a queued call yet, so the queue cannot be restated "+ + "in this script.\n"+ + " Change the microflow in Studio Pro, or remove the task queue from the call first "+ + "(the binding lives on the call activity, not the microflow).", + qualifiedName, len(queues), strings.Join(queues, ", "))) +} + +// queuedCallTargets walks a stored microflow document and returns the queue +// bound to each call that has one. +// +// The binding that matters is QueueSettings — a Queues$QueueSettings node whose +// own Queue property names the queue. The call's top-level Queue property is +// also read, because it is in the metamodel, but on its own it is inert: +// measured on 11.13, a call carrying only Queue (with QueueSettings null) draws +// no complaint from mx check at all, while one carrying QueueSettings does. +func queuedCallTargets(v any) []string { + var out []string + switch t := v.(type) { + case map[string]any: + if qs, ok := t["QueueSettings"].(map[string]any); ok && qs != nil { + name, _ := qs["Queue"].(string) + if name == "" { + name = "(unnamed queue)" + } + out = append(out, name) + } else if name, ok := t["Queue"].(string); ok && name != "" { + // Present without QueueSettings: not something Mendix acts on, but + // still authored state that the rewrite would drop. + out = append(out, name) + } + for _, val := range t { + out = append(out, queuedCallTargets(val)...) + } + case []any: + for _, el := range t { + out = append(out, queuedCallTargets(el)...) + } + } + return dedupeStrings(out) +} + +func dedupeStrings(in []string) []string { + if len(in) < 2 { + return in + } + seen := make(map[string]bool, len(in)) + out := in[:0] + for _, s := range in { + if seen[s] { + continue + } + seen[s] = true + out = append(out, s) + } + return out +} diff --git a/mdl/executor/validate_queued_calls_test.go b/mdl/executor/validate_queued_calls_test.go new file mode 100644 index 000000000..b44d4066c --- /dev/null +++ b/mdl/executor/validate_queued_calls_test.go @@ -0,0 +1,160 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// storedCall builds the shape a stored MicroflowCall has once a task queue is +// bound to it in Studio Pro. The binding lives two levels down (Queues$QueueSettings +// inside the call's QueueSettings property), so the walk has to be recursive. +func storedCall(queueSettings map[string]any, topLevelQueue any) map[string]any { + return map[string]any{ + "$Type": "Microflows$Microflow", + "ObjectCollection": map[string]any{ + "Objects": []any{ + map[string]any{ + "$Type": "Microflows$ActionActivity", + "Action": map[string]any{ + "$Type": "Microflows$MicroflowCall", + "Microflow": "Q.Target", + "Queue": topLevelQueue, + "QueueSettings": queueSettings, + }, + }, + }, + }, + } +} + +func TestQueuedCallTargets_FindsQueueSettings(t *testing.T) { + got := queuedCallTargets(storedCall(map[string]any{ + "$Type": "Queues$QueueSettings", + "Queue": "Q.MyQueue", + }, nil)) + if len(got) != 1 || got[0] != "Q.MyQueue" { + t.Fatalf("queuedCallTargets = %v, want [Q.MyQueue]", got) + } +} + +func TestQueuedCallTargets_NoBinding(t *testing.T) { + if got := queuedCallTargets(storedCall(nil, nil)); len(got) != 0 { + t.Fatalf("queuedCallTargets = %v, want none for an unqueued call", got) + } +} + +// A call can carry a bare Queue with QueueSettings null. Measured on 11.13 that +// is inert (mx check does not complain), but it is still authored state, and a +// rewrite would drop it. +func TestQueuedCallTargets_BareQueue(t *testing.T) { + got := queuedCallTargets(storedCall(nil, "Q.MyQueue")) + if len(got) != 1 || got[0] != "Q.MyQueue" { + t.Fatalf("queuedCallTargets = %v, want [Q.MyQueue]", got) + } +} + +func TestQueuedCallTargets_Dedupes(t *testing.T) { + doc := map[string]any{"Objects": []any{ + map[string]any{"QueueSettings": map[string]any{"Queue": "Q.A"}}, + map[string]any{"QueueSettings": map[string]any{"Queue": "Q.A"}}, + map[string]any{"QueueSettings": map[string]any{"Queue": "Q.B"}}, + }} + got := queuedCallTargets(doc) + if len(got) != 2 { + t.Fatalf("queuedCallTargets = %v, want 2 distinct queues", got) + } +} + +// TestCheckNoQueuedCalls_Refuses is the guard against the data loss itself: a +// CREATE OR REPLACE of a microflow with a queued call used to succeed and write +// QueueSettings back as null, so mx check went from CE1613 to 0 errors by +// deleting the user's configuration. +func TestCheckNoQueuedCalls_Refuses(t *testing.T) { + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + GetRawUnitFunc: func(id model.ID) (map[string]any, error) { + return storedCall(map[string]any{"Queue": "Q.MyQueue"}, nil), nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb)) + + err := checkNoQueuedCalls(ctx, "mf-1", "Q.ACT_Caller") + if err == nil { + t.Fatal("expected a refusal for a microflow with a queued call") + } + msg := err.Error() + for _, want := range []string{"Q.ACT_Caller", "Q.MyQueue", "task queue"} { + if !strings.Contains(msg, want) { + t.Errorf("error message missing %q:\n%s", want, msg) + } + } +} + +func TestCheckNoQueuedCalls_AllowsUnqueued(t *testing.T) { + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + GetRawUnitFunc: func(id model.ID) (map[string]any, error) { + return storedCall(nil, nil), nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb)) + if err := checkNoQueuedCalls(ctx, "mf-1", "Q.ACT_Caller"); err != nil { + t.Fatalf("unqueued microflow must still be rewritable: %v", err) + } +} + +// An unreadable unit is not this guard's business — the rewrite path reports its +// own errors, and failing here would block writes for an unrelated reason. +func TestCheckNoQueuedCalls_UnreadableUnitDoesNotBlock(t *testing.T) { + ctx, _ := newMockCtx(t) // default mock: GetRawUnit is not configured, so it errors + if err := checkNoQueuedCalls(ctx, "mf-1", "Q.ACT_Caller"); err != nil { + t.Fatalf("unreadable unit must not block the write: %v", err) + } +} + +// TestCreateOrModifyMicroflow_RefusesQueuedCall drives the guard through the +// real CREATE OR MODIFY path. This is the test that fails if the call site in +// execCreateMicroflow is removed — checkNoQueuedCalls passing on its own proves +// nothing about whether anything calls it. +func TestCreateOrModifyMicroflow_RefusesQueuedCall(t *testing.T) { + mod := mkModule("Q") + h := mkHierarchy(mod) + + existing := µflows.Microflow{Name: "ACT_Caller"} + existing.ID = "mf-existing" + existing.ContainerID = mod.ID + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListMicroflowsFunc: func() ([]*microflows.Microflow, error) { + return []*microflows.Microflow{existing}, nil + }, + GetRawUnitFunc: func(id model.ID) (map[string]any, error) { + return storedCall(map[string]any{"Queue": "Q.MyQueue"}, nil), nil + }, + UpdateMicroflowFunc: func(mf *microflows.Microflow) error { + t.Error("UpdateMicroflow must not run: the rewrite would drop the queue binding") + return nil + }, + } + + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + err := execCreateMicroflow(ctx, &ast.CreateMicroflowStmt{ + Name: ast.QualifiedName{Module: "Q", Name: "ACT_Caller"}, + CreateOrModify: true, + }) + if err == nil { + t.Fatal("expected a refusal, got a successful rewrite") + } + if !strings.Contains(err.Error(), "Q.MyQueue") { + t.Errorf("error should name the queue that would be lost:\n%s", err) + } +} diff --git a/mdl/executor/validate_scheduled_events.go b/mdl/executor/validate_scheduled_events.go new file mode 100644 index 000000000..32c37d90b --- /dev/null +++ b/mdl/executor/validate_scheduled_events.go @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Check-time (no-project) validation for CREATE SCHEDULED EVENT. +// +// The repeat rule is a polymorphic child with eight variants that differ in +// which fields they carry, so most of what can go wrong — a field that belongs +// to a different repeat, an hour of 99, a misspelled weekday — is decidable from +// the statement alone. Running it here means a plain `mxcli check` reports it, +// instead of the script passing check and failing at exec. +package executor + +import ( + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// ValidateScheduledEvents reports (MDL-SCHED01) a CREATE SCHEDULED EVENT whose +// properties do not describe a schedule Mendix can store. +// +// It reuses the executor's own builder, so check and exec cannot drift: there is +// one implementation of what a valid statement is, and this pass is the same +// function exec calls before writing. +func ValidateScheduledEvents(prog *ast.Program) []linter.Violation { + var out []linter.Violation + for _, stmt := range prog.Statements { + s, ok := stmt.(*ast.CreateScheduledEventStmt) + if !ok { + continue + } + if _, err := scheduledEventFromStmt(s); err != nil { + out = append(out, linter.Violation{ + RuleID: "MDL-SCHED01", + Severity: linter.SeverityError, + Message: err.Error(), + Suggestion: "Each Repeat takes only its own fields — see `mxcli syntax scheduled-event` " + + "for the field list of each one.", + }) + } + } + return out +} diff --git a/mdl/exprcheck/unknown_funcs_test.go b/mdl/exprcheck/unknown_funcs_test.go index 3905801a7..be428c8f3 100644 --- a/mdl/exprcheck/unknown_funcs_test.go +++ b/mdl/exprcheck/unknown_funcs_test.go @@ -11,10 +11,10 @@ func TestUnknownFunctionCalls(t *testing.T) { wantSuggat string // expected suggestion (if any) }{ {"randomInt(9)", "randomInt", "random"}, - {"round(random() * 8)", "", ""}, // all known - {"toUpperCase($x)", "", ""}, // known - {"secondsBetween($a, $b)", "", ""}, // known - {"$a + length($s)", "", ""}, // known nested + {"round(random() * 8)", "", ""}, // all known + {"toUpperCase($x)", "", ""}, // known + {"secondsBetween($a, $b)", "", ""}, // known + {"$a + length($s)", "", ""}, // known nested {"if $a then floor($b) else ceil($c)", "", ""}, {"totallyMadeUpFn($x)", "totallyMadeUpFn", ""}, // no close match } @@ -41,13 +41,13 @@ func TestSourceRejectedForIntegerTarget(t *testing.T) { src string want bool }{ - {"$a div $b", true}, // arithmetic Decimal - {"secondsBetween($d1, $d2)", true}, // Decimal-returning func - {"random()", true}, // Decimal-returning func - {"round(random() * 8)", false}, // rounding → accepted - {"floor($a div $b)", false}, // rounding → accepted - {"$a + $b", false}, // Integer arithmetic - {"length($s)", false}, // Integer-returning func + {"$a div $b", true}, // arithmetic Decimal + {"secondsBetween($d1, $d2)", true}, // Decimal-returning func + {"random()", true}, // Decimal-returning func + {"round(random() * 8)", false}, // rounding → accepted + {"floor($a div $b)", false}, // rounding → accepted + {"$a + $b", false}, // Integer arithmetic + {"length($s)", false}, // Integer-returning func {"calendarMonthsBetween($d1, $d2)", false}, // Integer-returning func {"", false}, } diff --git a/mdl/grammar/MDLLexer.g4 b/mdl/grammar/MDLLexer.g4 index 61a832e43..ea8995d3c 100644 --- a/mdl/grammar/MDLLexer.g4 +++ b/mdl/grammar/MDLLexer.g4 @@ -354,6 +354,9 @@ READONLY: R E A D O N L Y; ATTRIBUTES: A T T R I B U T E S; FILTERTYPE: F I L T E R T Y P E; IMAGE: I M A G E; +QUEUE: Q U E U E; +QUEUES: Q U E U E S; +SCHEDULED: S C H E D U L E D; COLLECTION: C O L L E C T I O N S?; // accept singular + plural ("collection(s)") JAR: J A R; DEPENDENCY: D E P E N D E N C Y; diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index cebf49dd0..2d4657fc1 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -115,6 +115,8 @@ createStatement | createUserRoleStatement | createDemoUserStatement | createImageCollectionStatement + | createQueueStatement + | createScheduledEventStatement | createJsonStructureStatement | createImportMappingStatement | createExportMappingStatement @@ -126,6 +128,7 @@ createStatement | createKnowledgeBaseStatement | createAgentStatement | createNanoflowStatement + | createMenuStatement ) ; @@ -298,6 +301,14 @@ navMenuItemDef | MENU_KW STRING_LITERAL (ICON qualifiedName)? LPAREN navMenuItemDef* RPAREN SEMICOLON? ; +// A standalone menu document (Menus$MenuDocument) — the reusable menu a menu +// widget points at, as opposed to the menu inside a navigation profile. Both are +// built from the same items, so this reuses navMenuItemDef rather than defining a +// second item syntax. +createMenuStatement + : MENU_KW qualifiedName LPAREN navMenuItemDef* RPAREN + ; + dropStatement : DROP ENTITY qualifiedName | DROP ASSOCIATION qualifiedName @@ -307,8 +318,11 @@ dropStatement | DROP NANOFLOW qualifiedName | DROP PAGE qualifiedName | DROP SNIPPET qualifiedName + | DROP MENU_KW qualifiedName | DROP MODULE qualifiedName | DROP NOTEBOOK qualifiedName + | DROP QUEUE qualifiedName + | DROP SCHEDULED EVENT qualifiedName | DROP JAVA ACTION qualifiedName | DROP JAVASCRIPT ACTION qualifiedName | DROP INDEX qualifiedName ON qualifiedName diff --git a/mdl/grammar/domains/MDLCatalog.g4 b/mdl/grammar/domains/MDLCatalog.g4 index f5e606fae..3fd502643 100644 --- a/mdl/grammar/domains/MDLCatalog.g4 +++ b/mdl/grammar/domains/MDLCatalog.g4 @@ -35,6 +35,8 @@ showStatement | showOrList CONSTANT VALUES (IN (qualifiedName | IDENTIFIER))? | showOrList LAYOUTS (IN (qualifiedName | IDENTIFIER))? | showOrList NOTEBOOKS (IN (qualifiedName | IDENTIFIER))? + | showOrList QUEUES (IN (qualifiedName | IDENTIFIER))? + | showOrList SCHEDULED EVENTS (IN (qualifiedName | IDENTIFIER))? | showOrList JAVA ACTIONS (IN (qualifiedName | IDENTIFIER))? | showOrList JAVASCRIPT ACTIONS (IN (qualifiedName | IDENTIFIER))? | showOrList IMAGE COLLECTION (IN (qualifiedName | IDENTIFIER))? @@ -146,6 +148,7 @@ describeStatement | DESCRIBE PAGE qualifiedName | DESCRIBE SNIPPET qualifiedName | DESCRIBE BUILDING BLOCK qualifiedName + | DESCRIBE MENU_KW qualifiedName | DESCRIBE LAYOUT qualifiedName | DESCRIBE ENUMERATION qualifiedName | DESCRIBE CONSTANT qualifiedName @@ -162,6 +165,8 @@ describeStatement | DESCRIBE STYLING ON (PAGE | SNIPPET) qualifiedName (WIDGET IDENTIFIER)? // DESCRIBE STYLING ON PAGE Module.Page [WIDGET name] | DESCRIBE CATALOG DOT (catalogTableName) // DESCRIBE CATALOG.ENTITIES | DESCRIBE BUSINESS EVENT SERVICE qualifiedName // DESCRIBE BUSINESS EVENT SERVICE Module.Name + | DESCRIBE QUEUE qualifiedName // DESCRIBE QUEUE Module.Name + | DESCRIBE SCHEDULED EVENT qualifiedName // DESCRIBE SCHEDULED EVENT Module.Name | DESCRIBE DATABASE CONNECTION qualifiedName // DESCRIBE DATABASE CONNECTION Module.Name | DESCRIBE SETTINGS (CONFIGURATION STRING_LITERAL)? // DESCRIBE SETTINGS [CONFIGURATION 'Default'] | DESCRIBE FRAGMENT FROM PAGE qualifiedName WIDGET identifierOrKeyword // DESCRIBE FRAGMENT FROM PAGE Module.Page WIDGET name @@ -216,6 +221,7 @@ catalogTableName | CONSTANTS // keyword token — must be listed explicitly | OBJECTS // keyword token — must be listed explicitly | COMMUNITIES // keyword token (SHOW COMMUNITIES) — must be listed explicitly for CATALOG.COMMUNITIES + | QUEUES // keyword token (SHOW QUEUES) — must be listed explicitly for CATALOG.QUEUES | SOURCE_KW // For CATALOG.SOURCE FTS table | ODATA // For CATALOG.ODATA_CLIENTS and CATALOG.ODATA_SERVICES (via IDENTIFIER) | IDENTIFIER // For tables like activities, xpath_expressions, projects, snapshots, refs, strings, odata_clients, odata_services, java_actions diff --git a/mdl/grammar/domains/MDLDomainModel.g4 b/mdl/grammar/domains/MDLDomainModel.g4 index e7e31d312..c86129b2a 100644 --- a/mdl/grammar/domains/MDLDomainModel.g4 +++ b/mdl/grammar/domains/MDLDomainModel.g4 @@ -331,6 +331,51 @@ enumerationOption | FOLDER STRING_LITERAL // place the enumeration in a module folder (Bug 12b) ; +// ============================================================================= +// TASK QUEUE CREATION +// ============================================================================= + +/** + * CREATE [OR REPLACE|MODIFY] QUEUE Module.Name ( Parallelism: 3, ClusterWide: true ); + * + * Parallelism is stored by Mendix as an EXPRESSION string + * (Queues$BasicQueueConfig.ParallelismExpression), so it accepts a number or a + * quoted expression. + */ +createQueueStatement + : QUEUE qualifiedName queueBody? + ; + +queueBody + : LPAREN (queueProperty (COMMA queueProperty)* COMMA?)? RPAREN + ; + +queueProperty + : identifierOrKeyword COLON (NUMBER_LITERAL | STRING_LITERAL | booleanLiteral | identifierOrKeyword) + ; + +// ============================================================================= +// SCHEDULED EVENT CREATION +// ============================================================================= +// +// The repeat rule is a property (Repeat: Daily) plus the fields that rule uses, +// rather than an English clause, because the eight ScheduledEvents$*Schedule +// variants differ in WHICH fields they carry — a labelled property list keeps +// the storage's own vocabulary and lets the executor reject a field that does +// not belong to the chosen repeat. + +createScheduledEventStatement + : SCHEDULED EVENT qualifiedName scheduledEventBody? + ; + +scheduledEventBody + : LPAREN (scheduledEventProperty (COMMA scheduledEventProperty)* COMMA?)? RPAREN + ; + +scheduledEventProperty + : identifierOrKeyword COLON (qualifiedName | NUMBER_LITERAL | STRING_LITERAL | booleanLiteral | identifierOrKeyword) + ; + // ============================================================================= // IMAGE COLLECTION CREATION // ============================================================================= diff --git a/mdl/grammar/domains/MDLSettings.g4 b/mdl/grammar/domains/MDLSettings.g4 index 9db3b5d0f..b12db73a8 100644 --- a/mdl/grammar/domains/MDLSettings.g4 +++ b/mdl/grammar/domains/MDLSettings.g4 @@ -486,7 +486,7 @@ annotationParenValue */ keyword // DDL / DML - : ADD | ALTER | BATCH | BROWSER | CHANGE | CLOSE | COMMIT | CREATE | DECLARE | DELETE | DESCRIBE + : QUEUE | QUEUES | ADD | ALTER | BATCH | BROWSER | CHANGE | CLOSE | COMMIT | CREATE | DECLARE | DELETE | DESCRIBE | DOWNLOAD | DROP | EXECUTE | EXPORT | GENERATE | IMPORT | INSERT | INTO | MODIFY | MOVE | REFRESH | SYNCHRONIZE | UNSYNCHRONIZED | REMOVE | RENAME | REPLACE | RETRIEVE | RETURN | ROLLBACK | SET | UPDATE @@ -624,7 +624,7 @@ keyword | UNLOCK | UNPAUSE | WAIT | WORKFLOW | WORKFLOWS // Business events / settings - | BUSINESS | CONFIGURATION | EVENT | EVENTS | HANDLER | SETTINGS | SUBSCRIBE + | BUSINESS | CONFIGURATION | EVENT | EVENTS | HANDLER | SCHEDULED | SETTINGS | SUBSCRIBE // Code search / analysis | BACKGROUND | CALLERS | CALLEES | DEPTH | IMPACT | REFERENCES diff --git a/mdl/linter/context.go b/mdl/linter/context.go index e38e774fb..d0b08b438 100644 --- a/mdl/linter/context.go +++ b/mdl/linter/context.go @@ -814,32 +814,77 @@ func (ctx *LintContext) Snippets() iter.Seq[Snippet] { // ScheduledEvent represents a scheduled event document. type ScheduledEvent struct { - Name string - QualifiedName string - ModuleName string - MicroflowName string // qualified name of the microflow to execute + Name string + QualifiedName string + ModuleName string + MicroflowName string // qualified name of the microflow to execute + // IntervalSeconds is how often the event fires, derived from the Schedule + // child — NOT from the stored Interval/IntervalType pair, which Studio Pro + // writes and does not keep in sync with Schedule. Workflow Commons ships an + // event storing 0/"Minute" beside a DaySchedule of 01:00, so a rule keyed on + // the legacy pair would read it as "fires every 0 seconds". IntervalSeconds int - Enabled bool -} - -// intervalToSeconds converts a Mendix interval value and type to seconds. -// Returns 0 for unrecognised interval types (treated as "not convertible"). -func intervalToSeconds(interval int, intervalType string) int { - multipliers := map[string]int{ - "Second": 1, - "Minute": 60, - "Hour": 3600, - "Day": 86400, - "Week": 604800, - "Month": 2592000, - "Year": 31536000, + // Repeat is the schedule variant (Minute, Hour, Day, Week, MonthDate, + // MonthWeekday, YearDate, YearWeekday); empty when the event has no + // Schedule child. + Repeat string + // OnOverlap is DelayNext or SkipNext — the event's own concurrency control. + OnOverlap string + TimeZone string + Enabled bool +} + +// scheduleSeconds is the gap between runs implied by a schedule. +// +// The month and year figures are averages (30 and 365 days): the value is for +// thresholds and ordering ("anything that fires more often than a minute"), not +// for calendar arithmetic. +func scheduleSeconds(s *model.Schedule) int { + if s == nil { + return 0 + } + mult := s.Multiplier + if mult < 1 { + mult = 1 } - if mult, ok := multipliers[intervalType]; ok { - return interval * mult + const day = 86400 + switch s.Kind { + case model.ScheduleMinute: + return mult * 60 + case model.ScheduleHour: + return mult * 3600 + case model.ScheduleDay: + return day + case model.ScheduleWeek: + n := 0 + for _, on := range s.Weekdays { + if on { + n++ + } + } + if n == 0 { + return 7 * day + } + return (7 * day) / n + case model.ScheduleMonthDate, model.ScheduleMonthWeekday: + return mult * 30 * day + case model.ScheduleYearDate, model.ScheduleYearWeekday: + return 365 * day } return 0 } +// Queue represents a task queue document. +type Queue struct { + Name string + QualifiedName string + ModuleName string + // Parallelism is an EXPRESSION, not a number — Mendix stores it as a string, + // so a rule must not assume it parses as an integer. + Parallelism string + ClusterWide bool +} + // ScheduledEvents returns an iterator over all scheduled events (excluding system modules). // Returns an empty iterator if no reader is available. func (ctx *LintContext) ScheduledEvents() iter.Seq[ScheduledEvent] { @@ -889,12 +934,19 @@ func (ctx *LintContext) ScheduledEvents() iter.Seq[ScheduledEvent] { if mfName == "" { mfName = string(e.MicroflowID) } + repeat := "" + if e.Schedule != nil { + repeat = string(e.Schedule.Kind) + } se := ScheduledEvent{ Name: e.Name, QualifiedName: moduleName + "." + e.Name, ModuleName: moduleName, MicroflowName: mfName, - IntervalSeconds: intervalToSeconds(e.Interval, e.IntervalType), + IntervalSeconds: scheduleSeconds(e.Schedule), + Repeat: repeat, + OnOverlap: e.OnOverlap, + TimeZone: e.TimeZone, Enabled: e.Enabled, } if !yield(se) { @@ -981,6 +1033,43 @@ type DatabaseConnection struct { } // DatabaseConnections returns an iterator over all database connections (excluding system modules). +// Queues returns an iterator over all task queues (excluding platform modules). +// +// Backed by the catalog rather than the reader, so no LintReader change is +// needed; the catalog is already built whenever rules run. +func (ctx *LintContext) Queues() iter.Seq[Queue] { + return func(yield func(Queue) bool) { + rows, err := ctx.db.Query(fmt.Sprintf(` + SELECT q.Name, q.QualifiedName, q.ModuleName, q.Parallelism, q.ClusterWide + FROM queues q + LEFT JOIN modules m ON q.ModuleName = m.Name + WHERE %s + ORDER BY q.ModuleName, q.Name + `, notPlatformModule("m"))) + if err != nil { + ctx.recordQueryError("Queues", err) + return + } + defer rows.Close() + + for rows.Next() { + var q Queue + var clusterWide int + if err := rows.Scan(&q.Name, &q.QualifiedName, &q.ModuleName, &q.Parallelism, &clusterWide); err != nil { + ctx.recordQueryError("Queues (row scan)", err) + continue + } + q.ClusterWide = clusterWide != 0 + if ctx.IsExcluded(q.ModuleName) { + continue + } + if !yield(q) { + return + } + } + } +} + func (ctx *LintContext) DatabaseConnections() iter.Seq[DatabaseConnection] { return func(yield func(DatabaseConnection) bool) { rows, err := ctx.db.Query(fmt.Sprintf(` diff --git a/mdl/linter/starlark.go b/mdl/linter/starlark.go index 88623fa7f..a1d23e880 100644 --- a/mdl/linter/starlark.go +++ b/mdl/linter/starlark.go @@ -326,6 +326,7 @@ func (r *StarlarkRule) buildPredeclared() starlark.StringDict { "refs_from": starlark.NewBuiltin("refs_from", r.builtinRefsFrom), "attributes_for": starlark.NewBuiltin("attributes_for", r.builtinAttributesFor), "scheduled_events": starlark.NewBuiltin("scheduled_events", r.builtinScheduledEvents), + "queues": starlark.NewBuiltin("queues", r.builtinQueues), // Graph-analysis facts (populated by `refresh catalog communities`). "community_of": starlark.NewBuiltin("community_of", r.builtinCommunityOf), @@ -1016,15 +1017,43 @@ func databaseConnectionToStarlark(dc DatabaseConnection) starlark.Value { func scheduledEventToStarlark(se ScheduledEvent) starlark.Value { return starlarkstruct.FromStringDict(starlark.String("scheduled_event"), starlark.StringDict{ - "name": starlark.String(se.Name), - "qualified_name": starlark.String(se.QualifiedName), - "module_name": starlark.String(se.ModuleName), - "microflow_name": starlark.String(se.MicroflowName), + "name": starlark.String(se.Name), + "qualified_name": starlark.String(se.QualifiedName), + "module_name": starlark.String(se.ModuleName), + "microflow_name": starlark.String(se.MicroflowName), + // Derived from the Schedule child, not the legacy Interval/IntervalType + // pair — see ScheduledEvent.IntervalSeconds. "interval_seconds": starlark.MakeInt(se.IntervalSeconds), + "repeat": starlark.String(se.Repeat), + "on_overlap": starlark.String(se.OnOverlap), + "time_zone": starlark.String(se.TimeZone), "enabled": starlark.Bool(se.Enabled), }) } +// builtinQueues returns all task queues. +func (r *StarlarkRule) builtinQueues(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { + if r.ctx == nil { + return starlark.NewList(nil), nil + } + var result []starlark.Value + for q := range r.ctx.Queues() { + result = append(result, queueToStarlark(q)) + } + return starlark.NewList(result), nil +} + +func queueToStarlark(q Queue) starlark.Value { + return starlarkstruct.FromStringDict(starlark.String("queue"), starlark.StringDict{ + "name": starlark.String(q.Name), + "qualified_name": starlark.String(q.QualifiedName), + "module_name": starlark.String(q.ModuleName), + // An EXPRESSION string, not a number. + "parallelism": starlark.String(q.Parallelism), + "cluster_wide": starlark.Bool(q.ClusterWide), + }) +} + // builtinXPathExpressions returns all XPath expression entries from the catalog. func (r *StarlarkRule) builtinXPathExpressions(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { if r.ctx == nil { diff --git a/mdl/linter/starlark_scheduledevents_test.go b/mdl/linter/starlark_scheduledevents_test.go index d27e3e3b8..c4112c5cf 100644 --- a/mdl/linter/starlark_scheduledevents_test.go +++ b/mdl/linter/starlark_scheduledevents_test.go @@ -40,62 +40,127 @@ func (m *minimalReader) ListScheduledEvents() ([]*model.ScheduledEvent, error) { return nil, nil } -// TestIntervalToSeconds is a white-box test; we call it via the exported -// ScheduledEvents iterator rather than calling the unexported helper directly. -// The expected IntervalSeconds values verify all multipliers and the unknown-type fallback. -func TestIntervalToSeconds(t *testing.T) { +// TestScheduleSeconds checks the interval a rule sees, exercised through the +// exported ScheduledEvents iterator. +// +// It is derived from the Schedule child, NOT from the stored +// Interval/IntervalType pair: Studio Pro writes that pair and does not keep it +// in sync with Schedule — Workflow Commons 4.11.0 ships an event storing +// 0/"Minute" beside a DaySchedule of 01:00 — so a rule keyed on the legacy pair +// reads a daily job as firing every 0 seconds. TestScheduleSeconds_IgnoresStaleLegacyPair +// below is that exact document. +func TestScheduleSeconds(t *testing.T) { tests := []struct { - interval int - intervalType string - want int + name string + sched *model.Schedule + want int }{ - {1, "Second", 1}, - {2, "Minute", 120}, - {3, "Hour", 10800}, - {1, "Day", 86400}, - {1, "Week", 604800}, - {1, "Month", 2592000}, - {1, "Year", 31536000}, - {5, "Unknown", 0}, // unrecognised type → 0 - {5, "", 0}, // empty type → 0 - } - - containerID := model.ID("mod-1") + {"minutely x2", &model.Schedule{Kind: model.ScheduleMinute, Multiplier: 2}, 120}, + {"hourly x3", &model.Schedule{Kind: model.ScheduleHour, Multiplier: 3}, 10800}, + {"daily", &model.Schedule{Kind: model.ScheduleDay}, 86400}, + {"weekly, one day", &model.Schedule{Kind: model.ScheduleWeek, + Weekdays: [7]bool{false, true, false, false, false, false, false}}, 604800}, + // Two selected days means it fires twice a week. + {"weekly, two days", &model.Schedule{Kind: model.ScheduleWeek, + Weekdays: [7]bool{false, true, false, false, false, true, false}}, 302400}, + {"monthly", &model.Schedule{Kind: model.ScheduleMonthDate, Multiplier: 1}, 2592000}, + {"yearly", &model.Schedule{Kind: model.ScheduleYearDate}, 31536000}, + // An unstated multiplier is 1, not 0 — a 0 would read as "never". + {"multiplier defaults to 1", &model.Schedule{Kind: model.ScheduleHour}, 3600}, + {"no schedule", nil, 0}, + } + for _, tt := range tests { - reader := &minimalReader{ - listScheduledEvents: func() ([]*model.ScheduledEvent, error) { - return []*model.ScheduledEvent{{ - ContainerID: containerID, - Name: "SE", - Interval: tt.interval, - IntervalType: tt.intervalType, - Enabled: true, - }}, nil - }, - } + t.Run(tt.name, func(t *testing.T) { + if got := scheduledEventInterval(t, &model.ScheduledEvent{ + ContainerID: model.ID("mod-1"), Name: "SE", Schedule: tt.sched, Enabled: true, + }); got != tt.want { + t.Errorf("IntervalSeconds = %d, want %d", got, tt.want) + } + }) + } +} - cat, err := catalog.NewFromFile(filepath.Join(t.TempDir(), "cat.db")) - if err != nil { - t.Fatalf("NewFromFile: %v", err) - } - db := cat.CatalogDB() - if _, err := db.Exec( - `INSERT INTO modules_data (Id, Name, ProjectId, SnapshotId) VALUES (?,?,?,?)`, - string(containerID), "MyModule", "default", "s1", - ); err != nil { - t.Fatalf("insert module: %v", err) - } - cat.Close() +// TestScheduleSeconds_IgnoresStaleLegacyPair is the regression: this is the +// Workflow Commons document, whose legacy pair disagrees with its Schedule. +// Reading the pair gives 0; reading the Schedule gives a day. +func TestScheduleSeconds_IgnoresStaleLegacyPair(t *testing.T) { + got := scheduledEventInterval(t, &model.ScheduledEvent{ + ContainerID: model.ID("mod-1"), + Name: "SE_WorkflowAuditTrailRecord_CleanUp", + Interval: 0, + IntervalType: "Minute", + Schedule: &model.Schedule{Kind: model.ScheduleDay, HourOfDay: 1}, + Enabled: false, + }) + if got != 86400 { + t.Errorf("IntervalSeconds = %d, want 86400 — the legacy Interval/IntervalType pair must not be used", got) + } +} - ctx := linter.NewLintContext(cat, reader) - var got int - for se := range ctx.ScheduledEvents() { - got = se.IntervalSeconds +// TestScheduledEventExposesSchedule checks the fields a rule can branch on. +func TestScheduledEventExposesSchedule(t *testing.T) { + reader := &minimalReader{ + listScheduledEvents: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{{ + ContainerID: model.ID("mod-1"), Name: "SE", + Schedule: &model.Schedule{Kind: model.ScheduleMonthWeekday, Multiplier: 3}, + OnOverlap: "SkipNext", TimeZone: "Server", Enabled: true, + }}, nil + }, + } + ctx := linter.NewLintContext(newSingleModuleCatalog(t, model.ID("mod-1")), reader) + found := false + for se := range ctx.ScheduledEvents() { + found = true + if se.Repeat != "MonthWeekday" { + t.Errorf("Repeat = %q", se.Repeat) + } + if se.OnOverlap != "SkipNext" { + t.Errorf("OnOverlap = %q", se.OnOverlap) } - if got != tt.want { - t.Errorf("interval=%d type=%q: IntervalSeconds=%d, want %d", tt.interval, tt.intervalType, got, tt.want) + if se.TimeZone != "Server" { + t.Errorf("TimeZone = %q", se.TimeZone) } } + if !found { + t.Fatal("no scheduled event yielded") + } +} + +// scheduledEventInterval runs one event through the iterator and returns the +// interval a rule would see. +func scheduledEventInterval(t *testing.T, ev *model.ScheduledEvent) int { + t.Helper() + reader := &minimalReader{ + listScheduledEvents: func() ([]*model.ScheduledEvent, error) { + return []*model.ScheduledEvent{ev}, nil + }, + } + ctx := linter.NewLintContext(newSingleModuleCatalog(t, ev.ContainerID), reader) + got := 0 + for se := range ctx.ScheduledEvents() { + got = se.IntervalSeconds + } + return got +} + +// newSingleModuleCatalog builds a catalog holding one module, so the iterator +// can resolve the container to a module name. +func newSingleModuleCatalog(t *testing.T, containerID model.ID) *catalog.Catalog { + t.Helper() + cat, err := catalog.NewFromFile(filepath.Join(t.TempDir(), "cat.db")) + if err != nil { + t.Fatalf("NewFromFile: %v", err) + } + if _, err := cat.CatalogDB().Exec( + `INSERT INTO modules_data (Id, Name, ProjectId, SnapshotId) VALUES (?,?,?,?)`, + string(containerID), "MyModule", "default", "s1", + ); err != nil { + t.Fatalf("insert module: %v", err) + } + cat.Close() + return cat } func TestScheduledEvents_MicroflowNameResolution(t *testing.T) { @@ -169,11 +234,10 @@ func TestScheduledEvents_ExcludedModules(t *testing.T) { reader := &minimalReader{ listScheduledEvents: func() ([]*model.ScheduledEvent, error) { return []*model.ScheduledEvent{{ - ContainerID: containerID, - Name: "ExcludedSE", - Interval: 1, - IntervalType: "Day", - Enabled: true, + ContainerID: containerID, + Name: "ExcludedSE", + Schedule: &model.Schedule{Kind: model.ScheduleDay}, + Enabled: true, }}, nil }, } @@ -211,8 +275,8 @@ func TestScheduledEvents_IncludedModules(t *testing.T) { reader := &minimalReader{ listScheduledEvents: func() ([]*model.ScheduledEvent, error) { return []*model.ScheduledEvent{ - {ContainerID: modA, Name: "SE_A", Interval: 1, IntervalType: "Hour", Enabled: true}, - {ContainerID: modB, Name: "SE_B", Interval: 1, IntervalType: "Hour", Enabled: true}, + {ContainerID: modA, Name: "SE_A", Schedule: &model.Schedule{Kind: model.ScheduleHour, Multiplier: 1}, Enabled: true}, + {ContainerID: modB, Name: "SE_B", Schedule: &model.Schedule{Kind: model.ScheduleHour, Multiplier: 1}, Enabled: true}, }, nil }, } @@ -289,11 +353,15 @@ func TestStarlarkScheduledEventsBuiltin(t *testing.T) { reader := &minimalReader{ listScheduledEvents: func() ([]*model.ScheduledEvent, error) { return []*model.ScheduledEvent{{ - ContainerID: containerID, - Name: "DailySE", - MicroflowID: mfID, + ContainerID: containerID, + Name: "DailySE", + MicroflowID: mfID, + // The interval a rule sees comes from Schedule. The legacy pair + // is left deliberately inconsistent here (it claims 2 days) to + // prove the builtin does not read it. Interval: 2, IntervalType: "Day", + Schedule: &model.Schedule{Kind: model.ScheduleHour, Multiplier: 2}, Enabled: true, }}, nil }, @@ -343,7 +411,7 @@ func TestStarlarkScheduledEventsBuiltin(t *testing.T) { for _, want := range []string{ "se Billing.DailySE", "mf Billing.SUB_DailyJob", - "secs 172800", // 2 * 86400 + "secs 7200", // 2 hours, from the Schedule — NOT the 172800 the legacy pair implies "enabled yes", } { if !strings.Contains(joined, want) { diff --git a/mdl/scheduledevents/codec.go b/mdl/scheduledevents/codec.go new file mode 100644 index 000000000..95a353731 --- /dev/null +++ b/mdl/scheduledevents/codec.go @@ -0,0 +1,295 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package scheduledevents holds the BSON codec for scheduled events +// (ScheduledEvents$ScheduledEvent) and their polymorphic Schedule child. +// +// It lives outside both engines because both write this document, and the +// schedule has eight variants that differ in which fields they carry — a shape +// worth getting right once. The package depends only on the model types and the +// BSON driver, so either backend can call it. +package scheduledevents + +import ( + "fmt" + "time" + + "go.mongodb.org/mongo-driver/bson" + "go.mongodb.org/mongo-driver/bson/primitive" + + "github.com/mendixlabs/mxcli/mdl/bsonutil" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// TypeName is the BSON storage name for a scheduled event document. +const TypeName = "ScheduledEvents$ScheduledEvent" + +// ScheduleTypeNames maps a schedule kind to its BSON storage name. The variants +// differ in arity, not just in values, so the kind is dispatched before any +// field is read or written — see Serialize and ParseSchedule. +var ScheduleTypeNames = map[model.ScheduleKind]string{ + model.ScheduleMinute: "ScheduledEvents$MinuteSchedule", + model.ScheduleHour: "ScheduledEvents$HourSchedule", + model.ScheduleDay: "ScheduledEvents$DaySchedule", + model.ScheduleWeek: "ScheduledEvents$WeekSchedule", + model.ScheduleMonthDate: "ScheduledEvents$MonthDateSchedule", + model.ScheduleMonthWeekday: "ScheduledEvents$MonthWeekdaySchedule", + model.ScheduleYearDate: "ScheduledEvents$YearDateSchedule", + model.ScheduleYearWeekday: "ScheduledEvents$YearWeekdaySchedule", +} + +// WeekdayNames indexes model.Schedule.Weekdays: Sunday first, matching the +// Mendix Weekday enumeration and the BSON property names of WeekSchedule. +var WeekdayNames = [7]string{"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"} + +// Serialize writes the document in the shape Studio Pro produces. +// +// Pinned against four Studio Pro-authored events (Workflow Commons 4.11.0, OIDC +// SSO 4.6.0, SAML 4.2.1 ×2). All four agree on the property set and on three +// types the generated metamodel would lead you to get wrong: +// +// - every integer is BSON int64, while modelsdk/gen declares int32 — the same +// mismatch that made the READER fail on Studio Pro documents in issue #585; +// - StartDateTime is a BSON UTC datetime, while gen declares a string; +// - Microflow is a by-name reference (the qualified name), not an element ID. +// +// Interval/IntervalType are legacy siblings of Schedule that Studio Pro still +// writes and does NOT keep in sync — the Workflow Commons event stores +// Interval 0 / "Minute" next to a DaySchedule of 01:00. They are therefore +// carried through from the caller rather than derived from the schedule, so a +// round trip cannot invent a value Mendix did not have. +func Serialize(ev *model.ScheduledEvent) ([]byte, error) { + exportLevel := ev.ExportLevel + if exportLevel == "" { + exportLevel = "Hidden" + } + onOverlap := ev.OnOverlap + if onOverlap == "" { + // Every reference document uses DelayNext, and it is the safer default: + // SkipNext silently drops a run. + onOverlap = "DelayNext" + } + timeZone := ev.TimeZone + if timeZone == "" { + timeZone = "UTC" + } + var start time.Time + if ev.StartDateTime != nil { + start = ev.StartDateTime.UTC() + } + + doc := bson.D{ + {Key: "$ID", Value: bsonutil.IDToBsonBinary(string(ev.ID))}, + {Key: "$Type", Value: TypeName}, + {Key: "Documentation", Value: ev.Documentation}, + {Key: "Enabled", Value: ev.Enabled}, + {Key: "Excluded", Value: ev.Excluded}, + {Key: "ExportLevel", Value: exportLevel}, + {Key: "Interval", Value: int64(ev.Interval)}, + {Key: "IntervalType", Value: ev.IntervalType}, + {Key: "Microflow", Value: string(ev.MicroflowID)}, + {Key: "Name", Value: ev.Name}, + {Key: "OnOverlap", Value: onOverlap}, + } + if ev.Schedule != nil { + sched, err := SerializeSchedule(ev.Schedule) + if err != nil { + return nil, err + } + doc = append(doc, bson.E{Key: "Schedule", Value: sched}) + } + doc = append(doc, + bson.E{Key: "StartDateTime", Value: primitive.DateTime(start.UnixMilli())}, + bson.E{Key: "TimeZone", Value: timeZone}, + ) + + out, err := bson.Marshal(doc) + if err != nil { + return nil, fmt.Errorf("serialize scheduled event %q: %w", ev.Name, err) + } + return out, nil +} + +// SerializeSchedule writes the ScheduledEvents$*Schedule child. +// +// Each variant writes only the fields its type declares. Writing a field the +// type does not carry is the failure mode that produces a document mxbuild +// accepts and Studio Pro cannot open (System.InvalidOperationException at +// MprProperty), so this dispatches on the kind and never merges field sets. +// Keys are alphabetical within each variant, matching the observed documents. +func SerializeSchedule(s *model.Schedule) (bson.D, error) { + typeName, ok := ScheduleTypeNames[s.Kind] + if !ok { + return nil, fmt.Errorf("unknown schedule kind %q", s.Kind) + } + head := bson.D{ + {Key: "$ID", Value: bsonutil.IDToBsonBinary(types.GenerateID())}, + {Key: "$Type", Value: typeName}, + } + + var fields bson.D + switch s.Kind { + case model.ScheduleMinute: + fields = bson.D{{Key: "Multiplier", Value: int64(s.Multiplier)}} + case model.ScheduleHour: + fields = bson.D{ + {Key: "MinuteOffset", Value: int64(s.MinuteOffset)}, + {Key: "Multiplier", Value: int64(s.Multiplier)}, + } + case model.ScheduleDay: + fields = bson.D{ + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + } + case model.ScheduleWeek: + // Alphabetical, so the seven day flags interleave with the times rather + // than grouping — Friday, HourOfDay, MinuteOfHour, Monday, ... + fields = bson.D{ + {Key: "Friday", Value: s.Weekdays[5]}, + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + {Key: "Monday", Value: s.Weekdays[1]}, + {Key: "Saturday", Value: s.Weekdays[6]}, + {Key: "Sunday", Value: s.Weekdays[0]}, + {Key: "Thursday", Value: s.Weekdays[4]}, + {Key: "Tuesday", Value: s.Weekdays[2]}, + {Key: "Wednesday", Value: s.Weekdays[3]}, + } + case model.ScheduleMonthDate: + fields = bson.D{ + {Key: "DayOfMonth", Value: int64(s.DayOfMonth)}, + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + {Key: "MonthOffset", Value: int64(s.MonthOffset)}, + {Key: "Multiplier", Value: int64(s.Multiplier)}, + } + case model.ScheduleMonthWeekday: + fields = bson.D{ + {Key: "DaySelector", Value: s.DaySelector}, + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + {Key: "MonthOffset", Value: int64(s.MonthOffset)}, + {Key: "Multiplier", Value: int64(s.Multiplier)}, + {Key: "Weekday", Value: s.Weekday}, + } + case model.ScheduleYearDate: + fields = bson.D{ + {Key: "DayOfMonth", Value: int64(s.DayOfMonth)}, + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + {Key: "Month", Value: int64(s.Month)}, + } + case model.ScheduleYearWeekday: + fields = bson.D{ + {Key: "DaySelector", Value: s.DaySelector}, + {Key: "HourOfDay", Value: int64(s.HourOfDay)}, + {Key: "MinuteOfHour", Value: int64(s.MinuteOfHour)}, + {Key: "Month", Value: int64(s.Month)}, + {Key: "Weekday", Value: s.Weekday}, + } + } + return append(head, fields...), nil +} + +// Parse converts a stored document to the semantic type. MicroflowID holds the +// by-name microflow reference (BSON "Microflow"), not an element ID. +func Parse(doc bson.M, id, containerID model.ID) *model.ScheduledEvent { + ev := &model.ScheduledEvent{ContainerID: containerID} + ev.ID = id + ev.TypeName = TypeName + ev.Name, _ = doc["Name"].(string) + ev.Documentation, _ = doc["Documentation"].(string) + if mf, ok := doc["Microflow"].(string); ok { + ev.MicroflowID = model.ID(mf) + } + ev.Enabled, _ = doc["Enabled"].(bool) + ev.Excluded, _ = doc["Excluded"].(bool) + ev.ExportLevel, _ = doc["ExportLevel"].(string) + ev.OnOverlap, _ = doc["OnOverlap"].(string) + ev.TimeZone, _ = doc["TimeZone"].(string) + ev.IntervalType, _ = doc["IntervalType"].(string) + ev.Interval = anyInt(doc["Interval"]) + if dt, ok := doc["StartDateTime"].(primitive.DateTime); ok { + t := time.UnixMilli(int64(dt)).UTC() + ev.StartDateTime = &t + } + if sched, ok := doc["Schedule"].(bson.M); ok { + ev.Schedule = ParseSchedule(sched) + } + return ev +} + +// ParseSchedule dispatches on $Type before reading any field: the variants +// differ in arity, so inferring the kind from which keys are present would +// mis-read a schedule whose fields overlap another's. +func ParseSchedule(doc bson.M) *model.Schedule { + typeName, _ := doc["$Type"].(string) + var kind model.ScheduleKind + for k, name := range ScheduleTypeNames { + if name == typeName { + kind = k + break + } + } + if kind == "" { + return nil + } + s := &model.Schedule{Kind: kind} + switch kind { + case model.ScheduleMinute: + s.Multiplier = anyInt(doc["Multiplier"]) + case model.ScheduleHour: + s.Multiplier = anyInt(doc["Multiplier"]) + s.MinuteOffset = anyInt(doc["MinuteOffset"]) + case model.ScheduleDay: + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + case model.ScheduleWeek: + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + for i, day := range WeekdayNames { + s.Weekdays[i], _ = doc[day].(bool) + } + case model.ScheduleMonthDate: + s.Multiplier = anyInt(doc["Multiplier"]) + s.MonthOffset = anyInt(doc["MonthOffset"]) + s.DayOfMonth = anyInt(doc["DayOfMonth"]) + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + case model.ScheduleMonthWeekday: + s.Multiplier = anyInt(doc["Multiplier"]) + s.MonthOffset = anyInt(doc["MonthOffset"]) + s.DaySelector, _ = doc["DaySelector"].(string) + s.Weekday, _ = doc["Weekday"].(string) + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + case model.ScheduleYearDate: + s.Month = anyInt(doc["Month"]) + s.DayOfMonth = anyInt(doc["DayOfMonth"]) + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + case model.ScheduleYearWeekday: + s.Month = anyInt(doc["Month"]) + s.DaySelector, _ = doc["DaySelector"].(string) + s.Weekday, _ = doc["Weekday"].(string) + s.HourOfDay = anyInt(doc["HourOfDay"]) + s.MinuteOfHour = anyInt(doc["MinuteOfHour"]) + } + return s +} + +// anyInt accepts every numeric width a writer might have produced. Studio Pro +// writes int64; older mxcli builds and hand-made fixtures write int32 (#585). +func anyInt(v any) int { + switch n := v.(type) { + case int64: + return int(n) + case int32: + return int(n) + case int: + return n + case float64: + return int(n) + } + return 0 +} diff --git a/mdl/scheduledevents/codec_test.go b/mdl/scheduledevents/codec_test.go new file mode 100644 index 000000000..dce733cdb --- /dev/null +++ b/mdl/scheduledevents/codec_test.go @@ -0,0 +1,314 @@ +// SPDX-License-Identifier: Apache-2.0 + +package scheduledevents + +import ( + "encoding/hex" + "testing" + "time" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/model" +) + +// Three complete, unedited scheduled-event documents lifted out of Mendix's own +// marketplace modules — the only Studio Pro-authored references available +// without Studio Pro. Between them they cover both schedule variants that ship +// in Mendix modules (Day and Hour), both TimeZone values, both Enabled values, +// documented and undocumented, and a zero Interval. +// +// oidcHex OIDC SSO 4.6.0 CleanupOldAuthAttempts HourSchedule +// samlHex SAML 4.2.1 SE_LogCleanUp DaySchedule, TimeZone Server +// wfcHex Workflow Commons 4.11.0 SE_WorkflowAuditTrailRecord… DaySchedule, Interval 0 +const ( + oidcHex = "c00100000524494400100000000059533bc5d678bb40835cd6028304521f022454797065001f0000005363686564756c65644576656e7473245363686564756c65644576656e740002446f63756d656e746174696f6e00010000000008456e61626c65640001084578636c756465640000024578706f72744c6576656c000700000048696464656e0012496e74657276616c00010000000000000002496e74657276616c547970650005000000486f757200024d6963726f666c6f7700200000004f4944432e5355425f436c65616e75704f6c6441757468417474656d70747300024e616d650017000000436c65616e75704f6c6441757468417474656d70747300024f6e4f7665726c6170000a00000044656c61794e65787400035363686564756c65007100000005244944001000000000c154dede71230641b986faef8e304829022454797065001d0000005363686564756c65644576656e747324486f75725363686564756c6500124d696e7574654f6666736574001700000000000000124d756c7469706c696572000100000000000000000953746172744461746554696d650058a17738730100000254696d655a6f6e6500040000005554430000" + samlHex = "63020000052449440010000000007e890ddd9ec5a444b417ef5992c27101022454797065001f0000005363686564756c65644576656e7473245363686564756c65644576656e740002446f63756d656e746174696f6e00b80000005468697320616374696f6e2077696c6c20636c65616e757020746865206c6f676c696e65732e20546865206c6f677320656e74726965732077696c6c20626520617263686976656420666f722074686520616d6f756e74206f6620646179732073706563696669656420696e2074686520636f6e66696775726174696f6e2c20746865206c6f6720656e74726965732074686174207265616368656420746865206c696d69742077696c6c2062652064656c657465642e0008456e61626c65640000084578636c756465640000024578706f72744c6576656c000700000048696464656e0012496e74657276616c00010000000000000002496e74657276616c54797065000400000044617900024d6963726f666c6f77001500000053414d4c32302e53455f4c6f67436c65616e557000024e616d65000e00000053455f4c6f67436c65616e557000024f6e4f7665726c6170000a00000044656c61794e65787400035363686564756c65006f00000005244944001000000000df856dc51b45c24d89325077b18be82b022454797065001c0000005363686564756c65644576656e7473244461795363686564756c650012486f75724f66446179000400000000000000124d696e7574654f66486f7572000000000000000000000953746172744461746554696d6500002ea2b8390100000254696d655a6f6e6500070000005365727665720000" + wfcHex = "e1010000052449440010000000003d7bd7ba3c94a84d816c130543adf825022454797065001f0000005363686564756c65644576656e7473245363686564756c65644576656e740002446f63756d656e746174696f6e00010000000008456e61626c65640000084578636c756465640000024578706f72744c6576656c000700000048696464656e0012496e74657276616c00000000000000000002496e74657276616c5479706500070000004d696e75746500024d6963726f666c6f770034000000576f726b666c6f77436f6d6d6f6e732e53455f576f726b666c6f774175646974547261696c5265636f72645f436c65616e557000024e616d65002400000053455f576f726b666c6f774175646974547261696c5265636f72645f436c65616e557000024f6e4f7665726c6170000a00000044656c61794e65787400035363686564756c65006f0000000524494400100000000081a8e58306a48748bdc93d240d67c64b022454797065001c0000005363686564756c65644576656e7473244461795363686564756c650012486f75724f66446179000100000000000000124d696e7574654f66486f7572000000000000000000000953746172744461746554696d6500db6b4292900100000254696d655a6f6e6500040000005554430000" +) + +func mustDecode(t *testing.T, h string) bson.M { + t.Helper() + raw, err := hex.DecodeString(h) + if err != nil { + t.Fatalf("hex: %v", err) + } + var doc bson.M + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + return doc +} + +// TestParseStudioProDocuments checks the reader against the real documents, +// including the two properties the generated metamodel gets wrong: every +// integer is int64 (gen says int32) and StartDateTime is a BSON datetime (gen +// says string). +func TestParseStudioProDocuments(t *testing.T) { + tests := []struct { + name string + hexDoc string + want model.ScheduledEvent + wantSched model.Schedule + }{ + { + name: "OIDC CleanupOldAuthAttempts", + hexDoc: oidcHex, + want: model.ScheduledEvent{ + Name: "CleanupOldAuthAttempts", MicroflowID: "OIDC.SUB_CleanupOldAuthAttempts", + Enabled: true, Interval: 1, IntervalType: "Hour", + OnOverlap: "DelayNext", TimeZone: "UTC", ExportLevel: "Hidden", + }, + wantSched: model.Schedule{Kind: model.ScheduleHour, Multiplier: 1, MinuteOffset: 23}, + }, + { + name: "SAML SE_LogCleanUp", + hexDoc: samlHex, + want: model.ScheduledEvent{ + Name: "SE_LogCleanUp", MicroflowID: "SAML20.SE_LogCleanUp", + Enabled: false, Interval: 1, IntervalType: "Day", + OnOverlap: "DelayNext", TimeZone: "Server", ExportLevel: "Hidden", + }, + wantSched: model.Schedule{Kind: model.ScheduleDay, HourOfDay: 4, MinuteOfHour: 0}, + }, + { + name: "Workflow Commons cleanup", + hexDoc: wfcHex, + want: model.ScheduledEvent{ + Name: "SE_WorkflowAuditTrailRecord_CleanUp", + // Interval 0 / "Minute" alongside a DaySchedule of 01:00: Studio + // Pro does not keep the legacy pair in sync with Schedule, which + // is why the writer carries them through instead of deriving them. + MicroflowID: "WorkflowCommons.SE_WorkflowAuditTrailRecord_CleanUp", + Enabled: false, Interval: 0, IntervalType: "Minute", + OnOverlap: "DelayNext", TimeZone: "UTC", ExportLevel: "Hidden", + }, + wantSched: model.Schedule{Kind: model.ScheduleDay, HourOfDay: 1, MinuteOfHour: 0}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + ev := Parse(mustDecode(t, tt.hexDoc), "id-1", "mod-1") + if ev.Name != tt.want.Name { + t.Errorf("Name = %q, want %q", ev.Name, tt.want.Name) + } + if ev.MicroflowID != tt.want.MicroflowID { + t.Errorf("MicroflowID = %q, want %q", ev.MicroflowID, tt.want.MicroflowID) + } + if ev.Enabled != tt.want.Enabled { + t.Errorf("Enabled = %v", ev.Enabled) + } + if ev.Interval != tt.want.Interval { + t.Errorf("Interval = %d, want %d (stored as int64)", ev.Interval, tt.want.Interval) + } + if ev.IntervalType != tt.want.IntervalType { + t.Errorf("IntervalType = %q", ev.IntervalType) + } + if ev.OnOverlap != tt.want.OnOverlap { + t.Errorf("OnOverlap = %q", ev.OnOverlap) + } + if ev.TimeZone != tt.want.TimeZone { + t.Errorf("TimeZone = %q", ev.TimeZone) + } + if ev.ExportLevel != tt.want.ExportLevel { + t.Errorf("ExportLevel = %q", ev.ExportLevel) + } + if ev.StartDateTime == nil { + t.Error("StartDateTime not read — it is a BSON datetime, not the string gen declares") + } + if ev.Schedule == nil { + t.Fatal("Schedule not read") + } + if *ev.Schedule != tt.wantSched { + t.Errorf("Schedule = %+v, want %+v", *ev.Schedule, tt.wantSched) + } + }) + } +} + +// TestSerializeMatchesStudioProDocuments is the shape pin: read each real +// document, write it back, and require the result to be identical apart from +// the two generated $ID values. It compares raw BSON elements, so key ORDER and +// BSON TYPE (int64 vs int32, datetime vs string) are covered as well as values — +// none of which a field-by-field assertion would catch. +func TestSerializeMatchesStudioProDocuments(t *testing.T) { + for _, tt := range []struct{ name, hexDoc string }{ + {"OIDC", oidcHex}, + {"SAML", samlHex}, + {"WorkflowCommons", wfcHex}, + } { + t.Run(tt.name, func(t *testing.T) { + originalBytes, err := hex.DecodeString(tt.hexDoc) + if err != nil { + t.Fatalf("hex: %v", err) + } + ev := Parse(mustDecode(t, tt.hexDoc), "11111111-1111-1111-1111-111111111111", "mod-1") + + out, err := Serialize(ev) + if err != nil { + t.Fatalf("Serialize: %v", err) + } + assertSameExceptIDs(t, "", bson.Raw(originalBytes), bson.Raw(out)) + }) + } +} + +// assertSameExceptIDs compares two raw documents element by element, in order, +// requiring the same keys in the same positions with the same BSON types and +// values. $ID is compared for presence only: it is regenerated on every write. +func assertSameExceptIDs(t *testing.T, path string, want, got bson.Raw) { + t.Helper() + wantEls, err := want.Elements() + if err != nil { + t.Fatalf("%s: elements: %v", path, err) + } + gotEls, err := got.Elements() + if err != nil { + t.Fatalf("%s: elements: %v", path, err) + } + if len(wantEls) != len(gotEls) { + t.Fatalf("%s: wrote %d properties, Studio Pro wrote %d\n want %v\n got %v", + path, len(gotEls), len(wantEls), keysOf(wantEls), keysOf(gotEls)) + } + for i, we := range wantEls { + ge := gotEls[i] + if we.Key() != ge.Key() { + t.Errorf("%sproperty %d is %q, want %q (order differs)", path, i, ge.Key(), we.Key()) + continue + } + wv, gv := we.Value(), ge.Value() + if wv.Type != gv.Type { + t.Errorf("%s%s: BSON type %s, want %s", path, we.Key(), gv.Type, wv.Type) + continue + } + if we.Key() == "$ID" { + continue + } + if wv.Type == bson.TypeEmbeddedDocument { + assertSameExceptIDs(t, path+we.Key()+".", wv.Document(), gv.Document()) + continue + } + if !wv.Equal(gv) { + t.Errorf("%s%s = %v, want %v", path, we.Key(), gv, wv) + } + } +} + +func keysOf(els []bson.RawElement) []string { + out := make([]string, len(els)) + for i, e := range els { + out[i] = e.Key() + } + return out +} + +// TestSerializeSchedule_AllVariants covers the six variants no Mendix module +// ships, so the field sets are metamodel-derived rather than observed. The +// assertion that matters is that each variant writes ONLY its own fields: a +// merged field set is the shape that mxbuild accepts and Studio Pro refuses to +// open. +func TestSerializeSchedule_AllVariants(t *testing.T) { + full := model.Schedule{ + Multiplier: 2, MinuteOffset: 5, MonthOffset: 1, + HourOfDay: 6, MinuteOfHour: 30, DayOfMonth: 15, Month: 3, + DaySelector: "Last", Weekday: "Friday", + Weekdays: [7]bool{false, true, false, false, false, true, false}, + } + wantKeys := map[model.ScheduleKind][]string{ + model.ScheduleMinute: {"Multiplier"}, + model.ScheduleHour: {"MinuteOffset", "Multiplier"}, + model.ScheduleDay: {"HourOfDay", "MinuteOfHour"}, + model.ScheduleWeek: {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "HourOfDay", "MinuteOfHour"}, + model.ScheduleMonthDate: {"Multiplier", "MonthOffset", "DayOfMonth", "HourOfDay", "MinuteOfHour"}, + model.ScheduleMonthWeekday: {"Multiplier", "MonthOffset", "DaySelector", "Weekday", "HourOfDay", "MinuteOfHour"}, + model.ScheduleYearDate: {"Month", "DayOfMonth", "HourOfDay", "MinuteOfHour"}, + model.ScheduleYearWeekday: {"Month", "DaySelector", "Weekday", "HourOfDay", "MinuteOfHour"}, + } + + for kind, keys := range wantKeys { + t.Run(string(kind), func(t *testing.T) { + s := full + s.Kind = kind + doc, err := SerializeSchedule(&s) + if err != nil { + t.Fatalf("SerializeSchedule: %v", err) + } + got := map[string]bool{} + for _, e := range doc { + got[e.Key] = true + } + if !got["$ID"] || !got["$Type"] { + t.Error("missing $ID/$Type") + } + delete(got, "$ID") + delete(got, "$Type") + + for _, k := range keys { + if !got[k] { + t.Errorf("missing %s", k) + } + delete(got, k) + } + for k := range got { + t.Errorf("%s is not a property of %s — writing it produces a document Studio Pro cannot open", k, ScheduleTypeNames[kind]) + } + }) + } +} + +func TestSerializeSchedule_UnknownKindIsRefused(t *testing.T) { + if _, err := SerializeSchedule(&model.Schedule{Kind: "Fortnightly"}); err == nil { + t.Fatal("expected an error for an unknown schedule kind") + } +} + +// TestSerialize_Defaults documents the values supplied when the caller leaves +// them empty, all taken from the reference documents. +func TestSerialize_Defaults(t *testing.T) { + ev := &model.ScheduledEvent{Name: "SE_Plain", MicroflowID: "M.MF"} + ev.ID = "22222222-2222-2222-2222-222222222222" + + out, err := Serialize(ev) + if err != nil { + t.Fatalf("Serialize: %v", err) + } + var doc bson.M + if err := bson.Unmarshal(out, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + if doc["ExportLevel"] != "Hidden" { + t.Errorf("ExportLevel = %v, want Hidden", doc["ExportLevel"]) + } + if doc["OnOverlap"] != "DelayNext" { + t.Errorf("OnOverlap = %v, want DelayNext (SkipNext silently drops a run)", doc["OnOverlap"]) + } + if doc["TimeZone"] != "UTC" { + t.Errorf("TimeZone = %v, want UTC", doc["TimeZone"]) + } + if _, ok := doc["Interval"].(int64); !ok { + t.Errorf("Interval = %#v, want an int64 — Studio Pro writes int64, gen declares int32", doc["Interval"]) + } + if _, ok := doc["Schedule"]; ok { + t.Error("a nil schedule must not be written as an empty child") + } +} + +func TestSerialize_StartDateTimeRoundTrips(t *testing.T) { + want := time.Date(2024, 7, 8, 12, 12, 24, 923_000_000, time.UTC) + ev := &model.ScheduledEvent{Name: "SE", MicroflowID: "M.MF", StartDateTime: &want} + ev.ID = "33333333-3333-3333-3333-333333333333" + + out, err := Serialize(ev) + if err != nil { + t.Fatalf("Serialize: %v", err) + } + var doc bson.M + if err := bson.Unmarshal(out, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + got := Parse(doc, "id", "mod") + if got.StartDateTime == nil || !got.StartDateTime.Equal(want) { + t.Errorf("StartDateTime = %v, want %v", got.StartDateTime, want) + } +} diff --git a/mdl/types/navigation.go b/mdl/types/navigation.go index dc8187a5f..8fd704117 100644 --- a/mdl/types/navigation.go +++ b/mdl/types/navigation.go @@ -59,6 +59,26 @@ type NavMenuItem struct { Items []*NavMenuItem `json:"items,omitempty"` } +// MenuDocument is a standalone `Menus$MenuDocument` — a reusable menu that menu +// widgets point at, stored as its own document rather than inside a navigation +// profile. Atlas_Core ships two of them (Phone_Menu, Tablet_Menu). +// +// Its entries are the same `Menus$MenuItem` elements a navigation profile holds, +// so they are modelled as NavMenuItem rather than a parallel type — one item +// shape, one parser, one renderer. +type MenuDocument struct { + ID model.ID `json:"id"` + ContainerID model.ID `json:"containerId"` + Name string `json:"name"` + Documentation string `json:"documentation,omitempty"` + ExportLevel string `json:"exportLevel,omitempty"` + Excluded bool `json:"excluded,omitempty"` + Items []*NavMenuItem `json:"items,omitempty"` +} + +// GetName returns the menu document's name. +func (m *MenuDocument) GetName() string { return m.Name } + // NavOfflineEntity declares offline sync rules for an entity. type NavOfflineEntity struct { Entity string `json:"entity"` diff --git a/mdl/types/queue.go b/mdl/types/queue.go new file mode 100644 index 000000000..1560bd681 --- /dev/null +++ b/mdl/types/queue.go @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: Apache-2.0 + +package types + +import "github.com/mendixlabs/mxcli/model" + +// Queue is a Mendix task queue (Queues$Queue) — the configuration that governs +// how many instances of a queued microflow call run at once, and whether that +// limit is per-node or cluster-wide. +// +// The shape here follows four Studio Pro-authored queues from the Mendix +// Business Events module (Consumer_Queue, Consumer_Processor_Queue, +// Producer_Queue, Outbox_Cleanup_Queue), which agree exactly: +// +// { "$Type": "Queues$Queue", "Name": "Consumer_Queue", +// "Documentation": "", "Excluded": false, "ExportLevel": "Hidden", +// "Config": { "$Type": "Queues$BasicQueueConfig", +// "ClusterWide": false, "ParallelismExpression": "1" } } +// +// Two details are load-bearing and are not guesses: +// +// 1. Parallelism is stored as ParallelismExpression, a STRING holding a Mendix +// expression ("1"). Queues$BasicQueueConfig also declares an int32 +// `Parallelism`, but Studio Pro wrote it zero times out of four — so it is +// not written here either. +// 2. Config was Queues$BasicQueueConfig in all four, and it is the only config +// type in the metamodel, so there is no variant to dispatch on. +type Queue struct { + model.BaseElement + ContainerID model.ID `json:"containerId"` + Name string `json:"name"` + Documentation string `json:"documentation,omitempty"` + Excluded bool `json:"excluded,omitempty"` + ExportLevel string `json:"exportLevel,omitempty"` + + // Parallelism is the expression form (Config.ParallelismExpression), kept as + // a string because that is how Mendix stores it: usually a literal like "3", + // but any Mendix expression is valid. + Parallelism string `json:"parallelism,omitempty"` + // ClusterWide makes the parallelism limit apply across the cluster rather + // than per runtime instance (Config.ClusterWide). + ClusterWide bool `json:"clusterWide,omitempty"` +} + +// GetName returns the queue's name. +func (q *Queue) GetName() string { return q.Name } + +// GetContainerID returns the container ID. +func (q *Queue) GetContainerID() model.ID { return q.ContainerID } diff --git a/mdl/types/unit_types.go b/mdl/types/unit_types.go index b41b7cfeb..d6f2feff5 100644 --- a/mdl/types/unit_types.go +++ b/mdl/types/unit_types.go @@ -8,35 +8,35 @@ package types // when switching on UnitInfo.Type or unitCache entries. const ( // Core project structure - UnitTypeModule = "Projects$ModuleImpl" - UnitTypeModuleSettings = "Projects$ModuleSettings" - UnitTypeFolder = "Projects$Folder" + UnitTypeModule = "Projects$ModuleImpl" + UnitTypeModuleSettings = "Projects$ModuleSettings" + UnitTypeFolder = "Projects$Folder" UnitTypeProjectSettings = "Settings$ProjectSettings" // Domain model - UnitTypeDomainModel = "DomainModels$DomainModel" - UnitTypeEnumeration = "Enumerations$Enumeration" - UnitTypeConstant = "Constants$Constant" + UnitTypeDomainModel = "DomainModels$DomainModel" + UnitTypeEnumeration = "Enumerations$Enumeration" + UnitTypeConstant = "Constants$Constant" // Flows - UnitTypeMicroflow = "Microflows$Microflow" - UnitTypeNanoflow = "Microflows$Nanoflow" - UnitTypeRule = "Microflows$Rule" + UnitTypeMicroflow = "Microflows$Microflow" + UnitTypeNanoflow = "Microflows$Nanoflow" + UnitTypeRule = "Microflows$Rule" // Pages / UI - UnitTypePage = "Forms$Page" - UnitTypeLayout = "Forms$Layout" - UnitTypeSnippet = "Forms$Snippet" + UnitTypePage = "Forms$Page" + UnitTypeLayout = "Forms$Layout" + UnitTypeSnippet = "Forms$Snippet" // Java / JavaScript actions UnitTypeJavaAction = "JavaActions$JavaAction" UnitTypeJavaScriptAction = "JavaActions$JavaScriptAction" // Workflows - UnitTypeWorkflow = "Workflows$Workflow" + UnitTypeWorkflow = "Workflows$Workflow" // REST / OData / Business events - UnitTypePublishedRestService = "Rest$PublishedRestService" + UnitTypePublishedRestService = "Rest$PublishedRestService" UnitTypePublishedODataService = "ODataPublish$PublishedODataService" UnitTypeConsumedODataService = "Rest$ConsumedODataService" UnitTypeBusinessEvent = "BusinessEvents$" // prefix — multiple sub-types diff --git a/mdl/visitor/visitor_catalog_test.go b/mdl/visitor/visitor_catalog_test.go index b8599c969..8eba1f62b 100644 --- a/mdl/visitor/visitor_catalog_test.go +++ b/mdl/visitor/visitor_catalog_test.go @@ -68,6 +68,18 @@ func TestSelectFromCatalog(t *testing.T) { {"communities keyword table", "SELECT * FROM CATALOG.COMMUNITIES;"}, {"communities lowercase", "select * from catalog.communities;"}, {"communities with where", "SELECT AssetName FROM CATALOG.COMMUNITIES WHERE CommunityId = 1;"}, + // Same trap, hit again when QUEUES became a lexer keyword (SHOW QUEUES): + // CATALOG.QUEUES stopped parsing and the SELECT produced no statement and + // no output — no error, just silence. Every new plural keyword that names + // a catalog table has to be added to catalogTableName. + {"queues keyword table", "SELECT * FROM CATALOG.QUEUES;"}, + {"queues lowercase", "select * from catalog.queues;"}, + {"queues with where", "SELECT Name FROM CATALOG.QUEUES WHERE ClusterWide = 1;"}, + // SCHEDULED_EVENTS lexes as one IDENTIFIER (the underscore joins it), so + // the SCHEDULED keyword does not reach it — asserted so a future lexer + // change that splits it is caught here. + {"scheduled events table", "SELECT * FROM CATALOG.SCHEDULED_EVENTS;"}, + {"scheduled events with where", "SELECT Name FROM CATALOG.SCHEDULED_EVENTS WHERE Enabled = 1;"}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/mdl/visitor/visitor_entity.go b/mdl/visitor/visitor_entity.go index c4b043d6e..3d2d51932 100644 --- a/mdl/visitor/visitor_entity.go +++ b/mdl/visitor/visitor_entity.go @@ -801,6 +801,10 @@ func (b *Builder) ExitDropStatement(ctx *parser.DropStatementContext) { b.statements = append(b.statements, &ast.DropSnippetStmt{ Name: buildQualifiedName(names[0]), }) + } else if ctx.MENU_KW() != nil { + b.statements = append(b.statements, &ast.DropMenuStmt{ + Name: buildQualifiedName(names[0]), + }) } else if ctx.JAVASCRIPT() != nil && ctx.ACTION() != nil { b.statements = append(b.statements, &ast.DropJavaScriptActionStmt{ Name: buildQualifiedName(names[0]), @@ -829,6 +833,14 @@ func (b *Builder) ExitDropStatement(ctx *parser.DropStatementContext) { b.statements = append(b.statements, &ast.DropImageCollectionStmt{ Name: buildQualifiedName(names[0]), }) + } else if ctx.QUEUE() != nil { + b.statements = append(b.statements, &ast.DropQueueStmt{ + Name: buildQualifiedName(names[0]), + }) + } else if ctx.SCHEDULED() != nil && ctx.EVENT() != nil { + b.statements = append(b.statements, &ast.DropScheduledEventStmt{ + Name: buildQualifiedName(names[0]), + }) } else if ctx.MODEL() != nil { b.statements = append(b.statements, &ast.DropModelStmt{ Name: buildQualifiedName(names[0]), diff --git a/mdl/visitor/visitor_menu.go b/mdl/visitor/visitor_menu.go new file mode 100644 index 000000000..295466adc --- /dev/null +++ b/mdl/visitor/visitor_menu.go @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/grammar/parser" +) + +// ExitCreateMenuStatement handles CREATE [OR MODIFY] MENU Module.Name ( items ). +// +// The items reuse navMenuItemDef, the same rule CREATE NAVIGATION's MENU block +// uses, so buildNavMenuItemDef is reused verbatim — a menu item is written the +// same way wherever it appears. +func (b *Builder) ExitCreateMenuStatement(ctx *parser.CreateMenuStatementContext) { + qn := ctx.QualifiedName() + if qn == nil { + return + } + + stmt := &ast.CreateMenuStmt{Name: buildQualifiedName(qn)} + for _, itemCtx := range ctx.AllNavMenuItemDef() { + stmt.Items = append(stmt.Items, buildNavMenuItemDef(itemCtx)) + } + + if createStmt := findParentCreateStatement(ctx); createStmt != nil { + if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { + stmt.CreateOrModify = true + } + } + + b.statements = append(b.statements, stmt) +} diff --git a/mdl/visitor/visitor_query.go b/mdl/visitor/visitor_query.go index 79170dfae..8f2ca7f3f 100644 --- a/mdl/visitor/visitor_query.go +++ b/mdl/visitor/visitor_query.go @@ -204,6 +204,26 @@ func (b *Builder) ExitShowStatement(ctx *parser.ShowStatementContext) { } } b.statements = append(b.statements, stmt) + } else if ctx.QUEUES() != nil { + stmt := &ast.ShowQueuesStmt{} + if ctx.IN() != nil { + if qn := ctx.QualifiedName(); qn != nil { + stmt.Module = getQualifiedNameText(qn) + } else if id := ctx.IDENTIFIER(); id != nil { + stmt.Module = id.GetText() + } + } + b.statements = append(b.statements, stmt) + } else if ctx.SCHEDULED() != nil && ctx.EVENTS() != nil { + stmt := &ast.ShowScheduledEventsStmt{} + if ctx.IN() != nil { + if qn := ctx.QualifiedName(); qn != nil { + stmt.Module = getQualifiedNameText(qn) + } else if id := ctx.IDENTIFIER(); id != nil { + stmt.Module = id.GetText() + } + } + b.statements = append(b.statements, stmt) } else if ctx.LAYOUTS() != nil { stmt := &ast.ShowStmt{ObjectType: ast.ShowLayouts} if ctx.IN() != nil { @@ -694,6 +714,22 @@ func (b *Builder) ExitCatalogSelectQuery(ctx *parser.CatalogSelectQueryContext) // ExitDescribeStatement handles DESCRIBE ENTITY/ASSOCIATION/ENUMERATION/MODULE func (b *Builder) ExitDescribeStatement(ctx *parser.DescribeStatementContext) { + // DESCRIBE QUEUE Module.Name + if ctx.QUEUE() != nil { + if qn := ctx.QualifiedName(); qn != nil { + b.statements = append(b.statements, &ast.DescribeQueueStmt{Name: buildQualifiedName(qn)}) + } + return + } + + // DESCRIBE SCHEDULED EVENT Module.Name + if ctx.SCHEDULED() != nil && ctx.EVENT() != nil { + if qn := ctx.QualifiedName(); qn != nil { + b.statements = append(b.statements, &ast.DescribeScheduledEventStmt{Name: buildQualifiedName(qn)}) + } + return + } + // Handle DESCRIBE MODULE ROLE (uses qualifiedName) if ctx.MODULE() != nil && ctx.ROLE() != nil { if qn := ctx.QualifiedName(); qn != nil { @@ -1010,6 +1046,11 @@ func (b *Builder) ExitDescribeStatement(ctx *parser.DescribeStatementContext) { ObjectType: ast.DescribeBuildingBlock, Name: name, }) + } else if ctx.MENU_KW() != nil { + b.statements = append(b.statements, &ast.DescribeStmt{ + ObjectType: ast.DescribeMenu, + Name: name, + }) } else if ctx.SNIPPET() != nil { b.statements = append(b.statements, &ast.DescribeStmt{ ObjectType: ast.DescribeSnippet, diff --git a/mdl/visitor/visitor_queue.go b/mdl/visitor/visitor_queue.go new file mode 100644 index 000000000..a7ff58cc1 --- /dev/null +++ b/mdl/visitor/visitor_queue.go @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/grammar/parser" +) + +// ExitCreateQueueStatement builds a CreateQueueStmt from +// CREATE [OR REPLACE|MODIFY] QUEUE Module.Name ( ... ). +func (b *Builder) ExitCreateQueueStatement(ctx *parser.CreateQueueStatementContext) { + stmt := &ast.CreateQueueStmt{ + Name: buildQualifiedName(ctx.QualifiedName()), + Documentation: findDocCommentText(ctx), + } + if createStmt := findParentCreateStatement(ctx); createStmt != nil { + if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { + stmt.CreateOrModify = true + } + } + + if body := ctx.QueueBody(); body != nil { + bodyCtx := body.(*parser.QueueBodyContext) + for _, prop := range bodyCtx.AllQueueProperty() { + pc, ok := prop.(*parser.QueuePropertyContext) + if !ok || pc == nil { + continue + } + iok := pc.IdentifierOrKeyword(0) + if iok == nil { + continue + } + key := strings.ToLower(identifierOrKeywordText(iok)) + switch key { + case "parallelism": + stmt.Parallelism = queuePropertyText(pc) + case "clusterwide": + stmt.ClusterWide = strings.EqualFold(queuePropertyText(pc), "true") + case "exportlevel": + stmt.ExportLevel = queuePropertyText(pc) + case "documentation": + stmt.Documentation = queuePropertyText(pc) + } + } + } + + b.statements = append(b.statements, stmt) +} + +// queuePropertyText returns the value side of a queue property, unquoted. +// +// The value alternatives are NUMBER_LITERAL | STRING_LITERAL | booleanLiteral | +// identifierOrKeyword. The key is identifierOrKeyword(0), so an identifier value +// is index 1 — reading index 0 would echo the key back as the value. +func queuePropertyText(pc *parser.QueuePropertyContext) string { + if n := pc.NUMBER_LITERAL(); n != nil { + return n.GetText() + } + if s := pc.STRING_LITERAL(); s != nil { + return unquoteString(s.GetText()) + } + if bl := pc.BooleanLiteral(); bl != nil { + return bl.GetText() + } + if v := pc.IdentifierOrKeyword(1); v != nil { + return identifierOrKeywordText(v) + } + return "" +} diff --git a/mdl/visitor/visitor_queue_test.go b/mdl/visitor/visitor_queue_test.go new file mode 100644 index 000000000..831d9bba1 --- /dev/null +++ b/mdl/visitor/visitor_queue_test.go @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +func TestCreateQueue(t *testing.T) { + input := `CREATE QUEUE Ops.OrderProcessing ( + Parallelism: 3, + ClusterWide: true + );` + prog, errs := Build(input) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateQueueStmt) + if !ok { + t.Fatalf("expected CreateQueueStmt, got %T", prog.Statements[0]) + } + if stmt.Name.Module != "Ops" || stmt.Name.Name != "OrderProcessing" { + t.Errorf("Name = %+v", stmt.Name) + } + if stmt.Parallelism != "3" { + t.Errorf("Parallelism = %q, want 3", stmt.Parallelism) + } + if !stmt.ClusterWide { + t.Error("ClusterWide = false, want true") + } + if stmt.CreateOrModify { + t.Error("CreateOrModify set without OR MODIFY") + } +} + +// TestCreateQueue_ExpressionParallelism covers the reason Parallelism is a +// string all the way down: Mendix stores an expression, not a number. +func TestCreateQueue_ExpressionParallelism(t *testing.T) { + prog, errs := Build(`CREATE OR MODIFY QUEUE Ops.Q ( Parallelism: '$Config/Workers' );`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateQueueStmt) + if stmt.Parallelism != "$Config/Workers" { + t.Errorf("Parallelism = %q, want the unquoted expression", stmt.Parallelism) + } + if !stmt.CreateOrModify { + t.Error("CreateOrModify not set for CREATE OR MODIFY") + } +} + +func TestCreateQueue_Defaults(t *testing.T) { + prog, errs := Build(`CREATE QUEUE Ops.Q ();`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateQueueStmt) + if stmt.Parallelism != "" { + t.Errorf("Parallelism = %q, want empty (the backend supplies the default)", stmt.Parallelism) + } + if stmt.ClusterWide { + t.Error("ClusterWide defaulted to true") + } +} + +func TestDropQueue(t *testing.T) { + prog, errs := Build(`DROP QUEUE Ops.OrderProcessing;`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.DropQueueStmt) + if !ok { + t.Fatalf("expected DropQueueStmt, got %T", prog.Statements[0]) + } + if stmt.Name.String() != "Ops.OrderProcessing" { + t.Errorf("Name = %s", stmt.Name.String()) + } +} + +func TestShowAndDescribeQueues(t *testing.T) { + prog, errs := Build(`SHOW QUEUES; LIST QUEUES IN Ops; DESCRIBE QUEUE Ops.OrderProcessing;`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + if len(prog.Statements) != 3 { + t.Fatalf("expected 3 statements, got %d", len(prog.Statements)) + } + if _, ok := prog.Statements[0].(*ast.ShowQueuesStmt); !ok { + t.Errorf("statement 0 = %T, want ShowQueuesStmt", prog.Statements[0]) + } + s1, ok := prog.Statements[1].(*ast.ShowQueuesStmt) + if !ok { + t.Fatalf("statement 1 = %T, want ShowQueuesStmt", prog.Statements[1]) + } + if s1.Module != "Ops" { + t.Errorf("Module = %q, want Ops", s1.Module) + } + s2, ok := prog.Statements[2].(*ast.DescribeQueueStmt) + if !ok { + t.Fatalf("statement 2 = %T, want DescribeQueueStmt", prog.Statements[2]) + } + if s2.Name.String() != "Ops.OrderProcessing" { + t.Errorf("Name = %s", s2.Name.String()) + } +} + +// TestQueueKeywordStillUsableAsIdentifier guards the cost of adding QUEUE to the +// lexer: a new keyword stops being usable as an ordinary name unless it is also +// listed in the `keyword` rule. "queue" is a plausible attribute name. +func TestQueueKeywordStillUsableAsIdentifier(t *testing.T) { + prog, errs := Build(`CREATE ENTITY Ops.Job ( queue: String(100), queues: String(100) );`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateEntityStmt) + if !ok { + t.Fatalf("expected CreateEntityStmt, got %T", prog.Statements[0]) + } + if len(stmt.Attributes) != 2 { + t.Fatalf("expected 2 attributes, got %d", len(stmt.Attributes)) + } + if stmt.Attributes[0].Name != "queue" || stmt.Attributes[1].Name != "queues" { + t.Errorf("attribute names = %q, %q", stmt.Attributes[0].Name, stmt.Attributes[1].Name) + } +} diff --git a/mdl/visitor/visitor_scheduledevent.go b/mdl/visitor/visitor_scheduledevent.go new file mode 100644 index 000000000..5a52640b3 --- /dev/null +++ b/mdl/visitor/visitor_scheduledevent.go @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "strconv" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/grammar/parser" +) + +// ExitCreateScheduledEventStatement builds a CreateScheduledEventStmt from +// CREATE [OR REPLACE|MODIFY] SCHEDULED EVENT Module.Name ( ... ). +// +// Every property is carried through as written; the executor decides which ones +// the chosen Repeat actually has and rejects the rest. Numeric properties are +// stored as pointers so an explicit 0 (a real hour, minute or month offset) is +// distinguishable from an omitted one. +func (b *Builder) ExitCreateScheduledEventStatement(ctx *parser.CreateScheduledEventStatementContext) { + stmt := &ast.CreateScheduledEventStmt{ + Name: buildQualifiedName(ctx.QualifiedName()), + Documentation: findDocCommentText(ctx), + } + if createStmt := findParentCreateStatement(ctx); createStmt != nil { + if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { + stmt.CreateOrModify = true + } + } + + if body := ctx.ScheduledEventBody(); body != nil { + bodyCtx := body.(*parser.ScheduledEventBodyContext) + for _, prop := range bodyCtx.AllScheduledEventProperty() { + pc, ok := prop.(*parser.ScheduledEventPropertyContext) + if !ok || pc == nil { + continue + } + iok := pc.IdentifierOrKeyword(0) + if iok == nil { + continue + } + key := strings.ToLower(identifierOrKeywordText(iok)) + val := scheduledEventPropertyText(pc) + switch key { + case "microflow": + stmt.Microflow = val + case "repeat": + stmt.Repeat = val + case "multiplier": + stmt.Multiplier = parseIntPtr(val) + case "minuteoffset": + stmt.MinuteOffset = parseIntPtr(val) + case "monthoffset": + stmt.MonthOffset = parseIntPtr(val) + case "hourofday": + stmt.HourOfDay = parseIntPtr(val) + case "minuteofhour": + stmt.MinuteOfHour = parseIntPtr(val) + case "dayofmonth": + stmt.DayOfMonth = parseIntPtr(val) + case "month": + stmt.Month = parseIntPtr(val) + case "weekdays": + stmt.Weekdays = val + case "dayselector": + stmt.DaySelector = val + case "weekday": + stmt.Weekday = val + case "startdatetime": + stmt.StartDateTime = val + case "timezone": + stmt.TimeZone = val + case "onoverlap": + stmt.OnOverlap = val + case "enabled": + stmt.Enabled = parseBoolPtr(val) + case "excluded": + stmt.Excluded = parseBoolPtr(val) + case "exportlevel": + stmt.ExportLevel = val + case "documentation": + stmt.Documentation = val + } + } + } + + b.statements = append(b.statements, stmt) +} + +// scheduledEventPropertyText returns the value side of a property, unquoted. +// +// The key is identifierOrKeyword(0), so an identifier value is index 1 — +// reading index 0 would echo the key back as the value. +func scheduledEventPropertyText(pc *parser.ScheduledEventPropertyContext) string { + if qn := pc.QualifiedName(); qn != nil { + return getQualifiedNameText(qn) + } + if n := pc.NUMBER_LITERAL(); n != nil { + return n.GetText() + } + if s := pc.STRING_LITERAL(); s != nil { + return unquoteString(s.GetText()) + } + if bl := pc.BooleanLiteral(); bl != nil { + return bl.GetText() + } + if v := pc.IdentifierOrKeyword(1); v != nil { + return identifierOrKeywordText(v) + } + return "" +} + +// parseIntPtr returns nil for anything that is not an integer, so a malformed +// value is reported by the executor's validation rather than silently becoming 0. +func parseIntPtr(s string) *int { + n, err := strconv.Atoi(strings.TrimSpace(s)) + if err != nil { + return nil + } + return &n +} + +func parseBoolPtr(s string) *bool { + switch strings.ToLower(strings.TrimSpace(s)) { + case "true": + v := true + return &v + case "false": + v := false + return &v + } + return nil +} diff --git a/mdl/visitor/visitor_scheduledevent_test.go b/mdl/visitor/visitor_scheduledevent_test.go new file mode 100644 index 000000000..0a091ed75 --- /dev/null +++ b/mdl/visitor/visitor_scheduledevent_test.go @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +func TestCreateScheduledEvent(t *testing.T) { + input := `CREATE SCHEDULED EVENT Ops.NightlyCleanup ( + Microflow: Ops.SE_Cleanup, + Repeat: Daily, + HourOfDay: 4, + MinuteOfHour: 0, + TimeZone: Server, + Enabled: true, + StartDateTime: '2026-01-01T04:00:00Z' + );` + prog, errs := Build(input) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateScheduledEventStmt) + if !ok { + t.Fatalf("expected CreateScheduledEventStmt, got %T", prog.Statements[0]) + } + if stmt.Name.String() != "Ops.NightlyCleanup" { + t.Errorf("Name = %s", stmt.Name.String()) + } + if stmt.Microflow != "Ops.SE_Cleanup" { + t.Errorf("Microflow = %q — a qualified name must survive as one", stmt.Microflow) + } + if stmt.Repeat != "Daily" { + t.Errorf("Repeat = %q", stmt.Repeat) + } + if stmt.HourOfDay == nil || *stmt.HourOfDay != 4 { + t.Errorf("HourOfDay = %v", stmt.HourOfDay) + } + // 0 is a real minute, so an explicit 0 must not be indistinguishable from + // an omitted property — that is why these fields are pointers. + if stmt.MinuteOfHour == nil || *stmt.MinuteOfHour != 0 { + t.Errorf("MinuteOfHour = %v, want an explicit 0", stmt.MinuteOfHour) + } + if stmt.TimeZone != "Server" { + t.Errorf("TimeZone = %q", stmt.TimeZone) + } + if stmt.Enabled == nil || !*stmt.Enabled { + t.Errorf("Enabled = %v", stmt.Enabled) + } + if stmt.StartDateTime != "2026-01-01T04:00:00Z" { + t.Errorf("StartDateTime = %q", stmt.StartDateTime) + } +} + +func TestCreateScheduledEvent_OmittedFieldsStayNil(t *testing.T) { + prog, errs := Build(`CREATE SCHEDULED EVENT Ops.E ( Microflow: Ops.MF, Repeat: Minutely, Multiplier: 5 );`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateScheduledEventStmt) + if stmt.Multiplier == nil || *stmt.Multiplier != 5 { + t.Errorf("Multiplier = %v", stmt.Multiplier) + } + if stmt.HourOfDay != nil { + t.Errorf("HourOfDay = %v, want nil for an omitted property", stmt.HourOfDay) + } + if stmt.Enabled != nil { + t.Errorf("Enabled = %v, want nil for an omitted property", stmt.Enabled) + } +} + +func TestCreateScheduledEvent_WeeklyAndSelectors(t *testing.T) { + prog, errs := Build(`CREATE OR MODIFY SCHEDULED EVENT Ops.E ( + Microflow: Ops.MF, + Repeat: MonthlyByWeekday, + Multiplier: 3, + DaySelector: Last, + Weekday: Friday, + Weekdays: 'Monday, Friday' + );`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateScheduledEventStmt) + if !stmt.CreateOrModify { + t.Error("CreateOrModify not set for CREATE OR MODIFY") + } + if stmt.DaySelector != "Last" || stmt.Weekday != "Friday" { + t.Errorf("DaySelector/Weekday = %q/%q", stmt.DaySelector, stmt.Weekday) + } + if stmt.Weekdays != "Monday, Friday" { + t.Errorf("Weekdays = %q", stmt.Weekdays) + } +} + +func TestDropScheduledEvent(t *testing.T) { + prog, errs := Build(`DROP SCHEDULED EVENT Ops.NightlyCleanup;`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.DropScheduledEventStmt) + if !ok { + t.Fatalf("expected DropScheduledEventStmt, got %T", prog.Statements[0]) + } + if stmt.Name.String() != "Ops.NightlyCleanup" { + t.Errorf("Name = %s", stmt.Name.String()) + } +} + +func TestShowAndDescribeScheduledEvents(t *testing.T) { + prog, errs := Build(`SHOW SCHEDULED EVENTS; LIST SCHEDULED EVENTS IN Ops; DESCRIBE SCHEDULED EVENT Ops.NightlyCleanup;`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + if len(prog.Statements) != 3 { + t.Fatalf("expected 3 statements, got %d", len(prog.Statements)) + } + if _, ok := prog.Statements[0].(*ast.ShowScheduledEventsStmt); !ok { + t.Errorf("statement 0 = %T", prog.Statements[0]) + } + s1, ok := prog.Statements[1].(*ast.ShowScheduledEventsStmt) + if !ok { + t.Fatalf("statement 1 = %T", prog.Statements[1]) + } + if s1.Module != "Ops" { + t.Errorf("Module = %q", s1.Module) + } + if _, ok := prog.Statements[2].(*ast.DescribeScheduledEventStmt); !ok { + t.Errorf("statement 2 = %T", prog.Statements[2]) + } +} + +// TestScheduledKeywordStillUsableAsIdentifier guards the cost of adding +// SCHEDULED to the lexer: a new keyword stops being usable as an ordinary name +// unless it is also listed in the `keyword` rule. +func TestScheduledKeywordStillUsableAsIdentifier(t *testing.T) { + prog, errs := Build(`CREATE ENTITY Ops.Job ( scheduled: Boolean, event: String(50) );`) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateEntityStmt) + if len(stmt.Attributes) != 2 { + t.Fatalf("expected 2 attributes, got %d", len(stmt.Attributes)) + } + if stmt.Attributes[0].Name != "scheduled" || stmt.Attributes[1].Name != "event" { + t.Errorf("attribute names = %q, %q", stmt.Attributes[0].Name, stmt.Attributes[1].Name) + } +} diff --git a/model/types.go b/model/types.go index bdfde6ad1..908ec38a6 100644 --- a/model/types.go +++ b/model/types.go @@ -244,6 +244,63 @@ func (r *RegularExpression) GetContainerID() ID { return r.ContainerID } +// ScheduleKind identifies which ScheduledEvents$*Schedule variant a schedule is. +// The variants differ in which fields they carry, so the kind has to be decided +// before any field is read or written — see Schedule. +type ScheduleKind string + +const ( + ScheduleMinute ScheduleKind = "Minute" + ScheduleHour ScheduleKind = "Hour" + ScheduleDay ScheduleKind = "Day" + ScheduleWeek ScheduleKind = "Week" + ScheduleMonthDate ScheduleKind = "MonthDate" + ScheduleMonthWeekday ScheduleKind = "MonthWeekday" + ScheduleYearDate ScheduleKind = "YearDate" + ScheduleYearWeekday ScheduleKind = "YearWeekday" +) + +// Schedule is the repeat rule of a scheduled event — the polymorphic +// ScheduledEvents$Schedule child, flattened into one struct with Kind saying +// which variant it is. +// +// It is flat rather than eight types because the fields overlap heavily and the +// consumers (a formatter, a serializer, a describe) all switch on the kind +// anyway. Only the fields listed for a kind are written; the rest are ignored: +// +// Minute Multiplier +// Hour Multiplier, MinuteOffset +// Day HourOfDay, MinuteOfHour +// Week Weekdays, HourOfDay, MinuteOfHour +// MonthDate Multiplier, MonthOffset, DayOfMonth, HourOfDay, MinuteOfHour +// MonthWeekday Multiplier, MonthOffset, DaySelector, Weekday, HourOfDay, MinuteOfHour +// YearDate Month, DayOfMonth, HourOfDay, MinuteOfHour +// YearWeekday Month, DaySelector, Weekday, HourOfDay, MinuteOfHour +type Schedule struct { + Kind ScheduleKind `json:"kind"` + + // Multiplier is the repeat count ("every N minutes/hours/months"). + // Day, Week and the two Year variants have no multiplier in the metamodel. + Multiplier int `json:"multiplier,omitempty"` + // MinuteOffset is the minute within the hour, for the Hour variant only. + MinuteOffset int `json:"minuteOffset,omitempty"` + // MonthOffset selects which month of a multi-month cycle fires. + MonthOffset int `json:"monthOffset,omitempty"` + + HourOfDay int `json:"hourOfDay,omitempty"` + MinuteOfHour int `json:"minuteOfHour,omitempty"` + + // DayOfMonth is 1-31; Month is 1-12. + DayOfMonth int `json:"dayOfMonth,omitempty"` + Month int `json:"month,omitempty"` + + // Weekdays holds the seven per-day flags of the Week variant, Sunday first. + Weekdays [7]bool `json:"weekdays,omitempty"` + // DaySelector is First|Second|Third|Fourth|Last, Weekday is Sunday..Saturday. + DaySelector string `json:"daySelector,omitempty"` + Weekday string `json:"weekday,omitempty"` +} + // ScheduledEvent represents a scheduled event. type ScheduledEvent struct { BaseElement @@ -256,6 +313,15 @@ type ScheduledEvent struct { Interval int `json:"interval,omitempty"` IntervalType string `json:"intervalType,omitempty"` Enabled bool `json:"enabled"` + // Schedule is the repeat rule. Every Studio Pro-authored event carries one; + // nil means the document did not have the child. + Schedule *Schedule `json:"schedule,omitempty"` + // OnOverlap is SkipNext or DelayNext — what happens when a run is still + // going when the next one is due. This is a scheduled event's own + // concurrency control; it does not go through a task queue. + OnOverlap string `json:"onOverlap,omitempty"` + ExportLevel string `json:"exportLevel,omitempty"` + Excluded bool `json:"excluded,omitempty"` } // GetName returns the scheduled event's name. diff --git a/modelsdk/mpr/reader_units.go b/modelsdk/mpr/reader_units.go index 42d578b1e..294a0b78a 100644 --- a/modelsdk/mpr/reader_units.go +++ b/modelsdk/mpr/reader_units.go @@ -7,7 +7,6 @@ import ( "fmt" "os" "path/filepath" - "strings" "go.mongodb.org/mongo-driver/v2/bson" @@ -76,16 +75,21 @@ func (r *Reader) ListUnitsByType(typePrefix string) ([]UnitRef, error) { return result, nil } -// listUnitsByType returns all units matching the given type prefix. -func (r *Reader) listUnitsByType(typePrefix string) ([]rawUnit, error) { +// listUnitsByType returns all units of exactly the given storage type. An empty +// typeName returns every unit. +// +// Exact, not prefix: Mendix storage names nest (`Forms$Page` is a prefix of +// `Forms$PageTemplate`), so a prefix match silently folds one document type into +// another. See the note on the same function in sdk/mpr. +func (r *Reader) listUnitsByType(typeName string) ([]rawUnit, error) { if r.version == MPRVersionV2 { - return r.listUnitsByTypeV2(typePrefix) + return r.listUnitsByTypeV2(typeName) } - return r.listUnitsByTypeV1(typePrefix) + return r.listUnitsByTypeV1(typeName) } // listUnitsByTypeV1 handles MPR v1 format (contents in database). -func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { +func (r *Reader) listUnitsByTypeV1(typeName string) ([]rawUnit, error) { rows, err := r.db.Query(` SELECT UnitID, ContainerID, ContainmentName, Contents FROM Unit @@ -105,13 +109,13 @@ func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { return nil, fmt.Errorf("failed to scan unit row: %w", err) } - typeName := getTypeFromContents(contents) - if typePrefix == "" || strings.HasPrefix(typeName, typePrefix) { + unitType := getTypeFromContents(contents) + if typeName == "" || unitType == typeName { units = append(units, rawUnit{ ID: blobToUUID(unitID), ContainerID: blobToUUID(containerID), ContainmentName: containmentName, - Type: typeName, + Type: unitType, Contents: contents, }) } @@ -122,7 +126,7 @@ func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { // listUnitsByTypeV2 handles MPR v2 format (contents in mprcontents folder). // Uses caching to avoid reading every file for each query. -func (r *Reader) listUnitsByTypeV2(typePrefix string) ([]rawUnit, error) { +func (r *Reader) listUnitsByTypeV2(typeName string) ([]rawUnit, error) { if !r.unitCacheValid { if err := r.buildUnitCache(); err != nil { return nil, err @@ -132,7 +136,7 @@ func (r *Reader) listUnitsByTypeV2(typePrefix string) ([]rawUnit, error) { // Filter by type using cache, only read contents for matching units. var units []rawUnit for _, cu := range r.unitCache { - if typePrefix == "" || strings.HasPrefix(cu.Type, typePrefix) { + if typeName == "" || cu.Type == typeName { contents, err := r.readMprContents(cu.ID) if err != nil { continue diff --git a/modelsdk/widgets/augment_metadata_test.go b/modelsdk/widgets/augment_metadata_test.go index e18c12ada..5adc4b500 100644 --- a/modelsdk/widgets/augment_metadata_test.go +++ b/modelsdk/widgets/augment_metadata_test.go @@ -25,7 +25,7 @@ func TestReconcilePropertyMetadata(t *testing.T) { map[string]any{ "$Type": "CustomWidgets$WidgetPropertyType", "PropertyKey": "pagingPosition", - "Category": "General::Pagination", // stale + "Category": "General::Pagination", // stale "Caption": "Position of pagination", // stale "ValueType": map[string]any{ "$Type": "CustomWidgets$WidgetValueType", diff --git a/modelsdk/widgets/dirty_template_test.go b/modelsdk/widgets/dirty_template_test.go index 507c2842c..16fb696dd 100644 --- a/modelsdk/widgets/dirty_template_test.go +++ b/modelsdk/widgets/dirty_template_test.go @@ -168,7 +168,7 @@ func TestDirtyBindings_CleanIsClean(t *testing.T) { "Properties": []any{float64(2), map[string]any{ "$Type": "CustomWidgets$WidgetProperty", "Value": map[string]any{ - "$Type": "CustomWidgets$WidgetValue", + "$Type": "CustomWidgets$WidgetValue", "AttributeRef": nil, "DataSource": map[string]any{ // empty source slot — legitimate on data widgets "$Type": "CustomWidgets$CustomWidgetXPathSource", diff --git a/sdk/mpr/parser_enumeration.go b/sdk/mpr/parser_enumeration.go index dfaac3bc9..964ae548e 100644 --- a/sdk/mpr/parser_enumeration.go +++ b/sdk/mpr/parser_enumeration.go @@ -5,6 +5,7 @@ package mpr import ( "fmt" + "github.com/mendixlabs/mxcli/mdl/scheduledevents" "github.com/mendixlabs/mxcli/model" "go.mongodb.org/mongo-driver/bson" @@ -175,38 +176,17 @@ func (r *Reader) parseScheduledEvent(unitID, containerID string, contents []byte return nil, err } - var raw map[string]any + var raw bson.M if err := bson.Unmarshal(contents, &raw); err != nil { return nil, fmt.Errorf("failed to unmarshal BSON: %w", err) } - event := &model.ScheduledEvent{} - event.ID = model.ID(unitID) - event.TypeName = "ScheduledEvents$ScheduledEvent" - event.ContainerID = model.ID(containerID) - - if name, ok := raw["Name"].(string); ok { - event.Name = name - } - if doc, ok := raw["Documentation"].(string); ok { - event.Documentation = doc - } - if mfID, ok := raw["Microflow"].(string); ok { - event.MicroflowID = model.ID(mfID) - } - if enabled, ok := raw["Enabled"].(bool); ok { - event.Enabled = enabled - } - // Issue #585: Studio Pro stores Interval as BSON int64; extractInt - // also accepts int32/int/float64 emitted by other writers. - if _, ok := raw["Interval"]; ok { - event.Interval = extractInt(raw["Interval"]) - } - if intervalType, ok := raw["IntervalType"].(string); ok { - event.IntervalType = intervalType - } - - return event, nil + // Shared with the modelsdk engine so both read the same keys the shared + // writer produces — including the polymorphic Schedule child, which this + // parser used to drop, and StartDateTime, which Studio Pro stores as a BSON + // datetime. (Interval is int64 in Studio Pro documents; the codec accepts + // every numeric width — issue #585.) + return scheduledevents.Parse(raw, model.ID(unitID), model.ID(containerID)), nil } // resolveContents handles MPR v2 external file references. diff --git a/sdk/mpr/queues.go b/sdk/mpr/queues.go new file mode 100644 index 000000000..907690da8 --- /dev/null +++ b/sdk/mpr/queues.go @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "fmt" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" +) + +// Task queues (Queues$Queue). The document is small and flat apart from a single +// nested Config node, so it is read and written as raw BSON here rather than +// through a dedicated element type. +// +// The shape follows four Studio Pro-authored queues from the Mendix Business +// Events module. Note that Queues$BasicQueueConfig declares an int32 +// `Parallelism` in addition to `ParallelismExpression`, and Studio Pro wrote it +// in none of them — so only the expression is read and written. + +const queueUnitType = "Queues$Queue" + +// ListQueues reads every task queue in the project. +func (r *Reader) ListQueues() ([]*types.Queue, error) { + units, err := r.ListRawUnitsByType(queueUnitType) + if err != nil { + return nil, err + } + out := make([]*types.Queue, 0, len(units)) + for _, u := range units { + var doc bson.M + if err := bson.Unmarshal(u.Contents, &doc); err != nil { + return nil, fmt.Errorf("unmarshal queue %s: %w", u.ID, err) + } + q := &types.Queue{ContainerID: model.ID(u.ContainerID)} + q.ID = model.ID(u.ID) + q.TypeName = queueUnitType + q.Name, _ = doc["Name"].(string) + q.Documentation, _ = doc["Documentation"].(string) + q.Excluded, _ = doc["Excluded"].(bool) + q.ExportLevel, _ = doc["ExportLevel"].(string) + if cfg, ok := doc["Config"].(bson.M); ok { + q.Parallelism, _ = cfg["ParallelismExpression"].(string) + q.ClusterWide, _ = cfg["ClusterWide"].(bool) + } + out = append(out, q) + } + return out, nil +} + +// CreateQueue inserts a new task queue document. +func (w *Writer) CreateQueue(q *types.Queue) error { + if q == nil { + return fmt.Errorf("CreateQueue: nil queue") + } + if q.ID == "" { + q.ID = model.ID(generateUUID()) + } + contents, err := serializeQueueUnit(q) + if err != nil { + return err + } + return w.insertUnit(string(q.ID), string(q.ContainerID), "Documents", queueUnitType, contents) +} + +// UpdateQueue rewrites an existing task queue in place. +func (w *Writer) UpdateQueue(q *types.Queue) error { + if q == nil { + return fmt.Errorf("UpdateQueue: nil queue") + } + contents, err := serializeQueueUnit(q) + if err != nil { + return err + } + return w.UpdateRawUnit(string(q.ID), contents) +} + +// DeleteQueue removes a task queue by ID. +func (w *Writer) DeleteQueue(id string) error { + return w.deleteUnit(id) +} + +func serializeQueueUnit(q *types.Queue) ([]byte, error) { + parallelism := q.Parallelism + if parallelism == "" { + parallelism = "1" + } + exportLevel := q.ExportLevel + if exportLevel == "" { + exportLevel = "Hidden" + } + doc := bson.D{ + {Key: "$ID", Value: idToBsonBinary(string(q.ID))}, + {Key: "$Type", Value: queueUnitType}, + {Key: "Config", Value: bson.D{ + {Key: "$ID", Value: idToBsonBinary(generateUUID())}, + {Key: "$Type", Value: "Queues$BasicQueueConfig"}, + {Key: "ClusterWide", Value: q.ClusterWide}, + {Key: "ParallelismExpression", Value: parallelism}, + }}, + {Key: "Documentation", Value: q.Documentation}, + {Key: "Excluded", Value: q.Excluded}, + {Key: "ExportLevel", Value: exportLevel}, + {Key: "Name", Value: q.Name}, + } + return marshalUnitIDFirst(doc) +} diff --git a/sdk/mpr/reader_documents.go b/sdk/mpr/reader_documents.go index a8b735bb4..45f1e7f3c 100644 --- a/sdk/mpr/reader_documents.go +++ b/sdk/mpr/reader_documents.go @@ -1010,3 +1010,62 @@ func parseJarDependencyExclusion(raw map[string]any) *types.JarDependencyExclusi ArtifactID: extractString(raw["ArtifactId"]), } } + +// ListMenuDocuments returns all standalone Menus$MenuDocument documents. +// +// A menu document holds its entries in a Menus$MenuItemCollection rather than +// directly, but the entries themselves are ordinary Menus$MenuItem elements, so +// the recursive conversion reuses parseNavMenuItem. +func (r *Reader) ListMenuDocuments() ([]*types.MenuDocument, error) { + units, err := r.listUnitsByType("Menus$MenuDocument") + if err != nil { + return nil, err + } + + result := make([]*types.MenuDocument, 0, len(units)) + for _, u := range units { + var raw map[string]any + if err := bson.Unmarshal(u.Contents, &raw); err != nil { + return nil, fmt.Errorf("failed to parse menu document %s: %w", u.ID, err) + } + md := &types.MenuDocument{ + ID: model.ID(u.ID), + ContainerID: model.ID(u.ContainerID), + Name: extractString(raw["Name"]), + Documentation: extractString(raw["Documentation"]), + ExportLevel: extractString(raw["ExportLevel"]), + } + if b, ok := raw["Excluded"].(bool); ok { + md.Excluded = b + } + if coll, ok := raw["ItemCollection"].(map[string]any); ok { + for _, item := range extractBsonArray(coll["Items"]) { + if m, ok := item.(map[string]any); ok { + if mi := parseNavMenuItem(m); mi != nil { + md.Items = append(md.Items, mi) + } + } + } + } + result = append(result, md) + } + return result, nil +} + +// GetMenuDocumentByQualifiedName finds a menu document by module + name. +func (r *Reader) GetMenuDocumentByQualifiedName(moduleName, name string) (*types.MenuDocument, error) { + all, err := r.ListMenuDocuments() + if err != nil { + return nil, err + } + moduleMap, err := r.buildContainerModuleNameMap() + if err != nil { + return nil, err + } + for _, md := range all { + if md.Name == name && moduleMap[md.ContainerID] == moduleName { + return md, nil + } + } + return nil, fmt.Errorf("menu not found: %s.%s", moduleName, name) +} diff --git a/sdk/mpr/reader_units.go b/sdk/mpr/reader_units.go index 8422fc6f1..6f7c455f7 100644 --- a/sdk/mpr/reader_units.go +++ b/sdk/mpr/reader_units.go @@ -55,16 +55,29 @@ type rawUnit struct { Contents []byte } -// listUnitsByType returns all units matching the given type prefix. -func (r *Reader) listUnitsByType(typePrefix string) ([]rawUnit, error) { +// listUnitsByType returns all units of exactly the given storage type. An empty +// typeName returns every unit. +// +// The match is exact, and that is load-bearing rather than incidental: this used +// to be a prefix match, and `Forms$Page` is a prefix of `Forms$PageTemplate`, so +// ListPages swept in all 46 of Atlas_Web_Content's page templates. They then +// described as pages with an empty body — the template's content hangs off +// LayoutCall, which the page path does not read — so `show modules` reported 46 +// pages for a module with none, and anything comparing describe output judged a +// template unchanged without having looked at it. +// +// Mendix storage names nest this way in general (`Forms$Page` / +// `Forms$PageTemplate`), so a prefix match here is a trap for every future type, +// not a one-off. +func (r *Reader) listUnitsByType(typeName string) ([]rawUnit, error) { if r.version == MPRVersionV2 { - return r.listUnitsByTypeV2(typePrefix) + return r.listUnitsByTypeV2(typeName) } - return r.listUnitsByTypeV1(typePrefix) + return r.listUnitsByTypeV1(typeName) } // listUnitsByTypeV1 handles MPR v1 format (contents in database). -func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { +func (r *Reader) listUnitsByTypeV1(typeName string) ([]rawUnit, error) { rows, err := r.db.Query(` SELECT UnitID, ContainerID, ContainmentName, Contents FROM Unit @@ -84,13 +97,13 @@ func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { return nil, fmt.Errorf("failed to scan unit row: %w", err) } - typeName := getTypeFromContents(contents) - if typePrefix == "" || strings.HasPrefix(typeName, typePrefix) { + unitType := getTypeFromContents(contents) + if typeName == "" || unitType == typeName { units = append(units, rawUnit{ ID: blobToUUID(unitID), ContainerID: blobToUUID(containerID), ContainmentName: containmentName, - Type: typeName, + Type: unitType, Contents: contents, }) } @@ -101,7 +114,7 @@ func (r *Reader) listUnitsByTypeV1(typePrefix string) ([]rawUnit, error) { // listUnitsByTypeV2 handles MPR v2 format (contents in mprcontents folder). // Uses caching to avoid reading every file for each query. -func (r *Reader) listUnitsByTypeV2(typePrefix string) ([]rawUnit, error) { +func (r *Reader) listUnitsByTypeV2(typeName string) ([]rawUnit, error) { // Build cache if not valid if !r.unitCacheValid { if err := r.buildUnitCache(); err != nil { @@ -112,7 +125,7 @@ func (r *Reader) listUnitsByTypeV2(typePrefix string) ([]rawUnit, error) { // Filter by type using cache, only read contents for matching units var units []rawUnit for _, cu := range r.unitCache { - if typePrefix == "" || strings.HasPrefix(cu.Type, typePrefix) { + if typeName == "" || cu.Type == typeName { // Read contents from mprcontents folder // Note: cu.ID is already in the correct swapped format from blobToUUID contents, err := r.readMprContents(cu.ID) diff --git a/sdk/mpr/reader_units_type_test.go b/sdk/mpr/reader_units_type_test.go new file mode 100644 index 000000000..681d646b4 --- /dev/null +++ b/sdk/mpr/reader_units_type_test.go @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import "testing" + +// TestListUnitsByType_MatchesExactly is the regression guard for the page / +// page-template conflation. +// +// listUnitsByType used to match on a type *prefix*, and Mendix storage names +// nest: `Forms$Page` is a prefix of `Forms$PageTemplate`. So ListPages returned +// both, `show modules` reported the fixture's Atlas_Web_Content as having 46 +// pages when it has none, and every one of those templates described as a page +// with an empty body — the template's content hangs off LayoutCall, which the +// page path never reads. Anything comparing describe output therefore judged a +// template unchanged without having looked inside it. +// +// The assertion is deliberately about the *pair*: a test that only counted +// Forms$Page would pass against the prefix match too, because the miscount was +// caused by the other type being swept in. +func TestListUnitsByType_MatchesExactly(t *testing.T) { + r, err := Open(copyProject(t, "../../testdata/expr-checker", "minimal.mpr")) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + + pages, err := r.listUnitsByType("Forms$Page") + if err != nil { + t.Fatalf("listUnitsByType(Forms$Page): %v", err) + } + templates, err := r.listUnitsByType("Forms$PageTemplate") + if err != nil { + t.Fatalf("listUnitsByType(Forms$PageTemplate): %v", err) + } + + if len(pages) == 0 || len(templates) == 0 { + t.Fatalf("fixture should hold both types; got %d pages, %d templates", + len(pages), len(templates)) + } + + // Neither query may return a unit of the other type. + for _, u := range pages { + if u.Type != "Forms$Page" { + t.Fatalf("querying Forms$Page returned a %s — the match is by prefix, not exact", u.Type) + } + } + for _, u := range templates { + if u.Type != "Forms$PageTemplate" { + t.Fatalf("querying Forms$PageTemplate returned a %s", u.Type) + } + } + + // And the page query must not be the union of the two. + all, err := r.listUnitsByType("") + if err != nil { + t.Fatalf("listUnitsByType(\"\"): %v", err) + } + if len(all) <= len(pages)+len(templates) { + t.Fatalf("the empty type should return every unit; got %d, with %d pages + %d templates", + len(all), len(pages), len(templates)) + } +} + +// TestListPages_ExcludesPageTemplates checks the symptom the user actually sees, +// one layer up from the cause. +func TestListPages_ExcludesPageTemplates(t *testing.T) { + r, err := Open(copyProject(t, "../../testdata/expr-checker", "minimal.mpr")) + if err != nil { + t.Fatalf("Open: %v", err) + } + defer r.Close() + + pageUnits, err := r.listUnitsByType("Forms$Page") + if err != nil { + t.Fatalf("listUnitsByType: %v", err) + } + pages, err := r.ListPages() + if err != nil { + t.Fatalf("ListPages: %v", err) + } + if len(pages) != len(pageUnits) { + t.Errorf("ListPages returned %d pages for %d Forms$Page units — page templates are being counted as pages", + len(pages), len(pageUnits)) + } + + templates, err := r.ListPageTemplates() + if err != nil { + t.Fatalf("ListPageTemplates: %v", err) + } + if len(templates) == 0 { + t.Fatal("page templates must still be readable under their own type") + } + byName := make(map[string]bool, len(pages)) + for _, p := range pages { + byName[p.Name] = true + } + for _, tpl := range templates { + if byName[tpl.Name] { + t.Errorf("%q is reported as both a page and a page template", tpl.Name) + } + } +} diff --git a/sdk/mpr/scheduledevents.go b/sdk/mpr/scheduledevents.go new file mode 100644 index 000000000..21aaa1fb0 --- /dev/null +++ b/sdk/mpr/scheduledevents.go @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "fmt" + + sched "github.com/mendixlabs/mxcli/mdl/scheduledevents" + "github.com/mendixlabs/mxcli/model" +) + +// Scheduled events (ScheduledEvents$ScheduledEvent) for the legacy engine. +// +// The document shape lives in mdl/scheduledevents so both engines write exactly +// the same bytes — the schedule child has eight variants that differ in which +// fields they carry, and two copies of that dispatch would eventually disagree. +// +// The legacy READER for scheduled events is parseScheduledEvent (in +// parser_enumeration.go), which predates this and covers only the flat legacy +// fields; the write path here round-trips through the shared codec. + +// CreateScheduledEvent inserts a new scheduled event document. +func (w *Writer) CreateScheduledEvent(ev *model.ScheduledEvent) error { + if ev == nil { + return fmt.Errorf("CreateScheduledEvent: nil event") + } + if ev.ID == "" { + ev.ID = model.ID(generateUUID()) + } + contents, err := sched.Serialize(ev) + if err != nil { + return err + } + return w.insertUnit(string(ev.ID), string(ev.ContainerID), "Documents", sched.TypeName, contents) +} + +// UpdateScheduledEvent rewrites an existing scheduled event in place. +func (w *Writer) UpdateScheduledEvent(ev *model.ScheduledEvent) error { + if ev == nil { + return fmt.Errorf("UpdateScheduledEvent: nil event") + } + contents, err := sched.Serialize(ev) + if err != nil { + return err + } + return w.UpdateRawUnit(string(ev.ID), contents) +} + +// DeleteScheduledEvent removes a scheduled event by ID. +func (w *Writer) DeleteScheduledEvent(id string) error { + return w.deleteUnit(id) +} diff --git a/sdk/mpr/writer_units.go b/sdk/mpr/writer_units.go index d9e26cd85..69c7f3336 100644 --- a/sdk/mpr/writer_units.go +++ b/sdk/mpr/writer_units.go @@ -198,6 +198,21 @@ func (w *Writer) updateUnit(unitID string, contents []byte) error { // UpdateRawUnit saves raw BSON bytes for a unit, bypassing deserialization. // Used by ALTER PAGE to modify the BSON widget tree directly. +// AddRawUnit inserts a unit verbatim: same contents, same containment name, no +// re-encoding. It is the primitive a module transplant is built from. +// +// Copying a unit wholesale is safe because element `$ID` pointers do not cross +// unit boundaries — measured at 0 of 9,910 in a real project (PROPOSAL +// marketplace_module_upgrade §4) — and cross-unit references are qualified-name +// strings. Rewriting the contents, by contrast, would risk exactly the +// intra-unit pointer inconsistency ADR-0008 forbids. +// +// The caller owns uniqueness of unitID. Inserting an ID the project already +// holds is a caller error, not something this can repair. +func (w *Writer) AddRawUnit(unitID, containerID, containmentName, unitType string, contents []byte) error { + return w.insertUnit(unitID, containerID, containmentName, unitType, contents) +} + func (w *Writer) UpdateRawUnit(unitID string, contents []byte) error { return w.updateUnit(unitID, contents) } diff --git a/sdk/versions/mendix-11.yaml b/sdk/versions/mendix-11.yaml index 323a607e2..7771e22e2 100644 --- a/sdk/versions/mendix-11.yaml +++ b/sdk/versions/mendix-11.yaml @@ -139,6 +139,24 @@ features: odata_client: min_version: "10.0.0" + agent_documents: + # Agent Editor documents (Model, Knowledge Base, Consumed MCP Service, + # Agent). Studio Pro's agent editor is an extension and the documents are + # stored as custom blobs, so mxbuild validates nothing about them -- these + # entries are the only version gate mxcli has. + agent_model: + min_version: "11.9.0" + mdl: "CREATE MODEL Module.Name (Provider: ..., Key: Module.ApiKey)" + agent_knowledge_base: + min_version: "11.9.0" + mdl: "CREATE KNOWLEDGE BASE Module.Name (Provider: ..., Key: Module.KBKey)" + agent_consumed_mcp_service: + min_version: "11.9.0" + mdl: "CREATE CONSUMED MCP SERVICE Module.Name (ProtocolVersion: ...)" + agent: + min_version: "11.9.0" + mdl: "CREATE AGENT Module.Name (UsageType: Task|Chat, Model: ..., SystemPrompt: '...')" + workflows: basic: min_version: "9.0.0" diff --git a/sdk/versions/registry_test.go b/sdk/versions/registry_test.go index 491dfd8e0..69fda6a49 100644 --- a/sdk/versions/registry_test.go +++ b/sdk/versions/registry_test.go @@ -240,3 +240,36 @@ func TestDisplayName(t *testing.T) { t.Errorf("DisplayName() = %q, want %q", got, "view entities") } } + +// TestAgentDocumentsAreGated is the guard for the version gap reported in +// mxcli-formula1 FINDINGS §53: `show features` listed nothing for agents or MCP, +// so there was no checkFeature() gate and an older project got no actionable +// error. +// +// This matters more than a normal version gate because nothing downstream +// catches it. Agent Editor documents are stored as custom blobs, mxbuild does +// not validate them (it contains no agent-editor strings at all), so an agent +// authored against a pre-11.9 project builds green and simply cannot be opened. +func TestAgentDocumentsAreGated(t *testing.T) { + reg, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + docs := []string{"agent_model", "agent_knowledge_base", "agent_consumed_mcp_service", "agent"} + for _, name := range docs { + t.Run(name, func(t *testing.T) { + // Available on a project new enough for the agent editor... + if !reg.IsAvailable("agent_documents", name, SemVer{Major: 11, Minor: 9, Patch: 0}) { + t.Errorf("%s should be available on 11.9.0", name) + } + // ...and refused below it, which is the half that was missing. + if reg.IsAvailable("agent_documents", name, SemVer{Major: 11, Minor: 8, Patch: 0}) { + t.Errorf("%s must not be reported available on 11.8.0", name) + } + if reg.IsAvailable("agent_documents", name, SemVer{Major: 10, Minor: 24, Patch: 0}) { + t.Errorf("%s must not be reported available on 10.24.0", name) + } + }) + } +} diff --git a/vscode-mdl/package.json b/vscode-mdl/package.json index 73a3030e4..7499ed69e 100644 --- a/vscode-mdl/package.json +++ b/vscode-mdl/package.json @@ -152,7 +152,7 @@ }, { "command": "mendix.openElement", - "when": "view == mendixProjectTree && viewItem =~ /^(entity|microflow|nanoflow|page|snippet|enumeration|association|workflow|constant|layout|javaaction|javascriptaction|scheduledevent|buildingblock|pagetemplate|imagecollection|jsonstructure|importmapping|exportmapping|restclient|businesseventservice|databaseconnection|publishedrestservice|odataclient|odataservice|modulerole|userrole|projectsecurity|demouser|agent|aimodel|knowledgebase|consumedmcpservice|datatransformer)$/", + "when": "view == mendixProjectTree && viewItem =~ /^(entity|microflow|nanoflow|page|snippet|enumeration|association|workflow|constant|layout|javaaction|javascriptaction|scheduledevent|queue|buildingblock|pagetemplate|imagecollection|jsonstructure|importmapping|exportmapping|restclient|businesseventservice|databaseconnection|publishedrestservice|odataclient|odataservice|modulerole|userrole|projectsecurity|demouser|agent|aimodel|knowledgebase|consumedmcpservice|datatransformer)$/", "group": "mendix@6" }, { diff --git a/vscode-mdl/src/extension.ts b/vscode-mdl/src/extension.ts index 79f56824a..289638bcf 100644 --- a/vscode-mdl/src/extension.ts +++ b/vscode-mdl/src/extension.ts @@ -118,7 +118,7 @@ export function activate(context: vscode.ExtensionContext) { } // Try the given type first, then fallback to other common types - const fallbackTypes = ['entity', 'microflow', 'nanoflow', 'page', 'enumeration', 'snippet', 'constant', 'javaaction', 'javascriptaction', 'scheduledevent', 'buildingblock', 'pagetemplate', 'imagecollection', 'businesseventservice', 'databaseconnection', 'publishedrestservice', 'workflow', 'layout', 'importmapping', 'exportmapping', 'restclient', 'jsonstructure', 'agent', 'aimodel', 'knowledgebase', 'consumedmcpservice', 'datatransformer']; + const fallbackTypes = ['entity', 'microflow', 'nanoflow', 'page', 'enumeration', 'snippet', 'constant', 'javaaction', 'javascriptaction', 'scheduledevent', 'queue', 'buildingblock', 'pagetemplate', 'imagecollection', 'businesseventservice', 'databaseconnection', 'publishedrestservice', 'workflow', 'layout', 'importmapping', 'exportmapping', 'restclient', 'jsonstructure', 'agent', 'aimodel', 'knowledgebase', 'consumedmcpservice', 'datatransformer']; const typesToTry = [type, ...fallbackTypes.filter(t => t !== type)]; for (const tryType of typesToTry) { diff --git a/vscode-mdl/src/projectTreeProvider.ts b/vscode-mdl/src/projectTreeProvider.ts index 57c5d9da7..317b0e33e 100644 --- a/vscode-mdl/src/projectTreeProvider.ts +++ b/vscode-mdl/src/projectTreeProvider.ts @@ -253,6 +253,8 @@ export class MendixProjectTreeProvider implements vscode.TreeDataProvider = { 'JavaAction': 'javaaction', 'Constant': 'constant', 'ScheduledEvent': 'scheduledevent', + 'Queue': 'queue', 'ODataClient': 'odataclient', 'ODataService': 'odataservice', };