Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
47171ad
feat(pluggable-widgets-mcp): introduce pluggable-widgets-mcp
rahmanunver Dec 5, 2025
fb2996f
refactor(pluggable-widgets-mcp): fix prompt timeout, refactor schema,…
rahmanunver Dec 17, 2025
e2a6b1b
feat(pluggable-widgets-mcp): wip
rahmanunver Dec 30, 2025
1338456
feat(pluggable-widgets-mcp): addition of resources and build tools, d…
rahmanunver Jan 14, 2026
8e16723
feat(pluggable-widgets-mcp): refactor security guardrails into a sing…
rahmanunver Jan 15, 2026
b649610
feat(pluggable-widgets-mcp): introduce pluggable-widgets-mcp
rahmanunver Dec 5, 2025
051f519
feat(pluggable-widgets-mcp): add code generation tool, handle working…
rahmanunver Jan 19, 2026
53801f5
feat(pluggable-widgets-mcp): update readme and agents.md, add securit…
rahmanunver Jan 20, 2026
8141f42
feat(pluggable-widgets-mcp): add changelog
rahmanunver Jan 20, 2026
424be98
feat(pluggable-widgets-mcp): remove redundant prettier config
rahmanunver Jan 20, 2026
884de20
fix(pluggable-widgets-mcp): fix property types, resource URIs, and ad…
rahmanunver Feb 23, 2026
f069b3b
feat(pluggable-widgets-mcp): use custom generator-widget with non-int…
rahmanunver Feb 25, 2026
0833277
feat(pluggable-widgets-mcp): add project tools, session state, and de…
rahmanunver Feb 27, 2026
2711875
refactor(pluggable-widgets-mcp): extract shared sandbox utility and a…
rahmanunver Feb 27, 2026
5a2db95
test(pluggable-widgets-mcp): add vitest infrastructure and unit tests
rahmanunver Feb 27, 2026
ac78216
fix(pluggable-widgets-mcp): fix E2E pipeline — name passing, scaffold…
rahmanunver Feb 27, 2026
a6cb1c0
fix(xml-generator): force required=true for primitive property types
rahmanunver Mar 4, 2026
2c1c144
fix(tsx-generator): use _props param in editorPreview when no label prop
rahmanunver Mar 4, 2026
ed70e62
fix(scaffolding): skip scaffold when widget directory already exists
rahmanunver Mar 4, 2026
d9746cc
feat(build): extract formatBuildSuccessResponse, chain build to deplo…
rahmanunver Mar 4, 2026
941c16d
feat(code-generation): add detectTemplateMismatch, reorder next steps…
rahmanunver Mar 4, 2026
6cda1b1
feat(server): add protocol logger and session lifecycle instrumentation
rahmanunver Mar 4, 2026
abbc548
test: add formatBuildSuccessResponse, detectTemplateMismatch, scenari…
rahmanunver Mar 4, 2026
3fbd7b3
chore: add mpk-analyzer utility, widget-patterns doc, update .gitignore
rahmanunver Mar 4, 2026
d060aca
build(mcp): drive the widget generator through a Yeoman adapter
rahmanunver Jul 29, 2026
c02fc57
refactor(mcp): delete modules that do not earn their place
rahmanunver Jul 29, 2026
f86769a
fix(security): anchor the sandbox to the Mendix project directory
rahmanunver Jul 29, 2026
4808fbc
refactor(server): make HTTP stateless and fix the transport lifecycle
rahmanunver Jul 29, 2026
701cd1c
feat(tools): replace the code-generation pair with set-widget-properties
rahmanunver Jul 29, 2026
e1c38e6
test(mcp): tighten the response contract, add coverage, rewrite the docs
rahmanunver Jul 29, 2026
2f7a5ff
docs(mcp): document how the server is evaluated, add repo-local skills
rahmanunver Jul 29, 2026
a22c4dd
fix(generators): point widget XML at the schema that actually exists
rahmanunver Jul 31, 2026
a4c514a
test(mcp): add an end-to-end suite that drives a real server process
rahmanunver Jul 31, 2026
5f4d8d9
fix(mcp): keep e2e golden files out of prettier
rahmanunver Jul 31, 2026
8537876
docs(mcp): report the measured spread rather than the best run
rahmanunver Jul 31, 2026
5889dc9
fix(mcp): restore workspace glob for pluggable-widgets-mcp package
rahmanunver Aug 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Golden files must stay byte-identical to what the generator emits. Formatting them makes the
# comparison assert prettier's opinion instead of the server's output.
packages/pluggable-widgets-mcp/src/__e2e__/goldens
packages/tools/generator-widget/generators/app/templates
packages/tools/pluggable-widgets-tools/tests/projects
packages/pluggableWidgets/*/typings
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"prepare-release": "pnpm --filter @mendix/automation-utils run prepare-release",
"publish-marketplace": "turbo run publish-marketplace",
"release": "turbo run release",
"start:mcp": "pnpm --filter @mendix/pluggable-widgets-mcp run start",
"test": "turbo run test --continue --concurrency 1",
"update-screenshots": "node automation/scripts/update-screenshots.mjs",
"update-screenshots-local": "node automation/scripts/update-screenshots-local.mjs",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---
name: mcp-server-test
description: Use after any change to pluggable-widgets-mcp to verify it still works end to end and to record how fast it is. Runs unit tests, real-process end-to-end tests, and optionally an open-ended run where a model uses the server unaided. Produces a pass/fail result plus a timing comparison against the previous run.
---

# Testing the pluggable-widgets-mcp server

Three layers, fastest first. Run them in order and stop at the first failure — a broken unit test
makes the end-to-end result meaningless.

The reader-facing explanation of what this measures and why lives in
`packages/pluggable-widgets-mcp/docs/evaluation.md`. This file is the operating procedure.

## Arguments

- no arguments — layers 1 and 2 (warm), plus the timing comparison
- `--cold` — adds the from-scratch run that produces the headline timing
- `--llm` — adds layer 3, the open-ended run
- `--only <layer>` — `unit` | `e2e` | `llm`

## Layer 1 — unit tests

```bash
cd packages/pluggable-widgets-mcp
npm run test
```

Run from the package directory, not the repo root. Expect all tests green in a few seconds.

**On failure:** stop. Report which specs failed and what they assert. Do not continue to layer 2.

## Layer 2 — end-to-end tests

```bash
cd packages/pluggable-widgets-mcp
npm run build # e2e drives dist/, not src/
npm run test:e2e
```

`npm run build` first is not optional — the harness spawns `node dist/index.js`, so a stale `dist/`
tests the previous version of the server and will happily pass while the change under test is broken.

For the from-scratch run:

```bash
E2E_COLD=1 npm run test:e2e
```

Cold mode installs dependencies for real and needs network. If it fails on network, say so plainly —
do not report it as a server failure.

**On a golden-file mismatch:** the diff is the finding. Read it before deciding anything. A changed
golden is either a regression or an intended change to XML generation; only the diff tells you
which. If it is intended, update the golden in the same commit as the generator change, never
separately.

**On failure:** report the failing spec, the assertion, and the actual value. Do not re-run hoping
for a different result.

## Layer 3 — open-ended run (`--llm`)

This measures whether the server is _usable_, not whether it is correct. It only means something if
you approach it genuinely cold.

**Rules — these are the whole point of the exercise:**

- Do **not** read `src/`, the tests, or this repository's documentation first.
- Connect to the server and read only what it tells any client: `tools/list`, the server
instructions returned at initialize, and the MCP resources it advertises.
- Work only from those. If you find yourself guessing at an argument, that is a finding — record it
rather than looking up the answer.

**Setup:** start the server against a scratch Mendix project.

```bash
MENDIX_PROJECT_DIR=<path to a Mendix project> node dist/index.js stdio
```

**The brief.** Pick one and treat it as a user request, nothing more:

- "Add a rating-stars widget to my app."
- "I need a badge that shows a status and changes colour."
- "Build me a collapsible panel I can put other widgets inside."

**Record, as you go:**

| | |
| -------------------------- | ------------------------------------------------------------------- |
| Calls made, in order | including ones that failed |
| Wrong turns | a tool called with bad arguments, a step done out of order, a retry |
| Where guidance was missing | anything you had to guess |
| Result | did a `.mpk` reach the project's `widgets/` folder? |
| Wall-clock | from first call to deployed widget |

**Report as findings, not a score.** "The description of `set-widget-properties` does not say
properties replace the previous set, so the first call dropped two properties" is useful. "7/10" is
not.

## Timing

Every end-to-end run appends a row to `packages/pluggable-widgets-mcp/docs/benchmarks/timings.jsonl`
tagged with the current commit. After layer 2, compare the latest row against the previous row of
the same mode and report the delta.

Report it in plain terms:

```
scratch → deployed .mpk 37.4s (cold) +1.2s vs 18fe583
rebuild after an edit 3.9s (warm) -0.1s vs 18fe583
```

A few seconds of drift on the install or build steps is noise — those are npm and the Mendix
toolchain, not this server. Movement in the server's own steps (`set-widget-properties`,
`write-widget-file`, `deploy-widget`) is not noise: those are measured in milliseconds, so a jump to
hundreds of milliseconds means something real changed.

## Reporting

Give the outcome in this order:

1. **Pass or fail**, and if failed, the single most important reason.
2. **Timings**, with the delta against the previous run.
3. **Findings** from layer 3, if it ran.
4. **What you did not run**, and why.

Never report a layer as passing without having run it. If cold mode was skipped, say it was skipped
rather than quoting the previous run's number as if it were fresh.
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
---
name: pluggable-widgets-mcp
description: Use when working on the pluggable-widgets-mcp package — adding tools, writing tests, debugging the server, or understanding the widget lifecycle pipeline.
---

## Project Location

`packages/pluggable-widgets-mcp/` in the `web-widgets` monorepo. All paths below are relative to it.

## Mental model — read this before changing anything

> The server does what is mechanically derivable. The client LLM does what requires judgment.
> The resources tell it how.

XML generation is the server's job: a deterministic transformation of a property model against a
fixed schema. **Component `.tsx` is not** — the model writes it via `write-widget-file`, guided by
`docs/widget-patterns.md`, which ships as an MCP resource. An in-process TSX generator used to exist
and was deleted; it emitted code that did not compile. Do not reintroduce one.

The server runs as a **child process of Mendix Studio Pro over STDIO**. HTTP is for MCP Inspector
debugging only: stateless, bound to `127.0.0.1`.

**The Mendix project directory is the single sandbox root.** Nothing derives from `process.cwd()` —
that used to move the security boundary depending on who spawned the process.

## Pipeline

```
get-project-info → create-widget → set-widget-properties → write-widget-file → build-widget → deploy-widget
(Yeoman) (XML) (model writes TSX) (.mpk) (→ project/widgets/)
```

Widgets scaffold into `{MENDIX_PROJECT_DIR}/widget-sources/<name>/`.

## Key entry points

| File | Role |
| ---------------------- | ---------------------------------------------------------------------------------- |
| `src/index.ts` | Entry point — validates `argv[2]` (`stdio` \| `http`), `--help` |
| `src/server/server.ts` | `createMcpServer()` — registers tools + resources |
| `src/tools/index.ts` | `registerAllTools(server, state)`, ordered by pipeline stage |
| `src/config.ts` | `getConfiguredProjectDir()`, `widgetSourcesDir()`, timeouts, `SERVER_INSTRUCTIONS` |

## Tool map (9 tools)

| File | Tools | Takes `state`? |
| ---------------------------- | ------------------------------------------------------------ | -------------- |
| `project.tools.ts` | `get-project-info`, `set-project-directory`, `deploy-widget` | Yes |
| `scaffolding.tools.ts` | `create-widget` | Yes |
| `widget-properties.tools.ts` | `set-widget-properties` | No |
| `file-operations.tools.ts` | `list-widget-files`, `read-widget-file`, `write-widget-file` | No |
| `build.tools.ts` | `build-widget` | Yes |

`SessionState` is `{ projectDir: string | undefined }` (`src/tools/session-state.ts`), seeded from
`MENDIX_PROJECT_DIR` and re-pointable by `set-project-directory`. Tools that resolve paths against
the sandbox or spawn processes take it; the rest are fenced by `validateFilePath` alone.

`set-widget-properties` is **declarative** — callers send the complete property set, not a diff. It
replaced `generate-widget-code` + `update-widget-properties`, which shared a
`.widget-definition.json` snapshot on disk that could disagree with the XML.

## Adding a new tool

```ts
import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";
import type { ToolResponse } from "@/tools/types";
import { createLogger } from "@/tools/utils/logger";
import { fail, ok } from "@/tools/utils/response";
import type { SessionState } from "@/tools/session-state";

const log = createLogger("my-feature");

const schema = z.object({ widgetPath: z.string().min(1).describe("Absolute path to the widget") });

export function registerMyFeatureTools(server: McpServer, state: SessionState): void {
server.registerTool(
"my-tool",
{ title: "My Tool", description: "What it does and what it returns.", inputSchema: schema },
async (args): Promise<ToolResponse> => {
if (!state.projectDir) {
return fail("ERR_PROJECT_NOT_CONFIGURED", "No Mendix project is configured.", {
suggestion: "Call set-project-directory."
});
}
return ok("Result text");
}
);
}
```

Then wire it into `registerAllTools` in `src/tools/index.ts`, at its pipeline position.

**Tool descriptions say what the tool does and returns — nothing about what to call next.** The
workflow lives once in `SERVER_INSTRUCTIONS` (`src/config.ts`). Retry policy prose does not belong in
a description: the server cannot enforce it, and in MCP the client owns the loop.

## Response contract

Exactly two constructors in `src/tools/utils/response.ts`:

```ts
ok(text)
fail(code, message, { suggestion?, file?, line?, column?, details? })
```

`ToolResponse` (`src/tools/types.ts`) is a **type alias, not an interface** — and that is
load-bearing. The SDK's handler signature expects a type carrying an index signature; TypeScript
grants aliases an implicit one, so it stays assignable while excess-property checking still catches a
misspelled `isError`. Changing it to an `interface` breaks every registration site.

Every failure path goes through `fail` with a code. Codes live in the `ErrorCode` union; add one only
when a tool actually emits it. `fail` renders `[ERR_CODE]` into the text, which is all the model sees.

## Path security

Two layers, both rooted at the project directory.

**Sandbox** (`src/tools/utils/sandbox.ts`) — `allowedRoots(state)`, `isPathAllowed(path, state)`,
`describeAllowedRoots(state)`. Roots are `state.projectDir` plus optional `MCP_EXTRA_ALLOWED_PATHS`
(split on `path.delimiter`, not `":"` — that would tear `C:\widgets` in two).

**Guardrails** (`src/security/guardrails.ts`) — `isPathWithinDirectory`, `isExtensionAllowed`,
`validateFilePath(widgetPath, filePath, checkExtension?)` (throws). Pass `checkExtension=true` for
writes. Containment is the whole traversal defence — `resolve()` collapses `../` before comparison,
so there is deliberately no substring test for `".."` (it rejected legitimate names like
`foo..bar.tsx`). Boundary comparisons use `path.sep`, never a hardcoded `"/"`.

Extensions: `.tsx .ts .xml .scss .css .json .md`, plus extensionless `package`/`tsconfig`/`eslintrc`
by **exact filename** and the dot-files `.gitignore .prettierrc .eslintrc .editorconfig`.

## The Yeoman generator — do not reintroduce CLI flags

`@mendix/generator-widget` (registry `^11.11.0`) has **no non-interactive CLI**: it declares zero
`this.option()` calls, `yeoman-generator@8`'s `prompt()` never reads `this.options`, and `.yo-rc.json`
prefill only touches `store: true` prompts and still prompts. A local fork added `--default` to work
around this; the registry version does not have it.

Instead `src/tools/utils/answer-adapter.ts` replaces the Yeoman environment's I/O layer — the
supported extension point. `AnswerAdapter` implements the full `QueuedAdapter` shape (`log`, `prompt`,
`queue`, `progress`, `close`, `abort`, `signal`) because `yeoman-environment` assigns it directly and
does **not** wrap a plain adapter.

Three properties that matter:

- Supplied answer wins → prompt default fills gaps → **missing-and-defaultless throws**
(`MissingAnswerError`), so an upstream prompt rename fails loudly instead of silently defaulting.
- Nothing writes to stdout. Under STDIO that channel is the JSON-RPC stream.
- Scaffolding **must** target a fresh empty directory. The generator's `end()` hook spawns builds with
`stdio: "inherit"` when it finds a populated `node_modules` — straight into the protocol channel.

The 14 prompt names are pinned in `src/tools/utils/__tests__/generator.test.ts`. They are the
generator's contract, not ours: `hasUnitTests`/`hasE2eTests` (not `unitTests`/`e2eTests`), and
`copyright` is deliberately unanswered so its own current-year default applies.

`runWidgetGenerator` (in-process, `skipInstall: true`, ~200 ms) and `runNpmInstall` (separate spawn,
own timeout) are separate steps with separately reported outcomes — a registry stall leaves a usable
scaffold.

## Build

`build-widget` success is **the child process's exit code**, never a substring of the output.
`parseBuildOutput` returns `Omit<BuildResult, "success">` so the compiler enforces that it does not
decide success. Failures return `file:line:column`; file contents are not embedded — `read-widget-file`
exists for that.

## Logging

`src/tools/utils/logger.ts` — `createLogger(tag)` → `.debug/.info/.warn/.error`. **Always stderr**;
Studio Pro captures it and that is the support log. Level via `MCP_LOG_LEVEL`. Distinct from
`src/tools/utils/notifications.ts`, which sends `notifications/message` to the _client_.

## Testing

**`npm run test` inside the package directory** — not `pnpm test` from the repo root.

That is the fast layer only. To verify a change end to end — real process, real scaffold, real build,
real `.mpk` — plus timings, use the `mcp-server-test` skill. `docs/evaluation.md` explains what each
layer proves and why.

```ts
import { createMcpTestContext, getResultText, isError } from "@/__test-utils__/mcp-test-harness";
import { registerMyFeatureTools } from "@/tools/my-feature.tools";

const { client, state, cleanup } = await createMcpTestContext(registerMyFeatureTools);
state.projectDir = dir; // most tools refuse without one
const result = await client.callTool({ name: "my-tool", arguments: { ... } });
expect(isError(result)).toBe(false);
await cleanup();
```

The harness runs a real client↔server pair over `InMemoryTransport`, so calls go through JSON-RPC
serialization and Zod validation, not direct handler invocation.

**Two traps:**

- The suite runs with `restoreMocks: true`. A `vi.fn()` created inside a `vi.mock` factory is reset
after the first test, leaving later tests with a stub returning `undefined`. Use **plain functions**
in module-mock factories.
- Scaffolding is no longer slow — `skipInstall: true` runs in ~200 ms, so prefer a real scaffold over
mocking `runWidgetGenerator` in new tests.

Fixtures: `createTempMendixProject()`, `createTempWidgetWithMpk()` (`src/__test-utils__/temp-dir.ts`).

## Commands

| Command | Effect |
| ---------------------------------------------- | ------------------------------------- |
| `npm run build` | `tsc` + `tsc-alias` + chmod |
| `npm run dev` | Watch via tsx |
| `npm run test` | `vitest run` |
| `npm run lint` | eslint |
| `node dist/index.js stdio` | Production transport |
| `MENDIX_PROJECT_DIR=… node dist/index.js http` | Inspector debugging, `127.0.0.1:3100` |

## Environment

| Variable | Purpose |
| ------------------------- | --------------------------------------------------------------------------------------- |
| `MENDIX_PROJECT_DIR` | The project. Also the sandbox root. Tools fail `ERR_PROJECT_NOT_CONFIGURED` without it. |
| `MCP_EXTRA_ALLOWED_PATHS` | Extra roots, platform-delimited. Dev only. |
| `MCP_LOG_LEVEL` | `debug` \| `info` \| `warn` \| `error` |
| `PORT` | HTTP port, default 3100 |

`@/` → `src/` (tsconfig paths; `vite-tsconfig-paths` in tests, `tsc-alias` in build).

## Deep reference

`.claude/skills/pluggable-widgets-mcp/reference.md` — full module inventory, test inventory,
transport details, MPK structure.
Loading
Loading