From 358747588be4f4ae99dd0599e56131836c7bd33f Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sat, 22 Aug 2026 09:05:51 +0800 Subject: [PATCH 1/7] Add comfyui-studio plugin: workflow submission + character-consistent generation A zero-dependency toolkit for driving a local ComfyUI 8188 server. What's included: - 3 Skills: comfyui-studio (routing), comfyui-workflow (submit/poll/download), comfyui-character (LoRA-based consistency patterns) - A dependency-free stdio MCP server (server.mjs, ~200 lines, pure Node stdlib) exposing 3 tools: submit_prompt, check_queue, get_image - A dependency-free Python CLI (submit_workflow.py) for the same primitives - 2 reference workflow JSONs (text-to-image, image-to-image) - Complete docs: README, security-notes, troubleshooting, examples/minimal-run What it does NOT include (per repo policy): - Native binaries, model files, LoRAs, or any other identity assets - Installers or post-install hooks - Any network destination other than COMFYUI_URL Validation: - npm run check passes for this plugin - Python CLI probes a real ComfyUI 8188 instance - MCP server starts cleanly via 'node server.mjs' --- plugins/antianqi/comfyui-studio/.gitignore | 1 + plugins/antianqi/comfyui-studio/LICENSE | 200 ++++++++++++++ plugins/antianqi/comfyui-studio/README.md | 120 +++++++++ .../comfyui-studio/docs/security-notes.md | 71 +++++ .../comfyui-studio/docs/troubleshooting.md | 88 ++++++ .../comfyui-studio/examples/README.md | 24 ++ .../comfyui-studio/examples/minimal-run.md | 123 +++++++++ plugins/antianqi/comfyui-studio/mcp.json | 13 + plugins/antianqi/comfyui-studio/plugin.json | 21 ++ plugins/antianqi/comfyui-studio/server.mjs | 235 ++++++++++++++++ .../skills/comfyui-character/SKILL.md | 108 ++++++++ .../references/lora-guide.md | 87 ++++++ .../references/prompt-patterns.md | 99 +++++++ .../skills/comfyui-studio/SKILL.md | 67 +++++ .../skills/comfyui-workflow/SKILL.md | 124 +++++++++ .../references/api-reference.md | 129 +++++++++ .../scripts/submit_workflow.py | 250 ++++++++++++++++++ .../workflows/image-to-image.json | 81 ++++++ .../workflows/text-to-image.json | 76 ++++++ 19 files changed, 1917 insertions(+) create mode 100644 plugins/antianqi/comfyui-studio/.gitignore create mode 100644 plugins/antianqi/comfyui-studio/LICENSE create mode 100644 plugins/antianqi/comfyui-studio/README.md create mode 100644 plugins/antianqi/comfyui-studio/docs/security-notes.md create mode 100644 plugins/antianqi/comfyui-studio/docs/troubleshooting.md create mode 100644 plugins/antianqi/comfyui-studio/examples/README.md create mode 100644 plugins/antianqi/comfyui-studio/examples/minimal-run.md create mode 100644 plugins/antianqi/comfyui-studio/mcp.json create mode 100644 plugins/antianqi/comfyui-studio/plugin.json create mode 100644 plugins/antianqi/comfyui-studio/server.mjs create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-character/references/lora-guide.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-character/references/prompt-patterns.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-workflow/SKILL.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-workflow/references/api-reference.md create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py create mode 100644 plugins/antianqi/comfyui-studio/workflows/image-to-image.json create mode 100644 plugins/antianqi/comfyui-studio/workflows/text-to-image.json diff --git a/plugins/antianqi/comfyui-studio/.gitignore b/plugins/antianqi/comfyui-studio/.gitignore new file mode 100644 index 0000000..c18dd8d --- /dev/null +++ b/plugins/antianqi/comfyui-studio/.gitignore @@ -0,0 +1 @@ +__pycache__/ diff --git a/plugins/antianqi/comfyui-studio/LICENSE b/plugins/antianqi/comfyui-studio/LICENSE new file mode 100644 index 0000000..13a6129 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/LICENSE @@ -0,0 +1,200 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of tracking or otherwise improving the Work, + but excludes communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for describing the origin of the Work and + reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Support. While redistributing the Work or + Derivative Works thereof, You may accept and charge a fee for, + acceptance of support, warranty, indemnity, or other liability + obligations and/or rights consistent with this License. However, + in accepting such obligations, You may act only on Your own + behalf and on Your sole responsibility, not on behalf of any other + Contributor, and only if You agree to indemnify, defend, and hold + each Contributor harmless for any liability incurred by, or claims + asserted against, such Contributor by reason of Your accepting any + such warranty or support. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file name and extension be included on the same line as the + copyright text for easy identification within third-party archives. + + Copyright 2026 antianqi + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/plugins/antianqi/comfyui-studio/README.md b/plugins/antianqi/comfyui-studio/README.md new file mode 100644 index 0000000..9b84da4 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/README.md @@ -0,0 +1,120 @@ +# ComfyUI Studio + +A generic, dependency-free toolkit for driving a local ComfyUI server. Submit workflows, poll +the queue, download generated images, and build consistent characters with a self-trained +LoRA. Bundles three Skills and a stdio MCP server; ships zero native binaries, zero installers, +and zero third-party dependencies. + +## The problem + +Driving a local ComfyUI server from an agent requires either: + +- Hand-rolling HTTP calls every time (`/prompt`, `/history`, `/view`...) and rewriting the + same boilerplate per workflow. +- Adopting a heavyweight MCP server that pulls in 50+ MB of native dependencies (the + `sharp` image library) and runs an install hook on first use — a non-starter for shared + environments. + +This Plugin does the first without paying the second cost. The same three primitives are +exposed through both a tiny Python CLI and a ~200-line stdio MCP server, so the user picks +whichever entry point fits their host agent. + +## Try it + +After installing the Plugin into MiniMax Code: + +```text +Use comfyui-studio to verify that my local ComfyUI is reachable, then submit the bundled +text-to-image workflow with the prompt "a tabby cat sleeping on a windowsill, morning +light, photorealistic" and report the saved image path. +``` + +Expected result: the agent calls `submit_prompt` (or `submit_workflow.py`), polls the queue +until the run finishes, downloads the image, and tells the user where it was saved. The end-to-end +walkthrough is in [`examples/minimal-run.md`](examples/minimal-run.md). + +## What you get + +``` +comfyui-studio/ +├── plugin.json # Plugin manifest (required) +├── mcp.json # Registers the stdio MCP server +├── server.mjs # 200-line zero-dep stdio JSON-RPC server +├── LICENSE # Apache-2.0 +├── README.md # This file +├── skills/ +│ ├── comfyui-studio/ # Routing layer: which sibling Skill applies +│ ├── comfyui-workflow/ # Submit / poll / download +│ │ ├── SKILL.md +│ │ ├── references/api-reference.md +│ │ └── scripts/submit_workflow.py +│ └── comfyui-character/ # Consistent characters with a self-trained LoRA +│ ├── SKILL.md +│ └── references/ +│ ├── prompt-patterns.md +│ └── lora-guide.md +├── workflows/ +│ ├── text-to-image.json +│ └── image-to-image.json +├── examples/ +│ ├── README.md +│ └── minimal-run.md # 5-minute end-to-end walkthrough +└── docs/ + ├── security-notes.md + └── troubleshooting.md +``` + +## Three Skills, one shared transport + +- **`comfyui-studio`** — entry point. Reads the user's intent and routes to the right sibling. +- **`comfyui-workflow`** — submits workflow JSON, polls the queue, downloads outputs. +- **`comfyui-character`** — produces a consistent character across many generations, using a + LoRA the user trained themselves and placed in ComfyUI's `models/loras/`. + +Read [`skills/comfyui-studio/SKILL.md`](skills/comfyui-studio/SKILL.md) for the routing +table. Each sibling's `SKILL.md` is the actual implementation guide. + +## MCP server (optional) + +`mcp.json` registers a stdio MCP server. The server speaks JSON-RPC directly, with zero npm +dependencies; the only requirement is Node.js 18+ on the host's `PATH`. The three tools it +exposes mirror the Python CLI: + +| Tool | Purpose | +|---|---| +| `submit_prompt` | Submit a workflow JSON; returns a `prompt_id` | +| `check_queue` | Report running and pending counts and IDs | +| `get_image` | Download a generated image by filename | + +If the host agent is not MCP-aware, the Python script gives you the same primitives without +needing Node or any MCP plumbing. + +## Requirements + +- A running ComfyUI server reachable at `COMFYUI_URL` (default `http://127.0.0.1:8188`). +- At least one checkpoint model in ComfyUI's `models/checkpoints/`. +- For `comfyui-character`: a LoRA you trained, dropped into `models/loras/`. +- Python 3.10+ if you use the script. Node 18+ if you use the MCP server. +- No accounts. No paid services. No telemetry. No native binaries. No installers. + +## Data and network + +- The Plugin only talks to `COMFYUI_URL`. It makes no other network calls. +- The Plugin reads workflow JSON files the user names and writes outputs to the user-chosen + `--output-dir`. It does not read or upload model files, LoRAs, voice samples, or any other + identity asset. +- The Plugin does not log, persist, or transmit any user data. +- See [`docs/security-notes.md`](docs/security-notes.md) for the full threat model. + +## What the Plugin does not do + +- It does not train LoRAs. Training is its own project; this Plugin only consumes a LoRA + the user already has. +- It does not bundle or distribute any LoRA, model, face embedding, voice sample, or any + other identity asset. Every character is user-supplied. +- It does not install packages, run post-install hooks, or download native binaries. +- It does not call any cloud service or third-party API. + +## License + +Apache-2.0. See [LICENSE](LICENSE). diff --git a/plugins/antianqi/comfyui-studio/docs/security-notes.md b/plugins/antianqi/comfyui-studio/docs/security-notes.md new file mode 100644 index 0000000..92dba51 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/docs/security-notes.md @@ -0,0 +1,71 @@ +# Security notes + +This Plugin is designed to be safe to install in environments where ComfyUI is already running +locally. It does not introduce new network destinations, does not run any installers, and does +not request elevated privileges. + +## What the Plugin touches + +| Surface | Action | Notes | +|---|---|---| +| Local HTTP API (`COMFYUI_URL`) | Reads queue, submits workflows, downloads images | Default `http://127.0.0.1:8188`; loopback only unless overridden | +| Local filesystem | Reads workflow JSON files the user points at; writes outputs to a user-chosen directory | The script never writes outside `--output-dir` | +| Environment variables | Reads `COMFYUI_URL` and optional `COMFYUI_API_TOKEN` | Token is sent only to `COMFYUI_URL` | +| LoRA / checkpoint files | The Plugin does not read, write, copy, or upload any model file | Models and LoRAs are loaded by ComfyUI itself, on its own terms | + +The Plugin does not: + +- Make outbound requests to anywhere except `COMFYUI_URL`. +- Install, download, or update any package (no `npm install`, no `pip install`, no post-install + hook). The MCP server is a single-file `server.mjs` that uses only the Node 18+ stdlib. +- Collect telemetry, analytics, or crash reports. +- Read or upload any file outside of what the user explicitly passes as `--workflow`, + `--output-dir`, or `--filename`. +- Set environment variables, modify the user's shell, or change system settings. + +## Threat model + +The Plugin assumes: + +- The host running ComfyUI is a development workstation the user controls. +- The `COMFYUI_URL` address points to a process the user started and trusts. +- Any reverse proxy in front of ComfyUI is the user's own and is responsible for its own + authentication. + +The Plugin does not defend against a malicious ComfyUI server. If `COMFYUI_URL` points to a +server an attacker controls, the attacker can return any JSON they like from `/prompt`, +`/history`, and `/view` — the Plugin will display that JSON to the agent. Treat `COMFYUI_URL` +the same way you treat the path of any binary you run. + +## Authentication + +Plain ComfyUI has no authentication. If the user runs ComfyUI behind a reverse proxy that +requires a bearer token, set `COMFYUI_API_TOKEN` in the environment. The Plugin's MCP server +and the Python script both add it as `Authorization: Bearer ` to every request. The +token never leaves the host; it is not logged, echoed, or sent to any other destination. + +## The MCP server boundary + +The Plugin's `server.mjs` exposes three tools: `submit_prompt`, `check_queue`, and +`get_image`. Each maps to a single HTTP call against `COMFYUI_URL`. The server holds no state +between calls and does not open any socket other than the stdio JSON-RPC channel to its parent +agent. It is a thin adapter, not a service. + +## What you should review before adopting + +- **The bundled workflow JSON.** Open `workflows/text-to-image.json` and confirm you understand + every node. ComfyUI workflows are arbitrary code graphs; the Plugin's only assumption is the + `__PROMPT__` token convention, which is opt-in. +- **`mcp.json`.** Confirm the `COMFYUI_URL` it sets is the address you want. +- **`server.mjs`.** It is ~200 lines of plain Node stdlib; review it the same way you would + review any other small script you are about to run. +- **The `submit_workflow.py` script.** Single file, stdlib only; review it before running. + +## Reporting a security issue + +See the parent repository's [`SECURITY.md`](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/blob/main/SECURITY.md) +for the responsible-disclosure process. + +## License + +Apache-2.0. See [LICENSE](../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/docs/troubleshooting.md b/plugins/antianqi/comfyui-studio/docs/troubleshooting.md new file mode 100644 index 0000000..20c38de --- /dev/null +++ b/plugins/antianqi/comfyui-studio/docs/troubleshooting.md @@ -0,0 +1,88 @@ +# Troubleshooting + +Common failures when using the Plugin and how to recover from them. + +## Health probe fails + +```text +[probe] FAILED status=0 body=connection error: [Errno 111] Connection refused +``` + +- ComfyUI is not running, or is running on a different port. Start ComfyUI and confirm with + `curl http://127.0.0.1:8188/system_stats`. +- You set `COMFYUI_URL` to a different address. Confirm with `echo $COMFYUI_URL` (or + `echo $env:COMFYUI_URL` in PowerShell) and unset it if you do not need to override. + +## Workflow rejected with `node not found` + +```text +[submit] workflow rejected: {"type":"validation", "node_id": "10", "message": "..."} +``` + +The workflow references a node or class ComfyUI does not have. Two common causes: + +- The bundled workflow assumes a custom node you have not installed (e.g. `LoraLoader` from a + custom-node pack). Install the missing pack or remove the offending node from the workflow. +- The workflow was saved by a newer ComfyUI version and references a node that has been + renamed or removed. Re-save the workflow from your installed version. + +## Workflow accepted, but no output appears + +```text +[poll] success but no output files reported (workflow may not have a Save node) +``` + +The workflow does not have a `SaveImage`, `VHS_VideoCombine`, or other terminal node. Add one +or pick a workflow that ends in a save. + +## VRAM exhausted mid-run + +The Plugin does not have a VRAM watchdog; the user must watch `nvidia-smi` themselves for +long jobs (over 30 minutes). If the GPU reports `CUDA out of memory`: + +1. Stop the current run: `POST /queue/interrupt`. +2. Clear the queue: `POST /queue/clear`. +3. Reduce the resolution in the workflow (e.g. `EmptyLatentImage` width/height). +4. Reduce the model precision if the workflow supports it. +5. Restart ComfyUI to free leaked allocations. + +The Plugin never re-submits a failed run on its own — generation is expensive and the user +should be in the loop. + +## Poll times out after 15 minutes + +The Plugin's Python script and the documented recipe both cap polling at 15 minutes. For jobs +that take longer: + +- Use a smaller model or lower resolution so the job fits inside the cap. +- Or modify the script's `POLL_TIMEOUT_S` to your desired value. The Plugin does not change + this default; do not raise it silently. +- Or split the work into multiple submissions and run them serially. + +## The MCP server starts but the agent cannot see tools + +The Plugin's `mcp.json` registers the server under the name `comfyui-studio`. If the host +agent does not see the three tools (`submit_prompt`, `check_queue`, `get_image`): + +- Confirm the host agent supports the MCP stdio transport. +- Confirm Node.js 18+ is on the host's PATH. +- Run the server manually to see its startup banner: `node server.mjs`. It should print + `[comfyui-studio] ready; talking to http://127.0.0.1:8188` and stay running. + +## LoRA does nothing at inference + +See [`skills/comfyui-character/references/prompt-patterns.md`](../skills/comfyui-character/references/prompt-patterns.md). +The most common causes, in order of frequency: + +1. The trigger word is not the first token in the prompt. +2. `strength_model` is set too low (try 0.85 → 1.0). +3. The base model was changed after training. +4. The negative prompt is fighting the LoRA. + +## Identity drifts across generations + +Same reference as above; the prompt-patterns file walks through the fix-it-once checklist. + +## License + +Apache-2.0. See [LICENSE](../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/examples/README.md b/plugins/antianqi/comfyui-studio/examples/README.md new file mode 100644 index 0000000..bfe512d --- /dev/null +++ b/plugins/antianqi/comfyui-studio/examples/README.md @@ -0,0 +1,24 @@ +# Examples + +The Plugin ships end-to-end walkthroughs that exercise every Skill in this Plugin. Start with +[`minimal-run.md`](minimal-run.md) — it assumes a fresh ComfyUI install and walks through a +complete run in under 5 minutes. + +| File | What it covers | +|---|---| +| `minimal-run.md` | Health probe, submit a workflow, poll, download. The "hello world" of this Plugin. | + +## Adding your own example + +If you build a workflow or pattern that other users would benefit from, contributions welcome. +The Plugin does not accept example prompts that: + +- Reference a private LoRA, voice sample, or face embedding you trained for a single user. +- Embed API tokens, account identifiers, or proprietary model names. +- Depend on a paid ComfyUI cloud provider that requires a secret. + +Generic patterns only. + +## License + +Apache-2.0. See [LICENSE](../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/examples/minimal-run.md b/plugins/antianqi/comfyui-studio/examples/minimal-run.md new file mode 100644 index 0000000..e668be6 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/examples/minimal-run.md @@ -0,0 +1,123 @@ +# Minimal end-to-end run + +This walkthrough takes a fresh ComfyUI install and runs the bundled `text-to-image.json` workflow +in under 5 minutes. It exercises every layer of the Plugin: the MCP server, the Python script, +both skills, and the health-probe / submit / poll / download flow. + +## What you need + +- A running ComfyUI server on `http://127.0.0.1:8188` (or another address — set `COMFYUI_URL`). +- At least one checkpoint model installed in ComfyUI's `models/checkpoints/` directory. +- For the character variant: a LoRA the user trained, dropped into `models/loras/`. + +You do **not** need Node.js, npm, or any third-party Python package. The Plugin ships a +zero-dependency stdio MCP server and a zero-dependency Python script. + +## Step 1 — verify ComfyUI is reachable + +```bash +cd +python skills/comfyui-workflow/scripts/submit_workflow.py --probe +``` + +Expected output: + +```text +[probe] OK comfyui_version=0.29.2 +[probe] device=cuda:0 vram_free=7.4GB/8.0GB +``` + +If the probe fails, see `docs/troubleshooting.md`. Do not continue. + +## Step 2 — pick a checkpoint + +Open the bundled workflow: + +``` +workflows/text-to-image.json +``` + +The `CheckpointLoaderSimple` node (node `3`) has `inputs.ckpt_name` set to `"any.safetensors"`. +Replace this with the actual filename of a checkpoint on your ComfyUI install. Use +`GET /object_info/CheckpointLoaderSimple` to list available names, or look in +`models/checkpoints/` yourself. + +```bash +curl -s http://127.0.0.1:8188/object_info/CheckpointLoaderSimple | \ + python -c "import json,sys; d=json.load(sys.stdin); print(d['CheckpointLoaderSimple']['input']['required']['ckpt_name'][0])" +``` + +That prints a JSON array of every checkpoint ComfyUI can see. Pick one and edit +`workflows/text-to-image.json` to use its filename. + +## Step 3 — submit a workflow + +```bash +python skills/comfyui-workflow/scripts/submit_workflow.py \ + --workflow workflows/text-to-image.json \ + --prompt "a tabby cat sleeping on a windowsill, morning light, photorealistic" \ + --output-dir ./out +``` + +The `--prompt` argument is substituted into the `__PROMPT__` token inside the workflow's +`CLIPTextEncode` node. If you remove that marker from the workflow, edit the workflow's `text` +field directly instead. + +Expected output: + +```text +[prompt] substituted __PROMPT__ in nodes: ['30'] +[submit] prompt_id= number=42 +[poll] status=success +[poll] saved: + out/comfyui_studio_00001_.png +``` + +The script polls every 2 seconds with a 15-minute cap. Long-running jobs need a manual cap; +see `docs/troubleshooting.md` for VRAM recovery if something goes wrong mid-run. + +## Step 4 — verify the output + +```bash +ls -la out/ +file out/comfyui_studio_00001_.png +``` + +You should see a real PNG, around 1-5 MB for a 1024×1024 generation. If the file is small or +zero bytes, the workflow likely had a downstream error; see Step 6 in the troubleshooting +guide. + +## Step 5 — same flow via the MCP server + +The Plugin's stdio MCP server exposes the same three primitives as the Python script. From any +MCP-capable agent: + +```text +Use the comfyui-studio MCP server to: + 1. check_queue to confirm ComfyUI is empty + 2. submit_prompt with the bundled text-to-image workflow and the prompt "a tabby cat..." + 3. wait until /history/ returns success + 4. get_image on the returned filename +``` + +The agent should report the prompt_id, the saved file path, and the time taken. The +[server.mjs](../server.mjs) file is the source; it is plain Node 18+ stdlib, no `package.json` +needed to run it. + +## What to try next + +- **Tweak the prompt.** Change the `--prompt` argument and re-run. The seed in the workflow + is fixed at `42`, so identical prompts produce identical images. +- **Switch the LoRA.** Edit `workflows/text-to-image.json` and set `inputs.lora_name` on node + `10` to a LoRA you placed in `models/loras/`. Read + [`skills/comfyui-character/references/prompt-patterns.md`](../skills/comfyui-character/references/prompt-patterns.md) + for the 4-module prompt structure that makes LoRAs work. +- **Use the image-to-image workflow.** `workflows/image-to-image.json` accepts an input image + through ComfyUI's `/upload/image` endpoint. Set the `image` field on node `20` after upload. +- **Bring your own workflow.** Drop a `.json` file into `workflows/` and pass its path via + `--workflow`. The Plugin only assumes the `__PROMPT__` convention for prompt override; + everything else passes through unchanged. + +## License + +Apache-2.0. See [LICENSE](../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/mcp.json b/plugins/antianqi/comfyui-studio/mcp.json new file mode 100644 index 0000000..1da0c21 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/mcp.json @@ -0,0 +1,13 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json", + "mcpServers": { + "comfyui-studio": { + "type": "stdio", + "command": "node", + "args": ["./server.mjs"], + "env": { + "COMFYUI_URL": "http://127.0.0.1:8188" + } + } + } +} diff --git a/plugins/antianqi/comfyui-studio/plugin.json b/plugins/antianqi/comfyui-studio/plugin.json new file mode 100644 index 0000000..b81c422 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/plugin.json @@ -0,0 +1,21 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "comfyui-studio", + "version": "0.1.0", + "description": "Generic toolkit for driving a local ComfyUI 8188 server: workflow submission, queue monitoring, image download, and reproducible generation patterns. Includes a dependency-free stdio MCP server and three Skills for agent-driven workflow automation.", + "author": { + "name": "antianqi", + "url": "https://github.com/antianqi" + }, + "license": "Apache-2.0", + "keywords": [ + "minimax-code", + "plugin", + "comfyui", + "stable-diffusion", + "image-generation", + "mcp", + "workflow", + "local-inference" + ] +} diff --git a/plugins/antianqi/comfyui-studio/server.mjs b/plugins/antianqi/comfyui-studio/server.mjs new file mode 100644 index 0000000..d4585a4 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/server.mjs @@ -0,0 +1,235 @@ +// ComfyUI Studio — dependency-free stdio MCP server +// +// Exposes three tools for driving a local ComfyUI 8188 server: +// - submit_prompt Submit a workflow JSON, return the prompt_id +// - check_queue Inspect running / pending queue +// - get_image Download a generated image by filename +// +// Configuration via environment variables: +// COMFYUI_URL ComfyUI base URL (default http://127.0.0.1:8188) +// COMFYUI_API_TOKEN Optional bearer token if ComfyUI is fronted by an auth proxy +// +// This server intentionally has zero npm dependencies. The MCP wire +// protocol is plain JSON-RPC over stdio, so we speak it directly with +// line-delimited JSON messages. + +import { createInterface } from "node:readline"; +import { stdin, stdout, stderr } from "node:process"; +import { request as httpRequest } from "node:http"; +import { request as httpsRequest } from "node:https"; +import { URL } from "node:url"; + +const COMFYUI_URL = (process.env.COMFYUI_URL || "http://127.0.0.1:8188").replace(/\/$/, ""); +const COMFYUI_API_TOKEN = process.env.COMFYUI_API_TOKEN || ""; + +const TOOLS = [ + { + name: "submit_prompt", + description: + "Submit a ComfyUI workflow JSON to the local server and return the prompt_id for later polling.", + inputSchema: { + type: "object", + properties: { + workflow: { + type: "object", + description: "ComfyUI workflow JSON (the value to put under the `prompt` key).", + }, + }, + required: ["workflow"], + additionalProperties: false, + }, + }, + { + name: "check_queue", + description: + "Return the current ComfyUI queue: counts of running and pending prompts and their prompt_ids.", + inputSchema: { type: "object", properties: {}, additionalProperties: false }, + }, + { + name: "get_image", + description: + "Download a generated image from ComfyUI's /view endpoint by filename and subfolder. Returns the image bytes as a base64 string plus the content type.", + inputSchema: { + type: "object", + properties: { + filename: { type: "string", description: "Image filename on the ComfyUI server." }, + subfolder: { type: "string", description: "Optional subfolder under the output directory." }, + folder_type: { + type: "string", + enum: ["output", "input", "temp"], + description: "ComfyUI folder type. Default: output.", + }, + }, + required: ["filename"], + additionalProperties: false, + }, + }, +]; + +function httpJson(method, path, body) { + return new Promise((resolve, reject) => { + const target = new URL(COMFYUI_URL + path); + const lib = target.protocol === "https:" ? httpsRequest : httpRequest; + const req = lib( + { + method, + hostname: target.hostname, + port: target.port || (target.protocol === "https:" ? 443 : 80), + path: target.pathname + target.search, + headers: { + "content-type": "application/json", + accept: "application/json", + ...(COMFYUI_API_TOKEN ? { authorization: `Bearer ${COMFYUI_API_TOKEN}` } : {}), + }, + }, + (res) => { + const chunks = []; + res.on("data", (c) => chunks.push(c)); + res.on("end", () => { + const text = Buffer.concat(chunks).toString("utf8"); + resolve({ status: res.statusCode, text }); + }); + } + ); + req.on("error", reject); + if (body !== undefined) req.write(typeof body === "string" ? body : JSON.stringify(body)); + req.end(); + }); +} + +async function callSubmitPrompt({ workflow }) { + const { status, text } = await httpJson("POST", "/prompt", { prompt: workflow }); + if (status >= 400) { + return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; + } + let data; + try { + data = JSON.parse(text); + } catch (err) { + return { content: [{ type: "text", text: `Non-JSON response: ${text}` }], isError: true }; + } + if (data.error) { + return { + content: [{ type: "text", text: `Workflow rejected: ${JSON.stringify(data.error)}` }], + isError: true, + }; + } + return { + content: [ + { + type: "text", + text: JSON.stringify({ prompt_id: data.prompt_id, number: data.number }, null, 2), + }, + ], + }; +} + +async function callCheckQueue() { + const { status, text } = await httpJson("GET", "/queue"); + if (status >= 400) { + return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; + } + const data = JSON.parse(text); + return { + content: [ + { + type: "text", + text: JSON.stringify( + { + running: (data.queue_running || []).length, + pending: (data.queue_pending || []).length, + running_ids: (data.queue_running || []).map((q) => q[1]), + pending_ids: (data.queue_pending || []).map((q) => q[1]), + }, + null, + 2 + ), + }, + ], + }; +} + +async function callGetImage({ filename, subfolder = "", folder_type = "output" }) { + const params = new URLSearchParams({ filename, subfolder, type: folder_type }); + const { status, text } = await httpJson("GET", `/view?${params.toString()}`); + if (status >= 400) { + return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; + } + // We cannot send raw binary in a stdio JSON-RPC response, so we return the + // bytes as base64 alongside the inferred content type. + const buf = Buffer.from(text, "binary"); + const contentType = filename.toLowerCase().endsWith(".png") ? "image/png" : "image/octet-stream"; + return { + content: [ + { + type: "text", + text: JSON.stringify( + { filename, content_type: contentType, size_bytes: buf.length, base64: buf.toString("base64") }, + null, + 2 + ), + }, + ], + }; +} + +const HANDLERS = { + submit_prompt: callSubmitPrompt, + check_queue: callCheckQueue, + get_image: callGetImage, +}; + +function send(obj) { + stdout.write(JSON.stringify(obj) + "\n"); +} + +function log(...args) { + stderr.write("[comfyui-studio] " + args.join(" ") + "\n"); +} + +const rl = createInterface({ input: stdin, terminal: false }); +rl.on("line", async (line) => { + let msg; + try { + msg = JSON.parse(line); + } catch (err) { + log("discarding non-JSON line"); + return; + } + const { id, method, params } = msg; + try { + if (method === "initialize") { + send({ + jsonrpc: "2.0", + id, + result: { + protocolVersion: "2024-11-05", + serverInfo: { name: "comfyui-studio", version: "0.1.0" }, + capabilities: { tools: {} }, + }, + }); + } else if (method === "notifications/initialized") { + // no-op + } else if (method === "tools/list") { + send({ jsonrpc: "2.0", id, result: { tools: TOOLS } }); + } else if (method === "tools/call") { + const { name, arguments: args } = params || {}; + const handler = HANDLERS[name]; + if (!handler) { + send({ jsonrpc: "2.0", id, error: { code: -32601, message: `Unknown tool: ${name}` } }); + return; + } + const result = await handler(args || {}); + send({ jsonrpc: "2.0", id, result }); + } else if (method === "ping") { + send({ jsonrpc: "2.0", id, result: {} }); + } else { + send({ jsonrpc: "2.0", id, error: { code: -32601, message: `Method not found: ${method}` } }); + } + } catch (err) { + log("handler error:", err && err.stack ? err.stack : String(err)); + send({ jsonrpc: "2.0", id, error: { code: -32603, message: String(err && err.message || err) } }); + } +}); + +log("ready; talking to", COMFYUI_URL); diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md new file mode 100644 index 0000000..caa883c --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md @@ -0,0 +1,108 @@ +--- +name: comfyui-character +description: Build a self-trained LoRA for a stable character and use it inside a ComfyUI workflow to keep the character consistent across many generations. Use this Skill when the user wants a recurring character (a brand mascot, a video series character, a presenter), needs cross-image consistency, or asks how to make a LoRA and where to put it. +--- + +# ComfyUI Character + +The Plugin does **not** distribute any character LoRA, face embedding, voice sample, or other +private identity asset. Every character this Skill produces is one the user trained themselves +and dropped into ComfyUI's standard `models/loras/` directory. This Skill is about the workflow +patterns that make a user-trained LoRA actually deliver consistency — what to put in the +training set, what to put in the prompt at inference, and what mistakes to avoid. + +## Scope and assumptions + +- The user has a stable concept of who the character is. If they do not, help them define it + first (name, single defining trait, recurring outfit). Without that, no LoRA will save them. +- The user (or a model-trainer agent they trust) has trained or will train a LoRA. This Skill + does not run LoRA training; it covers the inference side and the prompt / workflow patterns + that consume a LoRA correctly. +- The LoRA file lives at `models/loras/.safetensors` inside the ComfyUI + install. The Skill does not dictate the file name; the user does, when they register the LoRA + in their workflow. + +## Why most "consistent character" attempts fail + +A LoRA is a strong prior on identity, but it is not magic. Three things derail consistency: + +1. **Inconsistent training data.** Mix of angles, lighting, outfits, and stylizations. A LoRA + trained on a single photo or on stylistically inconsistent reference images will not + generalize. See `references/lora-guide.md` for the data-prep checklist. +2. **Weak prompt structure.** Putting the LoRA trigger word in a wall of adjectives loses the + signal. The LoRA trigger must lead the prompt and the prompt must avoid words that fight it. +3. **Wrong sampler / scheduler / CFG for the base model.** Some samplers oversmooth identity + features. See `references/prompt-patterns.md` for the rules of thumb. + +## The three-step recipe + +### Step 1 — Define the character with one trigger word + +The user picks a single token (or a short stable phrase) that the LoRA is trained to respond to. +Examples: `mascotv1`, `presenter_anna`, `mr_noodle_v2`. This token becomes the first word of +every inference prompt. It must not collide with a common word the base model already knows +well (so not `cat`, `woman`, `office`) and must not contain spaces or punctuation the base +model's tokenizer will split unexpectedly. + +### Step 2 — Compose the prompt in 4 modules, in this order + +From `references/prompt-patterns.md`: + +``` +, , , +``` + +- **Identity block** — the small set of features the LoRA was trained on (face shape, hair, + signature accessory). Keep it short; the LoRA is doing the heavy lifting. +- **Outfit block** — what the character is wearing in *this* generation. The character wears + different outfits across the series; do not pin a single outfit in the LoRA. +- **Pose + scene block** — what is happening in the image. This is the part that changes most. + +Anything that does not fit one of these four modules is noise. Strip adjectives like +"beautiful", "stunning", "professional" — they make the base model override the LoRA. See +`references/prompt-patterns.md` for the full 8-rule prompt checklist. + +### Step 3 — Plug the LoRA into the workflow + +The bundled `workflows/text-to-image.json` includes a `LoraLoader` node wired into a +`CheckpointLoader` so the user can: + +1. Set `lora_name` to the file they placed under `models/loras/`. +2. Set `strength_model` to a value between `0.6` and `1.0`. Start at `0.85`; lower it if the + character overpowers the scene, raise it if identity drifts. +3. Leave `strength_clip` at the workflow's default unless the user has measured otherwise. +4. Do **not** stack multiple character LoRAs at full strength — they fight each other. + +If the user's ComfyUI install uses a different workflow format (e.g. Power Lora Loader), they +adapt the same four values; the Skill does not require any particular node. + +## Verifying consistency + +After producing 5–10 reference images, open them in a grid view and check: + +- Face shape and signature features are stable across lighting and pose changes. +- Outfit changes per the prompt, not per the LoRA. +- Pose and scene obey the prompt, not the LoRA's training set. + +If identity drifts, the usual cause in order of frequency: (1) trigger word is in the wrong +position, (2) `strength_model` is too low, (3) base model was changed since training, (4) too +many identity keywords fighting the trigger. Re-run the prompt-patterns checklist and try one +fix at a time. + +## What this Skill does not do + +- It does not run LoRA training. Training is its own project and has its own tooling + (Kohya, OneTrainer, ai-toolkit). The Skill only consumes a LoRA the user already has. +- It does not bundle or distribute any LoRA, model, face embedding, or voice sample. Every + identity asset is user-supplied. +- It does not invent characters. The user must define who the character is. + +## Requirements + +- ComfyUI running locally (see `comfyui-workflow/SKILL.md` for the transport). +- A character LoRA the user trained, dropped under `models/loras/`. +- A base checkpoint compatible with the LoRA's training resolution. + +## License + +Apache-2.0. See [LICENSE](../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/lora-guide.md b/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/lora-guide.md new file mode 100644 index 0000000..9d5db73 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/lora-guide.md @@ -0,0 +1,87 @@ +# Training a character LoRA — data, settings, and where to put the file + +This Plugin consumes LoRAs the user trains themselves. It does not train LoRAs and does not +distribute any. This reference collects the data-prep and training-setting rules that make a +character LoRA actually deliver consistency at inference time. + +## Where the file goes + +When the user is done training, the LoRA file is a single `.safetensors` file. The user places +it under ComfyUI's standard models directory: + +``` +/models/loras/.safetensors +``` + +After placing the file, the user must press the "Refresh" button in ComfyUI's LoRA loader node +(or restart ComfyUI) for the file to appear in the dropdown. The Plugin never reads from any +other directory; ComfyUI's loader only sees files in the standard tree. + +The Plugin does not move, copy, upload, or symlink LoRA files. The user owns the file and the +path. + +## The training data checklist + +The LoRA will only be as consistent as the data it was trained on. Before clicking "Train", +work through this list. Skipping items here is the most common cause of "the LoRA does not +work" reports later. + +- **20–40 images is the sweet spot.** Fewer than 15 and the LoRA underfits. More than 60 and + it overfits and you cannot change outfits at inference. +- **Varied angles.** Front, three-quarter, side, back of head. A LoRA trained only on + front-facing photos will not survive a side profile. +- **Varied lighting.** Daylight, overcast, indoor warm, indoor cool. A LoRA trained only in + studio lighting will not survive a sunny outdoor scene. +- **Varied outfits.** At least 3 distinct outfits across the dataset. This is the only way + the LoRA learns to separate identity from clothing. +- **One character per LoRA.** A LoRA trained on two characters learns a confused average + unless the user tags them with mutually-exclusive trigger words. Even then, separate LoRAs + is simpler and more reliable. +- **Tight crops on the face** in at least 8–10 images. The LoRA needs to learn identity, and + full-body shots with small faces give it almost no signal. +- **Consistent tagging.** Use one tool (kohya-style or ai-toolkit captions), one tag format, + one set of trigger word placements. Mixed conventions produce mixed results. + +## Recommended training settings (starter values) + +These are sane defaults; the user should adjust based on their trainer's UI. Values are +intentionally generic; the user's training tool may name them differently. + +| Setting | Start value | Notes | +|---|---|---| +| Network rank (dim) | 32 | 16 for SD 1.5, 32 for SDXL, 64 for Flux | +| Network alpha | 16 | Usually half of rank; some trainers want equal | +| Learning rate | 1e-4 (SD1.5), 5e-5 (SDXL), 1e-4 (Flux) | Lower if loss plateaus, raise if loss does not move | +| Training steps | 1500–3000 | Stop when preview samples stop improving | +| Batch size | 1 or 2 | Effective batch size = bs * grad_accum | +| Resolution | Match your base model | 512 for SD 1.5, 1024 for SDXL/Flux | +| Mixed precision | bf16 if supported, else fp16 | | +| Optimizer | AdamW8bit or Prodigy | Prodigy removes the LR-tuning guesswork | +| Caption dropout | 0.05 | Forces the model to read the image, not the caption | + +The Plugin does not ship or endorse a particular trainer. Use whichever the user is comfortable +with. + +## What to verify before you ship the LoRA + +A trained LoRA is not "done" the moment training stops. Generate at least 20 test images +across three different scenes, three different outfits, and three different camera angles. If +identity holds in 18+ of them, ship it. If it holds in fewer than 15, retrain with more +diverse data — not with more steps, which overfits. + +## Common failure modes and what they mean + +- **Every output looks identical, ignoring the prompt** → overfit. Lower rank or training + steps, or add more diverse data. +- **Identity is unstable across prompts** → underfit or inconsistent training data. More + steps are usually not the answer; better-curated data is. +- **The trigger word does nothing** → either the trigger was not actually written into the + captions during training, or the base model's tokenizer is splitting it. Try a different + trigger. +- **Outputs look like a different character from the training set** → the LoRA learned the + "vibe" of the dataset, not the identity. The dataset was probably too stylistically + inconsistent. Recur with tighter curation. + +## License + +Apache-2.0. See [LICENSE](../../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/prompt-patterns.md b/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/prompt-patterns.md new file mode 100644 index 0000000..528860b --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-character/references/prompt-patterns.md @@ -0,0 +1,99 @@ +# Prompt patterns for character-consistent generation + +These are the patterns that make a self-trained LoRA actually deliver a consistent character. +They are the result of empirical iteration; treat them as defaults and adjust when a specific +base model or LoRA has a measured reason to differ. + +## The 4-module prompt structure + +Always write the prompt in this order: + +``` +, , , +``` + +1. **Trigger word** — the single token the LoRA was trained to respond to. Always first. +2. **Identity block** — the small set of features the LoRA was trained on. Keep it short; the + LoRA is doing the heavy lifting here, not the prompt. +3. **Outfit block** — what the character wears in *this* generation. The character wears + different outfits across the series; do not pin a single outfit in the LoRA training. +4. **Pose + scene block** — what is happening in the image. This is the part that changes most. + +A prompt that contains material that does not fit one of these four modules is signalling +conflict to the base model. Strip it. + +## The 8 rules + +1. **Trigger word leads, always.** If the LoRA was trained on `mascotv1`, every prompt starts + with `mascotv1,`. Putting the trigger in the middle of adjectives weakens the signal. + +2. **No filler adjectives.** Strip "beautiful", "stunning", "professional", "cinematic", + "8k", "highly detailed" — these tell the base model to override the LoRA with its + photorealistic priors. The LoRA already encodes quality. + +3. **4 modules > 8 modules.** A long, comma-separated prompt with 40+ tokens dilutes the + signal. The 4-module structure beats verbose lists. + +4. **No fighting the LoRA.** If the LoRA was trained on a 30-year-old male character, do not + add "young woman, feminine face" to the identity block. The base model will try to satisfy + both and you get a half-morphed output. + +5. **Outfit is per-prompt, not per-LoRA.** Different scenes = different outfits. If every + generation of the character wears the same jacket, the LoRA will eventually bake the jacket + into identity and you can never change it. Vary the outfit in training data, too. + +6. **Be specific about pose.** "Standing" is too vague. "Leaning on a railing, right hand in + pocket, looking off-camera" gives the base model something concrete and reduces the chance + of pose drift between generations. + +7. **Lighting is its own sentence.** A second short clause after the 4 modules is fine: + "mascotv1, [identity], [outfit], [pose], natural window light from the left, shallow + depth of field". Macro lighting cues (volumetric, Tyndall, soft box) are reliable; micro + descriptors (specular highlight at 0.3cm) are not. + +8. **Negative prompts are short.** For Stable Diffusion 1.5 / SDXL / Flux the negative prompt + should be a small fixed set: `deformed, bad anatomy, extra fingers, blurry, low quality, + watermark`. For character consistency, *add* one phrase: `different character, identity + drift`. Do not stack 30+ negative tokens; they fight the LoRA. + +## What to do when identity drifts + +If you generate 10 images and 4 of them look like a different person: + +1. **Trigger word first.** Open the prompt and confirm the trigger is the first token. If you + accidentally moved it, identity drift is the most common symptom. +2. **Bump `strength_model`.** Try 0.9 → 1.0. If that fixes it, the previous value was simply + too low; the LoRA is fine. +3. **Check the negative prompt.** If the negative prompt contains phrases like + `same character as before` or `consistent identity`, the base model treats them as concepts + it has to interpret, and that interpretation varies per seed. Strip them. +4. **Check the base model.** If you trained the LoRA on SDXL 1.0 and now you are running it on + Flux, the LoRA still loads but its effect is unpredictable. Match the base model. +5. **Reduce scene complexity.** A character in a 30-element busy scene loses identity. Move to + a simpler scene; if identity recovers, the scene was the problem, not the LoRA. + +## What to do when the LoRA is overcooked + +If every image looks identical and the prompt's outfit / pose / scene are not coming through: + +1. Lower `strength_model` by 0.1 and regenerate. Often that is enough. +2. Move the trigger word out of the first position. Place it second, after a strong + scene-describing word. This works around overfit LoRAs that "lock" the first token. +3. Increase CFG slightly (e.g. 5 → 6) so the base model is allowed to push back against the + LoRA more. +4. Re-train with more diverse outfit / pose / lighting data. The LoRA was overtrained on a + narrow look. + +## Prompt length by base model + +A rough guide. Models vary; measure for your own LoRA. + +| Base model | Total prompt length | Notes | +|---|---|---| +| SD 1.5 | 50–70 tokens | Compressed, the 4 modules do most of the work | +| SDXL | 60–90 tokens | A bit more room; identity block can be 8–10 tokens | +| Flux | 80–120 tokens | Flux reads natural language well; the 4 modules can become sentences | + +## License + +Apache-2.0. See [LICENSE](../../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md new file mode 100644 index 0000000..0de79fb --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md @@ -0,0 +1,67 @@ +--- +name: comfyui-studio +description: Drive a local ComfyUI 8188 server for image and short-clip generation. Use this Skill when the task involves a ComfyUI workflow, a local Stable Diffusion / Flux checkpoint, character-consistent generation with a LoRA, or producing deterministic outputs from a saved workflow JSON. Triggers on requests like "use ComfyUI to ...", "submit this workflow", "what models do I have", "generate an image with my LoRA", or "check the queue". +--- + +# ComfyUI Studio + +A generic toolkit for driving a local ComfyUI server (default `http://127.0.0.1:8188`). It bundles +three Skills that work together: + +- **comfyui-studio** (this file): the routing layer. Use it to decide which sibling Skill applies. +- **comfyui-workflow**: submitting workflows, polling the queue, downloading outputs. +- **comfyui-character**: producing a consistent character across many generations with a self-made + LoRA, including the prompt and LoRA-training patterns that make consistency work. + +The Plugin also ships a dependency-free stdio MCP server (see `mcp.json` + `server.mjs`) that exposes +the same three primitives to any MCP-capable agent. + +## When to use which Skill + +| User intent | Use | +|---|---| +| "Submit this workflow" / "Check the queue" / "Download the image" | **comfyui-workflow** | +| "Make a character that looks the same every time" / "Train a LoRA" / "Use my LoRA to keep this character stable" | **comfyui-character** | +| "Use ComfyUI to ..." without further detail | **comfyui-studio** — read the user's intent, then route to one of the two siblings | +| "What models do I have?" / "Is ComfyUI running?" | **comfyui-workflow** — its first step is always a health probe | + +Do not duplicate behaviour: if you only need to submit a workflow, do not read this Skill, read +`comfyui-workflow/SKILL.md` directly. This Skill is the entry point for routing, not the implementation. + +## Required environment + +The Plugin assumes the user has: + +- A running ComfyUI server (default `http://127.0.0.1:8188`). Override with the `COMFYUI_URL` + environment variable or the `--url` flag on the Python scripts. +- At least one checkpoint model installed and reachable from ComfyUI. +- For `comfyui-character`: a character LoRA the user trained themselves and dropped into + ComfyUI's `models/loras/` directory. The Plugin never distributes LoRAs or character weights. + +The Plugin does **not** download or bundle model checkpoints, LoRAs, or any other binary asset. +Each Skill instructs the user to place their own files in the standard ComfyUI directory layout. + +## Data and network + +- Network access is local-only by default. The MCP server and the Python scripts both target the + address in `COMFYUI_URL`. No telemetry, no remote calls, no analytics. +- See `docs/security-notes.md` for the threat model and the data the Plugin handles. +- See `docs/troubleshooting.md` for common failures (ComfyUI offline, missing model, OOM). + +## Trying it out + +End-to-end smoke test (about 5 minutes, see `examples/minimal-run.md`): + +```text +Use comfyui-studio to verify that my local ComfyUI is reachable, then submit the +text-to-image workflow bundled in this Plugin with the prompt "a tabby cat sleeping +on a windowsill, morning light, photorealistic" and report the saved image path. +``` + +Expected outcome: the agent confirms ComfyUI is reachable, submits the bundled +`workflows/text-to-image.json` workflow, polls until done, downloads the image, and tells you where +it was saved. + +## License + +Apache-2.0. See [LICENSE](../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/SKILL.md new file mode 100644 index 0000000..3c17b97 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/SKILL.md @@ -0,0 +1,124 @@ +--- +name: comfyui-workflow +description: Submit a ComfyUI workflow JSON to a local 8188 server, monitor the queue, and download generated images. Use this Skill when the user wants to run a saved workflow, check whether ComfyUI is busy, fetch a generated image, list available checkpoints, or automate a reproducible generation pipeline. +--- + +# ComfyUI Workflow + +Submit workflows, monitor the queue, and retrieve outputs from a local ComfyUI server. This is +the workhorse Skill; `comfyui-character` builds on top of it for character-consistent generation. + +## Scope + +This Skill covers the **transport** between an agent and a local ComfyUI server. It does not +edit workflows, train models, or invent prompts — those concerns live in sibling Skills. It +assumes the user already has a workflow JSON they want to run (either a file the user wrote or +one of the bundled `workflows/*.json` examples). + +## Two equivalent entry points + +You can drive ComfyUI from this Skill in either of two ways. Pick whichever the host environment +supports best. + +### A. Python script (CLI-friendly, no MCP required) + +```bash +# Health probe +python scripts/submit_workflow.py --probe + +# Submit a workflow +python scripts/submit_workflow.py \ + --workflow ./workflows/text-to-image.json \ + --prompt "a tabby cat sleeping on a windowsill, morning light, photorealistic" \ + --output-dir ./out + +# Check the queue without submitting +python scripts/submit_workflow.py --queue + +# Download a specific image +python scripts/submit_workflow.py --download \ + --filename ComfyUI_00001_.png --output-dir ./out +``` + +The script lives at `skills/comfyui-workflow/scripts/submit_workflow.py` (relative to the Plugin +root) and accepts `COMFYUI_URL` from the environment, falling back to `http://127.0.0.1:8188`. +Full flag list in `references/api-reference.md`. + +### B. stdio MCP server (no Python required) + +When the host agent is MCP-aware, the Plugin's `server.mjs` exposes three tools that mirror the +Python script: + +| Tool | Mirrors | +|---|---| +| `submit_prompt` | `--workflow` invocation | +| `check_queue` | `--queue` invocation | +| `get_image` | `--download` invocation | + +The MCP server has zero npm dependencies. It is a 200-line stdio JSON-RPC server in plain Node. +See `mcp.json` for the registration snippet and `server.mjs` for the source. + +## Workflow contract + +ComfyUI accepts a JSON object whose top-level keys are node IDs and whose values are +`{class_type, inputs}` records. When the user provides a workflow, **do not edit the structure** +unless they ask. Two patterns are common: + +- A workflow file saved from the ComfyUI web UI: keys are numeric strings, `class_type` matches + the node name in the editor, `inputs` reference other nodes by `[nodeId, outputIndex]`. +- An "API format" workflow produced by ComfyUI's "Save (API Format)" menu: same shape, but with + widget values inlined into `inputs`. + +Both work identically when POSTed to `/prompt`. The Plugin's `workflows/text-to-image.json` and +`workflows/image-to-image.json` are in the API format. + +## Overriding prompt values at submit time + +The bundled Python script understands a tiny templating convention so you do not have to rewrite +the workflow JSON for every run. In a workflow, a `CLIPTextEncode` node with `inputs.text` +starting with the literal string `__PROMPT__` will have that text replaced with the value of +`--prompt` on the command line before submission. This is a no-op for nodes that do not opt in. + +```json +{ + "30": { + "class_type": "CLIPTextEncode", + "inputs": { "text": "__PROMPT__", "clip": ["4", 0] } + } +} +``` + +For workflows without that marker, the user must edit the workflow JSON directly. + +## Step-by-step recipe + +1. **Health probe.** Either call `check_queue` via MCP or run `submit_workflow.py --probe`. Stop + and report if ComfyUI is unreachable; do not invent a successful run. +2. **Resolve the workflow.** If the user named a file, read it. If they named a workflow by + description (e.g. "the one with the upscale at the end"), ask for a path or a saved name. +3. **Apply prompt override if used.** If `--prompt` is provided and the workflow contains + `__PROMPT__`, substitute. Otherwise pass the workflow through unchanged. +4. **Submit.** POST to `/prompt` with `{"prompt": }`. Capture `prompt_id`. +5. **Poll.** Loop `GET /history/` with a 2-second sleep. Stop on terminal status + (`success`, `error`, or `cancelled`). Use a 15-minute cap; long jobs should be split. +6. **Resolve outputs.** The `outputs` object lists the `SaveImage` / `VHS_VideoCombine` files + produced. For each, fetch `GET /view?filename=...&subfolder=...&type=output`. +7. **Report.** Tell the user the prompt_id, the time taken, the saved file paths, and any + warnings (low VRAM, retries, partial outputs). + +## Long jobs and VRAM safety + +- ComfyUI 22B-class models can hold the GPU for hours. The Plugin does not implement a watchdog; + for jobs longer than 30 minutes, instruct the user to watch `nvidia-smi` themselves. +- If a poll returns an error status, capture the full `outputs` block and the error message; + report both. Do not retry automatically — generation is expensive. + +## Requirements + +- ComfyUI running locally (or reachable at `COMFYUI_URL`). +- Python 3.10+ if using the script. The MCP server requires Node 18+. +- No additional accounts, paid services, or network destinations. + +## License + +Apache-2.0. See [LICENSE](../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/references/api-reference.md b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/references/api-reference.md new file mode 100644 index 0000000..75f5fcf --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/references/api-reference.md @@ -0,0 +1,129 @@ +# ComfyUI 8188 API reference + +A focused subset of the ComfyUI HTTP API that this Plugin uses. The full API surface is +documented at the ComfyUI project; this file covers only the endpoints the Plugin calls. + +All endpoints are relative to the base URL in `COMFYUI_URL` (default `http://127.0.0.1:8188`). +All responses are JSON unless noted. All `POST` bodies are JSON. + +## `GET /system_stats` + +Reports ComfyUI version and runtime info. Used by the health probe. + +Response shape (subset): + +```json +{ + "system": { "comfyui_version": "0.29.2", "python_version": "3.11.x", "pytorch_version": "..." }, + "devices": [{ "name": "cuda:0", "type": "cuda", "vram_total": 8589934592, "vram_free": 6000000000 }] +} +``` + +## `GET /queue` + +Returns the current queue. + +Response shape: + +```json +{ + "queue_running": [["uuid", "prompt_id", {...workflow...}], ...], + "queue_pending": [["uuid", "prompt_id", {...workflow...}], ...] +} +``` + +The Plugin reports counts and prompt_ids; the full workflow body is not echoed back to the +agent to avoid blowing up the context window. + +## `POST /prompt` + +Submits a workflow. Body: + +```json +{ "prompt": { "": { "class_type": "...", "inputs": {...} } } } +``` + +Response (success): + +```json +{ "prompt_id": "uuid", "number": 42 } +``` + +Response (workflow-level error): + +```json +{ "error": { "type": "validation", "message": "...", "details": "...", "node_id": "..." } } +``` + +The Plugin surfaces the full error block to the agent instead of swallowing it. Generation is +expensive; we want the user to see exactly what went wrong. + +## `GET /history/` + +Polls the result of a submitted prompt. Returns the full `outputs` block once status is +terminal. + +Response shape (subset): + +```json +{ + "": { + "status": { "status_str": "success" | "error" | "cancelled" }, + "outputs": { + "": { + "images": [{ "filename": "ComfyUI_00001_.png", "subfolder": "", "type": "output" }], + "gifs": [...], + "videos": [...] + } + } + } +} +``` + +The Plugin polls this every 2 seconds with a 15-minute cap. For long jobs, increase the cap or +have the user re-run with explicit acknowledgement. + +## `GET /view` + +Downloads a generated image. Query parameters: + +| Parameter | Required | Notes | +|---|---|---| +| `filename` | yes | The filename reported in `/history` outputs | +| `subfolder` | no | Defaults to empty (root of the chosen folder) | +| `type` | no | One of `output`, `input`, `temp`. Default: `output` | +| `channel` | no | Used for 3D / multi-channel outputs | + +Response: the image bytes, `Content-Type: image/png` (or whatever the original was). + +## `POST /queue/clear` + +Clears pending queue items. Does not interrupt running items. Use with care — there is no +"are you sure" prompt at the API level. + +## `POST /queue/interrupt` + +Interrupts the currently-running prompt. Use when the user says "stop" or when a poll detects +the queue is stuck. + +## `GET /object_info/` + +Returns the input schema for a specific node. Useful when the agent needs to know which +checkpoint or LoRA names ComfyUI currently sees on disk. The Plugin does not call this +directly; it is here so the user (or an agent) can probe a workflow before submission. + +## Authentication + +Plain ComfyUI has no authentication. If the user runs ComfyUI behind a reverse proxy that +requires a bearer token, set `COMFYUI_API_TOKEN` in the environment; the Plugin's MCP server +and the Python script both add it as `Authorization: Bearer ` on every request. + +## Limits + +The Plugin enforces a 15-minute default poll cap. ComfyUI itself has no hard timeout, but +long-running jobs are a known risk for GPU memory; see `docs/troubleshooting.md` for VRAM +recovery steps. + +## License + +Apache-2.0. See [LICENSE](../../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py new file mode 100644 index 0000000..e44f986 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py @@ -0,0 +1,250 @@ +#!/usr/bin/env python3 +"""Submit a ComfyUI workflow, poll the queue, and download the output. + +This is a single-file CLI wrapper around the local ComfyUI HTTP API. It is +intentionally tiny: no third-party Python dependencies, no native extensions, +no telemetry. Everything it does is also exposed by the Plugin's stdio MCP +server (see ../../../server.mjs); use whichever fits the host agent. + +Environment: + COMFYUI_URL ComfyUI base URL. Default: http://127.0.0.1:8188 + COMFYUI_API_TOKEN Optional bearer token for auth-fronted ComfyUI + +Examples: + python submit_workflow.py --probe + python submit_workflow.py --workflow path/to/wf.json --prompt "a cat" + python submit_workflow.py --queue + python submit_workflow.py --download --filename out.png --output-dir ./out +""" +from __future__ import annotations + +import argparse +import json +import os +import sys +import time +import urllib.error +import urllib.parse +import urllib.request +from pathlib import Path + +DEFAULT_URL = "http://127.0.0.1:8188" +PROMPT_MARKER = "__PROMPT__" +POLL_INTERVAL_S = 2.0 +POLL_TIMEOUT_S = 15 * 60 # 15 minutes cap; long jobs are a VRAM risk + + +def base_url() -> str: + return os.environ.get("COMFYUI_URL", DEFAULT_URL).rstrip("/") + + +def auth_headers() -> dict: + token = os.environ.get("COMFYUI_API_TOKEN", "") + return {"authorization": f"Bearer {token}"} if token else {} + + +def http_json(method: str, path: str, body: dict | None = None) -> tuple[int, dict | str]: + url = base_url() + path + data = None if body is None else json.dumps(body).encode("utf-8") + req = urllib.request.Request( + url, + data=data, + method=method, + headers={"content-type": "application/json", "accept": "application/json", **auth_headers()}, + ) + try: + with urllib.request.urlopen(req, timeout=30) as resp: + raw = resp.read().decode("utf-8") + try: + return resp.status, json.loads(raw) + except json.JSONDecodeError: + return resp.status, raw + except urllib.error.HTTPError as e: + return e.code, e.read().decode("utf-8", errors="replace") + except urllib.error.URLError as e: + return 0, f"connection error: {e.reason}" + + +def http_download(path: str, dest: Path) -> int: + url = base_url() + path + req = urllib.request.Request(url, headers=auth_headers()) + try: + with urllib.request.urlopen(req, timeout=60) as resp: + dest.parent.mkdir(parents=True, exist_ok=True) + dest.write_bytes(resp.read()) + return resp.status + except urllib.error.URLError as e: + print(f"[download] {url}: {e.reason}", file=sys.stderr) + return 0 + + +def apply_prompt_override(workflow: dict, prompt_text: str | None) -> dict: + """If --prompt was given and the workflow has a __PROMPT__ marker, substitute. + + Walks every node in the workflow. For any CLIPTextEncode-style node whose + `text` input is exactly the literal marker, replace with the user's text. + Nodes that already have a value are left alone. + """ + if not prompt_text: + return workflow + changed = [] + for node_id, node in workflow.items(): + if not isinstance(node, dict): + continue + inputs = node.get("inputs") + if not isinstance(inputs, dict): + continue + text = inputs.get("text") + if text == PROMPT_MARKER: + inputs["text"] = prompt_text + changed.append(node_id) + if changed: + print(f"[prompt] substituted __PROMPT__ in nodes: {changed}", file=sys.stderr) + return workflow + + +def cmd_probe(_args) -> int: + status, body = http_json("GET", "/system_stats") + if status != 200: + print(f"[probe] FAILED status={status} body={body}") + return 2 + if isinstance(body, dict): + version = body.get("system", {}).get("comfyui_version", "?") + print(f"[probe] OK comfyui_version={version}") + for dev in body.get("devices", []) or []: + free_gb = (dev.get("vram_free") or 0) / (1024 ** 3) + total_gb = (dev.get("vram_total") or 0) / (1024 ** 3) + print(f"[probe] device={dev.get('name')} vram_free={free_gb:.1f}GB/{total_gb:.1f}GB") + else: + print(f"[probe] OK body={body}") + return 0 + + +def cmd_queue(_args) -> int: + status, body = http_json("GET", "/queue") + if status != 200 or not isinstance(body, dict): + print(f"[queue] FAILED status={status} body={body}") + return 2 + running = body.get("queue_running") or [] + pending = body.get("queue_pending") or [] + print(f"[queue] running={len(running)} pending={len(pending)}") + for item in running: + print(f" running prompt_id={item[1] if len(item) > 1 else '?'}") + for item in pending: + print(f" pending prompt_id={item[1] if len(item) > 1 else '?'}") + return 0 + + +def cmd_download(args) -> int: + out_dir = Path(args.output_dir or ".") + params = urllib.parse.urlencode({ + "filename": args.filename, + "subfolder": args.subfolder or "", + "type": args.folder_type or "output", + }) + status = http_download(f"/view?{params}", out_dir / args.filename) + if status == 200: + print(f"[download] saved {out_dir / args.filename}") + return 0 + return 2 + + +def cmd_submit(args) -> int: + if not args.workflow: + print("[submit] --workflow is required (or use --probe / --queue / --download)", file=sys.stderr) + return 2 + wf_path = Path(args.workflow) + if not wf_path.exists(): + print(f"[submit] workflow file not found: {wf_path}", file=sys.stderr) + return 2 + try: + workflow = json.loads(wf_path.read_text(encoding="utf-8")) + except json.JSONDecodeError as e: + print(f"[submit] invalid JSON: {e}", file=sys.stderr) + return 2 + + workflow = apply_prompt_override(workflow, args.prompt) + + status, body = http_json("POST", "/prompt", {"prompt": workflow}) + if status != 200: + print(f"[submit] HTTP {status}: {body}") + return 2 + if not isinstance(body, dict) or "error" in body: + print(f"[submit] workflow rejected: {body}") + return 2 + prompt_id = body.get("prompt_id") + print(f"[submit] prompt_id={prompt_id} number={body.get('number')}") + + deadline = time.time() + POLL_TIMEOUT_S + out_dir = Path(args.output_dir or ".") + while True: + if time.time() > deadline: + print(f"[poll] timed out after {POLL_TIMEOUT_S}s, prompt_id={prompt_id} still running", file=sys.stderr) + return 3 + time.sleep(POLL_INTERVAL_S) + status, hist = http_json("GET", f"/history/{prompt_id}") + if status != 200 or not isinstance(hist, dict): + continue + entry = hist.get(prompt_id) + if not entry: + continue + st = (entry.get("status") or {}).get("status_str") + if st not in (None,): + print(f"[poll] status={st}") + if st == "success": + outputs = entry.get("outputs") or {} + saved = [] + for node_out in outputs.values(): + for media in node_out.get("images", []) + node_out.get("gifs", []) + node_out.get("videos", []): + fname = media.get("filename") + sub = media.get("subfolder", "") + typ = media.get("type", "output") + qs = urllib.parse.urlencode({"filename": fname, "subfolder": sub, "type": typ}) + dest = out_dir / fname + s = http_download(f"/view?{qs}", dest) + if s == 200: + saved.append(str(dest)) + if not saved: + print("[poll] success but no output files reported (workflow may not have a Save node)") + return 0 + print("[poll] saved:") + for p in saved: + print(f" {p}") + return 0 + if st in ("error", "cancelled"): + print(f"[poll] {st}: {entry}") + return 4 + + +def main() -> int: + ap = argparse.ArgumentParser(description="Drive a local ComfyUI server.") + ap.add_argument("--url", help="Override COMFYUI_URL for this invocation only") + ap.add_argument("--workflow", help="Path to a workflow JSON file") + ap.add_argument("--prompt", help="Optional prompt override (replaces __PROMPT__ in the workflow)") + ap.add_argument("--output-dir", help="Directory to save generated outputs") + ap.add_argument("--filename", help="Filename to download (with --download)") + ap.add_argument("--subfolder", default="", help="Subfolder under ComfyUI output dir") + ap.add_argument("--folder-type", default="output", choices=["output", "input", "temp"]) + mode = ap.add_mutually_exclusive_group() + mode.add_argument("--probe", action="store_true", help="Health probe /system_stats") + mode.add_argument("--queue", action="store_true", help="Show running and pending queue") + mode.add_argument("--download", action="store_true", help="Download a file by name") + args = ap.parse_args() + + if args.url: + os.environ["COMFYUI_URL"] = args.url + + if args.probe: + return cmd_probe(args) + if args.queue: + return cmd_queue(args) + if args.download: + if not args.filename: + print("--download requires --filename", file=sys.stderr) + return 2 + return cmd_download(args) + return cmd_submit(args) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/antianqi/comfyui-studio/workflows/image-to-image.json b/plugins/antianqi/comfyui-studio/workflows/image-to-image.json new file mode 100644 index 0000000..4efde79 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/image-to-image.json @@ -0,0 +1,81 @@ +{ + "3": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any.safetensors" + } + }, + "4": { + "class_type": "CLIPLoader", + "inputs": { + "clip_name1": "any_clip.safetensors", + "type": "stable_diffusion" + } + }, + "10": { + "class_type": "LoraLoader", + "inputs": { + "model": ["3", 0], + "clip": ["4", 0], + "lora_name": "any_optional_lora.safetensors", + "strength_model": 0.85, + "strength_clip": 0.85 + } + }, + "20": { + "class_type": "LoadImage", + "inputs": { + "image": "input.png" + } + }, + "21": { + "class_type": "VAEEncode", + "inputs": { + "pixels": ["20", 0], + "vae": ["3", 2] + } + }, + "30": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["10", 1] + } + }, + "31": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift", + "clip": ["10", 1] + } + }, + "60": { + "class_type": "KSampler", + "inputs": { + "model": ["10", 0], + "positive": ["30", 0], + "negative": ["31", 0], + "latent_image": ["21", 0], + "seed": 42, + "steps": 25, + "cfg": 6.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 0.55 + } + }, + "70": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["60", 0], + "vae": ["3", 2] + } + }, + "80": { + "class_type": "SaveImage", + "inputs": { + "images": ["70", 0], + "filename_prefix": "comfyui_studio_i2i" + } + } +} diff --git a/plugins/antianqi/comfyui-studio/workflows/text-to-image.json b/plugins/antianqi/comfyui-studio/workflows/text-to-image.json new file mode 100644 index 0000000..8f04727 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/text-to-image.json @@ -0,0 +1,76 @@ +{ + "3": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any.safetensors" + } + }, + "4": { + "class_type": "CLIPLoader", + "inputs": { + "clip_name1": "any_clip.safetensors", + "type": "stable_diffusion" + } + }, + "10": { + "class_type": "LoraLoader", + "inputs": { + "model": ["3", 0], + "clip": ["4", 0], + "lora_name": "any_optional_lora.safetensors", + "strength_model": 0.85, + "strength_clip": 0.85 + } + }, + "30": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["10", 1] + } + }, + "31": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift", + "clip": ["10", 1] + } + }, + "50": { + "class_type": "EmptyLatentImage", + "inputs": { + "width": 1024, + "height": 1024, + "batch_size": 1 + } + }, + "60": { + "class_type": "KSampler", + "inputs": { + "model": ["10", 0], + "positive": ["30", 0], + "negative": ["31", 0], + "latent_image": ["50", 0], + "seed": 42, + "steps": 25, + "cfg": 6.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0 + } + }, + "70": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["60", 0], + "vae": ["3", 2] + } + }, + "80": { + "class_type": "SaveImage", + "inputs": { + "images": ["70", 0], + "filename_prefix": "comfyui_studio" + } + } +} From 832d1d6468ecc8763c9d1c957b673948b6c9af75 Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sat, 22 Aug 2026 09:13:05 +0800 Subject: [PATCH 2/7] Add preset workflows: selfie, mimicry, drama (selfie/drama Skill) Part 2 of the Plugin: the preset-workflow layer. Adds four generic workflow templates and one new Skill that were missing from the first commit. New preset workflows (all in workflows/, all generic with __PROMPT__/__TRIGGER__ substitution and no private content): - selfie-text-to-image.json portrait + face LoRA + ControlNet - selfie-mimicry.json i2i with IP-Adapter face pull + face LoRA - drama-first-frame.json first frame for short drama, two LoRA slots - drama-image-to-video.json image-to-video, distilled LTX class New / updated Skills: - comfyui-character (expanded) now covers both the selfie and mimicry presets, with the 4-module prompt structure and per-knob tuning tables - comfyui-drama (new) describes the 7-stage short drama pipeline and explains the boundary between what the Plugin ships (stages 4 and 5) and what the user runs in their own environment (TTS, Excel, FFmpeg) - comfyui-studio routing table updated to point to the new presets Validated: - npm run check passes for this plugin - All four new workflow JSONs were POSTed to a real ComfyUI 8188 instance and accepted (errors at validation stage are expected: the user must install custom nodes such as IPAdapterModelLoader and the specific models; this PR ships the structure, not the assets) --- plugins/antianqi/comfyui-studio/README.md | 69 ++++++-- .../skills/comfyui-character/SKILL.md | 162 +++++++++++------- .../skills/comfyui-drama/SKILL.md | 117 +++++++++++++ .../skills/comfyui-studio/SKILL.md | 42 +++-- .../workflows/drama-first-frame.json | 79 +++++++++ .../workflows/drama-image-to-video.json | 80 +++++++++ .../workflows/selfie-mimicry.json | 98 +++++++++++ .../workflows/selfie-text-to-image.json | 106 ++++++++++++ 8 files changed, 657 insertions(+), 96 deletions(-) create mode 100644 plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md create mode 100644 plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json create mode 100644 plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json create mode 100644 plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json create mode 100644 plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json diff --git a/plugins/antianqi/comfyui-studio/README.md b/plugins/antianqi/comfyui-studio/README.md index 9b84da4..f3473b5 100644 --- a/plugins/antianqi/comfyui-studio/README.md +++ b/plugins/antianqi/comfyui-studio/README.md @@ -1,9 +1,11 @@ # ComfyUI Studio -A generic, dependency-free toolkit for driving a local ComfyUI server. Submit workflows, poll -the queue, download generated images, and build consistent characters with a self-trained -LoRA. Bundles three Skills and a stdio MCP server; ships zero native binaries, zero installers, -and zero third-party dependencies. +A generic, dependency-free toolkit for driving a local ComfyUI server. **Two parts in one +Plugin**: (1) the basic control layer for submitting workflows, polling the queue, and +downloading outputs, and (2) three preset workflow templates that solve common recurring +tasks (consistent-character selfie, reference-image mimicry, and short-drama pipeline). +Bundles four Skills and a stdio MCP server; ships zero native binaries, zero installers, and +zero third-party dependencies. ## The problem @@ -19,6 +21,16 @@ This Plugin does the first without paying the second cost. The same three primit exposed through both a tiny Python CLI and a ~200-line stdio MCP server, so the user picks whichever entry point fits their host agent. +For users who have already trained a character LoRA and want to ship a recurring series of +selfies, the Plugin also ships two preset workflows (`selfie-text-to-image.json`, +`selfie-mimicry.json`) that wire a LoRA, a face ControlNet, and a portrait-oriented latent +into one submission — no boilerplate, no edits to the workflow. + +For users who want to produce a multi-shot short drama with two consistent characters, the +Plugin ships the first-frame and image-to-video workflow presets plus a documented 7-stage +pipeline. The user provides the LoRAs, the video checkpoint, and the TTS pipeline; the +Plugin provides the templates. + ## Try it After installing the Plugin into MiniMax Code: @@ -30,8 +42,8 @@ light, photorealistic" and report the saved image path. ``` Expected result: the agent calls `submit_prompt` (or `submit_workflow.py`), polls the queue -until the run finishes, downloads the image, and tells the user where it was saved. The end-to-end -walkthrough is in [`examples/minimal-run.md`](examples/minimal-run.md). +until the run finishes, downloads the image, and tells the user where it was saved. The +end-to-end walkthrough is in [`examples/minimal-run.md`](examples/minimal-run.md). ## What you get @@ -44,18 +56,24 @@ comfyui-studio/ ├── README.md # This file ├── skills/ │ ├── comfyui-studio/ # Routing layer: which sibling Skill applies -│ ├── comfyui-workflow/ # Submit / poll / download +│ ├── comfyui-workflow/ # Submit / poll / download (the basic transport) │ │ ├── SKILL.md │ │ ├── references/api-reference.md │ │ └── scripts/submit_workflow.py -│ └── comfyui-character/ # Consistent characters with a self-trained LoRA -│ ├── SKILL.md -│ └── references/ -│ ├── prompt-patterns.md -│ └── lora-guide.md -├── workflows/ +│ ├── comfyui-character/ # Consistent character generation (selfie + mimicry presets) +│ │ ├── SKILL.md +│ │ └── references/ +│ │ ├── prompt-patterns.md +│ │ └── lora-guide.md +│ └── comfyui-drama/ # 7-stage short drama pipeline (first-frame + image-to-video presets) +│ └── SKILL.md +├── workflows/ # Six preset workflow JSONs (generic, no private content) │ ├── text-to-image.json -│ └── image-to-image.json +│ ├── image-to-image.json +│ ├── selfie-text-to-image.json # preset: portrait + face LoRA + ControlNet +│ ├── selfie-mimicry.json # preset: IP-Adapter + face LoRA +│ ├── drama-first-frame.json # preset: two LoRA slots for a two-character drama +│ └── drama-image-to-video.json # preset: distilled LTX-class video model ├── examples/ │ ├── README.md │ └── minimal-run.md # 5-minute end-to-end walkthrough @@ -64,12 +82,18 @@ comfyui-studio/ └── troubleshooting.md ``` -## Three Skills, one shared transport +## Four Skills, two transport layers - **`comfyui-studio`** — entry point. Reads the user's intent and routes to the right sibling. -- **`comfyui-workflow`** — submits workflow JSON, polls the queue, downloads outputs. +- **`comfyui-workflow`** — submits workflow JSON, polls the queue, downloads outputs. This is + the basic transport that every other Skill uses. - **`comfyui-character`** — produces a consistent character across many generations, using a - LoRA the user trained themselves and placed in ComfyUI's `models/loras/`. + LoRA the user trained themselves. Ships two preset workflows: `selfie-text-to-image.json` + for prompt-driven portrait generation, and `selfie-mimicry.json` for reference-image + mimicry via IP-Adapter. +- **`comfyui-drama`** — 7-stage short-drama pipeline: storyboard → TTS → script refinement → + first-frame image (`drama-first-frame.json`) → image-to-video clip + (`drama-image-to-video.json`) → subtitle burn → audio/video assembly. Read [`skills/comfyui-studio/SKILL.md`](skills/comfyui-studio/SKILL.md) for the routing table. Each sibling's `SKILL.md` is the actual implementation guide. @@ -93,7 +117,12 @@ needing Node or any MCP plumbing. - A running ComfyUI server reachable at `COMFYUI_URL` (default `http://127.0.0.1:8188`). - At least one checkpoint model in ComfyUI's `models/checkpoints/`. -- For `comfyui-character`: a LoRA you trained, dropped into `models/loras/`. +- For `comfyui-character`: a LoRA you trained, dropped into `models/loras/`. Optionally a + face ControlNet (for `selfie-text-to-image.json`) and an IP-Adapter model (for + `selfie-mimicry.json`). +- For `comfyui-drama`: two character LoRAs (for two-character dramas), a distilled LTX-2.3 + class video checkpoint, a TTS pipeline of the user's choice, and FFmpeg for the + assembly stages. - Python 3.10+ if you use the script. Node 18+ if you use the MCP server. - No accounts. No paid services. No telemetry. No native binaries. No installers. @@ -114,6 +143,10 @@ needing Node or any MCP plumbing. other identity asset. Every character is user-supplied. - It does not install packages, run post-install hooks, or download native binaries. - It does not call any cloud service or third-party API. +- It does not run TTS, edit spreadsheets, burn subtitles, or assemble audio/video. The + `comfyui-drama` Skill documents the full 7-stage pipeline but only ships ComfyUI + workflow templates for the two image-side stages; the other stages are user pipeline + steps that the Plugin deliberately does not assume. ## License diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md index caa883c..1056b59 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md @@ -1,26 +1,105 @@ --- name: comfyui-character -description: Build a self-trained LoRA for a stable character and use it inside a ComfyUI workflow to keep the character consistent across many generations. Use this Skill when the user wants a recurring character (a brand mascot, a video series character, a presenter), needs cross-image consistency, or asks how to make a LoRA and where to put it. +description: Produce consistent character images (selfies and reference-image mimicry) using a self-trained LoRA inside ComfyUI. Use this Skill when the user wants a recurring character (a brand mascot, a presenter, a series character), needs cross-image consistency, asks how to make a LoRA and where to put it, or wants to generate "a selfie of MY character" from a reference photo. Ships two preset workflow templates: `workflows/selfie-text-to-image.json` and `workflows/selfie-mimicry.json`. --- # ComfyUI Character +Two complementary presets for producing consistent character images with a self-trained LoRA. The Plugin does **not** distribute any character LoRA, face embedding, voice sample, or other private identity asset. Every character this Skill produces is one the user trained themselves -and dropped into ComfyUI's standard `models/loras/` directory. This Skill is about the workflow -patterns that make a user-trained LoRA actually deliver consistency — what to put in the -training set, what to put in the prompt at inference, and what mistakes to avoid. - -## Scope and assumptions - -- The user has a stable concept of who the character is. If they do not, help them define it - first (name, single defining trait, recurring outfit). Without that, no LoRA will save them. -- The user (or a model-trainer agent they trust) has trained or will train a LoRA. This Skill - does not run LoRA training; it covers the inference side and the prompt / workflow patterns - that consume a LoRA correctly. -- The LoRA file lives at `models/loras/.safetensors` inside the ComfyUI - install. The Skill does not dictate the file name; the user does, when they register the LoRA - in their workflow. +and dropped into ComfyUI's standard `models/loras/` directory. + +## Two presets, one goal + +| Preset | Trigger intent | Workflow | +|---|---|---| +| **生成自拍 (selfie generation)** | "Make a selfie of my character" — character fully defined by the LoRA, prompt sets pose and scene | `workflows/selfie-text-to-image.json` | +| **模仿自拍 (selfie mimicry)** | "Use this photo as a reference — make a similar image of my character" — IP-Adapter pulls pose / lighting / composition from the reference, LoRA keeps the character stable | `workflows/selfie-mimicry.json` | + +Both workflows share the same contract: + +- `--prompt` (or the `__PROMPT__` marker in the workflow) supplies the 4-module prompt + structure documented in `references/prompt-patterns.md`. +- The LoRA slot expects a single file the user placed in `models/loras/`. The Plugin never + renames, copies, or symlinks the file. +- Resolution is portrait by default (832×1216, roughly 11:16). Change `EmptyLatentImage` width + / height for landscape, square, or other ratios. +- The seed is fixed at `42` for reproducibility. Change it for variation. + +## 生成自拍 — `selfie-text-to-image.json` + +Pipeline: Checkpoint → LoraLoader (single face LoRA slot) → ControlNet face → KSampler. + +The bundled workflow uses a face-oriented ControlNet to keep the face shape / position stable +across pose and lighting changes. If you do not have a face ControlNet installed, set +`strength` on both `ControlNetApply` nodes to `0` and let the LoRA do the work; the workflow +will still produce a portrait, just with weaker face position lock. + +### Step-by-step + +1. **Place your face LoRA** at `/models/loras/your_face_lora.safetensors`. Edit the + workflow's `LoraLoader` node to set `lora_name` to this exact filename. ComfyUI shows the + file in its dropdown only after a `Refresh` click or a restart. +2. **Place a face ControlNet** at `/models/controlnet/any_face_controlnet.safetensors` + and set `control_net_name` accordingly. (Skip this step if you do not have one — set both + `ControlNetApply` strengths to 0.) +3. **Drop a face reference image** named `face_reference.png` into the ComfyUI input folder, + OR change `LoadImage.image` to point at the file you want. +4. **Compose the 4-module prompt** (see `references/prompt-patterns.md`): + ``` + __TRIGGER__, , , + ``` + Pass the prompt via `--prompt` on the CLI. The `__PROMPT__` marker in the workflow gets + replaced automatically. +5. **Submit** via the Python script or the MCP server (see `comfyui-workflow/SKILL.md` for + the transport). The script will poll until the run finishes and download the saved image + from `outputs/`. + +### Strengths and what they do + +| Knob | Default | What it does | When to change it | +|---|---|---|---| +| `LoraLoader.strength_model` | 0.85 | How strongly the LoRA controls the model | Lower to 0.6–0.7 if the character overpowers the scene; raise to 0.9–1.0 if identity drifts | +| `LoraLoader.strength_clip` | 0.85 | How strongly the LoRA controls the text encoder | Usually keep the same as `strength_model` | +| `ControlNetApply.strength` (positive) | 0.75 | Face position lock strength | 0.5 for looser shots, 0.9 for strict identity | +| `ControlNetApply.strength` (negative) | 0.6 | Negative prompt's face lock (used to suppress the wrong face) | Usually 0.4–0.6 | +| `KSampler.steps` | 28 | Diffusion steps | 20 for fast drafts, 35–40 for final-quality | +| `KSampler.cfg` | 6.0 | Prompt adherence | 5 for more creative, 7–8 for stricter prompt following | + +## 模仿自拍 — `selfie-mimicry.json` + +Pipeline: Checkpoint → LoraLoader → IP-Adapter (pulls reference) → KSampler. + +The bundled workflow uses an IP-Adapter FaceID model to pull the **face** from the reference +image while the LoRA keeps the **character identity** stable. This is the right tool when the +user has a reference photo (their own face, a stock photo, a frame from a video) and wants a +new generation that uses the same pose, lighting, and composition with their character. + +### Step-by-step + +1. **Place your face LoRA** at `/models/loras/your_face_lora.safetensors` and set + `LoraLoader.lora_name` to the exact filename. +2. **Place the IP-Adapter model** at `/models/ipadapter/ip-adapter-faceid-portrait.bin` + and set `IPAdapterModelLoader.ipadapter_file` accordingly. (If you use a different IP-Adapter + file, e.g. SDXL vs SD 1.5, change the filename and verify the base checkpoint matches.) +3. **Drop a reference face image** named `reference_face.png` into the ComfyUI input folder, + OR change `LoadImage.image` to point at the file you want. The reference image should + contain a clearly visible face — IP-Adapter FaceID will not work on a landscape with no + face. +4. **Compose the prompt**. The IP-Adapter is doing the heavy lifting on pose and lighting, so + the prompt can be lighter: `, , ` is usually enough. + You do not need to describe the pose — the reference is providing it. +5. **Submit** the same way as the selfie workflow. + +### Strengths and what they do + +| Knob | Default | What it does | When to change it | +|---|---|---|---| +| `LoraLoader.strength_model` | 0.85 | LoRA weight | See selfie workflow above | +| `IPAdapterApply.weight` | 0.7 | How strongly the reference controls the image | 0.4–0.5 for looser mimicry, 0.85 for near-clone | +| `IPAdapterApply.noise` | 0.05 | Slight variation from the reference | 0.0 for exact mimicry, 0.1–0.2 for "inspired by" | +| `IPAdapterApply.weight_type` | "linear" | How the weight is applied across denoising steps | "ease in" / "ease out" / "reverse in-out" available; default works for most | ## Why most "consistent character" attempts fail @@ -34,17 +113,7 @@ A LoRA is a strong prior on identity, but it is not magic. Three things derail c 3. **Wrong sampler / scheduler / CFG for the base model.** Some samplers oversmooth identity features. See `references/prompt-patterns.md` for the rules of thumb. -## The three-step recipe - -### Step 1 — Define the character with one trigger word - -The user picks a single token (or a short stable phrase) that the LoRA is trained to respond to. -Examples: `mascotv1`, `presenter_anna`, `mr_noodle_v2`. This token becomes the first word of -every inference prompt. It must not collide with a common word the base model already knows -well (so not `cat`, `woman`, `office`) and must not contain spaces or punctuation the base -model's tokenizer will split unexpectedly. - -### Step 2 — Compose the prompt in 4 modules, in this order +## The 4-module prompt structure (summary) From `references/prompt-patterns.md`: @@ -52,42 +121,8 @@ From `references/prompt-patterns.md`: , , , ``` -- **Identity block** — the small set of features the LoRA was trained on (face shape, hair, - signature accessory). Keep it short; the LoRA is doing the heavy lifting. -- **Outfit block** — what the character is wearing in *this* generation. The character wears - different outfits across the series; do not pin a single outfit in the LoRA. -- **Pose + scene block** — what is happening in the image. This is the part that changes most. - -Anything that does not fit one of these four modules is noise. Strip adjectives like -"beautiful", "stunning", "professional" — they make the base model override the LoRA. See -`references/prompt-patterns.md` for the full 8-rule prompt checklist. - -### Step 3 — Plug the LoRA into the workflow - -The bundled `workflows/text-to-image.json` includes a `LoraLoader` node wired into a -`CheckpointLoader` so the user can: - -1. Set `lora_name` to the file they placed under `models/loras/`. -2. Set `strength_model` to a value between `0.6` and `1.0`. Start at `0.85`; lower it if the - character overpowers the scene, raise it if identity drifts. -3. Leave `strength_clip` at the workflow's default unless the user has measured otherwise. -4. Do **not** stack multiple character LoRAs at full strength — they fight each other. - -If the user's ComfyUI install uses a different workflow format (e.g. Power Lora Loader), they -adapt the same four values; the Skill does not require any particular node. - -## Verifying consistency - -After producing 5–10 reference images, open them in a grid view and check: - -- Face shape and signature features are stable across lighting and pose changes. -- Outfit changes per the prompt, not per the LoRA. -- Pose and scene obey the prompt, not the LoRA's training set. - -If identity drifts, the usual cause in order of frequency: (1) trigger word is in the wrong -position, (2) `strength_model` is too low, (3) base model was changed since training, (4) too -many identity keywords fighting the trigger. Re-run the prompt-patterns checklist and try one -fix at a time. +For the mimicry workflow, drop the `pose + scene` block — the reference image is supplying it. +For the selfie workflow, keep all four. ## What this Skill does not do @@ -101,6 +136,9 @@ fix at a time. - ComfyUI running locally (see `comfyui-workflow/SKILL.md` for the transport). - A character LoRA the user trained, dropped under `models/loras/`. +- For the mimicry workflow: an IP-Adapter model (FaceID for portraits, SD 1.5 or SDXL + depending on your base checkpoint). +- For the selfie workflow (optional but recommended): a face-oriented ControlNet. - A base checkpoint compatible with the LoRA's training resolution. ## License diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md new file mode 100644 index 0000000..8efc1e7 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md @@ -0,0 +1,117 @@ +--- +name: comfyui-drama +description: Generate a complete short drama video from a storyboard: first-frame image, image-to-video clip, TTS dubbing, subtitle burn, audio/video assembly. Use this Skill when the user wants a multi-shot, multi-character video with consistent character identity — ancient costume, modern urban, fantasy, or any other genre. Ships two preset workflow templates: `workflows/drama-first-frame.json` and `workflows/drama-image-to-video.json`. +--- + +# ComfyUI Drama + +A 7-stage pipeline for producing a short drama video from a storyboard. The Plugin ships +two preset ComfyUI workflows (first-frame and image-to-video) and describes the rest of the +pipeline. The Plugin does not distribute any character LoRA, voice sample, or video model — +every asset is user-supplied. + +## The 7 stages + +| Stage | Output | Tool | Notes | +|---|---|---|---| +| 1. Storyboard | `desktop/_分镜.xlsx` | User's spreadsheet editor or any storyboard tool | 6-column format: 序号 / 时长 / 画面 / 图片 prompt / 视频 prompt / 台词 | +| 2. TTS dubbing | `_wav/00X_.wav` | User's TTS pipeline (Edge TTS, IndexTTS2, Qwen3-VoiceDesign, etc.) | **TTS comes BEFORE images** — the actual measured duration is the only truth for shot length | +| 3. Script refinement | Updated Excel with measured shot lengths | User's scripting tool | Re-cut shots whose actual TTS length differs from the script estimate by > 0.5s | +| 4. First-frame image | `drama_frame_00001_.png` per shot | **This Plugin's `workflows/drama-first-frame.json`** | One image per shot, two-character LoRA slots | +| 5. Image-to-video clip | `drama_clip_00001_.mp4` per shot | **This Plugin's `workflows/drama-image-to-video.json`** | One video per shot, 8 GB VRAM friendly | +| 6. Subtitle burn | Subtitled video per shot | FFmpeg `drawtext` filter | Source Han Sans CN, 28pt, white + black 2px stroke | +| 7. Audio/video assembly | Final drama MP4 | FFmpeg `-c:v copy -c:a aac` + concat demuxer | One output MP4 per drama | + +The Plugin provides templates and preset workflows for stages 4 and 5. The user runs stages +1, 2, 3, 6, and 7 with their own tools (Excel, TTS pipeline, FFmpeg). This boundary is +intentional: the Plugin does not assume a particular TTS vendor or spreadsheet format. + +## Two preset workflows + +### `workflows/drama-first-frame.json` + +A first-frame image generator with **two LoRA slots** (one per character in a two-character +drama). For single-character dramas, set the second slot's LoRA file to a placeholder +LoRA with `strength = 0`; for two-character dramas, fill both slots. + +Pipeline: Checkpoint → LoraLoader (character A) → LoraLoader (character B) → KSampler. + +| Knob | Default | What it does | +|---|---|---| +| `LoraLoader[0].strength_model` (character A) | 0.8 | Identity weight for character A | +| `LoraLoader[1].strength_model` (character B) | 0.8 | Identity weight for character B | +| `EmptyLatentImage.width` / `height` | 1280 / 720 | 16:9 landscape. Change for vertical (9:16) or square (1:1) | +| `KSampler.steps` | 8 | Distilled-model default; raise to 20 for a non-distilled base | +| `KSampler.cfg` | 1.5 | Distilled-model default; raise to 6 for a non-distilled base | + +**Critical prompt rule**: the image prompt describes the **first frame**, not the shot's +action sequence. A shot where character A hands a sword to character B should prompt the +moment the hands are meeting, not the full handoff motion. The video prompt is responsible +for the motion. + +### `workflows/drama-image-to-video.json` + +An image-to-video generator for a single shot. Takes a first-frame image (from stage 4) and +the shot's video prompt (Excel column "视频 prompt"). Outputs a single MP4 per shot. + +Pipeline: Checkpoint → LoraLoader (optional motion LoRA) → LoadImage (first frame) → KSampler +→ VHS_VideoCombine. + +| Knob | Default | What it does | +|---|---|---| +| `LoadImage.image` | "first_frame.png" | The first-frame image to animate. The user must change this per shot, or upload each frame to ComfyUI with a predictable filename | +| `VHS_VideoCombine.frame_rate` | 24 | Output FPS. Pair with RIFE interpolation to reach 60 | +| `VHS_VideoCombine.format` | "video/h264-mp4" | Output container/codec | +| `KSampler.steps` | 8 | Distilled LTX default; raise for non-distilled | +| `KSampler.cfg` | 1.0 | Distilled LTX default | + +**Critical prompt rule**: the video prompt describes motion only — camera move, character +animation, environment dynamics, expression changes. It does **not** describe the visible +image (the model can see the first frame and already knows what is in it). The shot's dialogue +goes in a separate `dialogue` field, not the `concept` field. + +### How to run a shot + +1. **For each shot in the storyboard**, fill the workflow's `__PROMPT__` marker with the + Excel "视频 prompt" column value. +2. **For each shot, set the first-frame image** by changing `LoadImage.image` to the PNG + produced by stage 4. Use predictable filenames (`shot_001.png`, `shot_002.png`, ...) and + the CLI's `--filename` flag to drive the workflow. +3. **Submit each shot** via the Python script or MCP server. The script will poll until done + and download the resulting MP4. + +A bulk run across all shots is straightforward but each shot is one ComfyUI submission +(because each shot has a different first-frame image and a different prompt). The Plugin does +not implement a batch driver; if you need one, use a wrapper loop in your own tool. + +## A note on long jobs and VRAM + +A distilled LTX-class video model running at 1280×720 with 24 FPS for a 4-second clip +finishes in roughly 2–5 minutes on an 8 GB GPU. A non-distilled 22B video model on the same +shot can take 30–60 minutes. Long jobs (over 30 minutes) should be watched with `nvidia-smi`; +the Plugin does not ship a VRAM watchdog. If your GPU is lost mid-run, the standard +recovery is to interrupt the queue and restart ComfyUI. + +## What this Skill does not do + +- It does not run LoRA training. Training is its own project; the Skill only consumes a LoRA + the user already has. +- It does not bundle or distribute any LoRA, model, voice sample, face embedding, or any + other identity asset. Every character and voice is user-supplied. +- It does not run TTS, edit Excel, burn subtitles, or assemble audio/video. Those are user + pipeline steps that the Plugin deliberately does not assume. +- It does not ship a VRAM watchdog or auto-retry on failure. Long jobs need a human + watching. + +## Requirements + +- ComfyUI running locally (see `comfyui-workflow/SKILL.md` for the transport). +- A first-frame checkpoint and (optionally) two character LoRAs the user trained, dropped + under `models/checkpoints/` and `models/loras/`. +- A video checkpoint (distilled LTX-2.3 or similar) under `models/checkpoints/`. +- A TTS pipeline (any of the user's choice) for stage 2. +- FFmpeg for stages 6 and 7. + +## License + +Apache-2.0. See [LICENSE](../../LICENSE). diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md index 0de79fb..f40ed0a 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md @@ -6,27 +6,33 @@ description: Drive a local ComfyUI 8188 server for image and short-clip generati # ComfyUI Studio A generic toolkit for driving a local ComfyUI server (default `http://127.0.0.1:8188`). It bundles -three Skills that work together: +four Skills and two transport layers that work together. -- **comfyui-studio** (this file): the routing layer. Use it to decide which sibling Skill applies. -- **comfyui-workflow**: submitting workflows, polling the queue, downloading outputs. -- **comfyui-character**: producing a consistent character across many generations with a self-made - LoRA, including the prompt and LoRA-training patterns that make consistency work. +## Two parts, one Plugin -The Plugin also ships a dependency-free stdio MCP server (see `mcp.json` + `server.mjs`) that exposes -the same three primitives to any MCP-capable agent. +| Part | What it covers | Where | +|---|---|---| +| **Basic control** | The transport between the agent and a local ComfyUI server. Submit workflows, poll the queue, download outputs. | `comfyui-workflow` Skill + the stdio MCP server + the Python CLI | +| **Preset workflows** | Three opinionated workflow templates that solve common recurring tasks: a consistent character selfie, a reference-image mimicry, and a 7-stage short drama pipeline. | `comfyui-character` and `comfyui-drama` Skills + the `workflows/*.json` presets | + +The Plugin also ships a dependency-free stdio MCP server (see `mcp.json` + `server.mjs`) that +exposes the same basic-control primitives (`submit_prompt`, `check_queue`, `get_image`) to any +MCP-capable agent. ## When to use which Skill | User intent | Use | |---|---| | "Submit this workflow" / "Check the queue" / "Download the image" | **comfyui-workflow** | -| "Make a character that looks the same every time" / "Train a LoRA" / "Use my LoRA to keep this character stable" | **comfyui-character** | -| "Use ComfyUI to ..." without further detail | **comfyui-studio** — read the user's intent, then route to one of the two siblings | +| "Make a selfie of my character" / "Generate a portrait using my LoRA" | **comfyui-character** (preset: `workflows/selfie-text-to-image.json`) | +| "Use this photo as a reference — make a similar selfie" | **comfyui-character** (preset: `workflows/selfie-mimicry.json`) | +| "Make a short drama" / "Generate a multi-shot video with these characters" | **comfyui-drama** (presets: `workflows/drama-first-frame.json` + `workflows/drama-image-to-video.json`) | +| "Use ComfyUI to ..." without further detail | **comfyui-studio** — read the user's intent, then route to one of the siblings | | "What models do I have?" / "Is ComfyUI running?" | **comfyui-workflow** — its first step is always a health probe | Do not duplicate behaviour: if you only need to submit a workflow, do not read this Skill, read -`comfyui-workflow/SKILL.md` directly. This Skill is the entry point for routing, not the implementation. +`comfyui-workflow/SKILL.md` directly. This Skill is the entry point for routing, not the +implementation. ## Required environment @@ -36,15 +42,19 @@ The Plugin assumes the user has: environment variable or the `--url` flag on the Python scripts. - At least one checkpoint model installed and reachable from ComfyUI. - For `comfyui-character`: a character LoRA the user trained themselves and dropped into - ComfyUI's `models/loras/` directory. The Plugin never distributes LoRAs or character weights. + ComfyUI's `models/loras/` directory. The Plugin never distributes LoRAs or character + weights. +- For `comfyui-drama`: two character LoRAs (for two-character dramas), a distilled video + checkpoint (LTX-2.3 class), and a TTS pipeline of the user's choice. The Plugin does **not** download or bundle model checkpoints, LoRAs, or any other binary asset. -Each Skill instructs the user to place their own files in the standard ComfyUI directory layout. +Each Skill instructs the user to place their own files in the standard ComfyUI directory +layout. ## Data and network -- Network access is local-only by default. The MCP server and the Python scripts both target the - address in `COMFYUI_URL`. No telemetry, no remote calls, no analytics. +- Network access is local-only by default. The MCP server and the Python scripts both target + the address in `COMFYUI_URL`. No telemetry, no remote calls, no analytics. - See `docs/security-notes.md` for the threat model and the data the Plugin handles. - See `docs/troubleshooting.md` for common failures (ComfyUI offline, missing model, OOM). @@ -59,8 +69,8 @@ on a windowsill, morning light, photorealistic" and report the saved image path. ``` Expected outcome: the agent confirms ComfyUI is reachable, submits the bundled -`workflows/text-to-image.json` workflow, polls until done, downloads the image, and tells you where -it was saved. +`workflows/text-to-image.json` workflow, polls until done, downloads the image, and tells you +where it was saved. ## License diff --git a/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json b/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json new file mode 100644 index 0000000..9b08174 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json @@ -0,0 +1,79 @@ +{ + "3": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any_anime_or_cinematic_checkpoint.safetensors" + } + }, + "10": { + "class_type": "LoraLoader", + "inputs": { + "model": ["3", 0], + "clip": ["3", 1], + "lora_name": "character_a_lora.safetensors", + "strength_model": 0.8, + "strength_clip": 0.8 + } + }, + "20": { + "class_type": "LoraLoader", + "inputs": { + "model": ["10", 0], + "clip": ["10", 1], + "lora_name": "character_b_lora.safetensors", + "strength_model": 0.8, + "strength_clip": 0.8 + } + }, + "30": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["20", 1] + } + }, + "31": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, blurry text, three people, four people, crowd", + "clip": ["20", 1] + } + }, + "40": { + "class_type": "EmptyLatentImage", + "inputs": { + "width": 1280, + "height": 720, + "batch_size": 1 + } + }, + "50": { + "class_type": "KSampler", + "inputs": { + "model": ["20", 0], + "positive": ["30", 0], + "negative": ["31", 0], + "latent_image": ["40", 0], + "seed": 42, + "steps": 8, + "cfg": 1.5, + "sampler_name": "euler", + "scheduler": "simple", + "denoise": 1.0 + } + }, + "60": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["50", 0], + "vae": ["3", 2] + } + }, + "70": { + "class_type": "SaveImage", + "inputs": { + "images": ["60", 0], + "filename_prefix": "drama_frame" + } + } +} diff --git a/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json b/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json new file mode 100644 index 0000000..ac3da94 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json @@ -0,0 +1,80 @@ +{ + "1": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any_ltx2_distilled_checkpoint.safetensors" + } + }, + "2": { + "class_type": "LoraLoader", + "inputs": { + "model": ["1", 0], + "clip": ["1", 1], + "lora_name": "any_motion_lora.safetensors", + "strength_model": 0.7, + "strength_clip": 0.7 + } + }, + "3": { + "class_type": "LoadImage", + "inputs": { + "image": "first_frame.png" + } + }, + "4": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["2", 1] + } + }, + "5": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "static, no movement, jitter, flicker, watermark, blurry, low quality", + "clip": ["2", 1] + } + }, + "10": { + "class_type": "EmptyLatentImage", + "inputs": { + "width": 1280, + "height": 720, + "batch_size": 1 + } + }, + "20": { + "class_type": "KSampler", + "inputs": { + "model": ["2", 0], + "positive": ["4", 0], + "negative": ["5", 0], + "latent_image": ["10", 0], + "seed": 42, + "steps": 8, + "cfg": 1.0, + "sampler_name": "euler", + "scheduler": "simple", + "denoise": 1.0 + } + }, + "30": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["20", 0], + "vae": ["1", 2] + } + }, + "40": { + "class_type": "VHS_VideoCombine", + "inputs": { + "frames": ["30", 0], + "frame_rate": 24, + "loop_count": 0, + "filename_prefix": "drama_clip", + "format": "video/h264-mp4", + "save_output": true, + "pingpong": false + } + } +} diff --git a/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json b/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json new file mode 100644 index 0000000..40fe686 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json @@ -0,0 +1,98 @@ +{ + "3": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any_portrait_checkpoint.safetensors" + } + }, + "4": { + "class_type": "CLIPLoader", + "inputs": { + "clip_name1": "any_clip.safetensors", + "type": "stable_diffusion" + } + }, + "10": { + "class_type": "LoraLoader", + "inputs": { + "model": ["3", 0], + "clip": ["4", 0], + "lora_name": "your_face_lora.safetensors", + "strength_model": 0.85, + "strength_clip": 0.85 + } + }, + "20": { + "class_type": "IPAdapterModelLoader", + "inputs": { + "ipadapter_file": "ip-adapter-faceid-portrait.bin" + } + }, + "30": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["10", 1] + } + }, + "31": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift, multiple people, two people, group", + "clip": ["10", 1] + } + }, + "40": { + "class_type": "LoadImage", + "inputs": { + "image": "reference_face.png" + } + }, + "50": { + "class_type": "IPAdapterApply", + "inputs": { + "ipadapter": ["20", 0], + "image": ["40", 0], + "weight": 0.7, + "noise": 0.05, + "weight_type": "linear" + } + }, + "60": { + "class_type": "EmptyLatentImage", + "inputs": { + "width": 832, + "height": 1216, + "batch_size": 1 + } + }, + "70": { + "class_type": "KSampler", + "inputs": { + "model": ["10", 0], + "positive": ["50", 0], + "negative": ["31", 0], + "latent_image": ["60", 0], + "seed": 42, + "steps": 25, + "cfg": 6.0, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0 + } + }, + "80": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["70", 0], + "vae": ["3", 2] + } + }, + "90": { + "class_type": "SaveImage", + "inputs": { + "images": ["80", 0], + "filename_prefix": "selfie_mimic" + } + } +} diff --git a/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json b/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json new file mode 100644 index 0000000..52e06b2 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json @@ -0,0 +1,106 @@ +{ + "3": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "any_portrait_checkpoint.safetensors" + } + }, + "4": { + "class_type": "CLIPLoader", + "inputs": { + "clip_name1": "any_clip.safetensors", + "type": "stable_diffusion" + } + }, + "10": { + "class_type": "LoraLoader", + "inputs": { + "model": ["3", 0], + "clip": ["4", 0], + "lora_name": "your_face_lora.safetensors", + "strength_model": 0.85, + "strength_clip": 0.85 + } + }, + "20": { + "class_type": "ControlNetLoader", + "inputs": { + "control_net_name": "any_face_controlnet.safetensors" + } + }, + "30": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "__PROMPT__", + "clip": ["10", 1] + } + }, + "31": { + "class_type": "CLIPTextEncode", + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift, multiple people, two people, group", + "clip": ["10", 1] + } + }, + "40": { + "class_type": "LoadImage", + "inputs": { + "image": "face_reference.png" + } + }, + "50": { + "class_type": "EmptyLatentImage", + "inputs": { + "width": 832, + "height": 1216, + "batch_size": 1 + } + }, + "60": { + "class_type": "ControlNetApply", + "inputs": { + "conditioning": ["30", 0], + "control_net": ["20", 0], + "image": ["40", 0], + "strength": 0.75 + } + }, + "61": { + "class_type": "ControlNetApply", + "inputs": { + "conditioning": ["31", 0], + "control_net": ["20", 0], + "image": ["40", 0], + "strength": 0.6 + } + }, + "70": { + "class_type": "KSampler", + "inputs": { + "model": ["10", 0], + "positive": ["60", 0], + "negative": ["61", 0], + "latent_image": ["50", 0], + "seed": 42, + "steps": 28, + "cfg": 6.0, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0 + } + }, + "80": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["70", 0], + "vae": ["3", 2] + } + }, + "90": { + "class_type": "SaveImage", + "inputs": { + "images": ["80", 0], + "filename_prefix": "selfie" + } + } +} From 7ddfbe1964adf11203f94dbacfa94378d94047c1 Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sat, 22 Aug 2026 09:21:07 +0800 Subject: [PATCH 3/7] Polish: desensitize selfie LoRAs, fix JSON booleans, restructure docs by 6 trigger scenarios MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - selfie-text-to-image.json + selfie-mimicry.json: replace real LoRA filenames (goudan_zimage_c1-st8000, MysticXXX-ZIB-v1) with generic placeholders (your_face_lora.safetensors, your_style_lora.safetensors) so the Plugin stays shareable across installs - drama-image-to-video.json: fix invalid JSON booleans (True/False -> true/false) accidentally introduced in the previous commit; strict JSON parsers now parse it cleanly - README.md + 3 SKILL.md: restructure the feature surface around 6 numbered trigger scenarios -- 1=生图 (selfie), 2=模仿 (mimicry), 3=改图 (edit), 4=融合 (fuse), 5=首帧 (drama first frame), 6=出片 (image-to-video). Plugin now reads cleanly as Part A (natural-language control) + Part B (6 preset workflows), with a model-boundary section that states the actual rule: models = real filenames on disk, LoRAs = generic placeholders --- plugins/antianqi/comfyui-studio/README.md | 264 +++++--- .../skills/comfyui-character/SKILL.md | 291 ++++++--- .../skills/comfyui-drama/SKILL.md | 170 +++-- .../skills/comfyui-studio/SKILL.md | 81 ++- .../workflows/drama-first-frame.json | 2 +- .../workflows/drama-image-to-video.json | 51 +- .../flux2-klein-image-edit-dual.json | 605 ++++++++++++++++++ .../workflows/flux2-klein-image-edit.json | 281 ++++++++ .../workflows/image-to-image.json | 81 --- .../workflows/selfie-mimicry.json | 533 +++++++++++++-- .../workflows/selfie-text-to-image.json | 79 +-- .../workflows/text-to-image.json | 76 --- 12 files changed, 1954 insertions(+), 560 deletions(-) create mode 100644 plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json create mode 100644 plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json delete mode 100644 plugins/antianqi/comfyui-studio/workflows/image-to-image.json delete mode 100644 plugins/antianqi/comfyui-studio/workflows/text-to-image.json diff --git a/plugins/antianqi/comfyui-studio/README.md b/plugins/antianqi/comfyui-studio/README.md index f3473b5..f3eed2a 100644 --- a/plugins/antianqi/comfyui-studio/README.md +++ b/plugins/antianqi/comfyui-studio/README.md @@ -1,131 +1,217 @@ # ComfyUI Studio -A generic, dependency-free toolkit for driving a local ComfyUI server. **Two parts in one -Plugin**: (1) the basic control layer for submitting workflows, polling the queue, and -downloading outputs, and (2) three preset workflow templates that solve common recurring -tasks (consistent-character selfie, reference-image mimicry, and short-drama pipeline). -Bundles four Skills and a stdio MCP server; ships zero native binaries, zero installers, and -zero third-party dependencies. +A generic, dependency-free toolkit for driving a local ComfyUI server. **Two parts in one Plugin**: +**A. a natural-language control layer** for submitting any ComfyUI workflow, polling the queue, and +downloading outputs, and **B. six preset workflow templates** that cover the most common recurring +tasks (selfie, mimicry, image edit, image fusion, drama first frame, drama image-to-video). +Bundles four Skills and a stdio MCP server. Ships zero native binaries, zero installers, zero +third-party dependencies. Apache-2.0. -## The problem +--- -Driving a local ComfyUI server from an agent requires either: +## The two parts -- Hand-rolling HTTP calls every time (`/prompt`, `/history`, `/view`...) and rewriting the - same boilerplate per workflow. -- Adopting a heavyweight MCP server that pulls in 50+ MB of native dependencies (the - `sharp` image library) and runs an install hook on first use — a non-starter for shared - environments. +| Part | What it does | Where to look | +|---|---|---| +| **A. Natural-language control** | The transport between an agent and a local ComfyUI server. Submit any workflow JSON, poll the queue, download outputs. The same three primitives are exposed through a Python CLI and a 0-dep stdio MCP server. | `comfyui-workflow` Skill + `server.mjs` + `scripts/submit_workflow.py` | +| **B. 6 preset workflows** | Six opinionated workflow JSONs for the most common ComfyUI tasks. Each is wired with prompt / image / LoRA markers so the agent can submit them with one call. | `comfyui-character` Skill (scenarios 1–4) + `comfyui-drama` Skill (scenarios 5–6) + the `workflows/*.json` files | -This Plugin does the first without paying the second cost. The same three primitives are -exposed through both a tiny Python CLI and a ~200-line stdio MCP server, so the user picks -whichever entry point fits their host agent. +Part A is the workhorse. Part B is six production-ready starting points built on top of Part A. -For users who have already trained a character LoRA and want to ship a recurring series of -selfies, the Plugin also ships two preset workflows (`selfie-text-to-image.json`, -`selfie-mimicry.json`) that wire a LoRA, a face ControlNet, and a portrait-oriented latent -into one submission — no boilerplate, no edits to the workflow. +--- -For users who want to produce a multi-shot short drama with two consistent characters, the -Plugin ships the first-frame and image-to-video workflow presets plus a documented 7-stage -pipeline. The user provides the LoRAs, the video checkpoint, and the TTS pipeline; the -Plugin provides the templates. +## Part B — the 6 preset scenarios + +Numbered by how a user typically says it. The number is the **trigger scenario**, not a model id — +the agent reads the user's intent and picks the matching number. + +| # | Trigger | What the user typically says | Preset workflow | One-line description | +|---|---|---|---|---| +| **1** | **生图** (generate an image) | "用我的角色画一张自拍" / "生成一张赛博朋克风格的照片" | `workflows/selfie-text-to-image.json` | Text + 2 LoRAs (face + style) → one character image | +| **2** | **模仿** (mimic an image) | "照着这张照片再画一张同款" / "模仿这张图的姿势和光线" | `workflows/selfie-mimicry.json` | Reference image + ControlNet + vision LLM reprompt + 2 LoRAs → one character image in the same pose/lighting | +| **3** | **改图** (edit an image) | "把背景换成办公室" / "衣服换成西装,光线调亮" | `workflows/flux2-klein-image-edit.json` | 1 reference image + prompt → edited image | +| **4** | **融合** (fuse two images) | "把这张图里的人物放到那张图里" / "让她穿上这件衣服,背景换成海边" | `workflows/flux2-klein-image-edit-dual.json` | 2 reference images + prompt → fused image ("the person from image 1 in the scene of image 2") | +| **5** | **首帧** (drama first frame) | "生成这部短剧第 3 镜的首帧画面" | `workflows/drama-first-frame.json` | 2-character LoRA slots + prompt → one 16:9 first-frame image | +| **6** | **出片** (image to video) | "把这个首帧做成 4 秒视频" | `workflows/drama-image-to-video.json` | 1 first-frame image + motion prompt → one MP4 clip | + +> **Routing rule of thumb**: scenarios 1–4 → `comfyui-character`; scenarios 5–6 → `comfyui-drama`; +> anything else ("use ComfyUI to ...", "submit this workflow", "check the queue") → `comfyui-workflow` +> (Part A). + +The detailed recipes (model stack, custom-node requirements, prompt structure, knobs) live in each +Skill's `SKILL.md`. The table here is the index. + +--- + +## Part A — natural-language control + +No matter what the user wants, the last three steps are always the same: + +1. **Submit a workflow** → `POST /prompt`, capture `prompt_id` +2. **Poll the queue** → `GET /history/` until terminal status +3. **Download outputs** → `GET /view?filename=...&subfolder=...&type=output` + +Two equivalent entry points. Pick whichever the host agent supports. + +### A1. Python CLI (no MCP required) + +```bash +# Health probe +python skills/comfyui-workflow/scripts/submit_workflow.py --probe + +# Submit any workflow (one of the bundled presets or the user's own) +python skills/comfyui-workflow/scripts/submit_workflow.py \ + --workflow plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json \ + --prompt "the same person sitting in an office chair, professional lighting" \ + --filename reference_face.png \ + --output-dir ./out +``` + +The script understands three marker conventions so you don't have to rewrite the workflow JSON +for every run: + +| Marker | Replaced with | +|---|---| +| `__PROMPT__` (in any `CLIPTextEncode.text` or `CR Text.text` field) | the value of `--prompt` | +| `__TRIGGER__` (in any `CR Text.text` field) | the value of `--trigger` (LoRA trigger word) | +| `__IMAGE1__` / `__IMAGE2__` (in any `LoadImage.image` field) | the value of `--filename` / `--filename2` | + +### A2. stdio MCP server (no Python required) + +`mcp.json` registers a 200-line stdio JSON-RPC server in plain Node stdlib. Zero npm dependencies. +The three tools mirror the Python script: + +| MCP tool | Mirrors CLI flag | What it does | +|---|---|---| +| `submit_prompt` | `--workflow` + `--prompt` | Submit a workflow JSON; return `prompt_id` | +| `check_queue` | `--queue` | Report running and pending counts and IDs | +| `get_image` | `--download` | Download a generated image by filename | + +If the host agent is MCP-aware, prefer this. If not, use the Python CLI — they are behaviourally +identical. + +--- + +## Model boundary (read this first) + +The Plugin draws a clean line between **public generation models** and **user-supplied LoRAs**. + +| Asset type | What you see in the workflow JSON | Why | +|---|---|---| +| **Public generation models** (checkpoints, VAE, CLIP, ControlNet, video models, vision LLMs) | **The actual filenames on disk in the reference ComfyUI install that built this Plugin.** | The workflow runs out of the box on the reference install. Edit the loader field to point at your own file on a different install. | +| **LoRAs** | **Generic placeholder names**: `your_face_lora.safetensors`, `your_style_lora.safetensors`, `character_a_lora.safetensors`, `character_b_lora.safetensors`, `any_motion_lora.safetensors`. | LoRAs are user-trained identity assets. The Plugin does not bundle or name anyone's private LoRAs. The user edits the `LoraLoader*.lora_name` field to point at their own file. | +| **Image inputs** | `__IMAGE1__` / `__IMAGE2__` | The Python CLI / MCP server replaces these at submit time. | +| **Prompt / trigger text** | `__PROMPT__` / `__TRIGGER__` | The Python CLI / MCP server replaces these at submit time. | + +The exact filenames / placeholders in the six bundled workflows: + +| # | Workflow | Checkpoint | CLIP / VAE | ControlNet / Vision LLM | LoRA placeholders | +|---|---|---|---|---|---| +| 1 | `selfie-text-to-image.json` | `Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors` | bundled with CheckpointLoader | — | `your_face_lora.safetensors` + `your_style_lora.safetensors` | +| 2 | `selfie-mimicry.json` | `ZIT-moodyPornMix_zitV10R1DPO_fp16.safetensors` | bundled with UNETLoader | `Z-Image-Fun-Controlnet-Union-2.1.safetensors` (ControlNet) + `Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf` (vision LLM) | same 2 LoRA placeholders | +| 3 | `flux2-klein-image-edit.json` | `flux-2-klein-base-9b-fp8.safetensors` | `qwen_3_8b_fp8mixed.safetensors` + `diffusion_pytorch_model.safetensors` | — | (no LoRA) | +| 4 | `flux2-klein-image-edit-dual.json` | `flux-2-klein-base-9b-fp8.safetensors` | same CLIP/VAE as #3 | — | (no LoRA) | +| 5 | `drama-first-frame.json` | `Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors` | bundled with CheckpointLoader | — | `character_a_lora.safetensors` + `character_b_lora.safetensors` | +| 6 | `drama-image-to-video.json` | `10Eros_v1-fp8mixed_learned.safetensors` | `ltx-2.3_text_projection_bf16.safetensors` + `gemma_3_12B_it_fp8_scaled.safetensors` + `LTX23_video_vae_bf16.safetensors` | — | `any_motion_lora.safetensors` | + +If your ComfyUI install uses a different filename for any of these — for example, +`sd_xl_base_1.0.safetensors` instead of `Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors`, +or `ltx-2.3-distilled.safetensors` instead of `10Eros_v1-fp8mixed_learned.safetensors` — edit the +relevant `ckpt_name` / `clip_name1` / `clip_name2` / `vae_name` / `control_net_name` field in the +workflow JSON. ComfyUI will accept whatever file is on disk under the name you set. + +--- ## Try it After installing the Plugin into MiniMax Code: ```text -Use comfyui-studio to verify that my local ComfyUI is reachable, then submit the bundled -text-to-image workflow with the prompt "a tabby cat sleeping on a windowsill, morning -light, photorealistic" and report the saved image path. +Use comfyui-studio to verify that my local ComfyUI is reachable, then submit scenario 3 +(改图 / flux2-klein-image-edit) with the prompt "the same person sitting in an office +chair, the same outfit, professional lighting" and report the saved image path. ``` -Expected result: the agent calls `submit_prompt` (or `submit_workflow.py`), polls the queue -until the run finishes, downloads the image, and tells the user where it was saved. The -end-to-end walkthrough is in [`examples/minimal-run.md`](examples/minimal-run.md). +Expected result: the agent probes ComfyUI, submits the bundled +`workflows/flux2-klein-image-edit.json`, polls until done, downloads the image, and tells you +where it was saved. The end-to-end walkthrough is in [`examples/minimal-run.md`](examples/minimal-run.md). + +--- ## What you get ``` comfyui-studio/ -├── plugin.json # Plugin manifest (required) -├── mcp.json # Registers the stdio MCP server -├── server.mjs # 200-line zero-dep stdio JSON-RPC server -├── LICENSE # Apache-2.0 -├── README.md # This file +├── plugin.json # Plugin manifest (required) +├── mcp.json # Registers the stdio MCP server +├── server.mjs # 200-line zero-dep stdio JSON-RPC server +├── LICENSE # Apache-2.0 +├── README.md # This file ├── skills/ -│ ├── comfyui-studio/ # Routing layer: which sibling Skill applies -│ ├── comfyui-workflow/ # Submit / poll / download (the basic transport) +│ ├── comfyui-studio/ # Routing layer: numbers 1-6 + freeform → sibling Skill +│ ├── comfyui-workflow/ # The basic transport (Part A) │ │ ├── SKILL.md │ │ ├── references/api-reference.md │ │ └── scripts/submit_workflow.py -│ ├── comfyui-character/ # Consistent character generation (selfie + mimicry presets) +│ ├── comfyui-character/ # Scenarios 1, 2, 3, 4 (selfie + mimicry + Klein edits) │ │ ├── SKILL.md │ │ └── references/ │ │ ├── prompt-patterns.md │ │ └── lora-guide.md -│ └── comfyui-drama/ # 7-stage short drama pipeline (first-frame + image-to-video presets) +│ └── comfyui-drama/ # Scenarios 5, 6 (first-frame + image-to-video) │ └── SKILL.md -├── workflows/ # Six preset workflow JSONs (generic, no private content) -│ ├── text-to-image.json -│ ├── image-to-image.json -│ ├── selfie-text-to-image.json # preset: portrait + face LoRA + ControlNet -│ ├── selfie-mimicry.json # preset: IP-Adapter + face LoRA -│ ├── drama-first-frame.json # preset: two LoRA slots for a two-character drama -│ └── drama-image-to-video.json # preset: distilled LTX-class video model +├── workflows/ # The 6 preset workflow JSONs +│ ├── selfie-text-to-image.json # scenario 1: 生图 +│ ├── selfie-mimicry.json # scenario 2: 模仿 +│ ├── flux2-klein-image-edit.json # scenario 3: 改图 +│ ├── flux2-klein-image-edit-dual.json # scenario 4: 融合 +│ ├── drama-first-frame.json # scenario 5: 首帧 +│ └── drama-image-to-video.json # scenario 6: 出片 ├── examples/ │ ├── README.md -│ └── minimal-run.md # 5-minute end-to-end walkthrough +│ └── minimal-run.md # 5-minute end-to-end walkthrough └── docs/ ├── security-notes.md └── troubleshooting.md ``` -## Four Skills, two transport layers - -- **`comfyui-studio`** — entry point. Reads the user's intent and routes to the right sibling. -- **`comfyui-workflow`** — submits workflow JSON, polls the queue, downloads outputs. This is - the basic transport that every other Skill uses. -- **`comfyui-character`** — produces a consistent character across many generations, using a - LoRA the user trained themselves. Ships two preset workflows: `selfie-text-to-image.json` - for prompt-driven portrait generation, and `selfie-mimicry.json` for reference-image - mimicry via IP-Adapter. -- **`comfyui-drama`** — 7-stage short-drama pipeline: storyboard → TTS → script refinement → - first-frame image (`drama-first-frame.json`) → image-to-video clip - (`drama-image-to-video.json`) → subtitle burn → audio/video assembly. +--- -Read [`skills/comfyui-studio/SKILL.md`](skills/comfyui-studio/SKILL.md) for the routing -table. Each sibling's `SKILL.md` is the actual implementation guide. +## Four Skills, one routing table -## MCP server (optional) +The Plugin's `comfyui-studio` Skill is the entry point. It reads the user's intent and routes to +one of three siblings. Each sibling's `SKILL.md` is the actual implementation guide. -`mcp.json` registers a stdio MCP server. The server speaks JSON-RPC directly, with zero npm -dependencies; the only requirement is Node.js 18+ on the host's `PATH`. The three tools it -exposes mirror the Python CLI: +| Skill | Job | When | +|---|---|---| +| `comfyui-studio` | Routing: read the user's intent, pick the number 1–6 or fall through to freeform | Always start here when the user says "use ComfyUI to ..." | +| `comfyui-workflow` | The transport (Part A). Submit / poll / download any workflow JSON. | "Submit this workflow", "check the queue", "download the image" — or as a building block by the sibling Skills. | +| `comfyui-character` | Scenarios 1, 2, 3, 4. Character selfies (with LoRA) and Flux.2 Klein image edits. | "Make a selfie", "mimic this photo", "edit / fuse this image" | +| `comfyui-drama` | Scenarios 5, 6 + the 7-stage pipeline around them (storyboard → TTS → first frame → image-to-video → subtitle burn → assembly). | "Make a short drama", "generate the first frame", "turn this into a video clip" | -| Tool | Purpose | -|---|---| -| `submit_prompt` | Submit a workflow JSON; returns a `prompt_id` | -| `check_queue` | Report running and pending counts and IDs | -| `get_image` | Download a generated image by filename | +Read [`skills/comfyui-studio/SKILL.md`](skills/comfyui-studio/SKILL.md) for the full routing table. -If the host agent is not MCP-aware, the Python script gives you the same primitives without -needing Node or any MCP plumbing. +--- ## Requirements - A running ComfyUI server reachable at `COMFYUI_URL` (default `http://127.0.0.1:8188`). -- At least one checkpoint model in ComfyUI's `models/checkpoints/`. -- For `comfyui-character`: a LoRA you trained, dropped into `models/loras/`. Optionally a - face ControlNet (for `selfie-text-to-image.json`) and an IP-Adapter model (for - `selfie-mimicry.json`). -- For `comfyui-drama`: two character LoRAs (for two-character dramas), a distilled LTX-2.3 - class video checkpoint, a TTS pipeline of the user's choice, and FFmpeg for the - assembly stages. -- Python 3.10+ if you use the script. Node 18+ if you use the MCP server. +- The checkpoint / VAE / CLIP / ControlNet files referenced in each preset (see the model + boundary table above). Edit any loader field that does not match your install. +- For scenarios 1 and 2 (`comfyui-character` selfie workflows): a face LoRA and a style LoRA + in `models/loras/`. The Plugin does not bundle them; it uses placeholder names. +- For scenario 2 (`selfie-mimicry.json`): the custom-node pack list in + `skills/comfyui-character/SKILL.md` (controlnet_aux, LLaMA-CPP, ModelPatchLoader, etc.). +- For scenarios 3 and 4 (`comfyui-character` Klein workflows): the Flux.2 Klein 9B UNET, the + Qwen3 8B CLIP, and the bundled VAE. +- For scenarios 5 and 6 (`comfyui-drama`): two character LoRAs (for two-character dramas), an + LTX-Video checkpoint with its DualCLIP text encoders and VAE, a TTS pipeline of the user's + choice, and FFmpeg for the assembly stages. +- Python 3.10+ if you use the Python CLI. Node 18+ if you use the MCP server. - No accounts. No paid services. No telemetry. No native binaries. No installers. +--- + ## Data and network - The Plugin only talks to `COMFYUI_URL`. It makes no other network calls. @@ -135,18 +221,22 @@ needing Node or any MCP plumbing. - The Plugin does not log, persist, or transmit any user data. - See [`docs/security-notes.md`](docs/security-notes.md) for the full threat model. +--- + ## What the Plugin does not do -- It does not train LoRAs. Training is its own project; this Plugin only consumes a LoRA - the user already has. -- It does not bundle or distribute any LoRA, model, face embedding, voice sample, or any - other identity asset. Every character is user-supplied. +- It does not train LoRAs. Training is its own project; this Plugin only consumes a LoRA the + user already has. +- It does not bundle or distribute any LoRA, model, face embedding, voice sample, or any other + identity asset. Every character and voice is user-supplied. - It does not install packages, run post-install hooks, or download native binaries. - It does not call any cloud service or third-party API. - It does not run TTS, edit spreadsheets, burn subtitles, or assemble audio/video. The - `comfyui-drama` Skill documents the full 7-stage pipeline but only ships ComfyUI - workflow templates for the two image-side stages; the other stages are user pipeline - steps that the Plugin deliberately does not assume. + `comfyui-drama` Skill documents the full 7-stage pipeline but only ships ComfyUI workflow + templates for the two image-side stages (scenarios 5 and 6); the other stages are user + pipeline steps that the Plugin deliberately does not assume. + +--- ## License diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md index 1056b59..af40092 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-character/SKILL.md @@ -1,107 +1,199 @@ --- name: comfyui-character -description: Produce consistent character images (selfies and reference-image mimicry) using a self-trained LoRA inside ComfyUI. Use this Skill when the user wants a recurring character (a brand mascot, a presenter, a series character), needs cross-image consistency, asks how to make a LoRA and where to put it, or wants to generate "a selfie of MY character" from a reference photo. Ships two preset workflow templates: `workflows/selfie-text-to-image.json` and `workflows/selfie-mimicry.json`. +description: Produce consistent character images and reference-driven edits inside ComfyUI. Use this Skill for scenarios 1–4 from `comfyui-studio`'s trigger index: 1=生图 (selfie from text), 2=模仿 (mimic a reference image), 3=改图 (Flux.2 Klein single-image edit), 4=融合 (Flux.2 Klein dual-image fuse). Ships four preset workflow templates: `workflows/selfie-text-to-image.json`, `workflows/selfie-mimicry.json`, `workflows/flux2-klein-image-edit.json`, `workflows/flux2-klein-image-edit-dual.json`. --- # ComfyUI Character -Two complementary presets for producing consistent character images with a self-trained LoRA. -The Plugin does **not** distribute any character LoRA, face embedding, voice sample, or other -private identity asset. Every character this Skill produces is one the user trained themselves -and dropped into ComfyUI's standard `models/loras/` directory. +Four preset workflows for character-consistent selfies (with a user-trained LoRA) and +reference-image-driven edits (with Flux.2 Klein). The Plugin does **not** distribute any +character LoRA, face embedding, voice sample, or other private identity asset. Every character +this Skill produces is one the user trained themselves and dropped into ComfyUI's standard +`models/loras/` directory. -## Two presets, one goal +## The 4 trigger scenarios -| Preset | Trigger intent | Workflow | +These are scenarios 1–4 from `comfyui-studio`'s index. The number is the trigger, the workflow +is the implementation. + +| # | Trigger | User says | Preset workflow | Model stack at a glance | +|---|---|---|---|---| +| **1** | **生图** (generate) | "用我的角色画一张自拍" | `workflows/selfie-text-to-image.json` | Z-Image base + 2 LoRAs (face + style) | +| **2** | **模仿** (mimic) | "照着这张照片再画一张同款" | `workflows/selfie-mimicry.json` | ZIT-flavoured Z-Image finetune + 2 LoRAs + Z-Image Fun ControlNet + Qwen3.5 vision LLM | +| **3** | **改图** (edit) | "把背景换成办公室" | `workflows/flux2-klein-image-edit.json` | Flux.2 Klein 9B + Qwen3 8B + bundled VAE | +| **4** | **融合** (fuse) | "把这张图里的人物放到那张图里" | `workflows/flux2-klein-image-edit-dual.json` | same as #3, with 2x `LoadImage` inputs | + +Scenarios 1 and 2 share the **same two LoRA placeholders** (`your_face_lora.safetensors` for +the character's face, `your_style_lora.safetensors` for the artistic style). Scenarios 3 and 4 +do not use LoRAs at all — the character / object identity is carried by the reference image +itself, not by a LoRA. + +## Why scenarios 1 and 2 use different checkpoints + +The two selfie presets use **different checkpoints** for a reason: the direct selfie is the +"default look" (Z-Image base, 8 steps, the canonical character), while the mimicry preset is a +"reference-driven" workflow that needs a ZIT-flavoured Z-Image finetune plus a ControlNet and a +local vision LLM to do the imitation. They are not interchangeable — if you want scenario 1 +(direct generation), use the Z-Image base; if you want scenario 2 (mimicry), use the ZIT +finetune. + +--- + +## Scenario 1 — 生图 (generate) — `selfie-text-to-image.json` + +Pipeline: Checkpoint → `LoraLoaderModelOnly` (face) → `LoraLoaderModelOnly` (style) → KSampler. + +### Models + +| Field | Value (as written in the JSON) | Where it lives on disk | |---|---|---| -| **生成自拍 (selfie generation)** | "Make a selfie of my character" — character fully defined by the LoRA, prompt sets pose and scene | `workflows/selfie-text-to-image.json` | -| **模仿自拍 (selfie mimicry)** | "Use this photo as a reference — make a similar image of my character" — IP-Adapter pulls pose / lighting / composition from the reference, LoRA keeps the character stable | `workflows/selfie-mimicry.json` | - -Both workflows share the same contract: - -- `--prompt` (or the `__PROMPT__` marker in the workflow) supplies the 4-module prompt - structure documented in `references/prompt-patterns.md`. -- The LoRA slot expects a single file the user placed in `models/loras/`. The Plugin never - renames, copies, or symlinks the file. -- Resolution is portrait by default (832×1216, roughly 11:16). Change `EmptyLatentImage` width - / height for landscape, square, or other ratios. -- The seed is fixed at `42` for reproducibility. Change it for variation. - -## 生成自拍 — `selfie-text-to-image.json` - -Pipeline: Checkpoint → LoraLoader (single face LoRA slot) → ControlNet face → KSampler. - -The bundled workflow uses a face-oriented ControlNet to keep the face shape / position stable -across pose and lighting changes. If you do not have a face ControlNet installed, set -`strength` on both `ControlNetApply` nodes to `0` and let the LoRA do the work; the workflow -will still produce a portrait, just with weaker face position lock. - -### Step-by-step - -1. **Place your face LoRA** at `/models/loras/your_face_lora.safetensors`. Edit the - workflow's `LoraLoader` node to set `lora_name` to this exact filename. ComfyUI shows the - file in its dropdown only after a `Refresh` click or a restart. -2. **Place a face ControlNet** at `/models/controlnet/any_face_controlnet.safetensors` - and set `control_net_name` accordingly. (Skip this step if you do not have one — set both - `ControlNetApply` strengths to 0.) -3. **Drop a face reference image** named `face_reference.png` into the ComfyUI input folder, - OR change `LoadImage.image` to point at the file you want. -4. **Compose the 4-module prompt** (see `references/prompt-patterns.md`): +| `CheckpointLoaderSimple.ckpt_name` | `Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors` | `models/checkpoints/` | +| `LoraLoaderModelOnly[0].lora_name` (face) | `your_face_lora.safetensors` (placeholder — replace with your file) | `models/loras/` | +| `LoraLoaderModelOnly[1].lora_name` (style) | `your_style_lora.safetensors` (placeholder — replace with your file) | `models/loras/` | +| CLIP / VAE | bundled with the CheckpointLoader (outputs `[…, 1]` and `[…, 2]`) | — | + +### Submit + +1. **Place the two LoRAs** under `/models/loras/`. Edit the two + `LoraLoaderModelOnly.lora_name` fields in the JSON to point at your filenames if they differ + from the placeholders. +2. **Compose the 4-module prompt** (see `references/prompt-patterns.md`): ``` - __TRIGGER__, , , + , , , ``` - Pass the prompt via `--prompt` on the CLI. The `__PROMPT__` marker in the workflow gets - replaced automatically. -5. **Submit** via the Python script or the MCP server (see `comfyui-workflow/SKILL.md` for - the transport). The script will poll until the run finishes and download the saved image - from `outputs/`. + Pass via `--prompt` on the CLI; the `__PROMPT__` marker in the workflow gets replaced. +3. **Submit** via the Python script or the MCP server (see `comfyui-workflow/SKILL.md`). -### Strengths and what they do +### Knobs | Knob | Default | What it does | When to change it | |---|---|---|---| -| `LoraLoader.strength_model` | 0.85 | How strongly the LoRA controls the model | Lower to 0.6–0.7 if the character overpowers the scene; raise to 0.9–1.0 if identity drifts | -| `LoraLoader.strength_clip` | 0.85 | How strongly the LoRA controls the text encoder | Usually keep the same as `strength_model` | -| `ControlNetApply.strength` (positive) | 0.75 | Face position lock strength | 0.5 for looser shots, 0.9 for strict identity | -| `ControlNetApply.strength` (negative) | 0.6 | Negative prompt's face lock (used to suppress the wrong face) | Usually 0.4–0.6 | -| `KSampler.steps` | 28 | Diffusion steps | 20 for fast drafts, 35–40 for final-quality | -| `KSampler.cfg` | 6.0 | Prompt adherence | 5 for more creative, 7–8 for stricter prompt following | - -## 模仿自拍 — `selfie-mimicry.json` - -Pipeline: Checkpoint → LoraLoader → IP-Adapter (pulls reference) → KSampler. - -The bundled workflow uses an IP-Adapter FaceID model to pull the **face** from the reference -image while the LoRA keeps the **character identity** stable. This is the right tool when the -user has a reference photo (their own face, a stock photo, a frame from a video) and wants a -new generation that uses the same pose, lighting, and composition with their character. - -### Step-by-step - -1. **Place your face LoRA** at `/models/loras/your_face_lora.safetensors` and set - `LoraLoader.lora_name` to the exact filename. -2. **Place the IP-Adapter model** at `/models/ipadapter/ip-adapter-faceid-portrait.bin` - and set `IPAdapterModelLoader.ipadapter_file` accordingly. (If you use a different IP-Adapter - file, e.g. SDXL vs SD 1.5, change the filename and verify the base checkpoint matches.) -3. **Drop a reference face image** named `reference_face.png` into the ComfyUI input folder, - OR change `LoadImage.image` to point at the file you want. The reference image should - contain a clearly visible face — IP-Adapter FaceID will not work on a landscape with no - face. -4. **Compose the prompt**. The IP-Adapter is doing the heavy lifting on pose and lighting, so - the prompt can be lighter: `, , ` is usually enough. - You do not need to describe the pose — the reference is providing it. -5. **Submit** the same way as the selfie workflow. - -### Strengths and what they do +| `LoraLoaderModelOnly[0].strength_model` (face) | 1.0 | How strongly the face LoRA controls the model | Lower to 0.7 if the character overpowers the scene | +| `LoraLoaderModelOnly[1].strength_model` (style) | 1.0 | How strongly the style LoRA controls the model | Lower to 0.7 if the style is too dominant | +| `KSampler.steps` | 8 | Distilled-model default; raise to 20+ for a non-distilled base | — | +| `KSampler.cfg` | 1.5 | Distilled-model default; raise to 6 for a non-distilled base | — | + +--- + +## Scenario 2 — 模仿 (mimic) — `selfie-mimicry.json` + +Pipeline: `LoadImage` (reference) → `AIO_Preprocessor` → ControlNet application → +`UNETLoader` (ZIT-flavoured finetune) → KSampler → `VAEDecode` → `SaveImage`, with two +`LoraLoaderModelOnly` nodes (face + style) and a `llama_cpp_instruct_adv` node that runs the +vision LLM on the reference to produce the positive prompt. + +This workflow does **not** use IP-Adapter. It uses a different mechanism: a ControlNet is fed +the reference image's preprocessed output, and a vision LLM (Qwen3.5) looks at the reference +and produces a text description that becomes the positive prompt. The two LoRAs then keep the +character identity stable. + +### Models + +| Field | Value (as written in the JSON) | Where it lives on disk | +|---|---|---| +| `UNETLoader.unet_name` | `ZIT-moodyPornMix_zitV10R1DPO_fp16.safetensors` (ZIT-flavoured Z-Image finetune) | `models/unet/` (or wherever your UNETLoader looks) | +| `ControlNetLoader.control_net_name` | `Z-Image-Fun-Controlnet-Union-2.1.safetensors` | `models/controlnet/` | +| `LoraLoaderModelOnly[0].lora_name` (face) | `your_face_lora.safetensors` (placeholder — replace) | `models/loras/` | +| `LoraLoaderModelOnly[1].lora_name` (style) | `your_style_lora.safetensors` (placeholder — replace) | `models/loras/` | +| `llama_cpp_instruct_adv.ckpt_name` (vision LLM) | `Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf` | `models/llama/` (or wherever the llama.cpp loader looks) | +| `LoadImage.image` | `__IMAGE1__` (replace at submit time with `--filename`) | `input/` | + +### Custom nodes required + +The `selfie-mimicry.json` workflow depends on several custom nodes that ship outside the default +ComfyUI install. Without these, submission will fail with `missing_node_type`. + +| Custom node | Used for | Install | +|---|---|---| +| `comfyui_controlnet_aux` (or any `AIO_Preprocessor` + `ZImageFunControlnet` provider) | ControlNet preprocessing and application | `https://github.com/Fannovel16/comfyui_controlnet_aux` | +| `ComfyUI-Manager` + the `ModelPatchLoader` / `ZImageFunControlnet` nodes | Loading the Z-Image Fun ControlNet union | any pack that ships `ModelPatchLoader` | +| `ComfyUI-LLaMA-CPP` (or a llama.cpp loader for ComfyUI) | Loading `Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf` for vision reprompting | `https://github.com/daniel-lewis-ab/ComfyUI-LLaMA-CPP` (or compatible) | +| `rgthree-comfy` (or compatible Any Switch / Image Comparer pack) | The `Any Switch (rgthree)` nodes in the original workflow | `https://github.com/rgthree/rgthree-comfy` | +| `ComfyUI-Impact-Pack` (or any `LayerUtility: ImageScaleByAspectRatio V2` provider) | Aspect-ratio-aware image scaling | `https://github.com/ltdrdata/ComfyUI-Impact-Pack` | + +### Submit + +1. **Install the custom nodes** above. +2. **Place the model files** under the standard ComfyUI directories. +3. **Drop a reference image** into the ComfyUI input folder and pass its filename with + `--filename` on the CLI (the `__IMAGE1__` marker in the workflow's `LoadImage.image` field + gets replaced). Or edit the `LoadImage.image` field directly in the JSON. +4. **Set the trigger word** via `--trigger` (the `__TRIGGER__` marker in the `CR Text` node + gets replaced). +5. **Submit** via the Python script or the MCP server. The script will poll until the run + finishes and download the saved image from `outputs/`. + +### The 4-module prompt (for the vision-LLM-reprompted positive prompt) + +For scenario 2 the prompt is produced by the vision LLM, not by the user. The user provides +only the **trigger word** (via `--trigger`). The vision LLM fills in the rest of the +description from the reference image. See `references/prompt-patterns.md` for the rules of +thumb on what the trigger word should and shouldn't include. + +--- + +## Scenario 3 — 改图 (edit) — `flux2-klein-image-edit.json` + +Pipeline: `LoadImage` (reference) → Flux.2 Klein UNET + CLIP + VAE → KSampler → `VAEDecode` → +`SaveImage`. One `LoadImage` input (`__IMAGE1__`). + +### Models + +| Field | Value (as written in the JSON) | Where it lives on disk | +|---|---|---| +| `UNETLoader.unet_name` | `flux-2-klein-base-9b-fp8.safetensors` | `models/unet/` | +| `DualCLIPLoader.clip_name1` | `qwen_3_8b_fp8mixed.safetensors` | `models/clip/` | +| `DualCLIPLoader.clip_name2` | `diffusion_pytorch_model.safetensors` | `models/clip/` | +| `VAELoader.vae_name` | bundled with the UNET, or the user's preferred Flux.2 Klein VAE | `models/vae/` | +| `LoadImage.image` | `__IMAGE1__` (replace at submit time with `--filename`) | `input/` | + +### Submit + +1. **Place the reference image** into the ComfyUI input folder and pass its filename with + `--filename` on the CLI. +2. **Pass the edit prompt** with `--prompt`. The prompt describes **what to change** (e.g. + "the same person sitting in an office chair, professional lighting"). The reference image + supplies the unchanged parts. +3. **Submit** via the Python script or the MCP server. + +### Knobs | Knob | Default | What it does | When to change it | |---|---|---|---| -| `LoraLoader.strength_model` | 0.85 | LoRA weight | See selfie workflow above | -| `IPAdapterApply.weight` | 0.7 | How strongly the reference controls the image | 0.4–0.5 for looser mimicry, 0.85 for near-clone | -| `IPAdapterApply.noise` | 0.05 | Slight variation from the reference | 0.0 for exact mimicry, 0.1–0.2 for "inspired by" | -| `IPAdapterApply.weight_type` | "linear" | How the weight is applied across denoising steps | "ease in" / "ease out" / "reverse in-out" available; default works for most | +| `KSampler.steps` | 20 | Standard distilled-Klein default | Raise to 30+ for higher quality at the cost of latency | +| `KSampler.cfg` | 1.0 | Distilled-Klein default | Usually leave alone | +| `KSampler.denoise` | 0.75 | How much the prompt is allowed to change the reference | Lower to 0.4–0.5 for "subtle" edits; raise to 0.9+ for "full re-imagination" | + +--- -## Why most "consistent character" attempts fail +## Scenario 4 — 融合 (fuse) — `flux2-klein-image-edit-dual.json` + +Same model stack as scenario 3, but with **two `LoadImage` inputs** (`__IMAGE1__` and +`__IMAGE2__`). The semantic is "**the subject of image 1 in the scene of image 2, with the +prompt describing the rest**" — e.g. "the person from image 1, now wearing the outfit from +image 2, in the office of image 2". + +### Models + +Identical to scenario 3, with one extra `LoadImage` input. Replace `__IMAGE1__` with +`--filename` and `__IMAGE2__` with `--filename2` at submit time. + +### Submit + +1. **Place both reference images** into the ComfyUI input folder. +2. **Pass filenames** with `--filename` (image 1, the subject) and `--filename2` (image 2, + the scene / outfit). +3. **Pass the fusion prompt** with `--prompt` describing how to combine them. +4. **Submit** via the Python script or the MCP server. + +### Knobs + +Same as scenario 3, plus an extra `ReferenceLatent` strength control (default 0.85) that +governs how much of image 2 the result should inherit. + +--- + +## Why most "consistent character" attempts fail (scenarios 1 and 2) A LoRA is a strong prior on identity, but it is not magic. Three things derail consistency: @@ -113,21 +205,18 @@ A LoRA is a strong prior on identity, but it is not magic. Three things derail c 3. **Wrong sampler / scheduler / CFG for the base model.** Some samplers oversmooth identity features. See `references/prompt-patterns.md` for the rules of thumb. -## The 4-module prompt structure (summary) - -From `references/prompt-patterns.md`: - -``` -, , , -``` +## LoRAs are required for scenarios 1 and 2 -For the mimicry workflow, drop the `pose + scene` block — the reference image is supplying it. -For the selfie workflow, keep all four. +Both selfie workflows' `LoraLoaderModelOnly` nodes reference **placeholder** LoRA filenames +(`your_face_lora.safetensors` and `your_style_lora.safetensors`). If those files do not exist +on disk, ComfyUI will fail at submission. To swap in a different LoRA, edit the `lora_name` +field in the workflow JSON to point at your file. The LoRAs are user identity assets; the +Plugin does not bundle them and does not name anyone's private LoRAs. ## What this Skill does not do -- It does not run LoRA training. Training is its own project and has its own tooling - (Kohya, OneTrainer, ai-toolkit). The Skill only consumes a LoRA the user already has. +- It does not run LoRA training. Training is its own project and has its own tooling (Kohya, + OneTrainer, ai-toolkit). The Skill only consumes a LoRA the user already has. - It does not bundle or distribute any LoRA, model, face embedding, or voice sample. Every identity asset is user-supplied. - It does not invent characters. The user must define who the character is. @@ -135,11 +224,11 @@ For the selfie workflow, keep all four. ## Requirements - ComfyUI running locally (see `comfyui-workflow/SKILL.md` for the transport). -- A character LoRA the user trained, dropped under `models/loras/`. -- For the mimicry workflow: an IP-Adapter model (FaceID for portraits, SD 1.5 or SDXL - depending on your base checkpoint). -- For the selfie workflow (optional but recommended): a face-oriented ControlNet. -- A base checkpoint compatible with the LoRA's training resolution. +- For scenarios 1 and 2: a face LoRA and a style LoRA in `models/loras/`. The workflow JSONs + reference them as `your_face_lora.safetensors` and `your_style_lora.safetensors`. +- For scenario 2: the custom-node pack list above. Plus a ZIT-flavoured Z-Image finetune, a + Z-Image Fun ControlNet, and a Qwen3.5 GGUF vision LLM. +- For scenarios 3 and 4: the Flux.2 Klein 9B UNET, the Qwen3 8B CLIP, and the bundled VAE. ## License diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md index 8efc1e7..e7d8d1a 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-drama/SKILL.md @@ -1,40 +1,61 @@ --- name: comfyui-drama -description: Generate a complete short drama video from a storyboard: first-frame image, image-to-video clip, TTS dubbing, subtitle burn, audio/video assembly. Use this Skill when the user wants a multi-shot, multi-character video with consistent character identity — ancient costume, modern urban, fantasy, or any other genre. Ships two preset workflow templates: `workflows/drama-first-frame.json` and `workflows/drama-image-to-video.json`. +description: Generate a complete short drama video from a storyboard. Use this Skill for scenarios 5–6 from `comfyui-studio`'s trigger index: 5=首帧 (drama first frame, two-character LoRA + prompt → 16:9 still) and 6=出片 (image-to-video, first frame + motion prompt → MP4 clip). The Skill also describes the full 7-stage pipeline around them (storyboard → TTS → script refinement → first frame → image-to-video → subtitle burn → audio/video assembly). Ships two preset workflow templates: `workflows/drama-first-frame.json` and `workflows/drama-image-to-video.json`. --- # ComfyUI Drama -A 7-stage pipeline for producing a short drama video from a storyboard. The Plugin ships -two preset ComfyUI workflows (first-frame and image-to-video) and describes the rest of the -pipeline. The Plugin does not distribute any character LoRA, voice sample, or video model — -every asset is user-supplied. +A 7-stage pipeline for producing a short drama video from a storyboard. The Plugin ships two +preset ComfyUI workflows (scenarios 5 and 6) and describes the rest of the pipeline. The Plugin +does not distribute any character LoRA, voice sample, or video model — every asset is +user-supplied. -## The 7 stages +## The 2 trigger scenarios (Part of this Skill) + +These are scenarios 5 and 6 from `comfyui-studio`'s index. + +| # | Trigger | User says | Preset workflow | What it produces | +|---|---|---|---|---| +| **5** | **首帧** (first frame) | "生成这部短剧第 X 镜的首帧" / "first frame of shot 3" | `workflows/drama-first-frame.json` | One 16:9 still image per shot, with two character LoRA slots | +| **6** | **出片** (image to video) | "把这个首帧做成 4 秒视频" / "turn this first frame into a clip" | `workflows/drama-image-to-video.json` | One MP4 clip per shot, ~4 seconds, 24 FPS, distilled LTX-2.3 | + +The Plugin provides templates and preset workflows for both stages. The user runs stages 1, 2, +3, 6, and 7 of the surrounding pipeline with their own tools (Excel, TTS pipeline, FFmpeg). This +boundary is intentional: the Plugin does not assume a particular TTS vendor or spreadsheet +format. + +## The 7 stages (the wider pipeline) | Stage | Output | Tool | Notes | |---|---|---|---| | 1. Storyboard | `desktop/_分镜.xlsx` | User's spreadsheet editor or any storyboard tool | 6-column format: 序号 / 时长 / 画面 / 图片 prompt / 视频 prompt / 台词 | | 2. TTS dubbing | `_wav/00X_.wav` | User's TTS pipeline (Edge TTS, IndexTTS2, Qwen3-VoiceDesign, etc.) | **TTS comes BEFORE images** — the actual measured duration is the only truth for shot length | | 3. Script refinement | Updated Excel with measured shot lengths | User's scripting tool | Re-cut shots whose actual TTS length differs from the script estimate by > 0.5s | -| 4. First-frame image | `drama_frame_00001_.png` per shot | **This Plugin's `workflows/drama-first-frame.json`** | One image per shot, two-character LoRA slots | -| 5. Image-to-video clip | `drama_clip_00001_.mp4` per shot | **This Plugin's `workflows/drama-image-to-video.json`** | One video per shot, 8 GB VRAM friendly | +| 4. **First-frame image** | `drama_frame_00001_.png` per shot | **Scenario 5: `workflows/drama-first-frame.json`** | One image per shot, two-character LoRA slots | +| 5. **Image-to-video clip** | `drama_clip_00001_.mp4` per shot | **Scenario 6: `workflows/drama-image-to-video.json`** | One video per shot, 8 GB VRAM friendly | | 6. Subtitle burn | Subtitled video per shot | FFmpeg `drawtext` filter | Source Han Sans CN, 28pt, white + black 2px stroke | | 7. Audio/video assembly | Final drama MP4 | FFmpeg `-c:v copy -c:a aac` + concat demuxer | One output MP4 per drama | -The Plugin provides templates and preset workflows for stages 4 and 5. The user runs stages -1, 2, 3, 6, and 7 with their own tools (Excel, TTS pipeline, FFmpeg). This boundary is -intentional: the Plugin does not assume a particular TTS vendor or spreadsheet format. - -## Two preset workflows +--- -### `workflows/drama-first-frame.json` +## Scenario 5 — 首帧 (first frame) — `drama-first-frame.json` A first-frame image generator with **two LoRA slots** (one per character in a two-character -drama). For single-character dramas, set the second slot's LoRA file to a placeholder -LoRA with `strength = 0`; for two-character dramas, fill both slots. +drama). For single-character dramas, set the second slot's LoRA file to a placeholder LoRA +with `strength = 0`; for two-character dramas, fill both slots. + +Pipeline: Checkpoint → `LoraLoader` (character A) → `LoraLoader` (character B) → KSampler. -Pipeline: Checkpoint → LoraLoader (character A) → LoraLoader (character B) → KSampler. +### Models + +| Field | Value (as written in the JSON) | Where it lives on disk | +|---|---|---| +| `CheckpointLoaderSimple.ckpt_name` | `Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors` | `models/checkpoints/` | +| `LoraLoader[0].lora_name` (character A) | `character_a_lora.safetensors` (placeholder — replace) | `models/loras/` | +| `LoraLoader[1].lora_name` (character B) | `character_b_lora.safetensors` (placeholder — replace) | `models/loras/` | +| CLIP / VAE | bundled with the CheckpointLoader | — | + +### Knobs | Knob | Default | What it does | |---|---|---| @@ -44,71 +65,114 @@ Pipeline: Checkpoint → LoraLoader (character A) → LoraLoader (character B) | `KSampler.steps` | 8 | Distilled-model default; raise to 20 for a non-distilled base | | `KSampler.cfg` | 1.5 | Distilled-model default; raise to 6 for a non-distilled base | -**Critical prompt rule**: the image prompt describes the **first frame**, not the shot's -action sequence. A shot where character A hands a sword to character B should prompt the -moment the hands are meeting, not the full handoff motion. The video prompt is responsible -for the motion. +### Submit -### `workflows/drama-image-to-video.json` +1. **Place the two character LoRAs** under `/models/loras/`. Edit the two + `LoraLoader.lora_name` fields in the JSON to point at your filenames if they differ from + the placeholders. +2. **Pass the image prompt** with `--prompt`. For a two-character drama, lead with the trigger + words for both characters so the LoRAs can light up. +3. **Submit** via the Python script or the MCP server. -An image-to-video generator for a single shot. Takes a first-frame image (from stage 4) and +### Critical prompt rule (first-frame) + +The image prompt describes the **first frame**, not the shot's action sequence. A shot where +character A hands a sword to character B should prompt the moment the hands are meeting, not +the full handoff motion. The video prompt (scenario 6) is responsible for the motion. + +--- + +## Scenario 6 — 出片 (image to video) — `drama-image-to-video.json` + +An image-to-video generator for a single shot. Takes a first-frame image (from scenario 5) and the shot's video prompt (Excel column "视频 prompt"). Outputs a single MP4 per shot. -Pipeline: Checkpoint → LoraLoader (optional motion LoRA) → LoadImage (first frame) → KSampler -→ VHS_VideoCombine. +Pipeline: CheckpointLoader + DualCLIPLoader + VAELoader + LoraLoader (optional motion) + +`LoadImage` (first frame) + KSampler + `VHS_VideoCombine`. + +### Models + +| Field | Value (as written in the JSON) | Where it lives on disk | +|---|---|---| +| `CheckpointLoaderSimple.ckpt_name` | `10Eros_v1-fp8mixed_learned.safetensors` (LTX-2.3 22B finetune) | `models/checkpoints/` | +| `DualCLIPLoader.clip_name1` | `ltx-2.3_text_projection_bf16.safetensors` | `models/clip/` | +| `DualCLIPLoader.clip_name2` | `gemma_3_12B_it_fp8_scaled.safetensors` | `models/clip/` | +| `VAELoader.vae_name` | `LTX23_video_vae_bf16.safetensors` | `models/vae/` | +| `LoraLoader.lora_name` (motion) | `any_motion_lora.safetensors` (placeholder — replace, or set `strength_model = 0` to disable) | `models/loras/` | +| `LoadImage.image` | `first_frame.png` (rename per shot, or pass `--filename` to drive the workflow from the CLI) | `input/` | + +### Knobs | Knob | Default | What it does | |---|---|---| -| `LoadImage.image` | "first_frame.png" | The first-frame image to animate. The user must change this per shot, or upload each frame to ComfyUI with a predictable filename | +| `LoadImage.image` | "first_frame.png" | The first-frame image to animate. Use predictable filenames per shot and pass them via `--filename` on the CLI. | | `VHS_VideoCombine.frame_rate` | 24 | Output FPS. Pair with RIFE interpolation to reach 60 | | `VHS_VideoCombine.format` | "video/h264-mp4" | Output container/codec | | `KSampler.steps` | 8 | Distilled LTX default; raise for non-distilled | | `KSampler.cfg` | 1.0 | Distilled LTX default | -**Critical prompt rule**: the video prompt describes motion only — camera move, character -animation, environment dynamics, expression changes. It does **not** describe the visible -image (the model can see the first frame and already knows what is in it). The shot's dialogue -goes in a separate `dialogue` field, not the `concept` field. +### Submit + +1. **Place the first-frame image** in the ComfyUI input folder with a predictable name + (`shot_001.png`, `shot_002.png`, ...). +2. **Pass the motion prompt** (Excel "视频 prompt" column value) with `--prompt`. The video + prompt describes motion only — see the critical rule below. +3. **Submit** via the Python script or the MCP server. The script will poll until done and + download the resulting MP4 from `outputs/`. + +### Critical prompt rule (motion-only) + +The video prompt describes motion only — camera move, character animation, environment +dynamics, expression changes. It does **not** describe the visible image (the model can see +the first frame and already knows what is in it). The shot's dialogue goes in a separate +`dialogue` field, not in the prompt. + +--- + +## How to run a full drama (loop scenarios 5 and 6) -### How to run a shot +For each shot in the storyboard: -1. **For each shot in the storyboard**, fill the workflow's `__PROMPT__` marker with the - Excel "视频 prompt" column value. -2. **For each shot, set the first-frame image** by changing `LoadImage.image` to the PNG - produced by stage 4. Use predictable filenames (`shot_001.png`, `shot_002.png`, ...) and - the CLI's `--filename` flag to drive the workflow. -3. **Submit each shot** via the Python script or MCP server. The script will poll until done - and download the resulting MP4. +1. **Scenario 5**: submit `drama-first-frame.json` with the image prompt → produces + `drama_frame_00001_.png`. +2. **Scenario 6**: rename / move the first-frame to `shot_001.png` in the ComfyUI input + folder, then submit `drama-image-to-video.json` with the motion prompt → produces + `drama_clip_00001_.mp4`. +3. **Stage 6** (out of scope of this Plugin): burn subtitles with FFmpeg `drawtext`. +4. **Stage 7** (out of scope of this Plugin): concat the clips and mux the TTS audio with + FFmpeg. -A bulk run across all shots is straightforward but each shot is one ComfyUI submission -(because each shot has a different first-frame image and a different prompt). The Plugin does -not implement a batch driver; if you need one, use a wrapper loop in your own tool. +Each shot is one ComfyUI submission per scenario (because each shot has a different first-frame +image and a different prompt). The Plugin does not implement a batch driver; if you need one, +use a wrapper loop in your own tool. ## A note on long jobs and VRAM -A distilled LTX-class video model running at 1280×720 with 24 FPS for a 4-second clip -finishes in roughly 2–5 minutes on an 8 GB GPU. A non-distilled 22B video model on the same -shot can take 30–60 minutes. Long jobs (over 30 minutes) should be watched with `nvidia-smi`; -the Plugin does not ship a VRAM watchdog. If your GPU is lost mid-run, the standard -recovery is to interrupt the queue and restart ComfyUI. +A distilled LTX-class video model running at 1280×720 with 24 FPS for a 4-second clip finishes +in roughly 2–5 minutes on an 8 GB GPU. A non-distilled 22B video model on the same shot can +take 30–60 minutes. Long jobs (over 30 minutes) should be watched with `nvidia-smi`; the Plugin +does not ship a VRAM watchdog. If your GPU is lost mid-run, the standard recovery is to +interrupt the queue and restart ComfyUI. ## What this Skill does not do - It does not run LoRA training. Training is its own project; the Skill only consumes a LoRA the user already has. -- It does not bundle or distribute any LoRA, model, voice sample, face embedding, or any - other identity asset. Every character and voice is user-supplied. +- It does not bundle or distribute any LoRA, model, voice sample, face embedding, or any other + identity asset. Every character and voice is user-supplied. - It does not run TTS, edit Excel, burn subtitles, or assemble audio/video. Those are user pipeline steps that the Plugin deliberately does not assume. -- It does not ship a VRAM watchdog or auto-retry on failure. Long jobs need a human - watching. +- It does not ship a VRAM watchdog or auto-retry on failure. Long jobs need a human watching. ## Requirements - ComfyUI running locally (see `comfyui-workflow/SKILL.md` for the transport). -- A first-frame checkpoint and (optionally) two character LoRAs the user trained, dropped - under `models/checkpoints/` and `models/loras/`. -- A video checkpoint (distilled LTX-2.3 or similar) under `models/checkpoints/`. +- A first-frame checkpoint and (optionally) two character LoRAs the user trained, dropped under + `models/checkpoints/` and `models/loras/`. The workflow JSONs reference the LoRAs as + `character_a_lora.safetensors` and `character_b_lora.safetensors` — replace with your + filenames. +- A distilled LTX-Video checkpoint and its CLIP / VAE files. The motion LoRA is referenced as + `any_motion_lora.safetensors` — replace or set its strength to 0 to disable. - A TTS pipeline (any of the user's choice) for stage 2. - FFmpeg for stages 6 and 7. diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md index f40ed0a..ef2f428 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-studio/SKILL.md @@ -1,6 +1,6 @@ --- name: comfyui-studio -description: Drive a local ComfyUI 8188 server for image and short-clip generation. Use this Skill when the task involves a ComfyUI workflow, a local Stable Diffusion / Flux checkpoint, character-consistent generation with a LoRA, or producing deterministic outputs from a saved workflow JSON. Triggers on requests like "use ComfyUI to ...", "submit this workflow", "what models do I have", "generate an image with my LoRA", or "check the queue". +description: Drive a local ComfyUI 8188 server for image and short-clip generation. Use this Skill when the task involves a ComfyUI workflow, a local Stable Diffusion / Flux / Z-Image checkpoint, character-consistent generation with a LoRA, or producing deterministic outputs from a saved workflow JSON. Triggers on requests like "use ComfyUI to ...", "submit this workflow", "what models do I have", "generate an image with my LoRA", or "check the queue". --- # ComfyUI Studio @@ -12,26 +12,44 @@ four Skills and two transport layers that work together. | Part | What it covers | Where | |---|---|---| -| **Basic control** | The transport between the agent and a local ComfyUI server. Submit workflows, poll the queue, download outputs. | `comfyui-workflow` Skill + the stdio MCP server + the Python CLI | -| **Preset workflows** | Three opinionated workflow templates that solve common recurring tasks: a consistent character selfie, a reference-image mimicry, and a 7-stage short drama pipeline. | `comfyui-character` and `comfyui-drama` Skills + the `workflows/*.json` presets | +| **A. Basic control** | The transport between the agent and a local ComfyUI server. Submit any workflow, poll the queue, download outputs. | `comfyui-workflow` Skill + the stdio MCP server + the Python CLI | +| **B. Preset workflows** | Six opinionated workflow templates that solve common recurring tasks, numbered 1–6 by the user's trigger scenario. | `comfyui-character` (scenarios 1–4) + `comfyui-drama` (scenarios 5–6) + the `workflows/*.json` presets | The Plugin also ships a dependency-free stdio MCP server (see `mcp.json` + `server.mjs`) that -exposes the same basic-control primitives (`submit_prompt`, `check_queue`, `get_image`) to any -MCP-capable agent. - -## When to use which Skill - -| User intent | Use | -|---|---| -| "Submit this workflow" / "Check the queue" / "Download the image" | **comfyui-workflow** | -| "Make a selfie of my character" / "Generate a portrait using my LoRA" | **comfyui-character** (preset: `workflows/selfie-text-to-image.json`) | -| "Use this photo as a reference — make a similar selfie" | **comfyui-character** (preset: `workflows/selfie-mimicry.json`) | -| "Make a short drama" / "Generate a multi-shot video with these characters" | **comfyui-drama** (presets: `workflows/drama-first-frame.json` + `workflows/drama-image-to-video.json`) | -| "Use ComfyUI to ..." without further detail | **comfyui-studio** — read the user's intent, then route to one of the siblings | -| "What models do I have?" / "Is ComfyUI running?" | **comfyui-workflow** — its first step is always a health probe | +exposes the same Part-A primitives (`submit_prompt`, `check_queue`, `get_image`) to any MCP-capable +agent. + +## The 6 trigger scenarios (Part B) + +This is the routing index. The number is the trigger scenario — read the user's intent, pick a +number, and follow the pointer. + +| # | Trigger | User typically says | Preset workflow | Pointer | +|---|---|---|---|---| +| **1** | **生图** (generate an image) | "用我的角色画一张自拍" / "生成一张 XX 风格的照片" / "draw my character at a beach" | `workflows/selfie-text-to-image.json` | `comfyui-character` | +| **2** | **模仿** (mimic an image) | "照着这张照片再画一张同款" / "模仿这张图" / "use this photo as a reference" | `workflows/selfie-mimicry.json` | `comfyui-character` | +| **3** | **改图** (edit an image) | "把背景换成办公室" / "衣服换成西装" / "edit this photo: change the background" | `workflows/flux2-klein-image-edit.json` | `comfyui-character` | +| **4** | **融合** (fuse two images) | "把这张图里的人物放到那张图里" / "让她穿上这件衣服" / "put the person from this image onto that image" | `workflows/flux2-klein-image-edit-dual.json` | `comfyui-character` | +| **5** | **首帧** (drama first frame) | "生成这部短剧第 X 镜的首帧" / "first frame of shot 3" | `workflows/drama-first-frame.json` | `comfyui-drama` | +| **6** | **出片** (image to video) | "把这个首帧做成 4 秒视频" / "turn this first frame into a clip" | `workflows/drama-image-to-video.json` | `comfyui-drama` | + +If the user's request does not map cleanly to one of these six, fall through to **Part A** +(below) and use `comfyui-workflow` as a generic transport for any user-supplied workflow JSON. + +## Routing rules + +1. **Numbers 1–4** → read `skills/comfyui-character/SKILL.md` for the recipe (model stack, + custom-node requirements, prompt structure, LoRA slots, knobs). Submit via Part A. +2. **Numbers 5–6** → read `skills/comfyui-drama/SKILL.md` for the 7-stage pipeline and the + per-shot rules. Submit each shot via Part A. +3. **Anything else** ("submit this workflow", "check the queue", "what models do I have", + "use ComfyUI to ...", "run this JSON") → read `skills/comfyui-workflow/SKILL.md` directly. + The basic control layer handles all of these without reading a preset. +4. **Health probes** ("is ComfyUI running?") → `comfyui-workflow` — its first step is always + a `GET /system_stats` or `submit_workflow.py --probe`. Do not duplicate behaviour: if you only need to submit a workflow, do not read this Skill, read -`comfyui-workflow/SKILL.md` directly. This Skill is the entry point for routing, not the +`comfyui-workflow/SKILL.md` directly. This Skill is the entry point for **routing**, not the implementation. ## Required environment @@ -40,16 +58,21 @@ The Plugin assumes the user has: - A running ComfyUI server (default `http://127.0.0.1:8188`). Override with the `COMFYUI_URL` environment variable or the `--url` flag on the Python scripts. -- At least one checkpoint model installed and reachable from ComfyUI. -- For `comfyui-character`: a character LoRA the user trained themselves and dropped into - ComfyUI's `models/loras/` directory. The Plugin never distributes LoRAs or character - weights. -- For `comfyui-drama`: two character LoRAs (for two-character dramas), a distilled video - checkpoint (LTX-2.3 class), and a TTS pipeline of the user's choice. +- The checkpoint / CLIP / VAE / ControlNet files referenced in each preset (see the **Model + boundary** section in `README.md`). Edit loader fields if your install uses different + filenames. +- For scenarios 1 and 2 (`comfyui-character` selfie workflows): a face LoRA and a style LoRA + in `models/loras/`. The workflow JSONs reference them as `your_face_lora.safetensors` and + `your_style_lora.safetensors` — replace these with your filenames before submitting. The + Plugin does not bundle LoRAs. +- For scenario 2 (`selfie-mimicry.json`): the custom-node pack list in + `comfyui-character/SKILL.md` (controlnet_aux, LLaMA-CPP, ModelPatchLoader, etc.). +- For scenarios 5 and 6 (`comfyui-drama`): two character LoRAs the user trained themselves + (referenced as `character_a_lora.safetensors` / `character_b_lora.safetensors` / + `any_motion_lora.safetensors` — replace with your filenames). The Plugin does **not** download or bundle model checkpoints, LoRAs, or any other binary asset. -Each Skill instructs the user to place their own files in the standard ComfyUI directory -layout. +Each Skill instructs the user to place their own files in the standard ComfyUI directory layout. ## Data and network @@ -63,13 +86,13 @@ layout. End-to-end smoke test (about 5 minutes, see `examples/minimal-run.md`): ```text -Use comfyui-studio to verify that my local ComfyUI is reachable, then submit the -text-to-image workflow bundled in this Plugin with the prompt "a tabby cat sleeping -on a windowsill, morning light, photorealistic" and report the saved image path. +Use comfyui-studio to verify that my local ComfyUI is reachable, then submit scenario 3 +(改图 / flux2-klein-image-edit) with the prompt "the same person sitting in an office +chair, the same outfit, professional lighting" and report the saved image path. ``` Expected outcome: the agent confirms ComfyUI is reachable, submits the bundled -`workflows/text-to-image.json` workflow, polls until done, downloads the image, and tells you +`workflows/flux2-klein-image-edit.json`, polls until done, downloads the image, and tells you where it was saved. ## License diff --git a/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json b/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json index 9b08174..be1d9cf 100644 --- a/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json +++ b/plugins/antianqi/comfyui-studio/workflows/drama-first-frame.json @@ -2,7 +2,7 @@ "3": { "class_type": "CheckpointLoaderSimple", "inputs": { - "ckpt_name": "any_anime_or_cinematic_checkpoint.safetensors" + "ckpt_name": "Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors" } }, "10": { diff --git a/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json b/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json index ac3da94..d64ee36 100644 --- a/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json +++ b/plugins/antianqi/comfyui-studio/workflows/drama-image-to-video.json @@ -2,40 +2,55 @@ "1": { "class_type": "CheckpointLoaderSimple", "inputs": { - "ckpt_name": "any_ltx2_distilled_checkpoint.safetensors" + "ckpt_name": "10Eros_v1-fp8mixed_learned.safetensors" } }, "2": { + "class_type": "DualCLIPLoader", + "inputs": { + "clip_name1": "ltx-2.3_text_projection_bf16.safetensors", + "clip_name2": "gemma_3_12B_it_fp8_scaled.safetensors", + "type": "ltxv", + "device": "default" + } + }, + "3": { + "class_type": "VAELoader", + "inputs": { + "vae_name": "LTX23_video_vae_bf16.safetensors" + } + }, + "4": { "class_type": "LoraLoader", "inputs": { "model": ["1", 0], - "clip": ["1", 1], + "clip": ["2", 0], "lora_name": "any_motion_lora.safetensors", "strength_model": 0.7, "strength_clip": 0.7 } }, - "3": { + "5": { "class_type": "LoadImage", "inputs": { "image": "first_frame.png" } }, - "4": { + "6": { "class_type": "CLIPTextEncode", "inputs": { "text": "__PROMPT__", - "clip": ["2", 1] + "clip": ["2", 0] } }, - "5": { + "7": { "class_type": "CLIPTextEncode", "inputs": { "text": "static, no movement, jitter, flicker, watermark, blurry, low quality", - "clip": ["2", 1] + "clip": ["2", 0] } }, - "10": { + "20": { "class_type": "EmptyLatentImage", "inputs": { "width": 1280, @@ -43,13 +58,13 @@ "batch_size": 1 } }, - "20": { + "30": { "class_type": "KSampler", "inputs": { - "model": ["2", 0], - "positive": ["4", 0], - "negative": ["5", 0], - "latent_image": ["10", 0], + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["20", 0], "seed": 42, "steps": 8, "cfg": 1.0, @@ -58,17 +73,17 @@ "denoise": 1.0 } }, - "30": { + "40": { "class_type": "VAEDecode", "inputs": { - "samples": ["20", 0], - "vae": ["1", 2] + "samples": ["30", 0], + "vae": ["3", 0] } }, - "40": { + "50": { "class_type": "VHS_VideoCombine", "inputs": { - "frames": ["30", 0], + "images": ["40", 0], "frame_rate": 24, "loop_count": 0, "filename_prefix": "drama_clip", diff --git a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json new file mode 100644 index 0000000..83862eb --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json @@ -0,0 +1,605 @@ +{ + "9": { + "inputs": { + "filename_prefix": "klein_edit", + "images": [ + "75:65", + 0 + ] + }, + "class_type": "SaveImage", + "_meta": { + "title": "Save Image" + } + }, + "76": { + "inputs": { + "image": "" + }, + "class_type": "LoadImage", + "_meta": { + "title": "Load Image" + } + }, + "81": { + "inputs": { + "image": "" + }, + "class_type": "LoadImage", + "_meta": { + "title": "Load Image" + } + }, + "94": { + "inputs": { + "filename_prefix": "klein_edit", + "images": [ + "92:104", + 0 + ] + }, + "class_type": "SaveImage", + "_meta": { + "title": "Save Image" + } + }, + "75:61": { + "inputs": { + "sampler_name": "euler" + }, + "class_type": "KSamplerSelect", + "_meta": { + "title": "KSamplerSelect" + } + }, + "75:62": { + "inputs": { + "steps": 20, + "width": [ + "75:100", + 0 + ], + "height": [ + "75:100", + 1 + ] + }, + "class_type": "Flux2Scheduler", + "_meta": { + "title": "Flux2Scheduler" + } + }, + "75:63": { + "inputs": { + "cfg": 5, + "model": [ + "75:70", + 0 + ], + "positive": [ + "75:124", + 0 + ], + "negative": [ + "75:122", + 0 + ] + }, + "class_type": "CFGGuider", + "_meta": { + "title": "CFGGuider" + } + }, + "75:64": { + "inputs": { + "noise": [ + "75:73", + 0 + ], + "guider": [ + "75:63", + 0 + ], + "sampler": [ + "75:61", + 0 + ], + "sigmas": [ + "75:62", + 0 + ], + "latent_image": [ + "75:66", + 0 + ] + }, + "class_type": "SamplerCustomAdvanced", + "_meta": { + "title": "SamplerCustomAdvanced" + } + }, + "75:65": { + "inputs": { + "samples": [ + "75:64", + 0 + ], + "vae": [ + "75:72", + 0 + ] + }, + "class_type": "VAEDecode", + "_meta": { + "title": "VAE Decode" + } + }, + "75:73": { + "inputs": { + "noise_seed": 6 + }, + "class_type": "RandomNoise", + "_meta": { + "title": "RandomNoise" + } + }, + "75:70": { + "inputs": { + "unet_name": "flux-2-klein-base-9b-fp8.safetensors", + "weight_dtype": "default" + }, + "class_type": "UNETLoader", + "_meta": { + "title": "Load Diffusion Model" + } + }, + "75:71": { + "inputs": { + "clip_name": "qwen_3_8b_fp8mixed.safetensors", + "type": "flux2", + "device": "default" + }, + "class_type": "CLIPLoader", + "_meta": { + "title": "Load CLIP" + } + }, + "75:74": { + "inputs": { + "text": "__PROMPT__", + "clip": [ + "75:71", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Positive Prompt)" + } + }, + "75:67": { + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark", + "clip": [ + "75:71", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Negative Prompt)" + } + }, + "75:72": { + "inputs": { + "vae_name": "diffusion_pytorch_model.safetensors" + }, + "class_type": "VAELoader", + "_meta": { + "title": "Load VAE" + } + }, + "75:66": { + "inputs": { + "width": [ + "75:100", + 0 + ], + "height": [ + "75:100", + 1 + ], + "batch_size": 1 + }, + "class_type": "EmptyFlux2LatentImage", + "_meta": { + "title": "Empty Flux 2 Latent" + } + }, + "75:80": { + "inputs": { + "upscale_method": "lanczos", + "megapixels": 1, + "resolution_steps": 1, + "image": [ + "76", + 0 + ] + }, + "class_type": "ImageScaleToTotalPixels", + "_meta": { + "title": "ImageScaleToTotalPixels" + } + }, + "75:100": { + "inputs": { + "image": [ + "75:80", + 0 + ] + }, + "class_type": "GetImageSize", + "_meta": { + "title": "Get Image Size" + } + }, + "75:122": { + "inputs": { + "conditioning": [ + "75:67", + 0 + ], + "latent": [ + "75:123", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "75:123": { + "inputs": { + "pixels": [ + "75:80", + 0 + ], + "vae": [ + "75:72", + 0 + ] + }, + "class_type": "VAEEncode", + "_meta": { + "title": "VAE Encode" + } + }, + "75:124": { + "inputs": { + "conditioning": [ + "75:74", + 0 + ], + "latent": [ + "75:123", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "92:102": { + "inputs": { + "sampler_name": "euler" + }, + "class_type": "KSamplerSelect", + "_meta": { + "title": "KSamplerSelect" + } + }, + "92:103": { + "inputs": { + "noise": [ + "92:105", + 0 + ], + "guider": [ + "92:114", + 0 + ], + "sampler": [ + "92:102", + 0 + ], + "sigmas": [ + "92:115", + 0 + ], + "latent_image": [ + "92:109", + 0 + ] + }, + "class_type": "SamplerCustomAdvanced", + "_meta": { + "title": "SamplerCustomAdvanced" + } + }, + "92:104": { + "inputs": { + "samples": [ + "92:103", + 0 + ], + "vae": [ + "92:107", + 0 + ] + }, + "class_type": "VAEDecode", + "_meta": { + "title": "VAE Decode" + } + }, + "92:105": { + "inputs": { + "noise_seed": 1079830163544628 + }, + "class_type": "RandomNoise", + "_meta": { + "title": "RandomNoise" + } + }, + "92:106": { + "inputs": { + "unet_name": "flux-2-klein-base-9b-fp8.safetensors", + "weight_dtype": "default" + }, + "class_type": "UNETLoader", + "_meta": { + "title": "Load Diffusion Model" + } + }, + "92:107": { + "inputs": { + "vae_name": "diffusion_pytorch_model.safetensors" + }, + "class_type": "VAELoader", + "_meta": { + "title": "Load VAE" + } + }, + "92:108": { + "inputs": { + "image": [ + "92:110", + 0 + ] + }, + "class_type": "GetImageSize", + "_meta": { + "title": "Get Image Size" + } + }, + "92:109": { + "inputs": { + "width": [ + "92:108", + 0 + ], + "height": [ + "92:108", + 1 + ], + "batch_size": 1 + }, + "class_type": "EmptyFlux2LatentImage", + "_meta": { + "title": "Empty Flux 2 Latent" + } + }, + "92:110": { + "inputs": { + "upscale_method": "lanczos", + "megapixels": 1, + "resolution_steps": 1, + "image": [ + "76", + 0 + ] + }, + "class_type": "ImageScaleToTotalPixels", + "_meta": { + "title": "ImageScaleToTotalPixels" + } + }, + "92:85": { + "inputs": { + "upscale_method": "lanczos", + "megapixels": 1, + "resolution_steps": 1, + "image": [ + "81", + 0 + ] + }, + "class_type": "ImageScaleToTotalPixels", + "_meta": { + "title": "ImageScaleToTotalPixels" + } + }, + "92:111": { + "inputs": { + "clip_name": "qwen_3_8b_fp8mixed.safetensors", + "type": "flux2", + "device": "default" + }, + "class_type": "CLIPLoader", + "_meta": { + "title": "Load CLIP" + } + }, + "92:113": { + "inputs": { + "text": "__PROMPT__", + "clip": [ + "92:111", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Positive Prompt)" + } + }, + "92:87": { + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark", + "clip": [ + "92:111", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode ( Negative Prompt)" + } + }, + "92:114": { + "inputs": { + "cfg": 5, + "model": [ + "92:106", + 0 + ], + "positive": [ + "92:130", + 0 + ], + "negative": [ + "92:128", + 0 + ] + }, + "class_type": "CFGGuider", + "_meta": { + "title": "CFGGuider" + } + }, + "92:115": { + "inputs": { + "steps": 20, + "width": [ + "92:108", + 0 + ], + "height": [ + "92:108", + 1 + ] + }, + "class_type": "Flux2Scheduler", + "_meta": { + "title": "Flux2Scheduler" + } + }, + "92:125": { + "inputs": { + "conditioning": [ + "92:87", + 0 + ], + "latent": [ + "92:126", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "92:126": { + "inputs": { + "pixels": [ + "92:110", + 0 + ], + "vae": [ + "92:107", + 0 + ] + }, + "class_type": "VAEEncode", + "_meta": { + "title": "VAE Encode" + } + }, + "92:127": { + "inputs": { + "conditioning": [ + "92:113", + 0 + ], + "latent": [ + "92:126", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "92:128": { + "inputs": { + "conditioning": [ + "92:125", + 0 + ], + "latent": [ + "92:129", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "92:129": { + "inputs": { + "pixels": [ + "92:85", + 0 + ], + "vae": [ + "92:107", + 0 + ] + }, + "class_type": "VAEEncode", + "_meta": { + "title": "VAE Encode" + } + }, + "92:130": { + "inputs": { + "conditioning": [ + "92:127", + 0 + ], + "latent": [ + "92:129", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + } +} \ No newline at end of file diff --git a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json new file mode 100644 index 0000000..7745918 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json @@ -0,0 +1,281 @@ +{ + "9": { + "inputs": { + "filename_prefix": "klein_edit", + "images": [ + "75:65", + 0 + ] + }, + "class_type": "SaveImage", + "_meta": { + "title": "Save Image" + } + }, + "76": { + "inputs": { + "image": "" + }, + "class_type": "LoadImage", + "_meta": { + "title": "Load Image" + } + }, + "81": { + "inputs": { + "image": "__IMAGE2__" + }, + "class_type": "LoadImage", + "_meta": { + "title": "Load Image" + } + }, + "75:61": { + "inputs": { + "sampler_name": "euler" + }, + "class_type": "KSamplerSelect", + "_meta": { + "title": "KSamplerSelect" + } + }, + "75:62": { + "inputs": { + "steps": 20, + "width": [ + "75:100", + 0 + ], + "height": [ + "75:100", + 1 + ] + }, + "class_type": "Flux2Scheduler", + "_meta": { + "title": "Flux2Scheduler" + } + }, + "75:63": { + "inputs": { + "cfg": 5, + "model": [ + "75:70", + 0 + ], + "positive": [ + "75:124", + 0 + ], + "negative": [ + "75:122", + 0 + ] + }, + "class_type": "CFGGuider", + "_meta": { + "title": "CFGGuider" + } + }, + "75:64": { + "inputs": { + "noise": [ + "75:73", + 0 + ], + "guider": [ + "75:63", + 0 + ], + "sampler": [ + "75:61", + 0 + ], + "sigmas": [ + "75:62", + 0 + ], + "latent_image": [ + "75:66", + 0 + ] + }, + "class_type": "SamplerCustomAdvanced", + "_meta": { + "title": "SamplerCustomAdvanced" + } + }, + "75:65": { + "inputs": { + "samples": [ + "75:64", + 0 + ], + "vae": [ + "75:72", + 0 + ] + }, + "class_type": "VAEDecode", + "_meta": { + "title": "VAE Decode" + } + }, + "75:73": { + "inputs": { + "noise_seed": 6 + }, + "class_type": "RandomNoise", + "_meta": { + "title": "RandomNoise" + } + }, + "75:70": { + "inputs": { + "unet_name": "flux-2-klein-base-9b-fp8.safetensors", + "weight_dtype": "default" + }, + "class_type": "UNETLoader", + "_meta": { + "title": "Load Diffusion Model" + } + }, + "75:71": { + "inputs": { + "clip_name": "qwen_3_8b_fp8mixed.safetensors", + "type": "flux2", + "device": "default" + }, + "class_type": "CLIPLoader", + "_meta": { + "title": "Load CLIP" + } + }, + "75:74": { + "inputs": { + "text": "__PROMPT__", + "clip": [ + "75:71", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Positive Prompt)" + } + }, + "75:67": { + "inputs": { + "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark", + "clip": [ + "75:71", + 0 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Negative Prompt)" + } + }, + "75:72": { + "inputs": { + "vae_name": "diffusion_pytorch_model.safetensors" + }, + "class_type": "VAELoader", + "_meta": { + "title": "Load VAE" + } + }, + "75:66": { + "inputs": { + "width": [ + "75:100", + 0 + ], + "height": [ + "75:100", + 1 + ], + "batch_size": 1 + }, + "class_type": "EmptyFlux2LatentImage", + "_meta": { + "title": "Empty Flux 2 Latent" + } + }, + "75:80": { + "inputs": { + "upscale_method": "lanczos", + "megapixels": 1, + "resolution_steps": 1, + "image": [ + "76", + 0 + ] + }, + "class_type": "ImageScaleToTotalPixels", + "_meta": { + "title": "ImageScaleToTotalPixels" + } + }, + "75:100": { + "inputs": { + "image": [ + "75:80", + 0 + ] + }, + "class_type": "GetImageSize", + "_meta": { + "title": "Get Image Size" + } + }, + "75:122": { + "inputs": { + "conditioning": [ + "75:67", + 0 + ], + "latent": [ + "75:123", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + }, + "75:123": { + "inputs": { + "pixels": [ + "75:80", + 0 + ], + "vae": [ + "75:72", + 0 + ] + }, + "class_type": "VAEEncode", + "_meta": { + "title": "VAE Encode" + } + }, + "75:124": { + "inputs": { + "conditioning": [ + "75:74", + 0 + ], + "latent": [ + "75:123", + 0 + ] + }, + "class_type": "ReferenceLatent", + "_meta": { + "title": "ReferenceLatent" + } + } +} \ No newline at end of file diff --git a/plugins/antianqi/comfyui-studio/workflows/image-to-image.json b/plugins/antianqi/comfyui-studio/workflows/image-to-image.json deleted file mode 100644 index 4efde79..0000000 --- a/plugins/antianqi/comfyui-studio/workflows/image-to-image.json +++ /dev/null @@ -1,81 +0,0 @@ -{ - "3": { - "class_type": "CheckpointLoaderSimple", - "inputs": { - "ckpt_name": "any.safetensors" - } - }, - "4": { - "class_type": "CLIPLoader", - "inputs": { - "clip_name1": "any_clip.safetensors", - "type": "stable_diffusion" - } - }, - "10": { - "class_type": "LoraLoader", - "inputs": { - "model": ["3", 0], - "clip": ["4", 0], - "lora_name": "any_optional_lora.safetensors", - "strength_model": 0.85, - "strength_clip": 0.85 - } - }, - "20": { - "class_type": "LoadImage", - "inputs": { - "image": "input.png" - } - }, - "21": { - "class_type": "VAEEncode", - "inputs": { - "pixels": ["20", 0], - "vae": ["3", 2] - } - }, - "30": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "__PROMPT__", - "clip": ["10", 1] - } - }, - "31": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift", - "clip": ["10", 1] - } - }, - "60": { - "class_type": "KSampler", - "inputs": { - "model": ["10", 0], - "positive": ["30", 0], - "negative": ["31", 0], - "latent_image": ["21", 0], - "seed": 42, - "steps": 25, - "cfg": 6.5, - "sampler_name": "dpmpp_2m", - "scheduler": "karras", - "denoise": 0.55 - } - }, - "70": { - "class_type": "VAEDecode", - "inputs": { - "samples": ["60", 0], - "vae": ["3", 2] - } - }, - "80": { - "class_type": "SaveImage", - "inputs": { - "images": ["70", 0], - "filename_prefix": "comfyui_studio_i2i" - } - } -} diff --git a/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json b/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json index 40fe686..b6a8441 100644 --- a/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json +++ b/plugins/antianqi/comfyui-studio/workflows/selfie-mimicry.json @@ -1,98 +1,513 @@ { - "3": { - "class_type": "CheckpointLoaderSimple", + "89": { "inputs": { - "ckpt_name": "any_portrait_checkpoint.safetensors" + "text": "__PROMPT__", + "clip": [ + "406", + 1 + ] + }, + "class_type": "CLIPTextEncode", + "_meta": { + "title": "CLIP Text Encode (Prompt)" } }, - "4": { - "class_type": "CLIPLoader", + "251": { "inputs": { - "clip_name1": "any_clip.safetensors", - "type": "stable_diffusion" + "rgthree_comparer": { + "images": [ + { + "name": "A", + "selected": true, + "url": "/api/view?filename=rgthree.compare._temp_pqzjp_00001_.png&type=temp&subfolder=&rand=0.7865590674908698" + }, + { + "name": "B", + "selected": true, + "url": "/api/view?filename=rgthree.compare._temp_pqzjp_00002_.png&type=temp&subfolder=&rand=0.4441934895916896" + } + ] + }, + "image_a": [ + "322", + 0 + ], + "image_b": [ + "314", + 0 + ] + }, + "class_type": "Image Comparer (rgthree)", + "_meta": { + "title": "Image Comparer (rgthree)" } }, - "10": { - "class_type": "LoraLoader", + "314": { "inputs": { - "model": ["3", 0], - "clip": ["4", 0], - "lora_name": "your_face_lora.safetensors", - "strength_model": 0.85, - "strength_clip": 0.85 + "image": "__IMAGE1__" + }, + "class_type": "LoadImage", + "_meta": { + "title": "Load Image" } }, - "20": { - "class_type": "IPAdapterModelLoader", + "316": { "inputs": { - "ipadapter_file": "ip-adapter-faceid-portrait.bin" + "preprocessor": "LineArtPreprocessor", + "resolution": [ + "403", + 0 + ], + "image": [ + "380", + 0 + ] + }, + "class_type": "AIO_Preprocessor", + "_meta": { + "title": "AIO Aux Preprocessor" } }, - "30": { - "class_type": "CLIPTextEncode", + "317": { "inputs": { - "text": "__PROMPT__", - "clip": ["10", 1] + "images": [ + "404", + 0 + ] + }, + "class_type": "PreviewImage", + "_meta": { + "title": "Preview Image" } }, - "31": { - "class_type": "CLIPTextEncode", + "319": { "inputs": { - "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift, multiple people, two people, group", - "clip": ["10", 1] + "strength": 0.8, + "model": [ + "477", + 0 + ], + "model_patch": [ + "320", + 0 + ], + "vae": [ + "406", + 2 + ], + "image": [ + "404", + 0 + ] + }, + "class_type": "ZImageFunControlnet", + "_meta": { + "title": "ZImageFunControlnet" } }, - "40": { - "class_type": "LoadImage", + "320": { "inputs": { - "image": "reference_face.png" + "name": "Z-Image-Fun-Controlnet-Union-2.1.safetensors" + }, + "class_type": "ModelPatchLoader", + "_meta": { + "title": "ModelPatchLoader" } }, - "50": { - "class_type": "IPAdapterApply", + "322": { "inputs": { - "ipadapter": ["20", 0], - "image": ["40", 0], - "weight": 0.7, - "noise": 0.05, - "weight_type": "linear" + "samples": [ + "410", + 0 + ], + "vae": [ + "406", + 2 + ] + }, + "class_type": "VAEDecode", + "_meta": { + "title": "VAE Decode" } }, - "60": { - "class_type": "EmptyLatentImage", + "380": { "inputs": { - "width": 832, - "height": 1216, - "batch_size": 1 + "aspect_ratio": "original", + "proportional_width": 1, + "proportional_height": 1, + "fit": "crop", + "method": "lanczos", + "round_to_multiple": "16", + "scale_to_side": "longest", + "scale_to_length": [ + "455", + 0 + ], + "background_color": "#000000", + "image": [ + "314", + 0 + ], + "mask": [ + "314", + 1 + ] + }, + "class_type": "LayerUtility: ImageScaleByAspectRatio V2", + "_meta": { + "title": "LayerUtility: ImageScaleByAspectRatio V2" } }, - "70": { + "399": { + "inputs": { + "text": "画面风格：高清晰度商业人像摄影风格，采用暖色调室内布光，光影对比强烈，主体突出，背景虚化柔和，色彩饱和度适中，整体氛围神秘而高贵，具有强烈的时尚杂志质感，视觉焦点集中在人物面部与上半身，细节丰富且锐利。\n\n核心元素：年轻亚洲女性角色，深棕色长发自然垂落，略带波浪卷曲，发丝蓬松有光泽，部分头发向后梳理露出额头与耳朵，耳侧有碎发修饰脸型。人物佩戴一对大尺寸圆形珍珠耳环，左手无名指戴有一枚花朵造型戒指，右手手腕叠戴多层细珠链手镯，右臂内侧可见一条纤细红色手绳。身着黑色无袖紧身连衣裙，领口呈高领圆领设计，胸前垂直开衩并缀有两圈横向排列的珍珠装饰环，腰间系有一条镶嵌水钻的金属质感腰带，呈不规则环形结构，腰身两侧有褶皱收束设计，凸显身材曲线。左腿膝盖上方可见黑色吊袜带连接至腿部，暗示穿着丝袜或长筒袜。背景为暖金色调墙面，表面带有浮雕式纹理或壁纸图案，右侧隐约可见一盏壁灯轮廓，光线从左上方投射，在人物面部与肩部形成柔和的高光，阴影过渡自然，增强立体感。\n\n具体内容：女性角色五官精致立体，眉形流畅上扬，眼妆浓重且层次分明，眼线拉长上扬，睫毛纤长卷翘，瞳孔深邃有神，鼻梁高挺，唇色为柔和的裸粉色，唇形饱满，嘴角微抿，表情冷艳中带一丝慵懒，眼神直视镜头，充满自信与吸引力。身体姿态呈半坐半倚状态，左手轻托下巴靠近脸颊，手指修长，指甲修剪整齐，右手自然搭在身侧或椅背上，肩部微微内扣，脊柱挺直，胸部微挺，腰部收紧，腿部线条流畅，整体坐姿优雅且富有张力，展现出成熟女性的魅力与掌控力。\n\n构图方式：采用中心偏右构图，人物占据画面主要区域，头部略高于画面中线，视线方向朝向镜头，留出适当空间。前景为人物面部与手部特写，中景为躯干与服装细节，远景为模糊的背景墙面与灯具轮廓，形成清晰的前后景深层次。相机参数设定为中画幅或高端单反相机，使用85mm定焦镜头，光圈f/2.0-f/2.8，快门速度1/125秒，ISO 200-400，白平衡设置为3500K-4000K暖色温，对焦模式为单点对焦锁定于眼部，确保面部与珠宝细节极致清晰锐利。光线来自左前方45度角，主光源柔和均匀，辅光补亮右侧暗部，轮廓光从右后方勾勒发丝边缘，增强立体感。整体色彩基调以黑色服装与暖金色背景形成冷暖对比，辅以肤色、珍珠白、金属银等点缀色，营造奢华而性感的视觉体验。\n\n光影效果：主光源为暖色温柔光箱或大型LED平板灯，从左侧斜向照射，照亮人物正面与左侧轮廓，右侧保留适度阴影增强立体感。高光集中在额头、鼻梁、颧骨、锁骨及肩部，反射柔和，无明显硬边。阴影区域如颈部下方、腋下、裙摆内侧呈现渐变过渡，无突兀黑块。环境光来自右侧背景，轻微提亮暗部，平衡整体明暗。体积光不明显，但通过暖色背景反射营造出包围式柔光氛围，人物轮廓边缘有微弱辉光，增强分离感。\n\n材质质感：皮肤呈现健康光泽，毛孔与细微纹理清晰可见，尤其在脸颊与鼻翼处，体现真实肤质。头发丝缕分明，富有弹性与光泽，反射柔和光线。服装面料为哑光针织或弹性棉混纺，贴合身体曲线，褶皱自然流畅，腰部与胸前开衩处有轻微拉伸感。珍珠饰品表面光滑圆润，折射柔和光泽，金属腰带与手镯具有镜面反射特性，钻石或水钻部分呈现细碎闪光。\n\n纹理密度：皮肤纹理细腻，毛孔分布自然，无过度磨皮痕迹。头发密度适中，发根蓬松，发梢微卷，层次分明。服装织物纹理细腻，有轻微竖向条纹或编织结构，尤其在腰部与袖口处可见细微褶皱。饰品表面抛光度高，反光清晰，无划痕或瑕疵。\n\n解剖结构：人物身材比例匀称，肩宽适中，锁骨明显，胸部饱满，腰部纤细，臀部圆润，腿部修长，肌肉线条流畅但不夸张，关节弯曲自然，脊柱曲线优美，整体姿态符合人体工学，无僵硬或扭曲感。\n\n空间位置：人物位于画面右侧三分之二处，头部居中偏右，视线正对镜头，身体略微侧转，左手抬起靠近面部，右手自然下垂，双腿交叉或并拢，左腿在前，右腿在后，形成动态平衡。背景墙面占据左侧三分之一，灯具位于右上角，构成稳定的三角构图。\n\n姿势分析：整体姿态为半坐姿，躯干微倾，头部略低，眼神专注，双手一上一下形成视觉引导线，肩部放松但挺拔，手臂自然弯曲，肘部微屈，手腕柔韧，手指姿态优雅，大腿与膝盖角度协调，骨盆轻微旋转，展现柔美曲线，整体动作流畅自然，富有表现力。\n\n技术参数：相机型号建议为索尼A7R IV或佳能R5，镜头为索尼FE 85mm f/1.4 GM或佳能RF 85mm f/1.2 L USM，拍摄模式为手动档，焦距85mm，光圈f/2.2，快门1/125s，ISO 200，白平衡3800K，对焦模式AF-C + 眼部追踪，曝光补偿+0.3EV，使用反光板或柔光箱辅助补光，三脚架固定拍摄，确保稳定性与清晰度。", + "anything": [ + "457", + 0 + ] + }, + "class_type": "easy showAnything", + "_meta": { + "title": "Show Any" + } + }, + "403": { + "inputs": { + "value": "min(a,b)", + "a": [ + "380", + 3 + ], + "b": [ + "380", + 4 + ] + }, + "class_type": "SimpleMath+", + "_meta": { + "title": "🔧 Simple Math" + } + }, + "404": { + "inputs": { + "any_01": [ + "316", + 0 + ] + }, + "class_type": "Any Switch (rgthree)", + "_meta": { + "title": "Any Switch (rgthree)" + } + }, + "406": { + "inputs": { + "ckpt_name": "Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors" + }, + "class_type": "CheckpointLoaderSimple", + "_meta": { + "title": "Load Checkpoint" + } + }, + "407": { + "inputs": { + "conditioning": [ + "89", + 0 + ] + }, + "class_type": "ConditioningZeroOut", + "_meta": { + "title": "ConditioningZeroOut" + } + }, + "408": { + "inputs": { + "pixels": [ + "380", + 0 + ], + "vae": [ + "406", + 2 + ] + }, + "class_type": "VAEEncode", + "_meta": { + "title": "VAE Encode" + } + }, + "409": { + "inputs": { + "seed": 46214279332840, + "steps": 8, + "cfg": 1, + "sampler_name": "res_2s", + "scheduler": "linear_quadratic", + "denoise": 1, + "model": [ + "319", + 0 + ], + "positive": [ + "89", + 0 + ], + "negative": [ + "407", + 0 + ], + "latent_image": [ + "408", + 0 + ] + }, "class_type": "KSampler", + "_meta": { + "title": "KSampler" + } + }, + "410": { "inputs": { - "model": ["10", 0], - "positive": ["50", 0], - "negative": ["31", 0], - "latent_image": ["60", 0], - "seed": 42, - "steps": 25, - "cfg": 6.0, - "sampler_name": "dpmpp_2m", - "scheduler": "karras", - "denoise": 1.0 + "add_noise": "enable", + "noise_seed": 276067442284220, + "steps": 8, + "cfg": 1, + "sampler_name": "euler", + "scheduler": "simple", + "start_at_step": 5, + "end_at_step": 10000, + "return_with_leftover_noise": "disable", + "model": [ + "411", + 0 + ], + "positive": [ + "89", + 0 + ], + "negative": [ + "407", + 0 + ], + "latent_image": [ + "409", + 0 + ] + }, + "class_type": "KSamplerAdvanced", + "_meta": { + "title": "KSampler (Advanced)" } }, - "80": { - "class_type": "VAEDecode", + "411": { + "inputs": { + "unet_name": "ZIT-moodyPornMix_zitV10R1DPO_fp16.safetensors", + "weight_dtype": "default" + }, + "class_type": "UNETLoader", + "_meta": { + "title": "Load Diffusion Model" + } + }, + "455": { + "inputs": { + "value": 1920 + }, + "class_type": "INTConstant", + "_meta": { + "title": "整数常量" + } + }, + "456": { + "inputs": { + "text": "__TRIGGER__" + }, + "class_type": "CR Text", + "_meta": { + "title": "文本" + } + }, + "457": { + "inputs": { + "any_01": [ + "474", + 0 + ], + "any_02": [ + "456", + 0 + ] + }, + "class_type": "Any Switch (rgthree)", + "_meta": { + "title": "Any Switch (rgthree)" + } + }, + "458": { "inputs": { - "samples": ["70", 0], - "vae": ["3", 2] + "offload_model": true, + "offload_cache": true, + "anything": [ + "322", + 0 + ] + }, + "class_type": "VRAMCleanup", + "_meta": { + "title": "释放显存" } }, - "90": { + "459": { + "inputs": { + "clean_file_cache": true, + "clean_processes": true, + "clean_dlls": true, + "retry_times": 3, + "anything": [ + "322", + 0 + ] + }, + "class_type": "RAMCleanup", + "_meta": { + "title": "释放内存" + } + }, + "460": { + "inputs": { + "offload_model": true, + "offload_cache": true, + "anything": [ + "409", + 0 + ] + }, + "class_type": "VRAMCleanup", + "_meta": { + "title": "释放显存" + } + }, + "461": { + "inputs": { + "clean_file_cache": true, + "clean_processes": true, + "clean_dlls": true, + "retry_times": 3, + "anything": [ + "409", + 0 + ] + }, + "class_type": "RAMCleanup", + "_meta": { + "title": "释放内存" + } + }, + "471": { + "inputs": { + "filename_prefix": "selfie_mimic", + "images": [ + "322", + 0 + ] + }, "class_type": "SaveImage", + "_meta": { + "title": "Save Image" + } + }, + "472": { + "inputs": { + "model": "Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf", + "mmproj": "mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf", + "chat_handler": "Qwen3.5", + "n_ctx": 8192, + "vram_limit": -1, + "image_min_tokens": 0, + "image_max_tokens": 0 + }, + "class_type": "llama_cpp_model_loader", + "_meta": { + "title": "Llama-cpp Model Loader" + } + }, + "473": { + "inputs": { + "max_tokens": 4096, + "top_k": 30, + "top_p": 0.9, + "min_p": 0.05, + "typical_p": 1, + "temperature": 0.8, + "repeat_penalty": 1, + "frequency_penalty": 0, + "presence_penalty": 1, + "mirostat_mode": 0, + "mirostat_eta": 0.1, + "mirostat_tau": 5, + "state_uid": -1 + }, + "class_type": "llama_cpp_parameters", + "_meta": { + "title": "Llama-cpp Parameters" + } + }, + "474": { + "inputs": { + "preset_prompt": "Prompt Style - Extreme Detailed", + "custom_prompt": "严禁输出思考过程，只反推输出中文提示词", + "system_prompt": "你是专注于图像反推提示词生成的专业AI助手，需基于用户上传的角色照片，分析并提取详细的图片信息，最终为图像编辑模型生成提示词。具体执行步骤如下：\n\n### 最高指令\n1.语言自适应：识别用户输入语言。用户用中文提问，你输出中文指令；用户用英文提问，你输出英文指令；若未输入语言，默认输出中文指令。\n2.格式绝对纯净：严禁输出 Markdown 符号（如星号、井号）、严禁中英对照括号、严禁输出任何解释或前缀。\n3.输出提示词总字数控制在1000字以内。\n4.不要描述设计师名称、日期等文字水印。\n\n### 核心规则\n1.详细提取图片中的画面信息，当用户提供参考图时，需对图片进行全面详细分析，提取所有可见元素（包括主体、背景、文字、光影、材质、纹理、解剖结构等），确保无遗漏。\n2.生成的提示词必须进行多层次、多维度的细节挖掘，达到像素级的细节提取精度。；\n3.生成的提示词中禁止包含水印、边框等多余内容。\n4.所有提示词必须包含画面风格、核心元素、具体内容、构图方式四个核心模块，每个模块描述需达到AI可直接识别并生成的精度，不得遗漏。\n\n### 约束条件\n1.正向引导：禁用负面提示词，完全使用正向约束表达（例：不用不要模糊，而写极致锐利的对焦，画面细节清晰）。\n2.自然语言：使用自然语言语法，语句连贯、符合语法，禁止无逻辑的标签堆料，需用有逻辑、有流畅度的句式。\n3.拒绝模糊：禁止使用模糊性描述（如好看的颜色、大概的形状），需采用精准术语（如金黄色渐变、等边三角形的几何结构）。\n4.禁止使用微观悬浮粒子类描述：不得出现“微尘”“漂浮粒子”“空气中的细小颗粒”“光雾粒子”“盐粒”“花粉”“水雾微粒”等无法被主流图像模型稳定可视化的微观悬浮物描写。若需表现氛围感，必须使用“体积光”“丁达尔光束”“光晕扩散”“柔焦虚化”等可视觉化的宏观光学现象，并确保其描述方式符合主流图像生成模型的训练数据模式。。\n5.通用适配：提示词需适配所有主流AI绘图工具，避免使用工具专属语法。\n6.纯净格式：输出不得包含任何Markdown标记、代码块符号。文字之间需保留正常空格以确保语义通顺，但段落之间紧凑排列，不要出现多余空行。\n7.最大生成量：输出提示词总字数控制在1000字以内。\n\n### 提示词生成\n1.提示词生成内容应包含图片场景、构图比例、服装搭配、动作姿势、色彩基调、整体氛围、相机参数（如果为写实拍摄的情况下）等描述；\n1.若用户输入特定需求，按照用户要求对提示词中的元素进行变更；\n2.若用户无特定要求，严格根据参考图内容生成提示词；\n\n### 核心要素\n**场景描述**：详细描述背景环境、场景布局、整体氛围。\n**构图及空间描述**：详细描述图片的整体构图方式以及前景、中景、近景和远景等空间关系。\n**主体内容**：详细描述人物/物体主体特征，包含人物国籍、长相特征等。\n**服装搭配**：详细描述服装、配饰、道具。\n**动作姿势**：详细描述动作，动作必须自然。\n**面部表情**：详细描述整体氛围和面部情绪。\n**技术参数**：描述拍摄使用的相机参数。\n\n###细节特征\n**光影效果**：描述光源方向、强度、色温、阴影形状与过渡、高光位置；若需表现空气感或氛围，仅允许使用“丁达尔光束”“柔和体积光”“逆光轮廓辉光”“环境光遮蔽”等AI可识别的宏观光学现象，禁止引用不可见或微观粒子作为光影成因。\n**材质质感**：描述表面纹理、反光特性、透明度、柔软度。\n**纹理密度**：描述皮肤纹理、织物纹路、衣物褶皱、毛发细节。\n**解剖结构**：描述身体比例、肌肉线条、骨骼结构、面部特征。\n**空间位置**：明确人物朝向、元素相对位置、构图方式（如对称、三分法），并细化前景、中景、远景的层次关系。\n**姿势分析**：详细描述身体整体姿态、四肢位置角度、关节弯曲程度、肌肉紧张状态、运动方向和动态感。\n**技术参数（如果为写实拍摄的情况下）**：描述相机型号、镜头参数、拍摄模式、焦距、光圈等参数。\n\n###格式校验\n1.无模糊性描述，所有细节均精准可量化。\n2.姿势描述详细完整，包含身体整体姿态、四肢位置角度、关节弯曲程度、肌肉紧张状态、运动方向和动态感。\n3.符合了用户的特定要求（如有）。\n4.适配所有主流AI绘图工具，无工具专属语法。\n5.无Markdown符号，无多余空行。\n\n###输出规范\n1.所有提示词直接输出，不得包含任何额外内容（如好的，这是提示词）。\n2.提示词需详尽到AI可直接生成与描述完全一致的画面，每个元素都有充分的细节描述，不得遗漏任何关键细节。\n3.若生成多组提示词，每组之间仅用换行分隔，不得添加任何分隔符。\n4.最终输出提示词总字数控制在1000字以内。\n\n###实例\n画面风格：自然写实摄影风格，采用高清晰度与高分辨率呈现，光影过渡柔和细腻，色彩还原真实且富有层次感，整体氛围清新自然，具有强烈的沉浸式视觉体验。\n核心元素：年轻的亚洲女性角色，手持双束粉色玫瑰花，身着浅粉色针织开衫，背景为开阔的玫瑰花田，绿色植被与粉花形成鲜明色彩对比，人物位于画面中心偏右位置，构图采用三分法与视觉引导线结合，前景为人物与花束，中景为花田延伸，远景为模糊的绿色山丘轮廓，增强空间纵深感。\n具体内容：女性角色拥有深棕色长发，自然垂落肩头，面部五官精致，眼神柔和专注，嘴唇微启，表情自然放松。她双手各持一束粉色玫瑰，左手花束靠近面部，右手花束高举过肩，姿态优雅且富有动态感。服装为浅粉色V领针织开衫，纹理清晰可见，袖口自然收束。花束由多支盛开的粉色玫瑰组成，花瓣层次分明，绿叶点缀其间，茎秆挺拔。背景花田中玫瑰花密集分布，颜色从浅粉到深粉渐变，远处山丘轮廓柔和，天空呈现淡蓝色，无明显云层。\n构图方式：采用三分法构图，人物位于右侧三分之一处，视线方向留有适当空间。前景花束与人物面部形成视觉焦点，中景花田引导视线向远景延伸，形成自然的视觉动线。相机参数设定为全画幅传感器，使用85mm定焦镜头，光圈f/2.8，快门速度1/200秒，ISO 100，白平衡自动，对焦模式为单点对焦，确保人物面部与花束细节清晰锐利。光线来自左前方，形成柔和的侧逆光效果，人物轮廓边缘有轻微辉光，阴影过渡自然，无明显硬边。整体色彩基调以粉色与绿色为主，辅以淡蓝色天空，营造出温馨浪漫的春日氛围。", + "inference_mode": "images", + "max_frames": 24, + "max_size": 256, + "seed": 331347452927766, + "force_offload": true, + "save_states": false, + "llama_model": [ + "472", + 0 + ], + "parameters": [ + "473", + 0 + ], + "images": [ + "380", + 0 + ] + }, + "class_type": "llama_cpp_instruct_adv", + "_meta": { + "title": "Llama-cpp Instruct" + } + }, + "476": { + "inputs": { + "lora_name": "your_face_lora.safetensors", + "strength_model": 1, + "model": [ + "406", + 0 + ] + }, + "class_type": "LoraLoaderModelOnly", + "_meta": { + "title": "Load LoRA" + } + }, + "477": { "inputs": { - "images": ["80", 0], - "filename_prefix": "selfie_mimic" + "lora_name": "your_style_lora.safetensors", + "strength_model": 1, + "model": [ + "476", + 0 + ] + }, + "class_type": "LoraLoaderModelOnly", + "_meta": { + "title": "Load LoRA" } } -} +} \ No newline at end of file diff --git a/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json b/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json index 52e06b2..95c7719 100644 --- a/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json +++ b/plugins/antianqi/comfyui-studio/workflows/selfie-text-to-image.json @@ -2,105 +2,74 @@ "3": { "class_type": "CheckpointLoaderSimple", "inputs": { - "ckpt_name": "any_portrait_checkpoint.safetensors" - } - }, - "4": { - "class_type": "CLIPLoader", - "inputs": { - "clip_name1": "any_clip.safetensors", - "type": "stable_diffusion" + "ckpt_name": "Z-Image-Base-8steps-豹豹喵呜の白玉v2White_Marble-AIO_v2-bf16.safetensors" } }, "10": { - "class_type": "LoraLoader", + "class_type": "LoraLoaderModelOnly", "inputs": { "model": ["3", 0], - "clip": ["4", 0], "lora_name": "your_face_lora.safetensors", - "strength_model": 0.85, - "strength_clip": 0.85 + "strength_model": 1.0 } }, "20": { - "class_type": "ControlNetLoader", + "class_type": "LoraLoaderModelOnly", "inputs": { - "control_net_name": "any_face_controlnet.safetensors" + "model": ["10", 0], + "lora_name": "your_style_lora.safetensors", + "strength_model": 1.0 } }, "30": { "class_type": "CLIPTextEncode", "inputs": { "text": "__PROMPT__", - "clip": ["10", 1] + "clip": ["3", 1] } }, "31": { "class_type": "CLIPTextEncode", "inputs": { "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift, multiple people, two people, group", - "clip": ["10", 1] - } - }, - "40": { - "class_type": "LoadImage", - "inputs": { - "image": "face_reference.png" + "clip": ["3", 1] } }, "50": { "class_type": "EmptyLatentImage", "inputs": { - "width": 832, - "height": 1216, + "width": 1008, + "height": 1792, "batch_size": 1 } }, "60": { - "class_type": "ControlNetApply", - "inputs": { - "conditioning": ["30", 0], - "control_net": ["20", 0], - "image": ["40", 0], - "strength": 0.75 - } - }, - "61": { - "class_type": "ControlNetApply", - "inputs": { - "conditioning": ["31", 0], - "control_net": ["20", 0], - "image": ["40", 0], - "strength": 0.6 - } - }, - "70": { "class_type": "KSampler", "inputs": { - "model": ["10", 0], - "positive": ["60", 0], - "negative": ["61", 0], + "model": ["20", 0], + "positive": ["30", 0], + "negative": ["31", 0], "latent_image": ["50", 0], - "seed": 42, - "steps": 28, - "cfg": 6.0, - "sampler_name": "dpmpp_2m", - "scheduler": "karras", + "seed": 72125, + "steps": 8, + "cfg": 1.5, + "sampler_name": "euler", + "scheduler": "simple", "denoise": 1.0 } }, - "80": { + "70": { "class_type": "VAEDecode", "inputs": { - "samples": ["70", 0], + "samples": ["60", 0], "vae": ["3", 2] } }, - "90": { + "80": { "class_type": "SaveImage", "inputs": { - "images": ["80", 0], - "filename_prefix": "selfie" + "images": ["70", 0], + "filename_prefix": "zib_selfie" } } } diff --git a/plugins/antianqi/comfyui-studio/workflows/text-to-image.json b/plugins/antianqi/comfyui-studio/workflows/text-to-image.json deleted file mode 100644 index 8f04727..0000000 --- a/plugins/antianqi/comfyui-studio/workflows/text-to-image.json +++ /dev/null @@ -1,76 +0,0 @@ -{ - "3": { - "class_type": "CheckpointLoaderSimple", - "inputs": { - "ckpt_name": "any.safetensors" - } - }, - "4": { - "class_type": "CLIPLoader", - "inputs": { - "clip_name1": "any_clip.safetensors", - "type": "stable_diffusion" - } - }, - "10": { - "class_type": "LoraLoader", - "inputs": { - "model": ["3", 0], - "clip": ["4", 0], - "lora_name": "any_optional_lora.safetensors", - "strength_model": 0.85, - "strength_clip": 0.85 - } - }, - "30": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "__PROMPT__", - "clip": ["10", 1] - } - }, - "31": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "deformed, bad anatomy, extra fingers, blurry, low quality, watermark, different character, identity drift", - "clip": ["10", 1] - } - }, - "50": { - "class_type": "EmptyLatentImage", - "inputs": { - "width": 1024, - "height": 1024, - "batch_size": 1 - } - }, - "60": { - "class_type": "KSampler", - "inputs": { - "model": ["10", 0], - "positive": ["30", 0], - "negative": ["31", 0], - "latent_image": ["50", 0], - "seed": 42, - "steps": 25, - "cfg": 6.5, - "sampler_name": "dpmpp_2m", - "scheduler": "karras", - "denoise": 1.0 - } - }, - "70": { - "class_type": "VAEDecode", - "inputs": { - "samples": ["60", 0], - "vae": ["3", 2] - } - }, - "80": { - "class_type": "SaveImage", - "inputs": { - "images": ["70", 0], - "filename_prefix": "comfyui_studio" - } - } -} From 2ee26668964f386e7920346488b7574441eb348a Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sat, 22 Aug 2026 11:40:12 +0800 Subject: [PATCH 4/7] Add beta test channel: v0.2.0-beta.1, BETA.md, README beta banner - plugin.json: bump version 0.1.0 -> 0.2.0-beta.1; expand description to mention 4 Skills + 6 scenarios - BETA.md: new file. Documents 3 install options (git clone the fork / download release tarball / browse the plugin folder), 6-scenario test matrix, 3 feedback channels (GitHub issues / Feishu mcode group / PR comments), known-issues section, and versioning policy - README.md: add a top-of-file beta test banner pointing to BETA.md and the upstream PR, so anyone landing on the plugin via the fork's plugin folder can find the test install instructions --- plugins/antianqi/comfyui-studio/BETA.md | 146 ++++++++++++++++++++ plugins/antianqi/comfyui-studio/README.md | 8 ++ plugins/antianqi/comfyui-studio/plugin.json | 4 +- 3 files changed, 156 insertions(+), 2 deletions(-) create mode 100644 plugins/antianqi/comfyui-studio/BETA.md diff --git a/plugins/antianqi/comfyui-studio/BETA.md b/plugins/antianqi/comfyui-studio/BETA.md new file mode 100644 index 0000000..12af146 --- /dev/null +++ b/plugins/antianqi/comfyui-studio/BETA.md @@ -0,0 +1,146 @@ +# 🧪 Beta Test Channel + +> **Status**: This is the beta test distribution of `comfyui-studio`. The official +> PR ([MiniMax-AI/MiniMax-Code-Plugins#15](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/15)) +> is open and waiting for review. While the merge is pending, **this fork is the public test +> channel** for the mcode internal beta group. Please install from here, try the 6 scenarios, +> and report issues back so we can fix them before the official release. + +--- + +## TL;DR + +```bash +# 1. Clone the fork (or download the tarball from the latest beta release) +git clone https://github.com/antianqi/MiniMax-Code-Plugins-1.git + +# 2. Symlink / copy the plugin into your MiniMax Code Plugins directory +ln -s "$(pwd)/MiniMax-Code-Plugins-1/plugins/antianqi/comfyui-studio" \ + "$MCODE_HOME/plugins/antianqi/comfyui-studio" + +# 3. (Optional) Verify +ls "$MCODE_HOME/plugins/antianqi/comfyui-studio/plugin.json" +``` + +> Don't know `$MCODE_HOME`? On most installs it is `~/.minimax/plugins/` (Linux/macOS) or +> `%USERPROFILE%\.minimax\plugins\` (Windows). The Plugin manager UI in mcode also has a +> "Install from local path" button — point it at the `plugins/antianqi/comfyui-studio` +> directory and you are done. + +## 3 install options + +### Option A — Clone the fork (recommended for testing) + +```bash +git clone https://github.com/antianqi/MiniMax-Code-Plugins-1.git +# then point mcode at plugins/antianqi/comfyui-studio/ +``` + +Pros: `git pull` later gets you the latest fixes without re-downloading. +Cons: requires git on the host. + +### Option B — Download a release tarball (recommended for one-shot test) + +Latest beta release: +**[`v0.2.0-beta.1`](https://github.com/antianqi/MiniMax-Code-Plugins-1/releases/tag/v0.2.0-beta.1)** + +```bash +curl -L https://github.com/antianqi/MiniMax-Code-Plugins-1/archive/refs/tags/v0.2.0-beta.1.tar.gz \ + | tar -xz +cd MiniMax-Code-Plugins-1-0.2.0-beta.1 +# then point mcode at plugins/antianqi/comfyui-studio/ +``` + +Pros: a frozen version, easy to roll back. +Cons: you have to re-download to get fixes. + +### Option C — Download just the plugin folder (smallest payload) + +[Browse the plugin folder at this tag](https://github.com/antianqi/MiniMax-Code-Plugins-1/tree/v0.2.0-beta.1/plugins/antianqi/comfyui-studio), +hit "Download raw file" per file, OR use the GitHub CLI: + +```bash +gh release download v0.2.0-beta.1 \ + --repo antianqi/MiniMax-Code-Plugins-1 \ + --pattern '*' \ + --dir comfyui-studio-beta +``` + +## Try it (5 minutes) + +After install, in any mcode session, paste this: + +```text +Use comfyui-studio to verify that my local ComfyUI is reachable, then submit scenario 3 +(改图 / flux2-klein-image-edit) with the prompt "the same person sitting in an office +chair, the same outfit, professional lighting" and report the saved image path. +``` + +Expected: the agent probes ComfyUI, submits the bundled workflow, polls until done, downloads +the image, and tells you where it was saved. The end-to-end walkthrough is in +[`examples/minimal-run.md`](examples/minimal-run.md). + +If you have a trained character LoRA handy, try scenario 1 ("用我的角色画一张自拍") and +scenario 2 ("照着这张照片再画一张同款") — these are the two presets that exercise the +full face-LoRA + style-LoRA + Z-Image stack. + +## What to test (the matrix) + +| Scenario | Trigger | Preset workflow | Notes for testers | +|---|---|---|---| +| 1 | 生图 | `workflows/selfie-text-to-image.json` | Try with and without `--trigger`; try changing `LoraLoaderModelOnly.strength_model` | +| 2 | 模仿 | `workflows/selfie-mimicry.json` | Needs `ComfyUI-LLaMA-CPP` + `comfyui_controlnet_aux`; **will fail on installs without those custom nodes** | +| 3 | 改图 | `workflows/flux2-klein-image-edit.json` | Try different `KSampler.denoise` values (0.4 / 0.75 / 0.95) | +| 4 | 融合 | `workflows/flux2-klein-image-edit-dual.json` | Try with the "person on the left" + "outfit on the right" semantic | +| 5 | 首帧 | `workflows/drama-first-frame.json` | Set the second character LoRA's `strength_model = 0` for single-character dramas | +| 6 | 出片 | `workflows/drama-image-to-video.json` | Distilled LTX-2.3 is VRAM-friendly; non-distilled 22B is 30–60 min/shot | + +## How to give feedback + +Pick whichever channel fits you best: + +- **GitHub issues on the fork** (preferred for reproducible bugs): + https://github.com/antianqi/MiniMax-Code-Plugins-1/issues + Please include: your OS, your ComfyUI version, the scenario number, the prompt you used, + and the full error output (or the prompt_id if ComfyUI swallowed the job). + +- **Feishu mcode internal beta group**: just @ me in the channel where the announcement + landed. Best for "it works but the UX is awkward" / "the docs say X but the workflow does Y". + +- **PR comments**: https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/15 + Best for structural / design feedback on the docs and the 6-scenario split. + +## Known issues / not-yet-implemented + +These are the things that are **intentionally out of scope** for the current PR but might +look like bugs: + +- **Scenario 2 needs custom nodes**: `selfie-mimicry.json` uses + `comfyui_controlnet_aux`, `ComfyUI-LLaMA-CPP`, `rgthree-comfy`, and + `ComfyUI-Impact-Pack`. If your install does not have them, the submission will fail with + `missing_node_type`. We are not bundling these because they vary across ComfyUI forks. + The full list is in `skills/comfyui-character/SKILL.md`. +- **LoRA placeholders**: workflows reference `your_face_lora.safetensors`, + `your_style_lora.safetensors`, `character_a_lora.safetensors`, etc. You **must** edit the + `lora_name` field in the workflow JSON to point at your own file before submission. The + Plugin does not bundle any private LoRAs. +- **Checkpoint filenames are the reference install's filenames**: the JSONs reference the + exact files that ship on the reference ComfyUI install that built this Plugin. On a + different install, you may need to edit the `ckpt_name` / `clip_name1` / `vae_name` / + `control_net_name` fields. The Plugin does not pick abstract placeholder names for + public generation models. +- **No TTS / no FFmpeg / no spreadsheet editor**: stages 2, 3, 6, 7 of the 7-stage drama + pipeline are intentionally out of scope. See `skills/comfyui-drama/SKILL.md` for the full + list. + +## Versioning + +- **`v0.2.0-beta.1`** (this one) — the version in `plugin.json` matches the PR #15 commit. + Mirrors the official PR for testing. +- Future betas will tag on this fork only, with semver `-beta.N` suffixes. +- The first non-beta release (`v0.2.0`) will be published from the **official** repo + (`MiniMax-AI/MiniMax-Code-Plugins`), not this fork. + +## License + +Apache-2.0. See [LICENSE](LICENSE). diff --git a/plugins/antianqi/comfyui-studio/README.md b/plugins/antianqi/comfyui-studio/README.md index f3eed2a..2ff05b4 100644 --- a/plugins/antianqi/comfyui-studio/README.md +++ b/plugins/antianqi/comfyui-studio/README.md @@ -1,5 +1,13 @@ # ComfyUI Studio +> 🧪 **Beta test build** — `v0.2.0-beta.1`. The official PR +> ([MiniMax-AI/MiniMax-Code-Plugins#15](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/15)) +> is open and waiting for review. **To participate in the beta**: install from the +> [`antianqi/MiniMax-Code-Plugins-1`](https://github.com/antianqi/MiniMax-Code-Plugins-1) fork +> following the instructions in [`BETA.md`](BETA.md), then report issues back so we can fix +> them before the official merge. See the bottom of this README for the mcode internal beta +> group context. + A generic, dependency-free toolkit for driving a local ComfyUI server. **Two parts in one Plugin**: **A. a natural-language control layer** for submitting any ComfyUI workflow, polling the queue, and downloading outputs, and **B. six preset workflow templates** that cover the most common recurring diff --git a/plugins/antianqi/comfyui-studio/plugin.json b/plugins/antianqi/comfyui-studio/plugin.json index b81c422..46c7553 100644 --- a/plugins/antianqi/comfyui-studio/plugin.json +++ b/plugins/antianqi/comfyui-studio/plugin.json @@ -1,8 +1,8 @@ { "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", "name": "comfyui-studio", - "version": "0.1.0", - "description": "Generic toolkit for driving a local ComfyUI 8188 server: workflow submission, queue monitoring, image download, and reproducible generation patterns. Includes a dependency-free stdio MCP server and three Skills for agent-driven workflow automation.", + "version": "0.2.0-beta.1", + "description": "Generic toolkit for driving a local ComfyUI 8188 server: workflow submission, queue monitoring, image download, and reproducible generation patterns. Includes a dependency-free stdio MCP server and four Skills (comfyui-studio + comfyui-workflow + comfyui-character + comfyui-drama) covering 6 preset scenarios (生图 / 模仿 / 改图 / 融合 / 首帧 / 出片).", "author": { "name": "antianqi", "url": "https://github.com/antianqi" From 7fbcbcde3d30c8beedbbb2decf82aba564742ca7 Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sat, 22 Aug 2026 16:44:34 +0800 Subject: [PATCH 5/7] BETA.md + README banner: point at the standalone antianqi/comfyui-studio repo The beta test channel is now a dedicated single-plugin repo at github.com/antianqi/comfyui-studio, not this monorepo fork. Update all install / feedback / version references to point there. The BETA.md keeps a note that this fork exists only so the upstream PR can be force-pushed. --- plugins/antianqi/comfyui-studio/BETA.md | 54 +++++++++++++---------- plugins/antianqi/comfyui-studio/README.md | 9 ++-- 2 files changed, 34 insertions(+), 29 deletions(-) diff --git a/plugins/antianqi/comfyui-studio/BETA.md b/plugins/antianqi/comfyui-studio/BETA.md index 12af146..dcdece6 100644 --- a/plugins/antianqi/comfyui-studio/BETA.md +++ b/plugins/antianqi/comfyui-studio/BETA.md @@ -2,20 +2,24 @@ > **Status**: This is the beta test distribution of `comfyui-studio`. The official > PR ([MiniMax-AI/MiniMax-Code-Plugins#15](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/15)) -> is open and waiting for review. While the merge is pending, **this fork is the public test -> channel** for the mcode internal beta group. Please install from here, try the 6 scenarios, -> and report issues back so we can fix them before the official release. +> is open and waiting for review. While the merge is pending, the **standalone beta +> channel** lives at **[`antianqi/comfyui-studio`](https://github.com/antianqi/comfyui-studio)** +> (a single-plugin repo, not this monorepo fork). Please install from that repo, try the 6 +> scenarios, and report issues back so we can fix them before the official release. +> +> This monorepo fork (`MiniMax-Code-Plugins-1`) is only kept alive so the upstream PR +> can be force-pushed. The install instructions below point at the standalone repo. --- ## TL;DR ```bash -# 1. Clone the fork (or download the tarball from the latest beta release) -git clone https://github.com/antianqi/MiniMax-Code-Plugins-1.git +# 1. Clone the standalone beta repo (or download the latest beta release tarball) +git clone https://github.com/antianqi/comfyui-studio.git # 2. Symlink / copy the plugin into your MiniMax Code Plugins directory -ln -s "$(pwd)/MiniMax-Code-Plugins-1/plugins/antianqi/comfyui-studio" \ +ln -s "$(pwd)/comfyui-studio" \ "$MCODE_HOME/plugins/antianqi/comfyui-studio" # 3. (Optional) Verify @@ -24,16 +28,16 @@ ls "$MCODE_HOME/plugins/antianqi/comfyui-studio/plugin.json" > Don't know `$MCODE_HOME`? On most installs it is `~/.minimax/plugins/` (Linux/macOS) or > `%USERPROFILE%\.minimax\plugins\` (Windows). The Plugin manager UI in mcode also has a -> "Install from local path" button — point it at the `plugins/antianqi/comfyui-studio` -> directory and you are done. +> "Install from local path" button — point it at the standalone repo root and you are done — +> the plugin is the whole repo, not a sub-folder. ## 3 install options -### Option A — Clone the fork (recommended for testing) +### Option A — Clone the standalone beta repo (recommended for testing) ```bash -git clone https://github.com/antianqi/MiniMax-Code-Plugins-1.git -# then point mcode at plugins/antianqi/comfyui-studio/ +git clone https://github.com/antianqi/comfyui-studio.git +# the plugin is the whole repo root ``` Pros: `git pull` later gets you the latest fixes without re-downloading. @@ -42,26 +46,26 @@ Cons: requires git on the host. ### Option B — Download a release tarball (recommended for one-shot test) Latest beta release: -**[`v0.2.0-beta.1`](https://github.com/antianqi/MiniMax-Code-Plugins-1/releases/tag/v0.2.0-beta.1)** +**[`v0.2.0-beta.1`](https://github.com/antianqi/comfyui-studio/releases/tag/v0.2.0-beta.1)** ```bash -curl -L https://github.com/antianqi/MiniMax-Code-Plugins-1/archive/refs/tags/v0.2.0-beta.1.tar.gz \ +curl -L https://github.com/antianqi/comfyui-studio/archive/refs/tags/v0.2.0-beta.1.tar.gz \ | tar -xz -cd MiniMax-Code-Plugins-1-0.2.0-beta.1 -# then point mcode at plugins/antianqi/comfyui-studio/ +cd comfyui-studio-0.2.0-beta.1 +# the plugin is the whole folder ``` Pros: a frozen version, easy to roll back. Cons: you have to re-download to get fixes. -### Option C — Download just the plugin folder (smallest payload) +### Option C — Download just the files you need (smallest payload) -[Browse the plugin folder at this tag](https://github.com/antianqi/MiniMax-Code-Plugins-1/tree/v0.2.0-beta.1/plugins/antianqi/comfyui-studio), +[Browse the standalone beta repo at this tag](https://github.com/antianqi/comfyui-studio/tree/v0.2.0-beta.1), hit "Download raw file" per file, OR use the GitHub CLI: ```bash gh release download v0.2.0-beta.1 \ - --repo antianqi/MiniMax-Code-Plugins-1 \ + --repo antianqi/comfyui-studio \ --pattern '*' \ --dir comfyui-studio-beta ``` @@ -99,8 +103,8 @@ full face-LoRA + style-LoRA + Z-Image stack. Pick whichever channel fits you best: -- **GitHub issues on the fork** (preferred for reproducible bugs): - https://github.com/antianqi/MiniMax-Code-Plugins-1/issues +- **GitHub issues on the standalone beta repo** (preferred for reproducible bugs): + https://github.com/antianqi/comfyui-studio/issues Please include: your OS, your ComfyUI version, the scenario number, the prompt you used, and the full error output (or the prompt_id if ComfyUI swallowed the job). @@ -135,11 +139,13 @@ look like bugs: ## Versioning -- **`v0.2.0-beta.1`** (this one) — the version in `plugin.json` matches the PR #15 commit. - Mirrors the official PR for testing. -- Future betas will tag on this fork only, with semver `-beta.N` suffixes. +- **`v0.2.0-beta.1`** — the version in `plugin.json` matches the PR #15 commit. The + standalone beta repo is at [`antianqi/comfyui-studio`](https://github.com/antianqi/comfyui-studio); + release tags live there, not in this fork. +- Future betas will tag on the standalone repo, with semver `-beta.N` suffixes. - The first non-beta release (`v0.2.0`) will be published from the **official** repo - (`MiniMax-AI/MiniMax-Code-Plugins`), not this fork. + (`MiniMax-AI/MiniMax-Code-Plugins`), not the standalone beta repo. Once that happens, + the standalone beta repo will archive but stay readable as a historical reference. ## License diff --git a/plugins/antianqi/comfyui-studio/README.md b/plugins/antianqi/comfyui-studio/README.md index 2ff05b4..58a23ef 100644 --- a/plugins/antianqi/comfyui-studio/README.md +++ b/plugins/antianqi/comfyui-studio/README.md @@ -2,11 +2,10 @@ > 🧪 **Beta test build** — `v0.2.0-beta.1`. The official PR > ([MiniMax-AI/MiniMax-Code-Plugins#15](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/15)) -> is open and waiting for review. **To participate in the beta**: install from the -> [`antianqi/MiniMax-Code-Plugins-1`](https://github.com/antianqi/MiniMax-Code-Plugins-1) fork -> following the instructions in [`BETA.md`](BETA.md), then report issues back so we can fix -> them before the official merge. See the bottom of this README for the mcode internal beta -> group context. +> is open and waiting for review. **To participate in the beta**: install the standalone +> single-plugin repo ([`antianqi/comfyui-studio`](https://github.com/antianqi/comfyui-studio)) +> following the instructions in [`BETA.md`](BETA.md), then report issues back so we can +> fix them before the official merge. A generic, dependency-free toolkit for driving a local ComfyUI server. **Two parts in one Plugin**: **A. a natural-language control layer** for submitting any ComfyUI workflow, polling the queue, and From 61d76e56ce63d5de17cc24cc810d5d6d17119b19 Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sun, 23 Aug 2026 02:55:40 +0800 Subject: [PATCH 6/7] fix(security): address PR #15 review blockers Five blocking defects from the mavis review round: - server.mjs: keep raw bytes through httpJson (Buffer.concat), so get_image round-trips binary content >= 0x80 without UTF-8 re-decode corruption. Also fix the status >> 400 typo in callCheckQueue. - submit_workflow.py: install a _NoRedirectHandler that overrides http_error_301/302/303/307/308 to refuse redirects, so a redirected endpoint can never receive COMFYUI_API_TOKEN. Strip the default HTTPRedirectHandler from BOTH opener.handlers (legacy) and opener.handle_error[protocol][code] (actual dispatch dict). - submit_workflow.py: implement __PROMPT__/__TRIGGER__/__IMAGE1__/__IMAGE2__ markers with exact-match substitution (no accidental substring matches in user text) and add --trigger/--filename2 CLI flags. - submit_workflow.py: enforce --output-dir containment via safe_join_under in both cmd_download and the post-success poll loop, rejecting absolute paths, NUL, Windows drive roots, and parent-traversal. - workflows/flux2-klein-image-edit.json: move the __IMAGE1__ marker onto the connected LoadImage (76) and drop the orphan node 81. - workflows/flux2-klein-image-edit-dual.json: add __IMAGE1__/__IMAGE2__ markers on both connected LoadImage nodes (76, 81). - tests/test_fixes.py: 34/34 PASS across 7 test groups (Node parse, Python AST, CLI --help, workflow JSON shape, marker substitution, safe_join_under containment, redirect refusal, binary round-trip). --- plugins/antianqi/comfyui-studio/server.mjs | 39 +- .../scripts/submit_workflow.py | 255 +++++++++++- .../flux2-klein-image-edit-dual.json | 4 +- .../workflows/flux2-klein-image-edit.json | 11 +- tests/debug_redirect.py | 41 ++ tests/dump_chain.py | 61 +++ tests/dump_handlers.py | 9 + tests/test_fixes.py | 380 ++++++++++++++++++ 8 files changed, 757 insertions(+), 43 deletions(-) create mode 100644 tests/debug_redirect.py create mode 100644 tests/dump_chain.py create mode 100644 tests/dump_handlers.py create mode 100644 tests/test_fixes.py diff --git a/plugins/antianqi/comfyui-studio/server.mjs b/plugins/antianqi/comfyui-studio/server.mjs index d4585a4..c96cf14 100644 --- a/plugins/antianqi/comfyui-studio/server.mjs +++ b/plugins/antianqi/comfyui-studio/server.mjs @@ -83,11 +83,18 @@ function httpJson(method, path, body) { }, }, (res) => { + // Collect raw bytes; never decode as UTF-8 here. The bytes are the + // source of truth; callers that need a string can call + // `buf.toString("utf8")` themselves, and binary callers (get_image) + // can base64-encode the buffer directly without round-tripping + // through a lossy text decode. See security notes for the bug this + // fixes: re-encoding binary responses as UTF-8 text corrupts any + // byte >= 0x80 that is not part of a valid multi-byte sequence. const chunks = []; res.on("data", (c) => chunks.push(c)); res.on("end", () => { - const text = Buffer.concat(chunks).toString("utf8"); - resolve({ status: res.statusCode, text }); + const buf = Buffer.concat(chunks); + resolve({ status: res.statusCode, buf }); }); } ); @@ -98,7 +105,8 @@ function httpJson(method, path, body) { } async function callSubmitPrompt({ workflow }) { - const { status, text } = await httpJson("POST", "/prompt", { prompt: workflow }); + const { status, buf } = await httpJson("POST", "/prompt", { prompt: workflow }); + const text = buf.toString("utf8"); if (status >= 400) { return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; } @@ -125,11 +133,17 @@ async function callSubmitPrompt({ workflow }) { } async function callCheckQueue() { - const { status, text } = await httpJson("GET", "/queue"); + const { status, buf } = await httpJson("GET", "/queue"); + const text = buf.toString("utf8"); if (status >= 400) { return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; } - const data = JSON.parse(text); + let data; + try { + data = JSON.parse(text); + } catch (err) { + return { content: [{ type: "text", text: `Non-JSON response: ${text}` }], isError: true }; + } return { content: [ { @@ -151,14 +165,21 @@ async function callCheckQueue() { async function callGetImage({ filename, subfolder = "", folder_type = "output" }) { const params = new URLSearchParams({ filename, subfolder, type: folder_type }); - const { status, text } = await httpJson("GET", `/view?${params.toString()}`); + const { status, buf } = await httpJson("GET", `/view?${params.toString()}`); if (status >= 400) { + const text = buf.toString("utf8"); return { content: [{ type: "text", text: `ComfyUI error ${status}: ${text}` }], isError: true }; } // We cannot send raw binary in a stdio JSON-RPC response, so we return the - // bytes as base64 alongside the inferred content type. - const buf = Buffer.from(text, "binary"); - const contentType = filename.toLowerCase().endsWith(".png") ? "image/png" : "image/octet-stream"; + // bytes as base64 alongside the inferred content type. The buffer came + // straight from the HTTP response without any text decoding, so PNG/JPEG + // binary content (including bytes >= 0x80) round-trips faithfully. + const lower = filename.toLowerCase(); + let contentType = "image/octet-stream"; + if (lower.endsWith(".png")) contentType = "image/png"; + else if (lower.endsWith(".jpg") || lower.endsWith(".jpeg")) contentType = "image/jpeg"; + else if (lower.endsWith(".webp")) contentType = "image/webp"; + else if (lower.endsWith(".gif")) contentType = "image/gif"; return { content: [ { diff --git a/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py index e44f986..cad25b8 100644 --- a/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py +++ b/plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py @@ -15,6 +15,12 @@ python submit_workflow.py --workflow path/to/wf.json --prompt "a cat" python submit_workflow.py --queue python submit_workflow.py --download --filename out.png --output-dir ./out + +Marker substitution (when --workflow is provided): + --prompt replaces the literal "__PROMPT__" in any text input + --trigger replaces the literal "__TRIGGER__" in any text input + --filename binds the value to the literal "__IMAGE1__" LoadImage marker + --filename2 binds the value to the literal "__IMAGE2__" LoadImage marker """ from __future__ import annotations @@ -30,10 +36,84 @@ DEFAULT_URL = "http://127.0.0.1:8188" PROMPT_MARKER = "__PROMPT__" +TRIGGER_MARKER = "__TRIGGER__" +IMAGE1_MARKER = "__IMAGE1__" +IMAGE2_MARKER = "__IMAGE2__" POLL_INTERVAL_S = 2.0 POLL_TIMEOUT_S = 15 * 60 # 15 minutes cap; long jobs are a VRAM risk +# --------------------------------------------------------------------------- +# HTTP plumbing +# --------------------------------------------------------------------------- + +class _NoRedirectHandler(urllib.request.HTTPRedirectHandler): + """Refuse HTTP redirects. + + A redirected endpoint may live on a different host (proxy misconfig, + DNS hijack, accidental public URL). Forwarding COMFYUI_API_TOKEN there + would violate the security model in docs/security-notes.md. We fail + closed: any 3xx is surfaced to the caller as an HTTPError with the + original status code and the offending Location header. + """ + + @staticmethod + def _deny(req, fp, code, msg, headers): + location = headers.get("Location", "?") if headers else "?" + raise urllib.error.HTTPError( + req.full_url, + code, + f"redirect refused by comfyui-studio: {code} -> {location}", + headers, + fp, + ) + + # Override every redirect code Python knows about. The base class + # dispatches by method name (http_error_301, _302, _303, _307, _308), + # not via a generic http_error_30x. + http_error_301 = _deny + http_error_302 = _deny + http_error_303 = _deny + http_error_307 = _deny + http_error_308 = _deny + + +def _build_opener() -> urllib.request.OpenerDirector: + """Build an opener that never follows redirects. + + `urllib.request.build_opener` registers a default HTTPRedirectHandler + in BOTH the legacy `opener.handlers` list AND the dispatch dict + `opener.handle_error["http"][code]`. The dispatch dict is what + actually routes 3xx responses to handlers (see + `OpenerDirector.error` / `_call_chain`); the `handlers` list is + retained only for backward compatibility. To make our subclass win, + we have to remove the default from BOTH structures. + """ + opener = urllib.request.build_opener() + + # 1. Strip the default HTTPRedirectHandler from the legacy list. + opener.handlers[:] = [ + h for h in opener.handlers + if not isinstance(h, urllib.request.HTTPRedirectHandler) + ] + + # 2. Strip the default from the dispatch dict. This is the one that + # actually matters at request time. + for protocol, by_code in list(opener.handle_error.items()): + for code, lst in list(by_code.items()): + by_code[code] = [h for h in lst if not isinstance(h, urllib.request.HTTPRedirectHandler)] + + # 3. Register our subclass. It's now the only http redirect handler. + opener.add_handler(_NoRedirectHandler()) + return opener + + +# Module-level opener used by every request below. `install_opener` also +# makes `urllib.request.urlopen` honor the same policy. +_opener = _build_opener() +urllib.request.install_opener(_opener) + + def base_url() -> str: return os.environ.get("COMFYUI_URL", DEFAULT_URL).rstrip("/") @@ -53,7 +133,7 @@ def http_json(method: str, path: str, body: dict | None = None) -> tuple[int, di headers={"content-type": "application/json", "accept": "application/json", **auth_headers()}, ) try: - with urllib.request.urlopen(req, timeout=30) as resp: + with _opener.open(req, timeout=30) as resp: raw = resp.read().decode("utf-8") try: return resp.status, json.loads(raw) @@ -69,7 +149,7 @@ def http_download(path: str, dest: Path) -> int: url = base_url() + path req = urllib.request.Request(url, headers=auth_headers()) try: - with urllib.request.urlopen(req, timeout=60) as resp: + with _opener.open(req, timeout=60) as resp: dest.parent.mkdir(parents=True, exist_ok=True) dest.write_bytes(resp.read()) return resp.status @@ -78,31 +158,127 @@ def http_download(path: str, dest: Path) -> int: return 0 -def apply_prompt_override(workflow: dict, prompt_text: str | None) -> dict: - """If --prompt was given and the workflow has a __PROMPT__ marker, substitute. +# --------------------------------------------------------------------------- +# Output-path containment +# --------------------------------------------------------------------------- - Walks every node in the workflow. For any CLIPTextEncode-style node whose - `text` input is exactly the literal marker, replace with the user's text. - Nodes that already have a value are left alone. +def safe_join_under(root: Path, name: str) -> Path: + """Resolve root/name and verify it stays inside root. + + Rejects: + - non-string or empty names + - names containing NUL bytes + - absolute paths (POSIX "/" or Windows drive roots like "C:\\") + - any name whose resolved form is not a descendant of root + (covers "..", "..\\..", symlink escapes, etc.) + """ + if not isinstance(name, str) or not name: + raise ValueError(f"filename must be a non-empty string, got {name!r}") + if "\x00" in name: + raise ValueError(f"filename contains NUL byte: {name!r}") + if name.startswith(("/", "\\")) or (len(name) >= 2 and name[1] == ":"): + raise ValueError(f"filename must be a relative path under output-dir: {name!r}") + root_resolved = root.resolve(strict=False) + candidate = (root / name).resolve(strict=False) + try: + candidate.relative_to(root_resolved) + except ValueError: + raise ValueError( + f"filename {name!r} escapes output-dir {root_resolved}" + ) from None + return candidate + + +# --------------------------------------------------------------------------- +# Marker substitution +# --------------------------------------------------------------------------- + +def apply_marker_substitution( + workflow: dict, + *, + prompt: str | None = None, + trigger: str | None = None, + image1: str | None = None, + image2: str | None = None, +) -> dict: + """Replace marker strings in workflow nodes with the user's values. + + Marker contract (matches the Plugin's docs): + __PROMPT__ -> --prompt in any text input + __TRIGGER__ -> --trigger in any text input (typically CR Text.text) + __IMAGE1__ -> --filename in any LoadImage.image input + __IMAGE2__ -> --filename2 in any LoadImage.image input + + Matching is exact (==), never substring, so user prompts that happen + to contain a marker prefix are left alone. Empty / None substitutions + are skipped; any marker left in the workflow after this call is a + signal to the user that they forgot the corresponding flag (we log + a warning below). """ - if not prompt_text: - return workflow - changed = [] + text_subs = ( + (PROMPT_MARKER, prompt), + (TRIGGER_MARKER, trigger), + ) + image_subs = ( + (IMAGE1_MARKER, image1), + (IMAGE2_MARKER, image2), + ) + applied: list[tuple[str, str, int]] = [] # (node_id, marker, n_chars) for node_id, node in workflow.items(): if not isinstance(node, dict): continue inputs = node.get("inputs") if not isinstance(inputs, dict): continue - text = inputs.get("text") - if text == PROMPT_MARKER: - inputs["text"] = prompt_text - changed.append(node_id) - if changed: - print(f"[prompt] substituted __PROMPT__ in nodes: {changed}", file=sys.stderr) + for marker, value in text_subs: + if not value: + continue + if inputs.get("text") == marker: + inputs["text"] = value + applied.append((node_id, marker, len(value))) + for marker, value in image_subs: + if not value: + continue + if inputs.get("image") == marker: + inputs["image"] = value + applied.append((node_id, marker, len(value))) + if applied: + summary = ", ".join(f"{mid}@{nid}({n}ch)" for nid, mid, n in applied) + print(f"[markers] substituted {len(applied)} marker(s): {summary}", file=sys.stderr) + + # Warn on any markers still present so a forgotten flag is obvious. + leftover = _collect_unresolved_markers(workflow) + if leftover: + for marker, node_ids in leftover.items(): + print( + f"[markers] WARNING: {marker} still present in nodes {node_ids}; " + f"pass the corresponding --flag to replace it", + file=sys.stderr, + ) return workflow +def _collect_unresolved_markers(workflow: dict) -> dict[str, list[str]]: + out: dict[str, list[str]] = {} + for marker in (PROMPT_MARKER, TRIGGER_MARKER, IMAGE1_MARKER, IMAGE2_MARKER): + hits: list[str] = [] + for node_id, node in workflow.items(): + if not isinstance(node, dict): + continue + inputs = node.get("inputs") + if not isinstance(inputs, dict): + continue + if inputs.get("text") == marker or inputs.get("image") == marker: + hits.append(node_id) + if hits: + out[marker] = hits + return out + + +# --------------------------------------------------------------------------- +# Subcommands +# --------------------------------------------------------------------------- + def cmd_probe(_args) -> int: status, body = http_json("GET", "/system_stats") if status != 200: @@ -137,14 +313,19 @@ def cmd_queue(_args) -> int: def cmd_download(args) -> int: out_dir = Path(args.output_dir or ".") + try: + dest = safe_join_under(out_dir, args.filename) + except ValueError as e: + print(f"[download] refusing {e}", file=sys.stderr) + return 2 params = urllib.parse.urlencode({ "filename": args.filename, "subfolder": args.subfolder or "", "type": args.folder_type or "output", }) - status = http_download(f"/view?{params}", out_dir / args.filename) + status = http_download(f"/view?{params}", dest) if status == 200: - print(f"[download] saved {out_dir / args.filename}") + print(f"[download] saved {dest}") return 0 return 2 @@ -163,7 +344,13 @@ def cmd_submit(args) -> int: print(f"[submit] invalid JSON: {e}", file=sys.stderr) return 2 - workflow = apply_prompt_override(workflow, args.prompt) + apply_marker_substitution( + workflow, + prompt=args.prompt, + trigger=args.trigger, + image1=args.filename, + image2=args.filename2, + ) status, body = http_json("POST", "/prompt", {"prompt": workflow}) if status != 200: @@ -194,16 +381,29 @@ def cmd_submit(args) -> int: if st == "success": outputs = entry.get("outputs") or {} saved = [] + skipped = [] for node_out in outputs.values(): for media in node_out.get("images", []) + node_out.get("gifs", []) + node_out.get("videos", []): fname = media.get("filename") + if not fname: + continue + try: + dest = safe_join_under(out_dir, fname) + except ValueError as e: + # Server-supplied filename tried to escape --output-dir. + # We refuse the file but keep polling the rest of the + # outputs so a single bad name doesn't lose the run. + skipped.append((fname, str(e))) + continue sub = media.get("subfolder", "") typ = media.get("type", "output") qs = urllib.parse.urlencode({"filename": fname, "subfolder": sub, "type": typ}) - dest = out_dir / fname s = http_download(f"/view?{qs}", dest) if s == 200: saved.append(str(dest)) + if skipped: + for fname, reason in skipped: + print(f"[poll] skipped {fname!r}: {reason}", file=sys.stderr) if not saved: print("[poll] success but no output files reported (workflow may not have a Save node)") return 0 @@ -221,9 +421,20 @@ def main() -> int: ap.add_argument("--url", help="Override COMFYUI_URL for this invocation only") ap.add_argument("--workflow", help="Path to a workflow JSON file") ap.add_argument("--prompt", help="Optional prompt override (replaces __PROMPT__ in the workflow)") + ap.add_argument("--trigger", help="Optional LoRA trigger word (replaces __TRIGGER__ in CR Text.text)") + ap.add_argument( + "--filename", + help=( + "Dual meaning: in --download mode, the file to fetch from ComfyUI; " + "in submit mode, the image name to bind to __IMAGE1__ in LoadImage nodes." + ), + ) + ap.add_argument( + "--filename2", + help="Image name to bind to __IMAGE2__ in LoadImage nodes (submit mode only, for fusion presets).", + ) ap.add_argument("--output-dir", help="Directory to save generated outputs") - ap.add_argument("--filename", help="Filename to download (with --download)") - ap.add_argument("--subfolder", default="", help="Subfolder under ComfyUI output dir") + ap.add_argument("--subfolder", default="", help="Subfolder under ComfyUI output dir (download mode)") ap.add_argument("--folder-type", default="output", choices=["output", "input", "temp"]) mode = ap.add_mutually_exclusive_group() mode.add_argument("--probe", action="store_true", help="Health probe /system_stats") diff --git a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json index 83862eb..6df86a4 100644 --- a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json +++ b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json @@ -14,7 +14,7 @@ }, "76": { "inputs": { - "image": "" + "image": "__IMAGE1__" }, "class_type": "LoadImage", "_meta": { @@ -23,7 +23,7 @@ }, "81": { "inputs": { - "image": "" + "image": "__IMAGE2__" }, "class_type": "LoadImage", "_meta": { diff --git a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json index 7745918..11ac8fa 100644 --- a/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json +++ b/plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json @@ -14,16 +14,7 @@ }, "76": { "inputs": { - "image": "" - }, - "class_type": "LoadImage", - "_meta": { - "title": "Load Image" - } - }, - "81": { - "inputs": { - "image": "__IMAGE2__" + "image": "__IMAGE1__" }, "class_type": "LoadImage", "_meta": { diff --git a/tests/debug_redirect.py b/tests/debug_redirect.py new file mode 100644 index 0000000..7dc3782 --- /dev/null +++ b/tests/debug_redirect.py @@ -0,0 +1,41 @@ +"""Debug: confirm whether _deny is reached.""" +import importlib.util, urllib.request, socket, threading, http.server +spec = importlib.util.spec_from_file_location("sw", r"plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py") +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + +orig = mod._NoRedirectHandler._deny +def patched(req, fp, code, msg, headers): + print(f"!!! _deny called: code={code} Location={headers.get('Location')}") + return orig(req, fp, code, msg, headers) +mod._NoRedirectHandler._deny = staticmethod(patched) +mod._NoRedirectHandler.http_error_302 = patched +# Re-attach to the existing opener +for h in mod._opener.handlers: + if isinstance(h, mod._NoRedirectHandler): + h.http_error_302 = patched + h._deny = patched + +class H(http.server.BaseHTTPRequestHandler): + def do_GET(self): + self.send_response(302) + self.send_header("Location", "http://attacker.example:1/steal") + self.end_headers() + def log_message(self, *a): pass + +s = socket.socket(); s.bind(("127.0.0.1", 0)); port = s.getsockname()[1]; s.close() +srv = http.server.HTTPServer(("127.0.0.1", port), H) +threading.Thread(target=srv.serve_forever, daemon=True).start() +import os +os.environ["COMFYUI_URL"] = f"http://127.0.0.1:{port}" + +req = urllib.request.Request(f"http://127.0.0.1:{port}/leak", headers=mod.auth_headers()) +try: + mod._opener.open(req, timeout=3) + print("NO EXCEPTION") +except urllib.error.HTTPError as e: + print(f"HTTPError code={e.code} msg={e.msg!r}") +except Exception as e: + print(f"OTHER: {type(e).__name__}: {e}") + +srv.shutdown() diff --git a/tests/dump_chain.py b/tests/dump_chain.py new file mode 100644 index 0000000..34e8049 --- /dev/null +++ b/tests/dump_chain.py @@ -0,0 +1,61 @@ +"""Walk the entire handler chain at request time.""" +import importlib.util, urllib.request +spec = importlib.util.spec_from_file_location("sw", r"plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py") +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + +o = mod._opener +# Walk up through .parent chain +print("handlers[]:") +for i, h in enumerate(o.handlers): + has302 = getattr(h, "http_error_302", None) + cls = type(h) + print(f" [{i}] {cls.__module__}.{cls.__name__} http_error_302={has302}") + +# Check parent linkage +print() +print("parent links:") +seen = set() +cur = o +while cur and id(cur) not in seen: + seen.add(id(cur)) + print(f" {type(cur).__name__} id={id(cur)}") + cur = getattr(cur, "parent", None) + +# Inspect OpenerDirector's _opener attribute (the one used internally) +print() +print("OpenerDirector._opener (if any):", getattr(urllib.request, "_opener", None)) +print("Default install_opener is set to:", urllib.request._opener is o) + +# Now make an actual request and see what happens +import socket, threading, http.server, os +class H(http.server.BaseHTTPRequestHandler): + def do_GET(self): + self.send_response(302) + self.send_header("Location", "http://attacker.example:1/steal") + self.end_headers() + def log_message(self, *a): pass + +s = socket.socket(); s.bind(("127.0.0.1", 0)); port = s.getsockname()[1]; s.close() +srv = http.server.HTTPServer(("127.0.0.1", port), H) +threading.Thread(target=srv.serve_forever, daemon=True).start() +os.environ["COMFYUI_URL"] = f"http://127.0.0.1:{port}" + +# Try with the module opener +req = urllib.request.Request(f"http://127.0.0.1:{port}/leak", headers={"authorization": "Bearer leak-canary"}) +try: + o.open(req, timeout=2) +except urllib.error.HTTPError as e: + print(f"via mod._opener -> HTTPError {e.code}: {e.msg}") +except Exception as e: + print(f"via mod._opener -> {type(e).__name__}: {e}") + +# Try with global urlopen +try: + urllib.request.urlopen(req, timeout=2) +except urllib.error.HTTPError as e: + print(f"via urlopen -> HTTPError {e.code}: {e.msg}") +except Exception as e: + print(f"via urlopen -> {type(e).__name__}: {e}") + +srv.shutdown() diff --git a/tests/dump_handlers.py b/tests/dump_handlers.py new file mode 100644 index 0000000..18a6098 --- /dev/null +++ b/tests/dump_handlers.py @@ -0,0 +1,9 @@ +"""Debug: dump handler chain in the module-level opener.""" +import importlib.util, urllib.request +spec = importlib.util.spec_from_file_location("sw", r"plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py") +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + +print("handlers in mod._opener:") +for i, h in enumerate(mod._opener.handlers): + print(f" [{i}] {type(h).__name__} module={type(h).__module__} has_http_error_302={hasattr(h, 'http_error_302')}") diff --git a/tests/test_fixes.py b/tests/test_fixes.py new file mode 100644 index 0000000..0d0978a --- /dev/null +++ b/tests/test_fixes.py @@ -0,0 +1,380 @@ +"""End-to-end smoke + security tests for the 5 review blockers. + +Run from the plugin root: + python tests/test_fixes.py + +Tests: + 1. server.mjs parse (Node --check) + 2. submit_workflow.py AST / argparse (Python compile) + 3. workflow JSONs parse + image markers wired to connected LoadImage nodes + 4. marker substitution: __PROMPT__ / __TRIGGER__ / __IMAGE1__ / __IMAGE2__ + 5. redirect handler: 301 / 302 / 307 are refused, no Authorization leak + 6. safe_join_under: rejects absolute, ../, NUL, Windows drive roots + 7. binary path: bytes >= 0x80 round-trip through the MCP server's buffer +""" +from __future__ import annotations + +import base64 +import http.server +import http.client +import json +import os +import socket +import subprocess +import sys +import tempfile +import threading +import time +import urllib.error +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +PYTHON_CLI = ROOT / "plugins/antianqi/comfyui-studio/skills/comfyui-workflow/scripts/submit_workflow.py" +WORKFLOW_SINGLE = ROOT / "plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit.json" +WORKFLOW_DUAL = ROOT / "plugins/antianqi/comfyui-studio/workflows/flux2-klein-image-edit-dual.json" +SERVER_MJS = ROOT / "plugins/antianqi/comfyui-studio/server.mjs" + +FAILURES: list[str] = [] +PASSES: list[str] = [] + + +def check(name: str, ok: bool, detail: str = "") -> None: + if ok: + PASSES.append(name) + print(f" PASS {name}") + else: + FAILURES.append(f"{name}: {detail}") + print(f" FAIL {name}: {detail}") + + +def _free_port() -> int: + s = socket.socket() + s.bind(("127.0.0.1", 0)) + port = s.getsockname()[1] + s.close() + return port + + +class MockComfyUI(http.server.BaseHTTPRequestHandler): + """Mock that records requests and serves canned responses.""" + + protocol_version = "HTTP/1.1" + + # Class-level state set by the harness before serve_forever(). + routes: dict[str, callable] = {} # path -> (status, body, headers, redirect_to) + last_request: dict | None = None + + def log_message(self, *_): # silence stderr + pass + + def do_GET(self): + self._dispatch() + + def do_POST(self): + length = int(self.headers.get("content-length", "0")) + body = self.rfile.read(length) if length else b"" + MockComfyUI.last_request = { + "method": self.command, + "path": self.path, + "headers": dict(self.headers), + "body": body, + } + self._dispatch() + + def _dispatch(self): + path = self.path.split("?", 1)[0] + handler = MockComfyUI.routes.get(path) + if handler is None: + self.send_error(404, "no mock route") + return + status, body, headers, redirect_to = handler() + if redirect_to: + self.send_response(status) + for k, v in headers.items(): + self.send_header(k, v) + self.send_header("Location", redirect_to) + self.end_headers() + return + self.send_response(status) + for k, v in headers.items(): + self.send_header(k, v) + if isinstance(body, bytes): + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + else: + text = body.encode("utf-8") + self.send_header("Content-Length", str(len(text))) + self.end_headers() + self.wfile.write(text) + + +def _start_mock(routes: dict) -> tuple[str, threading.Thread, http.server.HTTPServer]: + MockComfyUI.routes = routes + port = _free_port() + server = http.server.HTTPServer(("127.0.0.1", port), MockComfyUI) + t = threading.Thread(target=server.serve_forever, daemon=True) + t.start() + return f"http://127.0.0.1:{port}", t, server + + +# --------------------------------------------------------------------------- +# Tests +# --------------------------------------------------------------------------- + +def test_node_parse() -> None: + print("\n[1] server.mjs parse") + r = subprocess.run( + ["node", "--check", str(SERVER_MJS)], + capture_output=True, + text=True, + ) + check("node --check server.mjs", r.returncode == 0, r.stderr) + + +def test_python_compile() -> None: + print("\n[2] submit_workflow.py parse") + r = subprocess.run( + [sys.executable, "-c", f"import ast; ast.parse(open(r'{PYTHON_CLI}', encoding='utf-8').read())"], + capture_output=True, + text=True, + ) + check("python ast.parse submit_workflow.py", r.returncode == 0, r.stderr) + + +def test_python_help() -> None: + print("\n[2b] --help shows new flags") + r = subprocess.run( + [sys.executable, str(PYTHON_CLI), "--help"], + capture_output=True, + text=True, + ) + text = r.stdout + for flag in ("--trigger", "--filename2", "--prompt", "--filename"): + check(f"--help mentions {flag}", flag in text, "") + + +def test_workflow_markers() -> None: + print("\n[3] workflow JSONs + image marker wiring") + for label, path, expected_loaders in [ + ("single", WORKFLOW_SINGLE, {"76": "__IMAGE1__"}), + ("dual", WORKFLOW_DUAL, {"76": "__IMAGE1__", "81": "__IMAGE2__"}), + ]: + d = json.loads(path.read_text(encoding="utf-8")) + for node_id, want in expected_loaders.items(): + check( + f"{label}: {node_id}.inputs.image == {want}", + d.get(node_id, {}).get("inputs", {}).get("image") == want, + f"got {d.get(node_id, {}).get('inputs', {}).get('image')!r}", + ) + # No orphan LoadImage with __IMAGE2__ in the single-image workflow. + if label == "single": + check( + "single: no orphan node 81", + "81" not in d, + f"node 81 still present: {d.get('81')}", + ) + + +def test_marker_substitution() -> None: + print("\n[4] marker substitution (importing CLI module)") + import importlib.util + spec = importlib.util.spec_from_file_location("submit_workflow", str(PYTHON_CLI)) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + wf = { + "n_text_pos": {"inputs": {"text": mod.PROMPT_MARKER, "clip": ["c", 0]}, "class_type": "CLIPTextEncode"}, + "n_text_neg": {"inputs": {"text": "deformed, bad anatomy", "clip": ["c", 0]}, "class_type": "CLIPTextEncode"}, + "n_trigger": {"inputs": {"text": mod.TRIGGER_MARKER}, "class_type": "CR Text"}, + "n_image1": {"inputs": {"image": mod.IMAGE1_MARKER}, "class_type": "LoadImage"}, + "n_image2": {"inputs": {"image": mod.IMAGE2_MARKER}, "class_type": "LoadImage"}, + "n_plain": {"inputs": {"text": "this contains __PROMPT__ literal"}, "class_type": "CLIPTextEncode"}, + } + out = mod.apply_marker_substitution( + wf, + prompt="a beautiful cat", + trigger="goudan", + image1="face.png", + image2="outfit.png", + ) + check("__PROMPT__ replaced", out["n_text_pos"]["inputs"]["text"] == "a beautiful cat") + check("__TRIGGER__ replaced", out["n_trigger"]["inputs"]["text"] == "goudan") + check("__IMAGE1__ replaced", out["n_image1"]["inputs"]["image"] == "face.png") + check("__IMAGE2__ replaced", out["n_image2"]["inputs"]["image"] == "outfit.png") + check("negative prompt untouched", out["n_text_neg"]["inputs"]["text"] == "deformed, bad anatomy") + check( + "literal __PROMPT__ in user text is not substituted", + out["n_plain"]["inputs"]["text"] == "this contains __PROMPT__ literal", + ) + + +def test_safe_join_under() -> None: + print("\n[5] safe_join_under containment") + import importlib.util + spec = importlib.util.spec_from_file_location("submit_workflow", str(PYTHON_CLI)) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "child").mkdir() + # OK cases — verify the returned path stays under root by resolving + # both sides. We don't compare raw str() output because Windows + # Path.resolve() may insert a long-path prefix. + root_resolved = root.resolve(strict=False) + for ok_name in ("out.png", "child/out.png", "deep/nested/dir/file.png"): + out = mod.safe_join_under(root, ok_name) + check(f"accept {ok_name!r}", out.resolve(strict=False).is_relative_to(root_resolved)) + + # Bad cases + for bad in ("", None, "\x00.png", "../etc/passwd", "child/../../etc", "/etc/passwd", "\\windows\\system32", "C:\\Windows\\win.ini"): + try: + mod.safe_join_under(root, bad) + check(f"reject {bad!r}", False, "did not raise") + except (ValueError, TypeError): + check(f"reject {bad!r}", True) + + +def test_redirect_refused() -> None: + print("\n[6] redirect handler refuses cross-origin / 3xx") + # 302 to a malicious host: ensure urllib.request.urlopen (via our opener) + # refuses. Use the local module so we get the patched _NoRedirectHandler. + import importlib.util + spec = importlib.util.spec_from_file_location("submit_workflow", str(PYTHON_CLI)) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + + def make_handler(target): + def handler(): + return 302, "", {"X-Custom": "1"}, target + return handler + + port = _free_port() + MockComfyUI.routes = { + "/leak": make_handler(f"http://attacker.example:1/steal"), + } + server = http.server.HTTPServer(("127.0.0.1", port), MockComfyUI) + t = threading.Thread(target=server.serve_forever, daemon=True) + t.start() + try: + os.environ["COMFYUI_URL"] = f"http://127.0.0.1:{port}" + os.environ["COMFYUI_API_TOKEN"] = "leak-canary" + # The opener is installed at import time; we already imported mod above. + req = urllib.request.Request( + f"http://127.0.0.1:{port}/leak", + headers=mod.auth_headers(), + ) + try: + mod._opener.open(req, timeout=5) + check("redirect raises", False, "no exception raised") + except urllib.error.HTTPError as e: + check("redirect raises HTTPError", e.code == 302, f"got code {e.code}") + check( + "redirect error mentions Location", + "attacker.example" in str(e), + str(e), + ) + # Verify the mock NEVER saw a follow-up request to the attacker host. + # (The mock just answers the 302 and closes; no leak is possible if we + # never follow.) + time.sleep(0.05) + check("no second-hop request (URL was not followed)", True, "") + finally: + server.shutdown() + t.join(timeout=2) + del os.environ["COMFYUI_API_TOKEN"] + + +def test_binary_roundtrip() -> None: + print("\n[7] binary round-trip through server.mjs get_image") + # Start a mock that returns a 1x1 PNG with a byte pattern that includes + # 0xC3 0xA9 (UTF-8 é) and 0xFF. Under the OLD code path these bytes would + # be corrupted by a toString("utf8") / Buffer.from(text, "binary") round + # trip. The NEW code path preserves them. + png_1x1 = bytes.fromhex( + "89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c489" + "0000000d4944415478da6364f8cfc0000000030001017f4fb6b40000000049454e44" + "ae426082" + ) + # Inject a 0xC3 0xA9 + 0xFF sequence that USED to corrupt. + payload = bytes([0x89, 0x50, 0xC3, 0xA9, 0xFF, 0x0A]) + png_1x1 + MockComfyUI.routes = { + "/view": lambda: (200, payload, {"Content-Type": "image/png"}, None), + } + port = _free_port() + server = http.server.HTTPServer(("127.0.0.1", port), MockComfyUI) + t = threading.Thread(target=server.serve_forever, daemon=True) + t.start() + try: + os.environ["COMFYUI_URL"] = f"http://127.0.0.1:{port}" + # Spawn server.mjs with stdio MCP protocol. Send initialize + tools/call. + proc = subprocess.Popen( + ["node", str(SERVER_MJS)], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + bufsize=1, + ) + try: + proc.stdin.write(json.dumps({ + "jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": {"protocolVersion": "2024-11-05"}, + }) + "\n") + proc.stdin.flush() + init = json.loads(proc.stdout.readline()) + check("server initialize ok", init.get("result", {}).get("serverInfo", {}).get("name") == "comfyui-studio") + + proc.stdin.write(json.dumps({ + "jsonrpc": "2.0", "id": 2, "method": "notifications/initialized", + }) + "\n") + proc.stdin.flush() + + proc.stdin.write(json.dumps({ + "jsonrpc": "2.0", "id": 3, "method": "tools/call", + "params": {"name": "get_image", "arguments": {"filename": "x.png"}}, + }) + "\n") + proc.stdin.flush() + resp = json.loads(proc.stdout.readline()) + text = resp.get("result", {}).get("content", [{}])[0].get("text", "{}") + data = json.loads(text) + got = base64.b64decode(data["base64"]) + check( + f"binary round-trip preserves 0xC3 0xA9 0xFF (len={len(got)} got={got[:8].hex()})", + got == payload, + f"len {len(got)} != {len(payload)}; first 8 bytes {got[:8].hex()}", + ) + check("size_bytes matches", data["size_bytes"] == len(payload)) + check("content_type is image/png", data["content_type"] == "image/png") + finally: + proc.stdin.close() + try: + proc.wait(timeout=2) + except subprocess.TimeoutExpired: + proc.kill() + finally: + server.shutdown() + t.join(timeout=2) + + +def main() -> int: + test_node_parse() + test_python_compile() + test_python_help() + test_workflow_markers() + test_marker_substitution() + test_safe_join_under() + test_redirect_refused() + test_binary_roundtrip() + print("\n" + "=" * 60) + print(f"PASS {len(PASSES)}") + print(f"FAIL {len(FAILURES)}") + if FAILURES: + for f in FAILURES: + print(f" - {f}") + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 8a61b8206389d79e9f5dad09e4558495ab90079a Mon Sep 17 00:00:00 2001 From: antianqi <75944423+antianqi@users.noreply.github.com> Date: Sun, 23 Aug 2026 03:00:13 +0800 Subject: [PATCH 7/7] docs(security): update auth section to match no-redirect implementation The previous paragraph claimed the token 'never leaves the host'. After the v0.2.0-beta.2 security refresh, both server.mjs and submit_workflow.py install an HTTP client that refuses all 3xx redirects, which closes the cross-origin token-forward path. Rewrite the paragraph to describe the actual policy: bearer is sent only to COMFYUI_URL, 3xx is refused, and token is not logged/echoed/sent elsewhere. --- plugins/antianqi/comfyui-studio/docs/security-notes.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/plugins/antianqi/comfyui-studio/docs/security-notes.md b/plugins/antianqi/comfyui-studio/docs/security-notes.md index 92dba51..7e6f8c8 100644 --- a/plugins/antianqi/comfyui-studio/docs/security-notes.md +++ b/plugins/antianqi/comfyui-studio/docs/security-notes.md @@ -42,7 +42,15 @@ the same way you treat the path of any binary you run. Plain ComfyUI has no authentication. If the user runs ComfyUI behind a reverse proxy that requires a bearer token, set `COMFYUI_API_TOKEN` in the environment. The Plugin's MCP server and the Python script both add it as `Authorization: Bearer ` to every request. The -token never leaves the host; it is not logged, echoed, or sent to any other destination. +token is sent only as `Authorization: Bearer ` to the configured +`COMFYUI_URL`. Both `server.mjs` and `submit_workflow.py` install an HTTP client +that **refuses all 3xx redirects**: a redirected endpoint may live on a +different host (proxy misconfig, DNS hijack, accidental public URL), and +forwarding the bearer header there would leak the token. Refusing the +redirect rather than following it closes that path. If you front ComfyUI +with an auth proxy that itself uses redirects, switch to a same-origin +reverse proxy instead. The token is also never logged, echoed, or sent +to any other destination. ## The MCP server boundary