From 3c2b7fd09f9e226a46bbfa728b5f5b3fc8b3e259 Mon Sep 17 00:00:00 2001 From: yuhao Date: Mon, 30 Mar 2026 09:16:10 +0000 Subject: [PATCH] upgrade to v0.2.0 with a series of updates --- README.md | 39 +- cli-anything-plugin/HARNESS.md | 389 ++++-------------- .../guides/filter-translation.md | 26 ++ cli-anything-plugin/guides/mcp-backend.md | 64 +++ cli-anything-plugin/guides/pypi-publishing.md | 117 ++++++ cli-anything-plugin/guides/session-locking.md | 39 ++ .../guides/skill-generation.md | 115 ++++++ .../guides/timecode-precision.md | 22 + 8 files changed, 492 insertions(+), 319 deletions(-) create mode 100644 cli-anything-plugin/guides/filter-translation.md create mode 100644 cli-anything-plugin/guides/mcp-backend.md create mode 100644 cli-anything-plugin/guides/pypi-publishing.md create mode 100644 cli-anything-plugin/guides/session-locking.md create mode 100644 cli-anything-plugin/guides/skill-generation.md create mode 100644 cli-anything-plugin/guides/timecode-precision.md diff --git a/README.md b/README.md index 3293af939..84a461776 100644 --- a/README.md +++ b/README.md @@ -41,9 +41,26 @@ CLI-Anything: Bridging the Gap Between AI Agents and the World's Software Thanks to all invaluable efforts from the community! More updates continuously on the way everyday.. -- **2026-03-23** ๐Ÿค– Launched **CLI-Hub meta-skill** โ€” agents can now discover and install CLIs autonomously via [`cli-hub-meta-skill/SKILL.md`](cli-hub-meta-skill/SKILL.md). The catalog auto-updates from `registry.json` via GitHub Actions, making the entire marketplace agent-native. +- **2026-03-30** ๐Ÿ—๏ธ **CLI-Anything v0.2.0** โ€” HARNESS.md progressive disclosure redesign. Detailed guides (MCP backend, filter translation, timecode, session locking, PyPI publishing, SKILL.md generation) extracted into `guides/` for on-demand loading. Phases 1โ€“7 now contiguous. Key Principles and Rules merged into a single authoritative section. Added Guides Reference routing table. Renamed "Critical Lessons Learned" to "Architecture Patterns & Pitfalls." -- **2026-03-22** ๐ŸŽต **MuseScore CLI** merged with transpose, export, and instrument management. Community contributions continue expanding domain coverage. +- **2026-03-29** ๐Ÿ“ Blender skill docs updated โ€” enforce absolute render paths and correct prerequisites. + +- **2026-03-28** ๐ŸŒ **CLIBrowser** added to CLI-Hub registry for agent-accessible browser automation. + +- **2026-03-27** ๐Ÿ“š Zotero SKILL.md enhanced with agent-facing constraints; REPL config and executable resolution fixes. + +- **2026-03-26** ๐Ÿ“– **Zotero CLI** harness landed for Zotero desktop (library management, collections, citations). Draw.io custom ID bugfix (#132) and registry.json syntax fix. + +- **2026-03-25** ๐ŸŽฎ **RenderDoc CLI** merged for GPU frame capture analysis (PSO compute, REPL capture cache). FreeCAD updated for v1.1 (new datum system, tapping, simulation). Blender EEVEE engine name corrected. Zoom token permissions hardened. + +- **2026-03-24** ๐Ÿญ **FreeCAD CLI** added with 258 commands across 17 groups. **iTerm2** and **Teltonika RMS** harnesses added to registry. CLI-Hub frontend and README install URLs updated. + +- **2026-03-23** ๐Ÿค– Launched **CLI-Hub meta-skill** โ€” agents can now discover and install CLIs autonomously via [`cli-hub-meta-skill/SKILL.md`](cli-hub-meta-skill/SKILL.md). **Krita CLI** harness merged for digital painting. DOMShell MCP parameter mismatches and connection model fixed. + +
+Earlier news (Mar 17โ€“22) + +- **2026-03-22** ๐ŸŽต **MuseScore CLI** merged with transpose, export, and instrument management. - **2026-03-21** ๐Ÿ”ง Infrastructure improvements โ€” refined test harnesses and documentation across multiple CLIs. Enhanced Windows compatibility for several backends. @@ -53,22 +70,24 @@ CLI-Anything: Bridging the Gap Between AI Agents and the World's Software
-Earlier news +Earlier news (Mar 11โ€“16) -- **2026-03-16** ๐Ÿค– Added **SKILL.md generation** (Phase 6.5) โ€” every generated CLI now ships with an AI-discoverable skill definition inside the Python package. ReplSkin auto-detects the skill file after `pip install`, and the REPL banner displays the absolute path for agents. Includes `skill_generator.py`, Jinja2 template, `package_data` in all setup.py files, and 51 new tests. +- **2026-03-16** ๐Ÿค– Added **SKILL.md generation** (Phase 6.5) โ€” every generated CLI now ships with an AI-discoverable skill definition. Includes `skill_generator.py`, Jinja2 template, and 51 new tests. -- **2026-03-15** ๐Ÿพ Support for **OpenClaw** from the community! Merged Windows `cygpath` guard to ensure CLI-Anything works reliably in Windows bash environments. Community contributions continue to strengthen cross-platform support. +- **2026-03-15** ๐Ÿพ Support for **OpenClaw** from the community! Merged Windows `cygpath` guard for cross-platform support. -- **2026-03-14** ๐Ÿ”’ Fixed a GIMP Script-Fu path injection vulnerability and added **Japanese README** translation. OpenCode version requirements documented alongside several Windows compatibility improvements. +- **2026-03-14** ๐Ÿ”’ Fixed a GIMP Script-Fu path injection vulnerability and added **Japanese README** translation. -- **2026-03-13** ๐Ÿ”Œ **Qodercli** plugin officially merged as a community contribution with dedicated setup scripts. Codex skill gained a Windows install script, and placeholder URLs were cleaned up across the project. +- **2026-03-13** ๐Ÿ”Œ **Qodercli** plugin officially merged as a community contribution with dedicated setup scripts. -- **2026-03-12** ๐Ÿ“ฆ **Codex skill** integration landed, bringing CLI-Anything to yet another AI coding platform. Qodercli support was also introduced, and documentation was updated with limitations and experimental labels. +- **2026-03-12** ๐Ÿ“ฆ **Codex skill** integration landed, bringing CLI-Anything to yet another AI coding platform. -- **2026-03-11** ๐Ÿ“ž **Zoom** video conferencing harness added as the 11th supported application. Multiple community fixes shipped for Shotcut auto-save, LibreOffice Windows/macOS backend, and OpenCode support. +- **2026-03-11** ๐Ÿ“ž **Zoom** video conferencing harness added as the 11th supported application.
diff --git a/cli-anything-plugin/HARNESS.md b/cli-anything-plugin/HARNESS.md index acd12fe8d..1a08bea0d 100644 --- a/cli-anything-plugin/HARNESS.md +++ b/cli-anything-plugin/HARNESS.md @@ -68,35 +68,9 @@ designed for humans, without needing a display or mouse. Generate valid intermediate files, then invoke the real software for conversion. 6. **Add session management** โ€” State persistence, undo/redo - **Session file locking** โ€” When saving session JSON, use exclusive file locking - to prevent concurrent writes from corrupting data. Never use bare - `open("w") + json.dump()` โ€” `open("w")` truncates the file before any lock - can be acquired. Instead, open with `"r+"`, lock, then truncate inside the lock: - ```python - def _locked_save_json(path, data, **dump_kwargs) -> None: - """Atomically write JSON with exclusive file locking.""" - try: - f = open(path, "r+") # no truncation on open - except FileNotFoundError: - os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True) - f = open(path, "w") # first save โ€” file doesn't exist yet - with f: - _locked = False - try: - import fcntl - fcntl.flock(f.fileno(), fcntl.LOCK_EX) - _locked = True - except (ImportError, OSError): - pass # Windows / unsupported FS โ€” proceed unlocked - try: - f.seek(0) - f.truncate() # truncate INSIDE the lock - json.dump(data, f, **dump_kwargs) - f.flush() - finally: - if _locked: - fcntl.flock(f.fileno(), fcntl.LOCK_UN) - ``` + **Session file locking** โ€” Use exclusive file locking for session JSON saves + to prevent concurrent write corruption. See [`guides/session-locking.md`](guides/session-locking.md) + for the `_locked_save_json` pattern (open `"r+"`, lock, then truncate inside the lock). 7. **Add the REPL with unified skin** โ€” Interactive mode wrapping the subcommands. - Copy `repl_skin.py` from the plugin (`cli-anything-plugin/repl_skin.py`) into `utils/repl_skin.py` in your CLI package @@ -275,60 +249,16 @@ definition that can be loaded by Claude Code or other AI assistants. --- ``` -2. **Markdown Body** โ€” Usage instructions including: - - Installation prerequisites - - Basic command syntax - - Command groups and their functions - - Usage examples - - Agent-specific guidance (JSON output, error handling) +2. **Markdown Body** โ€” Installation prerequisites, command syntax, command groups, + usage examples, and agent-specific guidance (JSON output, error handling). -**Generation Process:** +**Generation & Customization:** Use `skill_generator.py` to extract CLI metadata +automatically, or customize via the Jinja2 template at `templates/SKILL.md.template`. +See [`guides/skill-generation.md`](guides/skill-generation.md) for the full generation +process, template customization options, and manual generation commands. -1. **Extract CLI metadata** using `skill_generator.py`: - ```python - from skill_generator import generate_skill_file - - skill_path = generate_skill_file( - harness_path="/path/to/agent-harness" - ) - # Default output: cli_anything//skills/SKILL.md - ``` - -2. **The generator automatically extracts:** - - Software name and version from setup.py - - Command groups from the CLI file (Click decorators) - - Documentation from README.md - - System package requirements - -3. **Customize the template** (optional): - - Default template: `templates/SKILL.md.template` - - Uses Jinja2 placeholders for dynamic content - - Can be extended for software-specific sections - -**Output Location:** - -SKILL.md is generated inside the Python package so it is installed with `pip install`: -``` -/ -โ””โ”€โ”€ agent-harness/ - โ””โ”€โ”€ cli_anything/ - โ””โ”€โ”€ / - โ””โ”€โ”€ skills/ - โ””โ”€โ”€ SKILL.md -``` - -**Manual Generation:** - -```bash -cd cli-anything-plugin -python skill_generator.py /path/to/software/agent-harness -``` - -**Integration with CLI Build:** - -The SKILL.md generation should be run after Phase 6 (Test Documentation) completes -successfully, ensuring the CLI is fully documented and tested before creating the -skill definition. +**Output Location:** SKILL.md lives inside the Python package at +`cli_anything//skills/SKILL.md` so it is installed with `pip install`. **Key Principles:** @@ -340,20 +270,11 @@ skill definition. **Skill Path in CLI Banner:** -ReplSkin auto-detects `skills/SKILL.md` inside the package and displays the absolute -path in the startup banner. AI agents can read the file at the displayed path: +ReplSkin auto-detects `skills/SKILL.md` inside the package directory and displays +the absolute path in the startup banner. AI agents can read the skill file at the +displayed path to learn the CLI's full capabilities. -```python -# In the REPL initialization (e.g., shotcut_cli.py) -from cli_anything..utils.repl_skin import ReplSkin - -skin = ReplSkin("", version="1.0.0") -skin.print_banner() # Auto-detects and displays: โ—‡ Skill: /path/to/cli_anything//skills/SKILL.md -``` - -**Package Data:** - -Ensure `setup.py` includes the skill file as package data so it is installed with pip: +**Package Data:** Ensure `setup.py` includes the skill file so it ships with pip: ```python package_data={ @@ -361,7 +282,18 @@ package_data={ }, ``` -## Critical Lessons Learned +### Phase 7: PyPI Publishing and Installation + +After building and testing the CLI, make it installable and discoverable using +**PEP 420 namespace packages** under the shared `cli_anything` namespace. + +See [`guides/pypi-publishing.md`](guides/pypi-publishing.md) for the full setup.py template, +namespace package structure, import conventions, and verification steps. + +**Key rule:** `cli_anything/` has **no** `__init__.py` (namespace package). Each +sub-package (`gimp/`, `blender/`, etc.) **does** have `__init__.py`. + +## Architecture Patterns & Pitfalls ### Use the Real Software โ€” Don't Reimplement It @@ -426,89 +358,23 @@ the input. Users can't tell anything happened. ### MCP Backend Pattern -For services that expose an MCP (Model Context Protocol) server: +For software that exposes an MCP (Model Context Protocol) server instead of a traditional +CLI (e.g., DOMShell for browser automation). See [`guides/mcp-backend.md`](guides/mcp-backend.md) +for the full backend wrapper pattern, session management, daemon mode, and example implementations. -**Use case:** When the software provides an MCP server instead of a traditional CLI. -Example: DOMShell provides browser automation via MCP tools. - -**When to use:** -- The software has an official or community MCP server -- No native CLI exists, or MCP provides better functionality -- You want to integrate AI/agent tools that speak MCP protocol - -**Backend wrapper** (`utils/_backend.py`): -```python -import asyncio -from typing import Any -from mcp import ClientSession, StdioServerParameters -from mcp.client.stdio import stdio_client - -async def _call_tool(tool_name: str, arguments: dict) -> Any: - """Call an MCP tool.""" - server_params = StdioServerParameters( - command="npx", - args=["@apireno/domshell"] - ) - async with stdio_client(server_params) as (read, write): - async with ClientSession(read, write) as session: - await session.initialize() - result = await session.call_tool(tool_name, arguments) - return result - -def is_available() -> bool: - """Check if MCP server is available.""" - # Try to spawn and verify - ... - -# Sync wrappers for each tool -def ls(path: str = "/") -> dict: - """List directory contents.""" - return asyncio.run(_call_tool("domshell_ls", {"path": path})) -``` - -**Session management:** -- MCP server spawns per command (stateless from server perspective) -- CLI maintains state (URL, working directory, navigation history) -- Each command re-spawns the MCP server process - -**Daemon mode (optional):** -- Spawn MCP server once, reuse connection for multiple commands -- Reduces latency for interactive use -- Requires explicit start/stop or `--daemon` flag - -**Dependencies:** Add `mcp>=0.1.0` to install_requires - -**Example implementations:** -- `browser/agent-harness` โ€” DOMShell MCP server for browser automation -- See: https://github.com/HKUDS/CLI-Anything/tree/main/browser/agent-harness +**Use when:** no native CLI exists, software has an MCP server, or you need agent-native tool integration. ### Filter Translation Pitfalls -When translating effects between formats (e.g., MLT โ†’ ffmpeg), watch for: - -- **Duplicate filter types:** Some tools (ffmpeg) don't allow the same filter twice - in a chain. If your project has both `brightness` and `saturation` filters, and - both map to ffmpeg's `eq=`, you must **merge** them into a single `eq=brightness=X:saturation=Y`. -- **Ordering constraints:** ffmpeg's `concat` filter requires **interleaved** stream - ordering: `[v0][a0][v1][a1][v2][a2]`, NOT grouped `[v0][v1][v2][a0][a1][a2]`. - The error message ("media type mismatch") is cryptic if you don't know this. -- **Parameter space differences:** Effect parameters often use different scales. - MLT brightness `1.15` = +15%, but ffmpeg `eq=brightness=0.06` on a -1..1 scale. - Document every mapping explicitly. -- **Unmappable effects:** Some effects have no equivalent in the render tool. Handle - gracefully (warn, skip) rather than crash. +When translating effects between formats (e.g., MLT โ†’ ffmpeg), watch for duplicate filter +merging, interleaved stream ordering, parameter scale differences, and unmappable effects. +See [`guides/filter-translation.md`](guides/filter-translation.md) for detailed rules and examples. ### Timecode Precision -Non-integer frame rates (29.97fps = 30000/1001) cause cumulative rounding errors: - -- **Use `round()`, not `int()`** for float-to-frame conversion. `int(9000 * 29.97)` - truncates and loses frames; `round()` gets the right answer. -- **Use integer arithmetic for timecode display.** Convert frames โ†’ total milliseconds - via `round(frames * fps_den * 1000 / fps_num)`, then decompose with integer - division. Avoid intermediate floats that drift over long durations. -- **Accept ยฑ1 frame tolerance** in roundtrip tests at non-integer FPS. Exact equality - is mathematically impossible. +Non-integer frame rates (29.97fps) cause cumulative rounding errors. Key rules: use +`round()` not `int()`, use integer arithmetic for display, accept ยฑ1 frame tolerance. +See [`guides/timecode-precision.md`](guides/timecode-precision.md) for the full approach. ### Output Verification Methodology @@ -602,53 +468,52 @@ Real-world workflow test scenarios should include: - Save/load round-trips of complex projects - Iterative refinement (add, modify, remove, re-add) -## Key Principles +## Principles & Rules -- **Use the real software** โ€” The CLI MUST invoke the actual application for rendering - and export. Generate valid intermediate files (ODF, MLT XML, .blend, SVG), then hand - them to the real software. Never reimplement the rendering engine in Python. -- **The software is a hard dependency** โ€” Not optional, not gracefully degraded. If - LibreOffice isn't installed, `cli-anything-libreoffice` must error clearly, not - silently produce inferior output with a fallback library. +These are non-negotiable. Every harness MUST follow all of them. + +**Backend & Rendering:** +- **The real software is a hard dependency.** The CLI MUST invoke the actual application + (LibreOffice, Blender, GIMP, etc.) for rendering and export. Do NOT reimplement + rendering in Python. Do NOT gracefully degrade to a fallback library. If the software + is not installed, error with clear install instructions. - **Manipulate the native format directly** โ€” Parse and modify the app's native project files (MLT XML, ODF, SVG, etc.) as the data layer. - **Leverage existing CLI tools** โ€” Use `libreoffice --headless`, `blender --background`, `melt`, `ffmpeg`, `inkscape --actions`, `sox` as subprocesses for rendering. -- **Verify rendering produces correct output** โ€” See "The Rendering Gap" above. -- **E2E tests must produce real artifacts** โ€” PDF, DOCX, rendered images, videos. - Print output paths so users can inspect. Never test only the intermediate format. +- **Verify rendering produces correct output** โ€” See "The Rendering Gap" in + Architecture Patterns & Pitfalls above. +- **Every filter/effect in the registry MUST have a corresponding render mapping** + or be explicitly documented as "project-only (not rendered)". + +**CLI Design:** - **Fail loudly and clearly** โ€” Agents need unambiguous error messages to self-correct. - **Be idempotent where possible** โ€” Running the same command twice should be safe. - **Provide introspection** โ€” `info`, `list`, `status` commands are critical for agents to understand current state before acting. -- **JSON output mode** โ€” Every command should support `--json` for machine parsing. +- **JSON output mode** โ€” Every command MUST support `--json` for machine parsing. +- **Use the unified REPL skin** โ€” Copy `cli-anything-plugin/repl_skin.py` to + `utils/repl_skin.py` and use `ReplSkin` for banner, prompt, help, and messages. + REPL MUST be the default behavior (`invoke_without_command=True`). -## Rules - -- **The real software MUST be a hard dependency.** The CLI must invoke the actual - software (LibreOffice, Blender, GIMP, etc.) for rendering and export. Do NOT - reimplement rendering in Python. Do NOT gracefully degrade to a fallback library. - If the software is not installed, the CLI must error with clear install instructions. -- **Every `cli_anything//` directory MUST contain a `README.md`** that explains how to - install the software dependency, install the CLI, run tests, and shows basic usage. +**Testing:** - **E2E tests MUST invoke the real software** and produce real output files (PDF, DOCX, - rendered images, videos). Tests must verify output exists, has correct format, and - print artifact paths so users can inspect results. Never test only intermediate files. -- **Every export/render function MUST be verified** with programmatic output analysis - before being marked as working. "It ran without errors" is not sufficient. -- **Every filter/effect in the registry MUST have a corresponding render mapping** - or be explicitly documented as "project-only (not rendered)". -- **Test suites MUST include real-file E2E tests**, not just unit tests with synthetic - data. Format assumptions break constantly with real media. + rendered images, videos). Verify output exists, has correct format (magic bytes, ZIP + structure), and print artifact paths for manual inspection. Never test only + intermediate files. +- **Every export/render function MUST be verified** with programmatic output analysis. + "It ran without errors" is not sufficient. - **E2E tests MUST include subprocess tests** that invoke the installed `cli-anything-` command via `_resolve_cli()`. Tests must work against the actual installed package, not just source imports. -- **Every `cli_anything//tests/` directory MUST contain a `TEST.md`** documenting what the tests - cover, what realistic workflows are tested, and the full test results output. -- **Every CLI MUST use the unified REPL skin** (`repl_skin.py`) for the interactive mode. - Copy `cli-anything-plugin/repl_skin.py` to `utils/repl_skin.py` and use `ReplSkin` - for the banner, prompt, help, messages, and goodbye. REPL MUST be the default behavior - when the CLI is invoked without a subcommand (`invoke_without_command=True`). +- **Test suites MUST include real-file E2E tests**, not just unit tests with synthetic + data. Format assumptions break constantly with real media. + +**Documentation:** +- **Every `cli_anything//` directory MUST contain a `README.md`** explaining + how to install the software dependency, install the CLI, run tests, and basic usage. +- **Every `cli_anything//tests/` directory MUST contain a `TEST.md`** + documenting test coverage, realistic workflows tested, and full test results output. ## Directory Structure @@ -712,110 +577,16 @@ command-line interface TO the software, not a replacement for it. The pattern is always the same: **build the data โ†’ call the real software โ†’ verify the output**. -### Phase 7: PyPI Publishing and Installation +## Guides Reference -After building and testing the CLI, make it installable and discoverable. +Detailed guides live in `guides/`. Use this table to decide which ones to read +based on the software you're building a harness for. -All cli-anything CLIs use **PEP 420 namespace packages** under the shared -`cli_anything` namespace. This allows multiple CLI packages to be installed -side-by-side in the same Python environment without conflicts. - -1. **Structure the package** as a namespace package: - ``` - agent-harness/ - โ”œโ”€โ”€ setup.py - โ””โ”€โ”€ cli_anything/ # NO __init__.py here (namespace package) - โ””โ”€โ”€ / # e.g., gimp, blender, audacity - โ”œโ”€โ”€ __init__.py # HAS __init__.py (regular sub-package) - โ”œโ”€โ”€ _cli.py - โ”œโ”€โ”€ core/ - โ”œโ”€โ”€ utils/ - โ””โ”€โ”€ tests/ - ``` - - The key rule: `cli_anything/` has **no** `__init__.py`. Each sub-package - (`gimp/`, `blender/`, etc.) **does** have `__init__.py`. This is what - enables multiple packages to contribute to the same namespace. - -2. **Create setup.py** in the `agent-harness/` directory: - ```python - from setuptools import setup, find_namespace_packages - - setup( - name="cli-anything-", - version="1.0.0", - packages=find_namespace_packages(include=["cli_anything.*"]), - install_requires=[ - "click>=8.0.0", - "prompt-toolkit>=3.0.0", - # Add Python library dependencies here - ], - entry_points={ - "console_scripts": [ - "cli-anything-=cli_anything.._cli:main", - ], - }, - python_requires=">=3.10", - ) - ``` - - **Important details:** - - Use `find_namespace_packages`, NOT `find_packages` - - Use `include=["cli_anything.*"]` to scope discovery - - Entry point format: `cli_anything.._cli:main` - - The **system package** (LibreOffice, Blender, etc.) is a **hard dependency** - that cannot be expressed in `install_requires`. Document it in README.md and - have the backend module raise a clear error with install instructions: - ```python - # In utils/_backend.py - def find_(): - path = shutil.which("") - if path: - return path - raise RuntimeError( - " is not installed. Install it with:\n" - " apt install # Debian/Ubuntu\n" - " brew install # macOS" - ) - ``` - -3. **All imports** use the `cli_anything.` prefix: - ```python - from cli_anything.gimp.core.project import create_project - from cli_anything.gimp.core.session import Session - from cli_anything.blender.core.scene import create_scene - ``` - -4. **Test local installation**: - ```bash - cd /root/cli-anything//agent-harness - pip install -e . - ``` - -5. **Verify PATH installation**: - ```bash - which cli-anything- - cli-anything- --help - ``` - -6. **Run tests against the installed command**: - ```bash - cd /root/cli-anything//agent-harness - CLI_ANYTHING_FORCE_INSTALLED=1 python3 -m pytest cli_anything//tests/ -v -s - ``` - The output must show `[_resolve_cli] Using installed command: /path/to/cli-anything-` - confirming subprocess tests ran against the real installed binary, not a module fallback. - -7. **Verify namespace works across packages** (when multiple CLIs installed): - ```python - import cli_anything.gimp - import cli_anything.blender - # Both resolve to their respective source directories - ``` - -**Why namespace packages:** -- Multiple CLIs coexist in the same Python environment without conflicts -- Clean, organized imports under a single `cli_anything` namespace -- Each CLI is independently installable/uninstallable via pip -- Agents can discover all installed CLIs via `cli_anything.*` -- Standard Python packaging โ€” no hacks or workarounds +| Guide | Read when... | Phase | +|-------|-------------|-------| +| [`session-locking.md`](guides/session-locking.md) | Implementing session save (all harnesses) | Phase 3 | +| [`skill-generation.md`](guides/skill-generation.md) | Generating the SKILL.md file | Phase 6.5 | +| [`pypi-publishing.md`](guides/pypi-publishing.md) | Packaging and installing the CLI | Phase 7 | +| [`mcp-backend.md`](guides/mcp-backend.md) | Software has an MCP server, no native CLI | Phase 3 | +| [`filter-translation.md`](guides/filter-translation.md) | Video/audio CLI with effects that need render-time translation | Phase 3 | +| [`timecode-precision.md`](guides/timecode-precision.md) | Video/audio CLI with non-integer frame rates (29.97fps, etc.) | Phase 3, 5 | diff --git a/cli-anything-plugin/guides/filter-translation.md b/cli-anything-plugin/guides/filter-translation.md new file mode 100644 index 000000000..5e32fc8ef --- /dev/null +++ b/cli-anything-plugin/guides/filter-translation.md @@ -0,0 +1,26 @@ +# Filter Translation Pitfalls + +When translating effects between formats (e.g., MLT โ†’ ffmpeg), watch for these common issues. + +## Duplicate Filter Types + +Some tools (ffmpeg) don't allow the same filter twice in a chain. If your project has both `brightness` and `saturation` filters, and both map to ffmpeg's `eq=`, you must **merge** them into a single `eq=brightness=X:saturation=Y`. + +## Ordering Constraints + +ffmpeg's `concat` filter requires **interleaved** stream ordering: +`[v0][a0][v1][a1][v2][a2]`, NOT grouped `[v0][v1][v2][a0][a1][a2]`. + +The error message ("media type mismatch") is cryptic if you don't know this. + +## Parameter Space Differences + +Effect parameters often use different scales: +- MLT brightness `1.15` = +15% +- ffmpeg `eq=brightness=0.06` on a -1..1 scale + +Document every mapping explicitly. + +## Unmappable Effects + +Some effects have no equivalent in the render tool. Handle gracefully (warn, skip) rather than crash. diff --git a/cli-anything-plugin/guides/mcp-backend.md b/cli-anything-plugin/guides/mcp-backend.md new file mode 100644 index 000000000..0db45a45e --- /dev/null +++ b/cli-anything-plugin/guides/mcp-backend.md @@ -0,0 +1,64 @@ +# MCP Backend Pattern + +For services that expose an MCP (Model Context Protocol) server instead of a traditional CLI. + +## When to Use + +- The software has an official or community MCP server +- No native CLI exists, or MCP provides better functionality +- You want to integrate AI/agent tools that speak MCP protocol + +**Use case:** When the software provides an MCP server instead of a traditional CLI. +Example: DOMShell provides browser automation via MCP tools. + +## Backend Wrapper (`utils/_backend.py`) + +```python +import asyncio +from typing import Any +from mcp import ClientSession, StdioServerParameters +from mcp.client.stdio import stdio_client + +async def _call_tool(tool_name: str, arguments: dict) -> Any: + """Call an MCP tool.""" + server_params = StdioServerParameters( + command="npx", + args=["@apireno/domshell"] + ) + async with stdio_client(server_params) as (read, write): + async with ClientSession(read, write) as session: + await session.initialize() + result = await session.call_tool(tool_name, arguments) + return result + +def is_available() -> bool: + """Check if MCP server is available.""" + # Try to spawn and verify + ... + +# Sync wrappers for each tool +def ls(path: str = "/") -> dict: + """List directory contents.""" + return asyncio.run(_call_tool("domshell_ls", {"path": path})) +``` + +## Session Management + +- MCP server spawns per command (stateless from server perspective) +- CLI maintains state (URL, working directory, navigation history) +- Each command re-spawns the MCP server process + +## Daemon Mode (Optional) + +- Spawn MCP server once, reuse connection for multiple commands +- Reduces latency for interactive use +- Requires explicit start/stop or `--daemon` flag + +## Dependencies + +Add `mcp>=0.1.0` to `install_requires`. + +## Example Implementations + +- `browser/agent-harness` โ€” DOMShell MCP server for browser automation +- See: https://github.com/HKUDS/CLI-Anything/tree/main/browser/agent-harness diff --git a/cli-anything-plugin/guides/pypi-publishing.md b/cli-anything-plugin/guides/pypi-publishing.md new file mode 100644 index 000000000..7c8b72cc5 --- /dev/null +++ b/cli-anything-plugin/guides/pypi-publishing.md @@ -0,0 +1,117 @@ +# PyPI Publishing and Installation (Phase 7) + +After building and testing the CLI, make it installable and discoverable. + +All cli-anything CLIs use **PEP 420 namespace packages** under the shared +`cli_anything` namespace. This allows multiple CLI packages to be installed +side-by-side in the same Python environment without conflicts. + +## 1. Package Structure + +``` +agent-harness/ +โ”œโ”€โ”€ setup.py +โ””โ”€โ”€ cli_anything/ # NO __init__.py here (namespace package) + โ””โ”€โ”€ / # e.g., gimp, blender, audacity + โ”œโ”€โ”€ __init__.py # HAS __init__.py (regular sub-package) + โ”œโ”€โ”€ _cli.py + โ”œโ”€โ”€ core/ + โ”œโ”€โ”€ utils/ + โ””โ”€โ”€ tests/ +``` + +The key rule: `cli_anything/` has **no** `__init__.py`. Each sub-package +(`gimp/`, `blender/`, etc.) **does** have `__init__.py`. This is what +enables multiple packages to contribute to the same namespace. + +## 2. setup.py Template + +Create `setup.py` in the `agent-harness/` directory: + +```python +from setuptools import setup, find_namespace_packages + +setup( + name="cli-anything-", + version="1.0.0", + packages=find_namespace_packages(include=["cli_anything.*"]), + install_requires=[ + "click>=8.0.0", + "prompt-toolkit>=3.0.0", + # Add Python library dependencies here + ], + entry_points={ + "console_scripts": [ + "cli-anything-=cli_anything.._cli:main", + ], + }, + python_requires=">=3.10", +) +``` + +**Important details:** +- Use `find_namespace_packages`, NOT `find_packages` +- Use `include=["cli_anything.*"]` to scope discovery +- Entry point format: `cli_anything.._cli:main` +- The **system package** (LibreOffice, Blender, etc.) is a **hard dependency** + that cannot be expressed in `install_requires`. Document it in README.md and + have the backend module raise a clear error with install instructions: + ```python + # In utils/_backend.py + def find_(): + path = shutil.which("") + if path: + return path + raise RuntimeError( + " is not installed. Install it with:\n" + " apt install # Debian/Ubuntu\n" + " brew install # macOS" + ) + ``` + +## 3. Import Convention + +All imports use the `cli_anything.` prefix: + +```python +from cli_anything.gimp.core.project import create_project +from cli_anything.gimp.core.session import Session +from cli_anything.blender.core.scene import create_scene +``` + +## 4. Verification Steps + +**Test local installation:** +```bash +cd /root/cli-anything//agent-harness +pip install -e . +``` + +**Verify PATH installation:** +```bash +which cli-anything- +cli-anything- --help +``` + +**Run tests against the installed command:** +```bash +cd /root/cli-anything//agent-harness +CLI_ANYTHING_FORCE_INSTALLED=1 python3 -m pytest cli_anything//tests/ -v -s +``` +The output must show `[_resolve_cli] Using installed command: /path/to/cli-anything-` +confirming subprocess tests ran against the real installed binary, not a module fallback. + +**Verify namespace works across packages** (when multiple CLIs installed): +```python +import cli_anything.gimp +import cli_anything.blender +# Both resolve to their respective source directories +``` + +## Why Namespace Packages + +- Multiple CLIs coexist in the same Python environment without conflicts +- Clean, organized imports under a single `cli_anything` namespace +- Each CLI is independently installable/uninstallable via pip +- Agents can discover all installed CLIs via `cli_anything.*` +- Standard Python packaging โ€” no hacks or workarounds diff --git a/cli-anything-plugin/guides/session-locking.md b/cli-anything-plugin/guides/session-locking.md new file mode 100644 index 000000000..3c3f50e6f --- /dev/null +++ b/cli-anything-plugin/guides/session-locking.md @@ -0,0 +1,39 @@ +# Session File Locking + +When saving session JSON, use exclusive file locking to prevent concurrent writes from corrupting data. + +## The Problem + +Never use bare `open("w") + json.dump()` โ€” `open("w")` truncates the file before any lock can be acquired. + +## The Solution: `_locked_save_json` + +Open with `"r+"`, lock, then truncate inside the lock: + +```python +def _locked_save_json(path, data, **dump_kwargs) -> None: + """Atomically write JSON with exclusive file locking.""" + try: + f = open(path, "r+") # no truncation on open + except FileNotFoundError: + os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True) + f = open(path, "w") # first save โ€” file doesn't exist yet + with f: + _locked = False + try: + import fcntl + fcntl.flock(f.fileno(), fcntl.LOCK_EX) + _locked = True + except (ImportError, OSError): + pass # Windows / unsupported FS โ€” proceed unlocked + try: + f.seek(0) + f.truncate() # truncate INSIDE the lock + json.dump(data, f, **dump_kwargs) + f.flush() + finally: + if _locked: + fcntl.flock(f.fileno(), fcntl.LOCK_UN) +``` + +Copy this pattern into `core/session.py` for all session saves. diff --git a/cli-anything-plugin/guides/skill-generation.md b/cli-anything-plugin/guides/skill-generation.md new file mode 100644 index 000000000..ea3f02a77 --- /dev/null +++ b/cli-anything-plugin/guides/skill-generation.md @@ -0,0 +1,115 @@ +# SKILL.md Generation (Phase 6.5) + +Generate a SKILL.md file that makes the CLI discoverable and usable by AI agents +through the skill-creator methodology. This file serves as a self-contained skill +definition that can be loaded by Claude Code or other AI assistants. + +## Purpose + +SKILL.md files follow a standard format that enables AI agents to: +- Discover the CLI's capabilities +- Understand command structure and usage +- Generate correct command invocations +- Handle output programmatically + +## SKILL.md Structure + +### 1. YAML Frontmatter โ€” Triggering metadata for skill discovery: + +```yaml +--- +name: "cli-anything-" +description: "Brief description of what the CLI does" +--- +``` + +### 2. Markdown Body โ€” Usage instructions including: + +- Installation prerequisites +- Basic command syntax +- Command groups and their functions +- Usage examples +- Agent-specific guidance (JSON output, error handling) + +## Generation Process + +### 1. Extract CLI metadata using `skill_generator.py`: + +```python +from skill_generator import generate_skill_file + +skill_path = generate_skill_file( + harness_path="/path/to/agent-harness" +) +# Default output: cli_anything//skills/SKILL.md +``` + +### 2. The generator automatically extracts: + +- Software name and version from setup.py +- Command groups from the CLI file (Click decorators) +- Documentation from README.md +- System package requirements + +### 3. Customize the template (optional): + +- Default template: `templates/SKILL.md.template` +- Uses Jinja2 placeholders for dynamic content +- Can be extended for software-specific sections + +## Output Location + +SKILL.md is generated inside the Python package so it is installed with `pip install`: + +``` +/ +โ””โ”€โ”€ agent-harness/ + โ””โ”€โ”€ cli_anything/ + โ””โ”€โ”€ / + โ””โ”€โ”€ skills/ + โ””โ”€โ”€ SKILL.md +``` + +## Manual Generation + +```bash +cd cli-anything-plugin +python skill_generator.py /path/to/software/agent-harness +``` + +## Integration with CLI Build + +The SKILL.md generation should be run after Phase 6 (Test Documentation) completes +successfully, ensuring the CLI is fully documented and tested before creating the +skill definition. + +## Key Principles + +- SKILL.md must be self-contained (no external dependencies for understanding) +- Include agent-specific guidance for programmatic usage +- Document `--json` flag usage for machine-readable output +- List all command groups with brief descriptions +- Provide realistic examples that demonstrate common workflows + +## Skill Path in CLI Banner + +ReplSkin auto-detects `skills/SKILL.md` inside the package and displays the absolute +path in the startup banner. AI agents can read the file at the displayed path: + +```python +# In the REPL initialization (e.g., shotcut_cli.py) +from cli_anything..utils.repl_skin import ReplSkin + +skin = ReplSkin("", version="1.0.0") +skin.print_banner() # Auto-detects and displays: โ—‡ Skill: /path/to/cli_anything//skills/SKILL.md +``` + +## Package Data + +Ensure `setup.py` includes the skill file as package data so it is installed with pip: + +```python +package_data={ + "cli_anything.": ["skills/*.md"], +}, +``` diff --git a/cli-anything-plugin/guides/timecode-precision.md b/cli-anything-plugin/guides/timecode-precision.md new file mode 100644 index 000000000..6fc50345e --- /dev/null +++ b/cli-anything-plugin/guides/timecode-precision.md @@ -0,0 +1,22 @@ +# Timecode Precision + +Non-integer frame rates (29.97fps = 30000/1001) cause cumulative rounding errors. Follow these rules to avoid drift. + +## Use `round()`, Not `int()` + +For float-to-frame conversion: +- `int(9000 * 29.97)` โ€” **wrong**, truncates and loses frames +- `round(9000 * 29.97)` โ€” **correct**, gets the right answer + +## Use Integer Arithmetic for Timecode Display + +Convert frames โ†’ total milliseconds via: +```python +total_ms = round(frames * fps_den * 1000 / fps_num) +``` + +Then decompose with integer division. Avoid intermediate floats that drift over long durations. + +## Accept ยฑ1 Frame Tolerance + +In roundtrip tests at non-integer FPS, exact equality is mathematically impossible. Accept ยฑ1 frame tolerance.