Comet is a Skills harness for coding agents: it turns a vague idea into a tracked, verified, and resumable workflow. After trying it on a small Node project, my verdict is straightforward: it does the job for structuring an agent, but it’s neither magical nor lightweight.
What is Comet?
Comet, the GitHub project by rpamis, describes itself as an “Agent Skill Harness For Turning Ideas Into Evaluated Workflows”. In plain terms, it doesn’t replace your coding agent. It gives it a framework: brief, phases, guardrails, state, evidence, resume, dashboard, and evaluation.
The tool is primarily aimed at long tasks where an assistant can lose the thread: adding a feature, evolving a spec, checking acceptance criteria, archiving what was done. Comet offers two workflows:
- Native, more direct, designed for strong models and short loops.
- Classic, more structured, inspired by OpenSpec and Superpowers.
It can also install Skills into coding agent platforms, such as Claude Code, and provides guard scripts to block transitions that happen too quickly.
| Element | What I observed |
|---|---|
| Version tested | 0.4.0-beta.14 from the GitHub repo |
| Stack | Node.js 22 required, pnpm, TypeScript, Vite for the dashboard |
| Interface | CLI + local web dashboard |
| Test case | Adding a /healthz HTTP endpoint in a small Node project |
| LLM | OpenAI proxy exported as required, not needed for the CLI runtime I tested |
| Result | Native workflow created, brief followed, artifacts hashed, real test executed, dashboard usable |
Installing Comet
I cloned the repo into /tmp/comet, read the README, the package.json, and the eval README. First friction point: my environment had Node 20, while Comet requires Node 22+. So I ran the commands with Node 22 via npx.
Another very concrete detail: /tmp was limited to 512 MB. The clone had to stay in /tmp, but the pnpm dependencies wouldn’t fit. I cleanly worked around it by placing the pnpm store and virtual store in a temporary folder outside the clone, then deleted those files afterwards.
The basic commands looked like this:
git clone https://github.com/rpamis/comet /tmp/comet
cd /tmp/comet
npx -y -p node@22 -p pnpm@10.18.3 pnpm install --ignore-scripts
npx -y -p node@22 -p pnpm@10.18.3 pnpm build
npx -y -p node@22 -p pnpm@10.18.3 pnpm test:runtime-smoke
The build compiled the Classic, Native, Entry, and TypeScript runtimes, along with the Vite dashboard. The runtime smoke test then passed 237 tests, with 1 test skipped. This isn’t just a binary that responds to a version command; the repo ships a real validation suite.
Comet in action on a real use case
To avoid a fake test, I created a small throwaway Node project: a health() function, then a realistic operator request: expose a /healthz endpoint with no external dependencies.
I initialized Comet in Native mode, project scope, Claude Code platform:
comet init . --yes --workflow native --platform claude --scope project --language en --json
comet doctor . --json
comet workflow resolve . --json
What Comet produced:
.comet/config.yaml, with the Native workflow selected..claude/skills/comet,.claude/skills/comet-native, and.claude/skills/comet-any.- a rule file
.claude/rules/comet-workflow-guard.md. - a hook router in the local Claude config.
- a folder
docs/comet/changes/add-health-endpoint/for the request.
Then I kicked off a Native change:
comet native new add-health-endpoint --language en
comet native status add-health-endpoint --details
First nice surprise: Comet refused to move forward as long as the brief was empty. The status reported brief-section-empty on the Outcome section. So I filled in the brief with outcome, scope, non-goals, acceptance examples, and verification expectations.
After confirmation, Comet advanced to the build phase. I then actually modified the small project: a native Node HTTP server, GET /healthz route, JSON response { "status": "ok" }, and a test that starts the server on an ephemeral port, calls the endpoint, checks the status, and shuts down the server.
The real test:
npm test
Useful output:
> test
> node test.js
health endpoint ok
Comet recorded a checkpoint with the artifacts index.js, test.js, and package.json, their sizes, and their hashes. The dashboard then showed the request in the verify phase, the complete implementation, 3 declared artifacts, 2 acceptance criteria, and a Git risks block.
What Comet actually produces
The product doesn’t just dump instructions into a README. It builds a tracking system around the task.
In my test, Comet produced:
- a structured brief, enforced by checks,
- a state machine with phase, review, approval, and verification result,
- checkpoints with a hashed manifest,
- an acceptance page extracted from the brief,
- a JSON and web dashboard,
- verification receipts.
The most interesting point, from a DevOps and security perspective, is traceability. When an agent says “I’m done”, Comet pushes you to answer: which artifact, which test, which evidence, which phase, which hash? That’s exactly the kind of discipline often missing in agentic workflows.
Comet’s limitations
My trial also revealed clear limitations.
First, the learning curve is real. The vocabulary is dense: Native, Classic, Shape, Build, Verify, receipts, fences, contract hash, implementation scope. For a freelancer who just wants to run an agent on a small fix, it’s too heavy.
Second, the verification phase is strict. In my throwaway project, the automated receipts did run npm test with exit code 0, but Comet marked them blocked, because the post-command snapshot fence didn’t match. In principle, this is healthy: the tool avoids validating evidence if the environment has shifted. In terms of user experience, it’s frustrating if the repo wasn’t prepared with a clean Git baseline from the start.
Finally, Comet is not a standalone autonomous agent ready to produce code all by itself from the CLI. It primarily serves as a harness around a Skills-compatible agent. The LLM lives in the coding agent platform, not necessarily within the comet command itself.
Does Comet really work?
Yes, for what I tested: installation, build, repo tests, project initialization, Native workflow creation, incomplete brief blocking, artifact tracking, dashboard, and verification of a real task.
No, if you expect a magic button that takes an idea and delivers a validated app without understanding the protocol. Comet imposes a method. That method can save time on long tasks, but it costs you upfront.
What I like:
- the guardrails before moving from one phase to another,
- the local dashboard, useful for seeing where a task stands,
- the hashes and manifests, good signals for auditing,
- the separation between Native and Classic,
- the commitment to evaluating Skills with metrics rather than gut feeling.
What I like less:
- abundant but intimidating documentation,
- Node 22 mandatory,
- sometimes very internals-focused messages,
- strict receipts that are hard to unblock in a dirty or poorly initialized repo,
- maximum value only with an agent already integrated into a compatible platform.
Should you adopt Comet?
My verdict: worth trying if you’re already running coding agents on long tasks, especially in teams or on projects where evidence matters. For a freelance security/DevOps use case, I can see it working well for infrastructure changes, risky refactors, fixes with acceptance criteria, and workflows where you need to resume after interruption.
I wouldn’t adopt it everywhere. For a small Nginx config fix or a 30-line script, Comet adds too much protocol. For an agent that needs to deliver content, modify code, test, and then leave behind a usable log, it becomes much more interesting.
In short: Comet is not the engine, it’s the rail. If your problem is that your agents derail, it deserves a real trial.
FAQ
Does Comet replace Claude Code or another agent?
No. Comet installs Skills, rules, hooks, and runtimes around an agent. It frames the work, but it doesn’t replace the model or the coding agent tool.
Can you use Comet without an OpenAI key?
For the CLI commands I tested, yes. The LLM becomes necessary when the Skills-compatible agent actually runs the generation workflow or when you launch certain evaluations.
Is Comet suitable for beginners?
Not really. The concept is powerful, but it already assumes you understand Git, coding agents, delivery phases, and test evidence. For a beginner, I’d suggest starting with Native mode on a very simple project.