Add ditto unpack CLI to turn clone result JSON into a project tree

The REST API returns a clone as a giant `files` map JSON blob with no
first-party way to use it — users were left staring at kilobytes of
escaped JSON with no path from "you got JSON" to "here's a project".

Add packages/cli: a zero-dependency `ditto` CLI whose `unpack` command
walks the files{} map from POST /v1/clones (or GET .../result) and
writes a real project tree to disk:

- text files written from inline content;
- binary assets materialized from inline base64, else fetched from their
  reference URL using $DITTO_API_URL / $DITTO_API_KEY (--no-fetch writes
  just the text tree and lists skipped assets);
- path-traversal guards (clone results are untrusted) and sha256
  integrity checks;
- reads JSON from stdin ("-") so a curl response pipes straight in;
- clear errors for queued jobs with no files yet, and a tip pointing at
  the /bundle endpoint when assets can't be fetched.

Document the pipe-from-curl one-liner prominently next to the REST
examples in README.md and docs/SERVICE.md, add a package README, a
Repository Map row, a root `unpack` script, and a CHANGELOG entry.
Covered by 9 tests (text tree, stdin, inline base64, live URL fetch with
auth + relative-URL resolution, --no-fetch, traversal refusal, queued-job
detection, sha256 mismatch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Michael Cole
2026-07-01 15:40:28 -04:00
co-authored by Claude Opus 4.8
parent 4de6e244a0
commit 638a0516c5
9 changed files with 589 additions and 5 deletions
+35 -4
View File
@@ -56,20 +56,50 @@ curl -sS -X POST "$DITTO_API_URL/v1/clones" \
}'
```
The service returns either an inline result or a queued job:
The service returns either a queued job or an inline result. A finished result
is a file map — every generated file keyed by its app-relative path:
```json
{ "jobId": "job_123", "status": "queued" }
{
"jobId": "job_123",
"status": "succeeded",
"files": {
"package.json": { "type": "text", "content": "{ ... }", "bytes": 812, "sha256": "..." },
"src/app/page.tsx": { "type": "text", "content": "export default ...", "bytes": 2048, "sha256": "..." },
"public/assets/logo.png": { "type": "binary", "url": ".../files/public/assets/logo.png", "bytes": 5123, "sha256": "..." }
}
}
```
Poll and download the generated app:
**Turn that JSON into a project on disk** with the official unpacker — pipe the
response straight in, no temp file:
```bash
curl -sS -X POST "$DITTO_API_URL/v1/clones" \
-H "authorization: Bearer $DITTO_API_KEY" \
-H "content-type: application/json" \
-d '{"url":"https://example.com/","options":{"mode":"single"}}' \
| npx ditto unpack - ./out
```
`ditto unpack <clone.json|-> <out-dir>` writes the text files inline and
materializes binary assets (inline base64, or fetched from their `url` using
`$DITTO_API_URL` / `$DITTO_API_KEY`). See
[`packages/cli`](packages/cli/README.md) for options.
If you got back a queued job (`{ "jobId": "job_123", "status": "queued" }`),
poll it, then unpack the finished result — or download the whole app as one
archive:
```bash
JOB_ID="job_123"
# poll status, then unpack the finished file map
curl -sS -H "authorization: Bearer $DITTO_API_KEY" \
"$DITTO_API_URL/v1/clones/$JOB_ID"
"$DITTO_API_URL/v1/clones/$JOB_ID/result" \
| npx ditto unpack - ./out
# ...or grab the whole app as a single archive
curl -L -H "authorization: Bearer $DITTO_API_KEY" \
"$DITTO_API_URL/v1/clones/$JOB_ID/bundle?format=tgz" \
-o ditto-clone.tgz
@@ -241,6 +271,7 @@ minting is intentional.
| --- | --- |
| `compiler/` | deterministic capture, inference, generation, and validation |
| `packages/core/` | compiler adapter and file-map helpers |
| `packages/cli/` | `ditto` CLI — unpack a clone result JSON into a project tree |
| `packages/api/` | Hono REST API and MCP server |
| `packages/db/` | Drizzle schema, migrations, repository, and queue wrapper |
| `packages/storage/` | local and S3/R2 artifact storage |