Canvas raster fallback, CDP rest-state screenshots, lottie overlay containment
- Visible canvases (>=48px) are captured as PNG stills at capture time (toDataURL, element-screenshot fallback for tainted/webgl) and emitted as cid-carrying <img> elements filling the captured box, so canvas-driven bands no longer render blank - Source AND clone full-page screenshots now capture via CDP Page.captureScreenshot with captureBeyondViewport: no scroll-stitching, no scroll events, so scroll-linked animations can't smear mid-scrub states into perceptual evidence (pixel-identical to the old path on static pages; Playwright fallback retained) - DittoLottie's runtime overlay stays position:absolute/inset:0 for its whole life and the host carries data-ditto-lottie with a scoped svg/canvas fit rule, so the runtime animation always fills the pinned per-viewport box instead of escaping to aspect height 259 tests pass (12 new), typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
e33354cfb5
commit
15d1e7a9e3
@@ -517,10 +517,102 @@ async function captureVideoStills(page: import("playwright").Page): Promise<Vide
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-page screenshot with robustness for heavy/animated pages. The default 30s
|
||||
* timeout is exceeded by tall SaaS pages (Playwright also waits for web fonts);
|
||||
* use a long timeout, freeze animations (also improves determinism), and retry.
|
||||
* As a last resort take a viewport-only shot so the file exists (the capture gate
|
||||
* Stage 2 — canvas raster fallback. A visible <canvas> (animated background, chart,
|
||||
* WebGL scene) is a runtime-drawn surface the clone cannot reproduce as DOM, so it
|
||||
* would render as an empty box. Rasterize each meaningful canvas to a PNG still under
|
||||
* a synthetic URL (the normal asset pipeline rewrites it to a local file); the node is
|
||||
* then marked with a `src` attr so generation emits the still as an <img> filling the
|
||||
* canvas's box. `toDataURL` is exact but THROWS for tainted canvases and for WebGL
|
||||
* contexts without preserveDrawingBuffer — those return a shot plan for the node-side
|
||||
* element-screenshot fallback (composited pixels, works regardless). A WebGL canvas may
|
||||
* also toDataURL to a blank image when the drawing buffer was already presented; that
|
||||
* blank is accepted as-is (detecting blankness would be heuristic, not deterministic).
|
||||
* The in-page part is exported for the fixture tests (page.evaluate'd directly).
|
||||
*/
|
||||
export type CanvasStillPlan = { stills: Array<{ url: string; dataUrl: string; sel: string }>; shots: Array<{ url: string; sel: string }> };
|
||||
export function captureCanvasStillsInPage(): CanvasStillPlan {
|
||||
const stills: CanvasStillPlan["stills"] = [];
|
||||
const shots: CanvasStillPlan["shots"] = [];
|
||||
const hash = (s: string): string => { let h = 5381; for (let i = 0; i < s.length; i++) h = ((h << 5) + h + s.charCodeAt(i)) >>> 0; return h.toString(36); };
|
||||
const canvases = Array.from(document.querySelectorAll("canvas"));
|
||||
let i = 0;
|
||||
for (const c of canvases) {
|
||||
const r = c.getBoundingClientRect();
|
||||
if (r.width < 48 || r.height < 48) continue; // not a meaningful painted surface
|
||||
const cs = getComputedStyle(c);
|
||||
if (cs.display === "none" || cs.visibility === "hidden" || parseFloat(cs.opacity || "1") < 0.05) continue;
|
||||
const idx = i++;
|
||||
c.setAttribute("data-clone-canvas", String(idx));
|
||||
const sel = `canvas[data-clone-canvas="${idx}"]`;
|
||||
const url = `https://clone-canvas.local/${idx}-${hash(c.id + "|" + c.width + "|" + c.height + "|" + idx)}.png`;
|
||||
let ok = false;
|
||||
try {
|
||||
const dataUrl = c.toDataURL("image/png"); // throws if tainted / WebGL buffer unreadable
|
||||
if (dataUrl.startsWith("data:image/png")) { stills.push({ url, dataUrl, sel }); ok = true; }
|
||||
} catch { /* fall through to the element-screenshot plan */ }
|
||||
if (!ok) shots.push({ url, sel });
|
||||
}
|
||||
return { stills, shots };
|
||||
}
|
||||
|
||||
async function captureCanvasStills(page: import("playwright").Page): Promise<CanvasStillPlan> {
|
||||
try {
|
||||
return await Promise.race([
|
||||
page.evaluate(captureCanvasStillsInPage),
|
||||
new Promise<CanvasStillPlan>((res) => setTimeout(() => res({ stills: [], shots: [] }), 12_000)),
|
||||
]);
|
||||
} catch {
|
||||
return { stills: [], shots: [] };
|
||||
}
|
||||
}
|
||||
|
||||
// Chromium refuses/truncates screenshots past a texture-size cap; keep clip dimensions
|
||||
// under a conservative bound so captureFullPageViaCDP fails cleanly (→ Playwright fallback)
|
||||
// instead of returning a truncated image on pathologically tall pages.
|
||||
const CDP_MAX_SHOT_DIMENSION = 16_384;
|
||||
|
||||
/**
|
||||
* Full-page screenshot via CDP `Page.captureScreenshot` with `captureBeyondViewport:true`.
|
||||
* Unlike Playwright's `fullPage:true` (which scroll-stitches — scrolling the page to render
|
||||
* each band, FIRING scroll events that drive scroll-linked animations, e.g. IX2 grow-on-scroll
|
||||
* or WAAPI scroll-timelines), CDP renders the whole page in ONE shot WITHOUT scrolling and
|
||||
* never fires scroll events. So the resulting still is the genuine at-rest (unscrolled) page,
|
||||
* matching the DOM snapshot the walk grades. Writes a PNG to `path`. Throws on any failure so
|
||||
* the caller can fall back to the Playwright path.
|
||||
*/
|
||||
export async function captureFullPageViaCDP(page: import("playwright").Page, path: string): Promise<void> {
|
||||
const client = await page.context().newCDPSession(page);
|
||||
try {
|
||||
const metrics = await client.send("Page.getLayoutMetrics") as {
|
||||
cssContentSize?: { width: number; height: number };
|
||||
contentSize?: { width: number; height: number };
|
||||
};
|
||||
const size = metrics.cssContentSize ?? metrics.contentSize;
|
||||
if (!size || !(size.width > 0) || !(size.height > 0)) throw new Error("cdp: empty content size");
|
||||
const width = Math.ceil(size.width);
|
||||
const height = Math.ceil(size.height);
|
||||
if (width > CDP_MAX_SHOT_DIMENSION || height > CDP_MAX_SHOT_DIMENSION) {
|
||||
throw new Error(`cdp: content ${width}x${height} exceeds max shot dimension`);
|
||||
}
|
||||
const shot = await client.send("Page.captureScreenshot", {
|
||||
format: "png",
|
||||
captureBeyondViewport: true,
|
||||
clip: { x: 0, y: 0, width, height, scale: 1 },
|
||||
}) as { data: string };
|
||||
if (!shot?.data) throw new Error("cdp: no screenshot data");
|
||||
writeBytes(path, Buffer.from(shot.data, "base64"));
|
||||
} finally {
|
||||
await client.detach().catch(() => { /* ignore */ });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-page screenshot with robustness for heavy/animated pages. Prefers CDP capture
|
||||
* (captureFullPageViaCDP — no scroll-stitch, so scroll-linked animations aren't scrubbed
|
||||
* and the still is the true at-rest page). On ANY CDP failure, falls back to Playwright's
|
||||
* `fullPage:true` (which scroll-stitches but freezes animations); the default 30s timeout
|
||||
* is exceeded by tall pages (Playwright also waits for web fonts), so use a long timeout and
|
||||
* retry. As a last resort take a viewport-only shot so the file exists (the capture gate
|
||||
* checks presence; a partial image still beats none).
|
||||
*/
|
||||
async function captureScreenshot(
|
||||
@@ -529,6 +621,13 @@ async function captureScreenshot(
|
||||
vw: number,
|
||||
log: (e: Record<string, unknown>) => void,
|
||||
): Promise<void> {
|
||||
try {
|
||||
await captureFullPageViaCDP(page, path);
|
||||
log({ event: "screenshot_cdp", viewport: vw });
|
||||
return;
|
||||
} catch (eCdp) {
|
||||
log({ event: "screenshot_cdp_fallback", viewport: vw, error: String(eCdp).slice(0, 200) });
|
||||
}
|
||||
for (let attempt = 0; attempt < 2; attempt++) {
|
||||
try {
|
||||
await page.screenshot({ path, fullPage: true, timeout: 90_000, animations: "disabled" });
|
||||
@@ -923,6 +1022,47 @@ export async function captureSite(opts: {
|
||||
await page.evaluate(() => window.scrollTo(0, 0));
|
||||
await page.waitForTimeout(80);
|
||||
}
|
||||
|
||||
// Stage 2: canvas raster fallback — same synthetic-URL mechanism as the video
|
||||
// stills above (captureCanvasStills). Bytes ride the normal asset pipeline; the
|
||||
// canvas node gets a `src` attr (whitelisted → survives the IR) that generation
|
||||
// reads to emit the still as an <img> carrying the canvas's cid/box. The attr is
|
||||
// stamped only AFTER bytes are stored, so a failed capture leaves the canvas
|
||||
// rendering as an empty box, same as before.
|
||||
const cplan = await captureCanvasStills(page);
|
||||
const stampCanvasSrc = (sel: string, u: string): Promise<void> =>
|
||||
page.evaluate(({ s, u: uu }) => { document.querySelector(s)?.setAttribute("src", uu); }, { s: sel, u }).catch(() => { /* ignore */ });
|
||||
for (const s of cplan.stills) {
|
||||
const comma = s.dataUrl.indexOf(",");
|
||||
if (comma < 0) continue;
|
||||
try {
|
||||
const buf = Buffer.from(s.dataUrl.slice(comma + 1), "base64");
|
||||
recordAsset(s.url, "image", "image/png", 200, "canvas-still");
|
||||
storeBytes(s.url, "image", buf);
|
||||
await stampCanvasSrc(s.sel, s.url);
|
||||
log({ event: "canvas_still", viewport: vw, sel: s.sel });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
for (const s of cplan.shots) {
|
||||
// Same reveal discipline as the video shots: a hidden canvas (entrance
|
||||
// animation not yet fired) would time out locator.screenshot's visibility wait.
|
||||
try {
|
||||
await page.evaluate(forceRevealForShot, s.sel);
|
||||
const buf = await page.locator(s.sel).first().screenshot({ type: "png", timeout: 5000, animations: "disabled" });
|
||||
recordAsset(s.url, "image", "image/png", 200, "canvas-still");
|
||||
storeBytes(s.url, "image", buf);
|
||||
await stampCanvasSrc(s.sel, s.url);
|
||||
log({ event: "canvas_still", viewport: vw, sel: s.sel });
|
||||
} catch (e) {
|
||||
log({ event: "canvas_still_error", viewport: vw, sel: s.sel, error: String(e).slice(0, 200) });
|
||||
} finally {
|
||||
await page.evaluate(restoreRevealForShot).catch(() => { /* ignore */ });
|
||||
}
|
||||
}
|
||||
if (cplan.shots.length) {
|
||||
await page.evaluate(() => window.scrollTo(0, 0));
|
||||
await page.waitForTimeout(80);
|
||||
}
|
||||
}
|
||||
|
||||
// Stage 4/5: stamp capture-ids before the canonical snapshot so the IR carries
|
||||
|
||||
@@ -391,6 +391,10 @@ export function propsList(node: IRNode, assetMap: Map<string, string>, sourceUrl
|
||||
|
||||
if (isVideo && !videoLocal && !props.some(([k]) => k === "preload")) props.push(["preload", JSON.stringify("none")]);
|
||||
|
||||
// A canvas emitted as its raster still (<img> — see resolveTag) is decorative
|
||||
// painted surface with no captured alt text; emit alt="" so the img is valid.
|
||||
if (node.tag === "canvas" && node.attrs.src && !props.some(([k]) => k === "alt")) props.push(["alt", JSON.stringify("")]);
|
||||
|
||||
if (node.rawHTML && node.tag === "svg") {
|
||||
// Strip the Stage-4 capture-id (`data-cid-cap`) the interaction pass stamps on
|
||||
// elements: it's internal instrumentation, render-inert, and would otherwise
|
||||
@@ -557,6 +561,12 @@ export function resolveTag(node: IRNode, insideInteractive: boolean, insideTable
|
||||
// children inside <iframe> are unrendered fallback content, so emit a <div> container
|
||||
// carrying the iframe's box/styles (CSS is keyed by cid, so the geometry is identical).
|
||||
if (tag === "iframe" && node.children.some((c) => !isTextChild(c))) tag = "div";
|
||||
// A canvas is runtime-drawn surface the clone cannot reproduce; when capture
|
||||
// rasterized it (canvas-still synthetic URL stamped as `src` — see
|
||||
// captureCanvasStillsInPage) emit the still as an <img> filling the canvas's box
|
||||
// (CSS is keyed by cid, so the geometry is identical). A canvas with no still
|
||||
// keeps rendering as an empty <canvas> box, same as before.
|
||||
if (tag === "canvas" && node.attrs.src) tag = "img";
|
||||
if (TABLE_SCOPED.has(tag) && !insideTable) tag = "div"; // orphan table element → neutral box
|
||||
if (violatesContentModel(node, tag)) tag = "div";
|
||||
return tag;
|
||||
|
||||
@@ -32,6 +32,11 @@ button, input, select, textarea, optgroup { font: inherit; color: inherit; backg
|
||||
button { cursor: pointer; }
|
||||
table { border-collapse: separate; border-spacing: 0; }
|
||||
img, picture, video, canvas, svg { display: inline-block; }
|
||||
/* Lottie runtime fit: the player re-mounts its svg/canvas into an absolute overlay that fills
|
||||
the host's captured (per-viewport, definite) box. Force that runtime media to fit the box so
|
||||
an aspect-mismatched viewBox (e.g. a portrait animation in a shorter, letterboxed source box)
|
||||
can't inflate past the pinned height. Scoped to the runtime-marked host only. */
|
||||
[data-ditto-lottie] > div > svg, [data-ditto-lottie] > div > canvas { width: 100%; height: 100%; display: block; }
|
||||
h1, h2, h3, h4, h5, h6, p, figure, blockquote, dl, dd { margin: 0; }
|
||||
/* Neutralize UA text defaults to inherit so the generator's default-skipping is
|
||||
correct: a property equal to its inherited value is simply not emitted. */
|
||||
|
||||
@@ -138,12 +138,19 @@ export default function DittoLottie({ spec }: { spec: LottieSpec }) {
|
||||
// Mount the live animation into an OVERLAY child, keeping the captured placeholder
|
||||
// frame in the DOM until lottie signals a successful load — a failed load (bad JSON,
|
||||
// network) then leaves the placeholder intact instead of erasing the container to blank.
|
||||
// The overlay stays position:absolute;inset:0 for its WHOLE life (not just pre-swap):
|
||||
// the host box carries the captured per-viewport height, and only an absolutely-filled
|
||||
// overlay inherits that definite box. A static-flow overlay has indefinite height, so
|
||||
// the player's svg (style height:100%) collapses to auto and inflates to the source
|
||||
// aspect (a portrait viewBox then oversizes far past the captured, letterboxed box).
|
||||
const mount = document.createElement("div");
|
||||
mount.style.position = "absolute";
|
||||
mount.style.inset = "0";
|
||||
mount.style.opacity = "0";
|
||||
const cs = getComputedStyle(el);
|
||||
if (cs.position === "static") el.style.position = "relative";
|
||||
// Mark the host so emitted CSS can force the runtime svg/canvas to fit the pinned box.
|
||||
el.setAttribute("data-ditto-lottie", "");
|
||||
el.appendChild(mount);
|
||||
const anim = lottie.loadAnimation({
|
||||
container: mount,
|
||||
@@ -171,8 +178,9 @@ export default function DittoLottie({ spec }: { spec: LottieSpec }) {
|
||||
const rendered = mount.querySelector("svg, canvas");
|
||||
if (rendered) rendered.setAttribute("data-cid", placeholderCid);
|
||||
}
|
||||
mount.style.position = "";
|
||||
mount.style.inset = "";
|
||||
// Do NOT clear position/inset: the overlay must keep filling the host's pinned box
|
||||
// (see above). Stripping them reverts it to static flow → indefinite height → the
|
||||
// player's height:100% svg inflates to its aspect and escapes the captured bbox.
|
||||
};
|
||||
// DOMLoaded fires only after the JSON parsed and the first frame rendered — the
|
||||
// right moment to reveal the live render and drop the placeholder. data_failed
|
||||
|
||||
@@ -5,6 +5,7 @@ import { join, extname, normalize } from "node:path";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { chromium } from "playwright";
|
||||
import { collectPage, type PageSnapshot } from "../capture/walker.js";
|
||||
import { captureFullPageViaCDP } from "../capture/capture.js";
|
||||
import { ensureDir, writeJSONCompact } from "../util/fsx.js";
|
||||
|
||||
const ESBUILD_SHIM =
|
||||
@@ -255,8 +256,17 @@ export async function renderApp(opts: {
|
||||
});
|
||||
const snapshot = await page.evaluate(collectPage);
|
||||
writeJSONCompact(join(opts.renderedDir, "dom", `dom-${vw}.json`), snapshot);
|
||||
// Screenshot via CDP (captureBeyondViewport, no scroll-stitch) to match the SOURCE
|
||||
// channel exactly — both sides must measure the same at-rest state. Fall back to
|
||||
// Playwright's fullPage stitch if CDP fails. The clone is static so scroll-stitch
|
||||
// matters less here, but symmetric capture is the requirement.
|
||||
try {
|
||||
await page.screenshot({ path: join(opts.renderedDir, "screenshots", `${vw}.png`), fullPage: true, timeout: 90_000, animations: "disabled" });
|
||||
const shotPath = join(opts.renderedDir, "screenshots", `${vw}.png`);
|
||||
try {
|
||||
await captureFullPageViaCDP(page, shotPath);
|
||||
} catch {
|
||||
await page.screenshot({ path: shotPath, fullPage: true, timeout: 90_000, animations: "disabled" });
|
||||
}
|
||||
} catch { /* ignore */ }
|
||||
return { viewport: vw, snapshot, runtimeErrors, failedResources: [...failedResources], httpStatus };
|
||||
} finally {
|
||||
|
||||
Reference in New Issue
Block a user