Color mode

Full SSG Determinism and Performance Audit

  • Date: 2026-11-07 (some measurements continued into early 2026-11-08)
  • Repository: Rerurate514/riebeckite
  • Branch: perf/ssg-determinism-and-optimization
  • Base: e88d403fd8796d39504257ba05c77c250486c181 (main at the time)
  • HEAD after the determinism fix: 6718e9877ea9fcfa30e62b075eff109396f1cbcd
  • Environment: Windows / Node.js v24.22.1 / pnpm v12.15.0 / Vite 8.3.5 (Rolldown 1.2.14)

Verdict

Area Status Summary
Phase 1 determinism READY Root cause identified and removed with a minimal ordering fix. Incremental and clean outputs matched byte-for-byte in all 20 scenarios for Starter/Showcase at 100 notes
Phase 2 measurement PARTIAL The 100-note, five-run batch completed. The 1,000 and 5,000 note runs did not. Session restarts and concurrent processes affected part of the measurements
Phase 3 optimization PARTIAL The strongest measured candidate (standalone SSR bundle reuse) was identified, but a new derived cache could not be validated safely enough in the available time, so no optimization was adopted
Overall PARTIAL The determinism fix is complete. Performance work stopped at candidate evaluation

Scope and Method

Benchmark

The official benchmarks/preset-ssg preset runner was used. It scaffolds real Starter and Showcase sites with scaffoldRiebeckiteSite and runs the HonoX/Vite SSG.

  • Each sample creates a fresh site and, when required, builds a baseline
  • It applies the scenario and runs an incremental build
  • It snapshots the incremental dist
  • It removes .riebeckite and dist, then runs a clean build
  • It snapshots the clean dist and compares paths, sizes, and SHA-256
  • --runs 5 repeats each scenario five times and records median, p95, min, max, and stdev
  • --retainOutputs true keeps the baseline, incremental, and clean output trees

Snapshots sort readdir by name and record each file as { bytes, sha256 }. Comparison checks both SHA-256 and size.

Measurement caveats

  • timingsMs.outputWrite is an additive sum of concurrent writes, not process wall-clock. It can exceed wall-clock and is not treated as a critical path.
  • spansMs also accumulates concurrent work. It is used as a stage breakdown, not as a wall-clock total.
  • The benchmark buildOnce runs only the server build and does not run the mode: "client" client build. It therefore contains fewer client assets than a real CLI build.
  • During the five-run 100-note batch, the environment restarted and other processes ran (benchmarks/chaos.mjs --size=1000 and test suites). The later wall-clock and p95 values include contention noise.

Phase 1: Determinism

Reproduction

The pre-fix code (equivalent to e88d403f) was run for Starter/100. Cold matched, but no-change, body edit, link edit, and publication change differed only in dist/index.js.

Scenario Incremental Clean Difference
cold 24,562 ms 23,956 ms none
no-change 12,732 ms 21,823 ms index.js only
single-body-edit 19,537 ms 25,629 ms index.js only
link-edit 19,983 ms 24,166 ms index.js only
publication-change 19,771 ms 22,117 ms index.js only

The pre-fix index.js values were:

  • Starter no-change: incremental 860,787 bytes / 54aa6017c36892fc257a44e45d3e5868417f948c548ab9309f099e52b22e62c1, clean 856,437 bytes / 42035779406ae441eaf905d4a8b7bf0f31ec6660ce25a5cc72ef3ef02f21f32d
  • Showcase publication-change: incremental 867,013 bytes / ad381a4c923a1744e678a60e3fcd4d24b40d77c00669dbc9651d44fd953624e8, clean 861,937 bytes / 4482d9466fcab2337692132758fe6c01c2457b86a95e92dd23672d8f3e5d2313

For Starter no-change, the baseline, incremental, and clean index.js files were retained and inspected. Baseline and clean were byte-identical; only the incremental file was 4,350 bytes larger. The first divergence is therefore the incremental Vite server bundle, before output-cache reuse.

Root cause

dist/index.js is the server entry generated by the @hono/vite-build Node adapter. Its virtual entry load() calls readdirSync on config.publicDir and config.build.outDir, then embeds each top-level entry as mainApp.use('/<name>', serveStatic(...)) in the server entry.

The riebeckite SSG plugin uses .riebeckite/build-output as its staging directory for the server build. Before the fix, buildStart copied the previous published dist into that staging directory when no client build artifacts existed, then returned. Because buildStart runs before the adapter's virtual entry load(), previous HTML, folders/*, record-*.html, and similar files became static paths in the server entry.

That changed Rolldown's module resolution and minified symbol ordering, which changed the size and hash of index.js. Minification was not the cause: the mismatch reproduced with minify: false, so that experiment was discarded.

Fix

Only packages/integrations/honox/src/ssg_plugin.ts was changed.

  1. configResolved empties the server staging directory (.riebeckite/build-output) before the adapter's load() runs.
  2. buildStart no longer copies the previous dist into staging.
  3. generateBundle restores client artifacts, runs collectBuildOutputPaths, loads the SSR module, and resolves the output change set before copying the previous dist into staging. The adapter therefore sees an empty staging directory, while the later SSG output pipeline can still reuse previous outputs.

Moving the copy after collectBuildOutputPaths and the SSR module load also prevents previous outputs from being classified as client assets in the output plan, and keeps staging clean when the SSR load fails (ownerConflictCount became 0).

An earlier attempt that used enforce: "pre" and rewrote config.build.outDir to staging in configResolved was rejected: it made three Vite server artifacts disappear from dist (missingInDistCount: 3).

Result after the fix

The full Phase 1 matrix (Starter/Showcase, 100 notes, 10 scenarios, one run each) is stored at benchmarks/results/ssg-phase1-final/2026-11-07T08-11-06-506Z-preset-ssg.json. All 20 rows reported equivalent: true and missingInDistCount: 0.

Preset Scenario Incremental (ms) Clean (ms) Planned / actual
Starter cold 19,544 19,479 263 / 263
Starter no-change 11,033 20,078 263 / 263
Starter full 16,484 19,113 263 / 263
Starter single-body-edit 16,729 19,313 263 / 263
Starter link-edit 16,586 18,933 263 / 263
Starter add 16,798 22,700 270 / 270
Starter delete 19,025 22,332 263 / 261
Starter rename 16,186 19,953 263 / 263
Starter publication-change 17,448 19,798 263 / 261
Starter draft-to-public 17,807 19,789 265 / 265
Showcase cold 41,963 40,821 415 / 415
Showcase no-change 34,950 41,100 415 / 415
Showcase full 40,121 41,516 415 / 415
Showcase single-body-edit 39,294 40,845 415 / 415
Showcase link-edit 38,909 41,099 415 / 415
Showcase add 38,800 40,630 422 / 422
Showcase delete 39,524 40,475 415 / 413
Showcase rename 40,225 39,936 415 / 415
Showcase publication-change 41,345 38,802 415 / 413
Showcase draft-to-public 38,776 39,847 417 / 417

Rows where actual < planned correspond to removed outputs; missingInDistCount is 0. No row had any output difference.

Phase 2: Performance

100 notes, five runs

Command (with RIEBECKITE_BENCHMARK_SKIP_BUILD=1 and a prior pnpm build:packages):

text
pnpm benchmark:presets -- --presets starter,showcase --sizes 100 \
  --scenarios cold,no-change,full,single-body-edit,link-edit,publication-change \
  --runs 5 --out benchmarks/results/ssg-phase2-final-100

Result: benchmarks/results/ssg-phase2-final-100/2026-11-07T14-18-15-985Z-preset-ssg.json

Latency and memory

Preset Scenario n Wall median (ms) p95 (ms) σ (ms) Clean median (ms) CPU median (ms) Peak RSS (MB)
Starter cold 5 25,233 34,664 4,019 25,786 44,563 1,107
Starter no-change 5 12,914 14,085 886 24,497 28,719 961
Starter full 5 21,310 42,574 9,132 25,630 38,579 1,006
Starter single-body-edit 5 21,434 23,222 983 24,004 39,438 1,015
Starter link-edit 5 20,461 22,191 1,521 22,708 37,656 1,025
Starter publication-change 5 18,162 18,806 559 21,483 34,515 995
Showcase cold 5 44,116 56,448 5,594 45,242 101,968 1,716
Showcase no-change 5 41,047 45,488 2,160 49,382 97,328 1,727
Showcase full 5 48,955 53,085 3,922 43,916 107,718 1,725
Showcase single-body-edit 5 40,612 45,073 2,363 42,164 93,781 1,746
Showcase link-edit 5 45,787 49,726 2,311 46,263 100,797 1,718
Showcase publication-change 5 44,137 56,780 5,907 50,150 100,156 1,708

Cache and stage breakdown

Values are medians of five runs. outputWrite is a concurrent sum and must not be compared with wall-clock.

Preset Scenario Cache hit / miss Content processed / reused Output rendered / reused SSR module load (ms) Output dependency resolution (ms) toSSG (ms)
Starter cold 0 / 256 105 / 0 212 / 0 3,453 6,488 6,982
Starter no-change 260 / 0 0 / 105 0 / 256 3,287 290 503
Starter full 0 / 256 0 / 105 212 / 0 3,510 447 6,541
Starter single-body-edit 260 / 0 1 / 104 212 / 10 3,533 1,055 6,370
Starter link-edit 260 / 0 1 / 104 212 / 10 3,357 940 5,705
Starter publication-change 260 / 0 99 / 6 210 / 10 2,947 1,398 5,104
Showcase cold 0 / 404 137 / 0 299 / 0 5,501 25,352 3,932
Showcase no-change 412 / 0 8 / 129 0 / 404 5,770 581 23,678
Showcase full 0 / 404 8 / 129 299 / 0 5,866 629 27,340
Showcase single-body-edit 412 / 0 9 / 128 205 / 182 5,180 21,319 2,809
Showcase link-edit 412 / 0 9 / 128 206 / 172 5,776 23,604 3,505
Showcase publication-change 412 / 0 108 / 29 296 / 62 5,509 22,636 3,677

Bottlenecks

  • Starter/100 no-change: of the 12.9 s wall median, SSR module load is about 3.3 s; the rest is output-cache validation and reuse. The standalone SSR bundle is rebuilt even when nothing changed.
  • Showcase/100: wall medians are 41–49 s. SSR module load is about 5–6 s and toSSG is about 24–28 s for no-change/full. With 404 planned outputs, route enumeration and HTML generation dominate.
  • cold / full: outputDependencyResolution costs about 6.5 s for Starter and 25 s for Showcase. This is a large fixed cost during full regeneration.
  • Memory: Starter/100 peaks around 1.0 GB and Showcase/100 around 1.7 GB. Linear extrapolation to 5,000 notes is unsafe, and Showcase is especially memory-constrained.
  • outputWrite: the reported value can exceed wall-clock (for example 547,736 ms). It is a concurrent sum; it is not yet valid evidence of a wall-clock bottleneck.

1,000 and 5,000 notes

The 1,000 and 5,000 note runs did not complete in this session. The five-run 100-note batch alone took about an hour, and the environment restarted during measurement with other processes running. Given Showcase/100 already peaks at about 1.7 GB, a five-run 5,000-note matrix is not realistic on this machine. Missing measurements are not treated as evidence that no problem exists.

Phase 3: Optimization Evaluation

Candidate: reuse the standalone SSR bundle

Measurements show ssrModuleLoad at about 3.3 s for Starter/100 and 5–6 s for Showcase/100. ssg/server/standalone.ts runs a nested Vite build to produce ssg-server.mjs and then imports it on every build. If the app's bundleFingerprint has not changed, the bundle could be reused.

The expected gain is roughly 10–25% of no-change wall-clock. It was not implemented because:

  • bundleFingerprint covers the app, config files, local workspace packages, and NODE_ENV / VITE_*, but not every programmatic Vite define, oxc, or plugin option.
  • A new derived cache would need additional validation for RIEBECKITE_SSG_FULL_REGENERATION, the client manifest, publication boundaries, and failure recovery.
  • Importing a stale bundle after a failed build could publish incorrect HTML. A safe design needs an atomic marker and rebuild-on-corruption behavior, which could not be validated sufficiently in the remaining time.

Rejected approaches

  • Disabling minification: minify: false still produced a mismatch. It is not the cause.
  • Rewriting build.outDir to staging in configResolved: three server artifacts disappeared from dist.
  • Limiting output-write concurrency: outputWriteTimeMs is an additive concurrent sum and exceeds wall-clock, so there is no wall-clock evidence for a gain.

Verification

Executed and PASS:

  • pnpm typecheck (3 projects)
  • pnpm build:packages (116 packages)
  • pnpm exec biome check on the changed TypeScript and benchmark files
  • node --import tsx --test --test-name-pattern "snapshot comparison" benchmarks/preset-ssg/runner.test.mjs (1 pass)
  • node --import tsx --test --test-reporter=spec --test-name-pattern "runner executes the clean output comparison" benchmarks/preset-ssg/runner.test.mjs (1 pass, 158.8 s)
  • All 20 Phase 1 scenarios (Starter/Showcase, 100 notes) with equivalent: true and missingInDistCount: 0
  • incremental_ssg.test.ts 8 pass; ssg_plugin.test.ts and standalone.test.ts 26 pass
  • Post-merge Starter/100 no-change, one run: equivalent: true, missingInDistCount: 0

Not executed or incomplete:

  • The full repository pnpm test (too long for this session)
  • 1,000 / 5,000 note measurements
  • Implementation and A/B measurement of an additional optimization

Risks

  • The fix depends on @hono/vite-build scanning config.build.outDir in its virtual entry. A change in that dependency could reintroduce the bug.
  • The copy in generateBundle must stay after client artifact restoration, collectBuildOutputPaths, the SSR module load, and output change-set resolution. Reordering it reintroduces output-plan contamination and staging residue on failure.
  • The benchmark omits the client build, so real-site wall-clock may be higher than reported.
  • The 100-note measurements were affected by concurrent processes. Use them for relative comparison in this environment, not as universal absolute values.
  • Showcase memory usage makes a five-run 5,000-note measurement impractical.

Next Steps

  1. Measure 1,000 notes at least once and compare scaling with 100 notes.
  2. Evaluate 5,000-note memory limits first; design streaming or partitioned generation if needed.
  3. Implement standalone SSR bundle reuse with a marker, rebuild-on-corruption, and full-regeneration semantics, then verify wall-clock with A/B runs.
  4. Add a benchmark that includes the client build to match real CLI behavior.
  5. Split outputWrite into concurrency and critical-path measurements.

Artifacts

  • Determinism fix: 567a8983; follow-up fix: 6718e987
  • Benchmark improvements: a83a296e
  • Phase 1 final: benchmarks/results/ssg-phase1-final/2026-11-07T08-11-06-506Z-preset-ssg.json
  • Phase 2 100 notes: benchmarks/results/ssg-phase2-final-100/2026-11-07T14-18-15-985Z-preset-ssg.json
  • Pre-fix reproduction: benchmarks/results/ssg-phase1/
  • Determinism regression test: benchmarks/preset-ssg/runner.test.mjs

Final Verdict

PARTIAL

The Phase 1 determinism fix is complete and byte-for-byte equality was verified for all 100-note scenarios. Phase 2 completed the five-run 100-note measurements, but the 1,000 / 5,000 note runs are missing. Phase 3 stopped at candidate evaluation; no optimization was adopted. Determinism is READY; performance improvement is PARTIAL.

Powered by Riebeckite

Local Graph

Nearby Notes