feat(review): add versioned JSON comparison reports

This commit is contained in:
Jan Petykiewicz 2026-11-15 10:38:48 -08:00
commit 248f6bcff7

185
docs/diff-reports.md Normal file
View file

@ -0,0 +1,185 @@
# Machine-readable diff reports
```sh
klamath diff before.gds after.gds --format json > report.json
klamath diff before.gds after.gds --format json --hierarchy graph
klamath diff before.gds after.gds --format json --top CHIP --layer 1/0
```
The command writes one UTF-8 JSON document followed by a newline to stdout.
Comparison diagnostics belong to that document. Output write failures are
reported on stderr and return 2; interrupted output is not a valid report.
JSON output currently belongs to `diff`; `dump`, `hierarchy`, and the Git
external driver retain their existing text interfaces. No external process is
used to produce a report.
The [version 1 JSON Schema](diff-report-v1.schema.json) defines the structure.
[Golden reports](../tests/golden/report-geometry.json) illustrate complete
evidence; other `report-*.json` goldens cover equality, metadata, incomplete
comparison, and failure. Their producer version is replaced with `test` so
package version changes do not alter the fixtures.
## Versions and determinism
`schema_version` describes the report structure. `comparison_profile` names the
equality rules, currently `klamath-object-v2`. `producer.version` identifies the
implementation. These versions are independent. Consumers interpreting equality
must understand the comparison profile as well as the schema.
Compatible additions may introduce optional fields. Consumers should ignore
unfamiliar additional fields and handle unfamiliar diagnostic codes and matching
rules without treating them as equality. Removed fields or changed meanings
require a new schema version. The schema intentionally permits additional fields.
For the same inputs, paths, options, and producer version, serialization is
deterministic. Cells, layer pairs, fields, properties, and graph edges are sorted.
Duplicate objects and properties remain separate occurrences. Reports contain
no generation timestamp or required content fingerprint.
## Result and exit status
| `status` | `result` | Exit code | Interpretation |
| --- | --- | --- | --- |
| `complete` | `equal` | 0 | Equal under the recorded profile and scope. |
| `complete` | `different` | 1 | Differences found. |
| `incomplete` | `different` | 2 | Definite differences found, with comparison limitations. |
| `incomplete` | `unknown` | 2 | Equality cannot be established. |
| `failed` | `unknown` | 2 | No comparison result is available. |
Unsupported records, future metadata versions, unresolved references, and
hierarchy cycles make the comparison incomplete. Malformed input, read errors,
invalid scopes, and invalid command arguments produce failed reports. CI must
check completeness as well as differences. An exit code of 1 is a normal
comparison outcome, including when stdout is redirected to a report file.
Library and cell timestamps are excluded from the comparison profile. A valid,
complete timestamp-only comparison returns 0. Raw record dumps remain available
for examining those timestamps.
`summary` counts the entire scoped comparison, before display selection. Its
`counts` is `exact` for a complete comparison and `observed` for an incomplete
one. Failed reports have a null summary and empty change arrays. Replacement
suggestions and affected ancestors never add to change counts.
## Scope, selection, and numbers
`scope` records the actual `top`, `cell`, and numeric `layer` selectors. Null
means unrestricted. The complete file is parsed before applying scope, so an
invalid record outside the selected subtree still fails the command.
`report_selection` records display-only `cell`, `layer`, and `max_objects`.
JSON has no default object limit. Explicit `--max-changes N` limits the total
number of emitted object occurrences across both sides. Removals precede
additions within each cell. `filtered_cells`, `filtered_objects`,
`limited_objects`, and `emitted_objects` explain evidence selection. `truncated`
is true if filtering or a limit omits any changed cell or object. Library and
layer-name changes, diagnostics, summaries, and the optional graph are retained.
A replacement is emitted only when both linked objects are included.
All aggregate counts, limits, and byte offsets are unsigned decimal **strings**.
This avoids loss of precision in JSON consumers using binary64 numbers.
Coordinates, numeric tags, layer/type pairs, widths, and array dimensions are
bounded integers represented as JSON numbers. Null means absent or unavailable;
zero and empty arrays do not imply absence.
`inputs.old` and `inputs.new` contain input paths, presence, and UNITS fields.
Presence is distinct from an empty library. The Rust API also supports absent
Git revisions using `Document::empty_like`; no units are invented for absent
inputs. Failed reports may have null presence and units. Argument failures can
lack input/selection context because the command could not be parsed.
Paths are UTF-8 when representable. Otherwise `path` is null and
`path_encoded_hex` preserves Rust's platform-native encoded path bytes, prefixed
with `0x`. This is platform-dependent evidence rather than a portable path name.
## Change evidence
`library_changes` records changed header fields as typed before/after arrays.
Library presence changes use the special field `presence`, a null tag, and
boolean before/after values. Units are not rescaled to a common grid.
`layer_name_changes` records numeric pairs with before/after names. A layer
pair is `{ "number": 1, "purpose": 0 }`, where purpose is DATATYPE, TEXTTYPE,
BOXTYPE, or NODETYPE according to the object kind. `layer_legend` contains
unchanged names available in the scope. Embedded annotation labels are excluded
from ordinary object changes. A metadata rename appears once.
Each `cell_changes` entry has its name, nullable before/after cell records,
attribute evidence, removed and added objects, and replacement suggestions.
A null cell side means the cell is absent in that scope. An existing empty
cell has a byte offset and an empty attribute array.
Each object retains:
- Its kind and numeric tag, numeric layer pair, target cell, and coordinate envelope.
- Its original source side and byte range `[byte_offset, end_offset)`.
- Typed field records and the sorted property multiset, including duplicates.
- Convenience fields for text/reference origin, path/text width, and reference
array dimensions. SREF dimensions are `[1, 1]`; the kind and retained fields
distinguish an SREF from a one-element AREF and explicit fields from omission.
Coordinates are cell-local integer database units. `bounds` contains
`[xmin, ymin, xmax, ymax]` of stored coordinates. It excludes path stroke width,
rendered text dimensions, and recursively transformed target geometry.
Equal envelopes do not establish geometric equality. Unit changes must be
considered before deriving physical movements or dimensions.
A retained field has a numeric `tag`, convenience `name`, `encoding`, decoded
`value`, and `payload_hex`. The numeric tag is authoritative; names are display
aids. XY values are arrays of `[x, y]` pairs. Other numeric fields are arrays
of their declared type. Unsupported encodings retain bytes with a null value.
String and property values are null if their bytes are not valid UTF-8;
`payload_hex` always preserves them without replacement characters.
Payloads follow the comparison profile: boundary/box rings use canonical start
and winding, string padding is removed, and fields/properties are sorted. They
are comparison evidence, not a verbatim serialization of the source records.
REAL8 payload encodings are retained exactly alongside their decoded values.
Explicit defaults and omitted fields remain distinct. Source ranges allow raw
record inspection when original encoding or order is needed.
Replacement suggestions link report-local removed/added IDs, with a `match_rule`,
typed `field_changes`, and `properties_changed`. Their explanation is optional
for consumers to display and must not be parsed. Current matching rules are
`unique_kind_layer_bounds`, `unique_kind_layer`, and `unique_kind_target`.
These are explanatory matches, not persistent identity or shape equivalence.
Ambiguous candidates stay independent removals and additions. IDs are unique
within a report and have no meaning across reports.
## Diagnostics and hierarchy
Diagnostics have `code`, `severity`, `message`, input `side`, original byte
offset, cell, layer, and target context. Unknown context is null. Messages are
for display; codes and context support automation. Current limitation codes are
`unsupported_record`, `unsupported_metadata_version`, `unresolved_reference`,
`hierarchy_cycle`, and `scope_mismatch`. Failure codes are `invalid_input`,
`io_error`, `invalid_scope`, and `command_error`. Fatal parser errors currently
retain their location in the human message; their structured byte offset is
null. Consumers must not depend on message wording.
`--hierarchy graph` includes the full scoped union graph of both revisions.
Otherwise `hierarchy` is null. Nodes are keyed by cell name, with per-side
states `included`, `outside_scope`, `unresolved`, or `absent`. Their `change`
is `direct`, `affected` through a descendant, or `unchanged`.
Edges record parent, target, before/after direct placement counts, and whether
the underlying reference records changed. AREF columns times rows contribute
symbolically; arrays are never expanded. Counts are direct edge multiplicities,
not recursively accumulated placements. Shared cells occur once. Removed edges
remain visible, and ancestors are computed over the union of both revisions.
Graph content is independent of display filters and object limits. Text depth
limits and ASCII/Mermaid choices do not apply to JSON graphs.
## Development
`cargo test --test review` checks report semantics, CLI behavior, and versioned
goldens. To intentionally regenerate report goldens after reviewing a contract
change, run:
```sh
KLAMATH_UPDATE_REPORT_GOLDENS=1 cargo test --test review machine_reports_match_versioned_goldens
```
Schema validators can validate the goldens or actual CLI output against
`diff-report-v1.schema.json`. Schema validation does not replace semantic checks
such as replacement links, summary totals, or status/exit-code agreement.