Implement native GDSII reading and writing with Arrow interchange, hierarchy scans and selective cell loading. Validate numeric packing and array pitches, and preserve normalized REAL8 encodings exactly within the supported range. Add standalone dump, structural diff, hierarchy, layer-name annotation and Git diff commands. Provide scoped text and JSON reports, deterministic comparison, and record-preserving metadata edits. Package the shared library and optional Python Arrow readers as pheon using CFFI ABI mode and Python-independent platform wheels. Include locked source builds, native API checks, cross-platform artifact workflows, documentation and regression coverage. Require Rust 1.88 or newer and Python 3.11 or newer. Validation: 87 native tests, 14 binding tests on Python 3.11/3.13/3.14, and 71 installed companion integration tests pass. Full, scanned and selected Arrow layouts match the previous schemas.
189 lines
10 KiB
Markdown
189 lines
10 KiB
Markdown
# Machine-readable diff reports
|
|
|
|
```sh
|
|
pheon diff before.gds after.gds --format json > report.json
|
|
pheon diff before.gds after.gds --format json --hierarchy graph
|
|
pheon diff before.gds after.gds --format json --top CHIP --layer 1/0
|
|
```
|
|
|
|
The command writes one UTF-8 JSON document followed by a newline to stdout.
|
|
Comparison diagnostics belong to that document. Output write failures are
|
|
reported on stderr and return 2; interrupted output is not a valid report.
|
|
JSON output currently belongs to `diff`; `dump`, `hierarchy`, and the Git
|
|
external driver retain their existing text interfaces. No external process is
|
|
used to produce a report.
|
|
|
|
The [version 1 JSON Schema](diff-report-v1.schema.json) defines the structure.
|
|
[Golden reports](../tests/golden/report-geometry.json) illustrate complete
|
|
evidence; other `report-*.json` goldens cover equality, metadata, incomplete
|
|
comparison, and failure. Their producer version is replaced with `test` so
|
|
package version changes do not alter the fixtures.
|
|
|
|
## Versions and determinism
|
|
|
|
Pheon emits schema version 1 with producer name `pheon`. This schema has not
|
|
had a public release. The comparison profile remains `klamath-object-v2`, and
|
|
the `KLAMATH_LAYER_NAMES_V1` GDS marker retains its existing meaning.
|
|
|
|
`schema_version` describes the report structure. `comparison_profile` names the
|
|
equality rules, currently `klamath-object-v2`. `producer.version` identifies the
|
|
implementation. These versions are independent. Consumers interpreting equality
|
|
must understand the comparison profile as well as the schema.
|
|
|
|
Compatible additions may introduce optional fields. Consumers should ignore
|
|
unfamiliar additional fields and handle unfamiliar diagnostic codes and matching
|
|
rules without treating them as equality. Removed fields or changed meanings
|
|
require a new schema version. The schema intentionally permits additional fields.
|
|
|
|
For the same inputs, paths, options, and producer version, serialization is
|
|
deterministic. Cells, layer pairs, fields, properties, and graph edges are sorted.
|
|
Duplicate objects and properties remain separate occurrences. Reports contain
|
|
no generation timestamp or required content fingerprint.
|
|
|
|
## Result and exit status
|
|
|
|
| `status` | `result` | Exit code | Interpretation |
|
|
| --- | --- | --- | --- |
|
|
| `complete` | `equal` | 0 | Equal under the recorded profile and scope. |
|
|
| `complete` | `different` | 1 | Differences found. |
|
|
| `incomplete` | `different` | 2 | Definite differences found, with comparison limitations. |
|
|
| `incomplete` | `unknown` | 2 | Equality cannot be established. |
|
|
| `failed` | `unknown` | 2 | No comparison result is available. |
|
|
|
|
Unsupported records, future metadata versions, unresolved references, and
|
|
hierarchy cycles make the comparison incomplete. Malformed input, read errors,
|
|
invalid scopes, and invalid command arguments produce failed reports. CI must
|
|
check completeness as well as differences. An exit code of 1 is a normal
|
|
comparison outcome, including when stdout is redirected to a report file.
|
|
|
|
Library and cell timestamps are excluded from the comparison profile. A valid,
|
|
complete timestamp-only comparison returns 0. Raw record dumps remain available
|
|
for examining those timestamps.
|
|
|
|
`summary` counts the entire scoped comparison, before display selection. Its
|
|
`counts` is `exact` for a complete comparison and `observed` for an incomplete
|
|
one. Failed reports have a null summary and empty change arrays. Replacement
|
|
suggestions and affected ancestors never add to change counts.
|
|
|
|
## Scope, selection, and numbers
|
|
|
|
`scope` records the actual `top`, `cell`, and numeric `layer` selectors. Null
|
|
means unrestricted. The complete file is parsed before applying scope, so an
|
|
invalid record outside the selected subtree still fails the command.
|
|
|
|
`report_selection` records display-only `cell`, `layer`, and `max_objects`.
|
|
JSON has no default object limit. Explicit `--max-changes N` limits the total
|
|
number of emitted object occurrences across both sides. Removals precede
|
|
additions within each cell. `filtered_cells`, `filtered_objects`,
|
|
`limited_objects`, and `emitted_objects` explain evidence selection. `truncated`
|
|
is true if filtering or a limit omits any changed cell or object. Library and
|
|
layer-name changes, diagnostics, summaries, and the optional graph are retained.
|
|
A replacement is emitted only when both linked objects are included.
|
|
|
|
All aggregate counts, limits, and byte offsets are unsigned decimal **strings**.
|
|
This avoids loss of precision in JSON consumers using binary64 numbers.
|
|
Coordinates, numeric tags, layer/type pairs, widths, and array dimensions are
|
|
bounded integers represented as JSON numbers. Null means absent or unavailable;
|
|
zero and empty arrays do not imply absence.
|
|
|
|
`inputs.old` and `inputs.new` contain input paths, presence, and UNITS fields.
|
|
Presence is distinct from an empty library. The Rust API also supports absent
|
|
Git revisions using `Document::empty_like`; no units are invented for absent
|
|
inputs. Failed reports may have null presence and units. Argument failures can
|
|
lack input/selection context because the command could not be parsed.
|
|
|
|
Paths are UTF-8 when representable. Otherwise `path` is null and
|
|
`path_encoded_hex` preserves Rust's platform-native encoded path bytes, prefixed
|
|
with `0x`. This is platform-dependent evidence rather than a portable path name.
|
|
|
|
## Change evidence
|
|
|
|
`library_changes` records changed header fields as typed before/after arrays.
|
|
Library presence changes use the special field `presence`, a null tag, and
|
|
boolean before/after values. Units are not rescaled to a common grid.
|
|
|
|
`layer_name_changes` records numeric pairs with before/after names. A layer
|
|
pair is `{ "number": 1, "purpose": 0 }`, where purpose is DATATYPE, TEXTTYPE,
|
|
BOXTYPE, or NODETYPE according to the object kind. `layer_legend` contains
|
|
unchanged names available in the scope. Embedded annotation labels are excluded
|
|
from ordinary object changes. A metadata rename appears once.
|
|
|
|
Each `cell_changes` entry has its name, nullable before/after cell records,
|
|
attribute evidence, removed and added objects, and replacement suggestions.
|
|
A null cell side means the cell is absent in that scope. An existing empty
|
|
cell has a byte offset and an empty attribute array.
|
|
|
|
Each object retains:
|
|
|
|
- Its kind and numeric tag, numeric layer pair, target cell, and coordinate envelope.
|
|
- Its original source side and byte range `[byte_offset, end_offset)`.
|
|
- Typed field records and the sorted property multiset, including duplicates.
|
|
- Convenience fields for text/reference origin, path/text width, and reference
|
|
array dimensions. SREF dimensions are `[1, 1]`; the kind and retained fields
|
|
distinguish an SREF from a one-element AREF and explicit fields from omission.
|
|
|
|
Coordinates are cell-local integer database units. `bounds` contains
|
|
`[xmin, ymin, xmax, ymax]` of stored coordinates. It excludes path stroke width,
|
|
rendered text dimensions, and recursively transformed target geometry.
|
|
Equal envelopes do not establish geometric equality. Unit changes must be
|
|
considered before deriving physical movements or dimensions.
|
|
|
|
A retained field has a numeric `tag`, convenience `name`, `encoding`, decoded
|
|
`value`, and `payload_hex`. The numeric tag is authoritative; names are display
|
|
aids. XY values are arrays of `[x, y]` pairs. Other numeric fields are arrays
|
|
of their declared type. Unsupported encodings retain bytes with a null value.
|
|
String and property values are null if their bytes are not valid UTF-8;
|
|
`payload_hex` always preserves them without replacement characters.
|
|
|
|
Payloads follow the comparison profile: boundary/box rings use canonical start
|
|
and winding, string padding is removed, and fields/properties are sorted. They
|
|
are comparison evidence, not a verbatim serialization of the source records.
|
|
REAL8 payload encodings are retained exactly alongside their decoded values.
|
|
Explicit defaults and omitted fields remain distinct. Source ranges allow raw
|
|
record inspection when original encoding or order is needed.
|
|
|
|
Replacement suggestions link report-local removed/added IDs, with a `match_rule`,
|
|
typed `field_changes`, and `properties_changed`. Their explanation is optional
|
|
for consumers to display and must not be parsed. Current matching rules are
|
|
`unique_kind_layer_bounds`, `unique_kind_layer`, and `unique_kind_target`.
|
|
These are explanatory matches, not persistent identity or shape equivalence.
|
|
Ambiguous candidates stay independent removals and additions. IDs are unique
|
|
within a report and have no meaning across reports.
|
|
|
|
## Diagnostics and hierarchy
|
|
|
|
Diagnostics have `code`, `severity`, `message`, input `side`, original byte
|
|
offset, cell, layer, and target context. Unknown context is null. Messages are
|
|
for display; codes and context support automation. Current limitation codes are
|
|
`unsupported_record`, `unsupported_metadata_version`, `unresolved_reference`,
|
|
`hierarchy_cycle`, and `scope_mismatch`. Failure codes are `invalid_input`,
|
|
`io_error`, `invalid_scope`, and `command_error`. Fatal parser errors currently
|
|
retain their location in the human message; their structured byte offset is
|
|
null. Consumers must not depend on message wording.
|
|
|
|
`--hierarchy graph` includes the full scoped union graph of both revisions.
|
|
Otherwise `hierarchy` is null. Nodes are keyed by cell name, with per-side
|
|
states `included`, `outside_scope`, `unresolved`, or `absent`. Their `change`
|
|
is `direct`, `affected` through a descendant, or `unchanged`.
|
|
|
|
Edges record parent, target, before/after direct placement counts, and whether
|
|
the underlying reference records changed. AREF columns times rows contribute
|
|
symbolically; arrays are never expanded. Counts are direct edge multiplicities,
|
|
not recursively accumulated placements. Shared cells occur once. Removed edges
|
|
remain visible, and ancestors are computed over the union of both revisions.
|
|
Graph content is independent of display filters and object limits. Text depth
|
|
limits and ASCII/Mermaid choices do not apply to JSON graphs.
|
|
|
|
## Development
|
|
|
|
`cargo test --test review` checks report semantics, CLI behavior, and versioned
|
|
goldens. To intentionally regenerate report goldens after reviewing a contract
|
|
change, run:
|
|
|
|
```sh
|
|
PHEON_UPDATE_REPORT_GOLDENS=1 cargo test --test review machine_reports_match_versioned_goldens
|
|
```
|
|
|
|
Schema validators can validate the goldens or actual CLI output against
|
|
`diff-report-v1.schema.json`. Schema validation does not replace semantic checks
|
|
such as replacement links, summary totals, or status/exit-code agreement.
|