pheon/README.md
Jan Petykiewicz a04a955e90 feat(review): add deterministic GDS dumps and structural comparisons
Read embedded layer-name labels, retain unsupported records, and report affected hierarchy as ASCII or Mermaid. Preserve duplicate objects, canonicalize polygon rings, and cover the first profile with ten fixture pairs and golden outputs.
2026-11-14 17:25:04 -08:00

144 lines
7.3 KiB
Markdown

# klamath-rs
A fast GDSII parser and writer in Rust, with Apache Arrow integration for efficient data handling.
## Overview
`klamath-rs` is designed to handle large GDSII files with high performance. It uses Apache Arrow as its primary in-memory data representation, allowing for efficient memory usage and easy integration with other tools in the Arrow ecosystem (like pandas, polars, or DuckDB via Python bindings).
## Documentation
- [Arrow Schema Documentation](ARROW_SCHEMA.md): Detailed information about the Arrow schema used for GDSII data.
## Features
- Fast parsing of GDSII files.
- Efficient memory representation using Apache Arrow.
- Support for common GDSII elements: SREF, AREF, Boundary, Path, Text, etc.
- Optimized "batch" representations for rectangles and boundaries sharing the same layer.
- C-compatible FFI for integration with other languages (e.g., Python).
## Usage
`read_path` exports a library through the Arrow C Data Interface. Allocate
initialized `ArrowArray` and `ArrowSchema` output structures, then check the
returned status: `0` means success and `-1` means failure. Only import the Arrow
outputs after success; the caller owns their release callbacks. Failed calls
leave the outputs unchanged. See `test.py` for a Python CFFI example.
`last_error_message()` returns a UTF-8, null-terminated error string, or null
after a successful parser call. Errors belong to the calling thread. Copy the
message before making another parser call on that thread, and before the thread
exits; calls on other threads do not invalidate it. Parser panics that unwind
are converted into errors.
For partial reads, call `scan_bytes` with a borrowed GDSII byte buffer to obtain
cell ranges and references without decoding geometry. Pass the desired ranges
to `read_cells_bytes` to load their full contents. Both calls use the same Arrow
outputs and status/error handling as `read_path`:
```c
int scan_bytes(const uint8_t *data, size_t size,
struct ArrowArray *array, struct ArrowSchema *schema);
int read_cells_bytes(const uint8_t *data, size_t size,
const uint64_t *ranges, size_t range_count,
struct ArrowArray *array, struct ArrowSchema *schema);
```
`ranges` contains alternating inclusive start and exclusive end offsets, with
two native-endian `uint64_t` values per cell. Each range must cover one complete
cell. A null `ranges` pointer is accepted when `range_count` is zero. Likewise,
null `data` is accepted for zero bytes and produces an ordinary parse error for
the empty input. Nonempty buffers must be readable for the duration of the call;
range values must be aligned. Exported Arrow data owns its storage independently
of the input. See [Arrow Schema Documentation](ARROW_SCHEMA.md) for the scan
schema, offset definitions, and ID assignment.
## Layout review CLI
The first review milestone provides a Rust CLI using the existing dependencies:
```sh
cargo build --bin klamath
target/debug/klamath dump chip.gds
target/debug/klamath dump chip.gds --records
target/debug/klamath diff old.gds new.gds --hierarchy ascii
target/debug/klamath diff old.gds new.gds --hierarchy mermaid --max-changes 20
target/debug/klamath hierarchy chip.gds --format mermaid --depth 4
```
The object dump is deterministic and omits source offsets. Record dumps preserve
record order and show offsets, tags, sizes, and values, including records the
object reviewer does not support. They can inspect an invalid metadata declaration
without invoking object-level validation. Zero padding after ENDLIB is accepted.
The fixed `klamath-object-v1` comparison profile:
- Compares the entire library, matching cells by name and layers by numeric pair.
- Ignores library/cell timestamps, cell order, element order, and property order.
- Normalizes boundary/box starting vertex and winding, retaining other vertices.
- Preserves duplicate objects and duplicate properties as multisets.
- Compares paths, labels, references, array endpoints/counts, properties, and all
retained flags exactly. Explicit defaults and their omission remain distinct;
SREFs and single-element AREFs remain distinct. REAL8 encodings are retained.
- Reports library fields, including UNITS, separately. It does not rescale files
into a shared grid, infer cell renames, or perform geometric Boolean operations.
- Uses bounding boxes as coordinate envelopes, not evidence of equality. Path
envelopes do not include stroke width; text/reference envelopes are coordinates,
not rendered or recursively transformed bounds.
- Returns an incomplete result for unsupported records, future metadata versions,
unresolved references, or cycles. These cannot produce an equality claim.
`diff` exits with **0** for equality under this profile, **1** for differences,
and **2** for invalid input or incomplete comparison. `--max-changes` limits the
number of object details displayed, not the comparison. Source offsets are
included in changed-object reports for tracing changes back to record dumps.
Hierarchy output distinguishes directly changed cells, affected ancestors, and
unchanged targets of modified references. It includes edges from both revisions,
so removed branches remain visible. Counts are direct edge multiplicities,
including arrays, rather than expanded placements. Shared definitions are marked
as already shown in text and appear once in Mermaid. `--depth` limits displayed
descendants only. The CLI emits Mermaid source, requiring no diagram renderer.
Layer names are read from labels in an existing cell. A label with the exact
string `KLAMATH_LAYER_NAMES_V1` identifies an annotation carrier `(layer, texttype)`
pair within that cell. All other labels on that same pair in that same cell must
be `layer:datatype:name` declarations, for example `1:0:WG`. The carrier pair is
not fixed; choose a suitable annotation layer in the design's technology.
Declarations split at the first two colons, allowing colons in names. Numeric
fields are unsigned 16-bit decimal integers. Names are nonempty UTF-8 strings.
Repeated identical declarations are accepted; conflicting declarations fail.
Names are collected across the library. Recognized metadata labels are omitted
from object comparison; mappings are compared separately and printed once per
numeric pair, so renaming a layer does not rewrite every shape description.
This milestone reads the sentinel convention. It does not yet write metadata,
import LayoutEditor's `LayerNames` extension, remap/extract layouts, or provide
top/layer scope filters. The separate review adapter borrows raw record payloads
and retains information normalized or omitted by the Arrow geometry reader;
the existing Arrow APIs are unchanged. Structured Rust change records feed the
text and hierarchy renderers.
After installing the executable on PATH, the canonical dump can serve as a Git
textconv. Add this attribute to the layout repository:
```gitattributes
*.gds diff=klamath
```
Configure the driver in Git config:
```gitconfig
[diff "klamath"]
textconv = klamath dump
binary = true
cachetextconv = true
```
This is a human review representation, not an applicable GDS patch. Invalidate
Git's textconv cache when changing the dump profile/tool version. For automated
equality checks, use `klamath diff` and its exit status directly.
`tests/review.rs` contains ten generated fixture pairs with explicit expected
results, plus record, metadata, malformed-input, hierarchy, and CLI regressions.