- Rust 92.8%
- Python 6.8%
- C 0.4%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| contrib/git | ||
| docs | ||
| klamath_rs_ext | ||
| src | ||
| tests | ||
| .gitignore | ||
| ARROW_SCHEMA.md | ||
| Cargo.toml | ||
| pyproject.toml | ||
| README.md | ||
| test.py | ||
klamath-rs
A fast GDSII parser and writer in Rust, with Apache Arrow integration for efficient data handling.
Overview
klamath-rs is designed to handle large GDSII files with high performance. It uses Apache Arrow as its primary in-memory data representation, allowing for efficient memory usage and easy integration with other tools in the Arrow ecosystem (like pandas, polars, or DuckDB via Python bindings).
Documentation
- Arrow Schema Documentation: Detailed information about the Arrow schema used for GDSII data.
Features
- Fast parsing of GDSII files.
- Efficient memory representation using Apache Arrow.
- Support for common GDSII elements: SREF, AREF, Boundary, Path, Text, etc.
- Optimized "batch" representations for rectangles and boundaries sharing the same layer.
- C-compatible FFI for integration with other languages (e.g., Python).
REAL8 writing uses normalized GDS encodings and represents every in-range f64
exactly. Finite magnitudes below 2^-260 underflow to canonical zero; nonfinite
values and magnitudes at or above 2^252 are rejected by the encoder.
Usage
read_path exports a library through the Arrow C Data Interface. Allocate
initialized ArrowArray and ArrowSchema output structures, then check the
returned status: 0 means success and -1 means failure. Only import the Arrow
outputs after success; the caller owns their release callbacks. Failed calls
leave the outputs unchanged. See test.py for a Python CFFI example.
last_error_message() returns a UTF-8, null-terminated error string, or null
after a successful parser call. Errors belong to the calling thread. Copy the
message before making another parser call on that thread, and before the thread
exits; calls on other threads do not invalidate it. Parser panics that unwind
are converted into errors.
For partial reads, call scan_bytes with a borrowed GDSII byte buffer to obtain
cell ranges and references without decoding geometry. Pass the desired ranges
to read_cells_bytes to load their full contents. Both calls use the same Arrow
outputs and status/error handling as read_path:
int scan_bytes(const uint8_t *data, size_t size,
struct ArrowArray *array, struct ArrowSchema *schema);
int read_cells_bytes(const uint8_t *data, size_t size,
const uint64_t *ranges, size_t range_count,
struct ArrowArray *array, struct ArrowSchema *schema);
ranges contains alternating inclusive start and exclusive end offsets, with
two native-endian uint64_t values per cell. Each range must cover one complete
cell. A null ranges pointer is accepted when range_count is zero. Likewise,
null data is accepted for zero bytes and produces an ordinary parse error for
the empty input. Nonempty buffers must be readable for the duration of the call;
range values must be aligned. Exported Arrow data owns its storage independently
of the input. See Arrow Schema Documentation for the scan
schema, offset definitions, and ID assignment.
Layout review CLI
The review CLI is a standalone Rust executable:
cargo build --bin klamath
target/debug/klamath dump chip.gds
target/debug/klamath dump chip.gds --records
target/debug/klamath diff old.gds new.gds --hierarchy ascii
target/debug/klamath diff old.gds new.gds --hierarchy mermaid --max-changes 20
target/debug/klamath hierarchy chip.gds --format mermaid --depth 4
target/debug/klamath diff old.gds new.gds --top CHIP --layer 1/0
target/debug/klamath diff old.gds new.gds --show-layer 1/0
target/debug/klamath diff old.gds new.gds --format json
target/debug/klamath diff old.gds new.gds --format json --hierarchy graph
target/debug/klamath name-layers chip.gds named.gds --cell CHIP --carrier 900/7 --name 1:0:WG
The object dump is deterministic and omits source offsets. Record dumps preserve record order and show offsets, tags, sizes, and values, including records the object reviewer does not support. They can inspect an invalid metadata declaration without invoking object-level validation. Zero padding after ENDLIB is accepted.
The fixed klamath-object-v2 comparison profile:
- Matches cells by name and layers by numeric pair. The default scope is the entire library; explicit selectors are recorded in each report.
- Ignores library/cell timestamps, cell order, element order, and property order.
- Normalizes boundary/box starting vertex and winding, retaining other vertices.
- Preserves duplicate objects and duplicate properties as multisets.
- Compares paths, labels, references, array endpoints/counts, properties, and all retained flags exactly. Explicit defaults and their omission remain distinct; SREFs and single-element AREFs remain distinct. REAL8 encodings are retained.
- Reports library fields, including UNITS, separately. It does not rescale files into a shared grid, infer cell renames, or perform geometric Boolean operations.
- Uses bounding boxes as coordinate envelopes, not evidence of equality. Path envelopes do not include stroke width; text/reference envelopes are coordinates, not rendered or recursively transformed bounds.
- Returns an incomplete result for unsupported records, future metadata versions, unresolved references, or cycles. These cannot produce an equality claim.
diff exits with 0 for equality under this profile, 1 for differences,
and 2 for invalid input or incomplete comparison. --max-changes limits the
number of object details displayed, not the comparison. Source offsets are
included in changed-object reports for tracing changes back to record dumps.
diff --format json emits a versioned report with typed before/after evidence,
structured diagnostics, and independent comparison status and completeness.
JSON includes all changes by default. Explicit display filters and --max-changes
record omitted evidence without changing totals or exit status. --hierarchy graph
adds the complete scoped reference graph, with symbolic array multiplicities;
--depth applies to text output only. Timestamp-only changes still return 0.
Input and argument errors also produce JSON when requested. See the
report contract and
version 1 JSON Schema.
Unmatched objects are paired for explanation only when candidates are unique by type/layer/bounding box, then by type/layer among the remaining objects. References additionally match by target. Ambiguous groups stay additions and removals. Reports explain reference movement, array-count changes, path widths, and polygons changed within the same bounds. Full before/after records remain visible, and duplicate counts remain significant. Unchanged layer names appear once as a legend for changed objects on those layers.
Comparison scope and display filters
--top NAME compares that cell and its reachable subtree independently in each
revision. --cell NAME compares the exact cell, including its reference records,
without comparing descendant contents. Combine the two to select a cell within
a subtree. A selector absent from both revisions is an error; absence on one
side is an addition/deletion.
--layer L/T selects objects on a numeric layer/type pair, plus references whose
targets use that layer transitively. Cells with no relevant content are omitted.
Unresolved references, cycles, and unsupported records with unknown layer effect
remain incomplete results. Unsupported objects on another known layer can be
excluded. Library fields, including units, remain compared. Unsupported layer-name
version diagnostics are always retained, since those declarations can affect any
layer or cell in the library. Scoped dumps and
hierarchy views accept the same selectors. Scope selection parses the complete
input: malformed record grammar and conflicting metadata are still errors.
--show-cell NAME and --show-layer L/T filter object details in a diff report
without changing comparison status, totals, metadata changes, or hierarchy.
References have no numeric layer and are hidden by --show-layer. The comparison
scope and any display-only filter are printed explicitly. The Rust API accepts
one combined Scope; applying another scope to an already scoped document is
rejected rather than losing reference coverage information.
Hierarchy output distinguishes directly changed cells, affected ancestors, and
unchanged targets of modified references. It includes edges from both revisions,
so removed branches remain visible. Counts are direct edge multiplicities,
including arrays, rather than expanded placements. Shared definitions are marked
as already shown in text and appear once in Mermaid. --depth limits displayed
descendants only. The CLI emits Mermaid source, requiring no diagram renderer.
Layer names are read from labels in an existing cell. A label with the exact
string KLAMATH_LAYER_NAMES_V1 identifies an annotation carrier (layer, texttype)
pair within that cell. All other labels on that same pair in that same cell must
be layer:datatype:name declarations, for example 1:0:WG. The carrier pair is
not fixed; choose a suitable annotation layer in the design's technology.
Declarations split at the first two colons, allowing colons in names. Numeric
fields are unsigned 16-bit decimal integers. Names are nonempty UTF-8 strings.
Repeated identical declarations are accepted; conflicting declarations fail.
Names are collected across the library. Recognized metadata labels are omitted
from object comparison; mappings are compared separately and printed once per
numeric pair, so renaming a layer does not rewrite every shape description.
name-layers INPUT OUTPUT --cell NAME --carrier L/T --name L:T:NAME ... writes
this convention. It consolidates existing recognized declarations into one legend
in the chosen existing cell, retains existing names not explicitly updated, and
overrides the named pairs. The chosen carrier must contain no ordinary design
objects in that cell. Unrelated records, timestamps, unknown records and trailing
padding are copied byte-for-byte. No structures or references are added. Output
must be a new file: the CLI refuses to overwrite an existing destination, including
the input. Invalid declarations, conflicting arguments, oversized labels and
future metadata versions fail before output creation.
The record-preserving review adapter retains information normalized or omitted
by the Arrow geometry reader; existing Arrow APIs are unchanged. Structured Rust
change records feed the text and hierarchy renderers. LayoutEditor's LayerNames
extension, layout extraction/remapping, and geometric Boolean comparison remain
outside the current CLI.
Git integration
Install the executable on PATH with cargo install --path . --bin klamath.
The templates in contrib/git/config and
contrib/git/attributes provide two alternative drivers.
For structural summaries and affected hierarchy, add this attribute:
*.gds diff=klamath
Configure the native external driver in Git config:
[diff "klamath"]
command = klamath git-diff --hierarchy ascii --
binary = true
trustExitCode = false
The adapter consumes Git's old/new temporary files directly, handles additions,
deletions and renames, and quotes display paths. It returns success after rendering
either equality or differences, as expected by the external-driver protocol.
Invalid or incomplete comparisons retain a nonzero exit and diagnostics. Scope
and display options can precede the final -- in the configured command. Existing
repositories using diff=layout can put the same command under [diff "layout"].
For ordinary Git line diffs, use canonical textconv instead:
*.gds diff=klamath-text
Configure the driver in Git config:
[diff "klamath-text"]
textconv = klamath dump
binary = true
cachetextconv = true
This is a human review representation, not an applicable GDS patch. Invalidate
Git's textconv cache when changing the dump profile/tool version. For automated
equality checks, use klamath diff and its exit status directly.
tests/review.rs contains ten initial fixture pairs, checked-in canonical/text/
Mermaid goldens, metadata byte-preservation checks, scope and report regressions,
and actual Git integration tests. It also checks near-limit polygon records,
2048-cell hierarchies and arrays representing over a billion placements without
expanding them. Run cargo test and cargo test --release for debug and optimized
validation. No KLayout installation or Python package is required.