- Rust 92.8%
- Python 6.8%
- C 0.4%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
Format the parser error reason and absolute byte offset without dumping the remaining input bytes. Calculate offsets correctly for errors inside bounded selected-cell ranges. |
||
| klamath_rs_ext | ||
| src | ||
| tests | ||
| .gitignore | ||
| ARROW_SCHEMA.md | ||
| Cargo.toml | ||
| pyproject.toml | ||
| README.md | ||
| test.py | ||
klamath-rs
A fast GDSII parser and writer in Rust, with Apache Arrow integration for efficient data handling.
Overview
klamath-rs is designed to handle large GDSII files with high performance. It uses Apache Arrow as its primary in-memory data representation, allowing for efficient memory usage and easy integration with other tools in the Arrow ecosystem (like pandas, polars, or DuckDB via Python bindings).
Documentation
- Arrow Schema Documentation: Detailed information about the Arrow schema used for GDSII data.
Features
- Fast parsing of GDSII files.
- Efficient memory representation using Apache Arrow.
- Support for common GDSII elements: SREF, AREF, Boundary, Path, Text, etc.
- Optimized "batch" representations for rectangles and boundaries sharing the same layer.
- C-compatible FFI for integration with other languages (e.g., Python).
Usage
read_path exports a library through the Arrow C Data Interface. Allocate
initialized ArrowArray and ArrowSchema output structures, then check the
returned status: 0 means success and -1 means failure. Only import the Arrow
outputs after success; the caller owns their release callbacks. Failed calls
leave the outputs unchanged. See test.py for a Python CFFI example.
last_error_message() returns a UTF-8, null-terminated error string, or null
after a successful parser call. Errors belong to the calling thread. Copy the
message before making another parser call on that thread, and before the thread
exits; calls on other threads do not invalidate it. Parser panics that unwind
are converted into errors.
For partial reads, call scan_bytes with a borrowed GDSII byte buffer to obtain
cell ranges and references without decoding geometry. Pass the desired ranges
to read_cells_bytes to load their full contents. Both calls use the same Arrow
outputs and status/error handling as read_path:
int scan_bytes(const uint8_t *data, size_t size,
struct ArrowArray *array, struct ArrowSchema *schema);
int read_cells_bytes(const uint8_t *data, size_t size,
const uint64_t *ranges, size_t range_count,
struct ArrowArray *array, struct ArrowSchema *schema);
ranges contains alternating inclusive start and exclusive end offsets, with
two native-endian uint64_t values per cell. Each range must cover one complete
cell. A null ranges pointer is accepted when range_count is zero. Likewise,
null data is accepted for zero bytes and produces an ordinary parse error for
the empty input. Nonempty buffers must be readable for the duration of the call;
range values must be aligned. Exported Arrow data owns its storage independently
of the input. See Arrow Schema Documentation for the scan
schema, offset definitions, and ID assignment.