Reuse library builders and cell traversal for full reads, lightweight hierarchy scans, and exact cell byte ranges. Preserve file offsets across readers and reject malformed ranges and FFI pointer/count inputs without aborting. Document scan schemas and ID assignment. Add regression coverage for Arrow round trips, transforms, subset ordering, empty selections, and invalid input.
55 lines
2.8 KiB
Markdown
55 lines
2.8 KiB
Markdown
# klamath-rs
|
|
|
|
A fast GDSII parser and writer in Rust, with Apache Arrow integration for efficient data handling.
|
|
|
|
## Overview
|
|
|
|
`klamath-rs` is designed to handle large GDSII files with high performance. It uses Apache Arrow as its primary in-memory data representation, allowing for efficient memory usage and easy integration with other tools in the Arrow ecosystem (like pandas, polars, or DuckDB via Python bindings).
|
|
|
|
## Documentation
|
|
|
|
- [Arrow Schema Documentation](ARROW_SCHEMA.md): Detailed information about the Arrow schema used for GDSII data.
|
|
|
|
## Features
|
|
|
|
- Fast parsing of GDSII files.
|
|
- Efficient memory representation using Apache Arrow.
|
|
- Support for common GDSII elements: SREF, AREF, Boundary, Path, Text, etc.
|
|
- Optimized "batch" representations for rectangles and boundaries sharing the same layer.
|
|
- C-compatible FFI for integration with other languages (e.g., Python).
|
|
|
|
## Usage
|
|
|
|
`read_path` exports a library through the Arrow C Data Interface. Allocate
|
|
initialized `ArrowArray` and `ArrowSchema` output structures, then check the
|
|
returned status: `0` means success and `-1` means failure. Only import the Arrow
|
|
outputs after success; the caller owns their release callbacks. Failed calls
|
|
leave the outputs unchanged. See `test.py` for a Python CFFI example.
|
|
|
|
`last_error_message()` returns a UTF-8, null-terminated error string, or null
|
|
after a successful parser call. Errors belong to the calling thread. Copy the
|
|
message before making another parser call on that thread, and before the thread
|
|
exits; calls on other threads do not invalidate it. Parser panics that unwind
|
|
are converted into errors.
|
|
|
|
For partial reads, call `scan_bytes` with a borrowed GDSII byte buffer to obtain
|
|
cell ranges and references without decoding geometry. Pass the desired ranges
|
|
to `read_cells_bytes` to load their full contents. Both calls use the same Arrow
|
|
outputs and status/error handling as `read_path`:
|
|
|
|
```c
|
|
int scan_bytes(const uint8_t *data, size_t size,
|
|
struct ArrowArray *array, struct ArrowSchema *schema);
|
|
int read_cells_bytes(const uint8_t *data, size_t size,
|
|
const uint64_t *ranges, size_t range_count,
|
|
struct ArrowArray *array, struct ArrowSchema *schema);
|
|
```
|
|
|
|
`ranges` contains alternating inclusive start and exclusive end offsets, with
|
|
two native-endian `uint64_t` values per cell. Each range must cover one complete
|
|
cell. A null `ranges` pointer is accepted when `range_count` is zero. Likewise,
|
|
null `data` is accepted for zero bytes and produces an ordinary parse error for
|
|
the empty input. Nonempty buffers must be readable for the duration of the call;
|
|
range values must be aligned. Exported Arrow data owns its storage independently
|
|
of the input. See [Arrow Schema Documentation](ARROW_SCHEMA.md) for the scan
|
|
schema, offset definitions, and ID assignment.
|