pheon/ARROW_SCHEMA.md

148 lines
6.2 KiB
Markdown
Raw Permalink Normal View History

2026-04-02 20:41:16 -07:00
# Arrow Schema Documentation
This project uses Apache Arrow to represent GDSII (Graphic Data System II) library data. The schema is optimized for memory efficiency and fast access, employing several packed representations for common data types.
## Top-Level Structure
The top-level object is a `StructArray` representing the entire GDSII library.
| Field | Type | Description |
|---|---|---|
| `meters_per_db_unit` | `Float64` | Database unit size in meters. |
| `user_units_per_db_unit` | `Float64` | Number of user units in one database unit. |
| `lib_name` | `Utf8` | The name of the GDSII library. |
| `mod_time` | `FixedSizeList(Int16, 6)` | Last modification time [year, month, day, hour, minute, second]. |
| `acc_time` | `FixedSizeList(Int16, 6)` | Last access time [year, month, day, hour, minute, second]. |
| `cells` | `List(Cell)` | A list of cells (structures) in the library. |
| `cell_names` | `List(Utf8)` | Names of all cells, indexed by their ID. |
| `layers` | `List(UInt32)` | List of packed layer/datatype pairs, indexed by their ID. |
## Cell Structure
Each cell in the `cells` list is a `StructArray`.
| Field | Type | Description |
|---|---|---|
| `id` | `UInt32` | Unique identifier for the cell. |
| `file_offset` | `UInt64` | Byte offset immediately after the cell's `STRNAME` record (the first element, or `ENDSTR` for an empty cell). Identical for full and selected reads. |
2026-04-02 20:41:16 -07:00
| `srefs` | `List(SRef)` | Simple structure references (no properties). |
| `arefs` | `List(ARef)` | Array references (no properties). |
| `sref_props` | `List(SRefProp)` | Structure references with properties. |
| `aref_props` | `List(ARefProp)` | Array references with properties. |
| `rect_batches` | `List(RectBatch)` | Batches of rectangular boundaries (optimized). |
| `boundary_batches` | `List(BoundaryBatch)` | Batches of generic boundaries. |
| `boundary_props` | `List(BoundaryProp)` | Individual boundaries with properties. |
| `paths` | `List(Path)` | Path elements. |
| `nodes` | `List(BoxNode)` | Node elements. |
| `boxes` | `List(BoxNode)` | Box elements. |
| `texts` | `List(Text)` | Text labels. |
## Hierarchy Scan Schema
`scan_library` and `scan_bytes` return the same library metadata and `cell_names`
as a full read, with the following `cells` schema and no `layers` field:
| Field | Type | Description |
|---|---|---|
| `id` | `UInt32` | Cell ID indexing this result's `cell_names`. |
| `struct_start_offset` | `UInt64` | Inclusive byte offset of the `BGNSTR` record. |
| `struct_end_offset` | `UInt64` | Exclusive byte offset immediately after `ENDSTR`. |
| `refs` | `List(ScanRef)` | SREFs and AREFs, including references with properties, sorted by target ID. |
Each scan reference has `target`, `invert_y`, `scale`, `angle_rad`, `xy`, `xy0`,
`xy1`, and `counts`, using the ARef types and packed encodings described below.
SREFs use zero basis vectors and counts `(1, 1)`. Properties are omitted from the
scan output; geometry and text are skipped without decoding.
Pass the half-open scan ranges to `read_selected_library` or `read_cells_bytes`
to obtain the full library schema for just those cells. Each range must contain
exactly one complete cell, from `BGNSTR` through `ENDSTR`. Cells appear in the
requested order. Cell and layer IDs are assigned within each result, so use
`cell_names` and `layers` from that result when resolving them; IDs can differ
from a full read or a scan. An empty selection returns metadata and empty lists.
The Rust random-access reader returns the original input slice as its remainder.
2026-04-02 20:41:16 -07:00
## Element Types
### Common Packed Encodings
- **Layer ID (`UInt32`)**: A packed representation of `(layer, datatype)`.
- `layer = (layer_id >> 16) as i16`
- `datatype = (layer_id & 0xFFFF) as i16`
- **XY Coordinate (`UInt64`)**: A packed representation of a single point `(x, y)`.
- `x = (xy >> 32) as i32`
- `y = (xy & 0xFFFFFFFF) as i32`
- **Array Counts (`UInt32`)**: A packed representation of `(columns, rows)`.
- `columns = (counts >> 16) as i16`
- `rows = (counts & 0xFFFF) as i16`
### Reference Elements (SRef / ARef)
References point to other cells in the library.
**SRef (Simple Reference)**:
- `target`: `UInt32` (Cell ID)
- `invert_y`: `Boolean` (Vertical flip)
- `scale`: `Float64` (Magnification)
- `angle_rad`: `Float64` (Rotation in radians)
- `xy`: `UInt64` (Packed origin point)
**ARef (Array Reference)**:
- Same as SRef, plus:
- `xy0`: `UInt64` (Packed column basis vector increment)
- `xy1`: `UInt64` (Packed row basis vector increment)
- `counts`: `UInt32` (Packed columns and rows)
Array increments must be exact signed 32-bit integer database-unit values. Full
reads, hierarchy scans, selected reads, and their FFI entry points reject arrays
whose endpoint/count division is fractional or outside that range. They never
round or truncate placements. The review CLI retains original endpoints and can
compare fractional arrays independently of this Arrow representation.
2026-04-02 20:41:16 -07:00
**SRefProp / ARefProp**:
- Same as above, plus a `properties` field.
### Boundary Elements
Boundaries are grouped into batches for efficiency when they share properties or are simple rectangles.
**RectBatch**:
- `layer`: `UInt32` (Packed Layer ID)
- `rects`: `List(Int32)` (Flattened list of `[min_x, min_y, max_x, max_y]` for each rectangle)
**BoundaryBatch**:
- `layer`: `UInt32` (Packed Layer ID)
- `vertices`: `List(Int32)` (Flattened list of all vertices for all boundaries in the batch)
- `vertex_offsets`: `List(UInt32)` (Start index for each boundary in the `vertices` list)
**BoundaryProp**:
- Individual boundary with `properties`.
### Other Elements
**Path**:
- `layer`: `UInt32`
- `path_type`: `Int16` (Optional)
- `extension_start`: `Int32` (Optional)
- `extension_end`: `Int32` (Optional)
- `width`: `Int32` (Optional; null when `WIDTH` is absent)
2026-04-02 20:41:16 -07:00
- `xy`: `List(Int32)` (centerline coordinates)
- `properties`: `List(Property)`
**Text**:
- `layer`: `UInt32`
- `presentation_horiz`: `UInt8`
- `presentation_vert`: `UInt8`
- `presentation_font`: `UInt8`
- `path_type`: `Int16`
- `width`: `Int32`
- `invert_y`: `Boolean`
- `mag`: `Float64`
- `angle_deg`: `Float64`
- `xy`: `UInt64` (Packed position)
- `string`: `Utf8`
- `properties`: `List(Property)`
**Property**:
- `key`: `Int16`
- `value`: `Utf8`