some docs

This commit is contained in:
Jan Petykiewicz 2026-04-02 20:41:16 -07:00
commit 7ddfc0cd7f

117
ARROW_SCHEMA.md Normal file
View file

@ -0,0 +1,117 @@
# Arrow Schema Documentation
This project uses Apache Arrow to represent GDSII (Graphic Data System II) library data. The schema is optimized for memory efficiency and fast access, employing several packed representations for common data types.
## Top-Level Structure
The top-level object is a `StructArray` representing the entire GDSII library.
| Field | Type | Description |
|---|---|---|
| `meters_per_db_unit` | `Float64` | Database unit size in meters. |
| `user_units_per_db_unit` | `Float64` | Number of user units in one database unit. |
| `lib_name` | `Utf8` | The name of the GDSII library. |
| `mod_time` | `FixedSizeList(Int16, 6)` | Last modification time [year, month, day, hour, minute, second]. |
| `acc_time` | `FixedSizeList(Int16, 6)` | Last access time [year, month, day, hour, minute, second]. |
| `cells` | `List(Cell)` | A list of cells (structures) in the library. |
| `cell_names` | `List(Utf8)` | Names of all cells, indexed by their ID. |
| `layers` | `List(UInt32)` | List of packed layer/datatype pairs, indexed by their ID. |
## Cell Structure
Each cell in the `cells` list is a `StructArray`.
| Field | Type | Description |
|---|---|---|
| `id` | `UInt32` | Unique identifier for the cell. |
| `file_offset` | `UInt64` | Byte offset of the cell in the original GDSII file. |
| `srefs` | `List(SRef)` | Simple structure references (no properties). |
| `arefs` | `List(ARef)` | Array references (no properties). |
| `sref_props` | `List(SRefProp)` | Structure references with properties. |
| `aref_props` | `List(ARefProp)` | Array references with properties. |
| `rect_batches` | `List(RectBatch)` | Batches of rectangular boundaries (optimized). |
| `boundary_batches` | `List(BoundaryBatch)` | Batches of generic boundaries. |
| `boundary_props` | `List(BoundaryProp)` | Individual boundaries with properties. |
| `paths` | `List(Path)` | Path elements. |
| `nodes` | `List(BoxNode)` | Node elements. |
| `boxes` | `List(BoxNode)` | Box elements. |
| `texts` | `List(Text)` | Text labels. |
## Element Types
### Common Packed Encodings
- **Layer ID (`UInt32`)**: A packed representation of `(layer, datatype)`.
- `layer = (layer_id >> 16) as i16`
- `datatype = (layer_id & 0xFFFF) as i16`
- **XY Coordinate (`UInt64`)**: A packed representation of a single point `(x, y)`.
- `x = (xy >> 32) as i32`
- `y = (xy & 0xFFFFFFFF) as i32`
- **Array Counts (`UInt32`)**: A packed representation of `(columns, rows)`.
- `columns = (counts >> 16) as i16`
- `rows = (counts & 0xFFFF) as i16`
### Reference Elements (SRef / ARef)
References point to other cells in the library.
**SRef (Simple Reference)**:
- `target`: `UInt32` (Cell ID)
- `invert_y`: `Boolean` (Vertical flip)
- `scale`: `Float64` (Magnification)
- `angle_rad`: `Float64` (Rotation in radians)
- `xy`: `UInt64` (Packed origin point)
**ARef (Array Reference)**:
- Same as SRef, plus:
- `xy0`: `UInt64` (Packed column basis vector increment)
- `xy1`: `UInt64` (Packed row basis vector increment)
- `counts`: `UInt32` (Packed columns and rows)
**SRefProp / ARefProp**:
- Same as above, plus a `properties` field.
### Boundary Elements
Boundaries are grouped into batches for efficiency when they share properties or are simple rectangles.
**RectBatch**:
- `layer`: `UInt32` (Packed Layer ID)
- `rects`: `List(Int32)` (Flattened list of `[min_x, min_y, max_x, max_y]` for each rectangle)
**BoundaryBatch**:
- `layer`: `UInt32` (Packed Layer ID)
- `vertices`: `List(Int32)` (Flattened list of all vertices for all boundaries in the batch)
- `vertex_offsets`: `List(UInt32)` (Start index for each boundary in the `vertices` list)
**BoundaryProp**:
- Individual boundary with `properties`.
### Other Elements
**Path**:
- `layer`: `UInt32`
- `path_type`: `Int16` (Optional)
- `extension_start`: `Int32` (Optional)
- `extension_end`: `Int32` (Optional)
- `width`: `Int32`
- `xy`: `List(Int32)` (centerline coordinates)
- `properties`: `List(Property)`
**Text**:
- `layer`: `UInt32`
- `presentation_horiz`: `UInt8`
- `presentation_vert`: `UInt8`
- `presentation_font`: `UInt8`
- `path_type`: `Int16`
- `width`: `Int32`
- `invert_y`: `Boolean`
- `mag`: `Float64`
- `angle_deg`: `Float64`
- `xy`: `UInt64` (Packed position)
- `string`: `Utf8`
- `properties`: `List(Property)`
**Property**:
- `key`: `Int16`
- `value`: `Utf8`