jan/masque
1
1
Fork 0
Code Issues Pull requests Releases Wiki Activity

[library] More library rework

Browse source
This commit is contained in:
Jan Petykiewicz 2026-07-13 12:24:33 -07:00
commit 0edd735a11
25 changed files with 2357 additions and 870 deletions
Download patch file Download diff file Expand all files Collapse all files

334
MIGRATION.md
View file

@ -1,7 +1,8 @@
# Migration Guide
This guide covers changes between the git tag `release` and the current tree.
At `release`, `masque.__version__` was `3.3`; the current tree reports `3.4`.
This guide covers changes between the `master` branch and the current tree.
Both `master` and the current tree report `masque.__version__ == '3.4'`; the
version string has not yet been bumped for these changes.
Most downstream changes are in `masque/builder/*`, but there are a few other
API changes that may require code updates.
@ -126,28 +127,40 @@ builder = Pather(...)
deferred = Pather(..., render='deferred')
```
Top-level imports from `masque` also continue to work.
The new `Pather` remains importable from both `masque` and `masque.builder`.
The removed `Builder` and `RenderPather` names are no longer exported from
either location.
`Pather` now defaults to `render='auto'`, so plain construction replaces the
old `Builder` behavior. Use `Pather(..., render='deferred')` where you
previously used `RenderPather`.
## `BasicTool` was replaced
## `SimpleTool` was removed and `AutoTool` registration changed
`BasicTool` is no longer exported. Use `AutoTool` for reusable straight, bend,
transition, and S-bend primitives.
`SimpleTool` is no longer exported. `AutoTool` remains, but its old public
descriptor classes and constructor-oriented configuration were replaced by
registration methods. Use it for generated straights and S-bends and reusable
bends, U-turns, and transitions.
### Old `BasicTool`
### Old `AutoTool`
```python
from masque.builder.tools import BasicTool
from masque.builder import AutoTool
tool = BasicTool(
straight=(make_straight, 'input', 'output'),
bend=(lib.abstract('bend'), 'input', 'output'),
tool = AutoTool(
straights=[
AutoTool.Straight('m1wire', make_straight, 'input', 'output'),
],
bends=[
AutoTool.Bend(lib.abstract('bend'), 'input', 'output'),
],
sbends=[],
transitions={
'm2wire': (lib.abstract('via'), 'top', 'bottom'),
('m2wire', 'm1wire'): AutoTool.Transition(
lib.abstract('via'), 'top', 'bottom'
),
},
default_out_ptype='m1wire',
)
```
@ -166,12 +179,44 @@ tool = (
The key differences are:
- `BasicTool` -> `AutoTool`
- `straight=(fn, in_name, out_name)` -> `add_straight(fn, ptype, in_name)`
- `bend=(abstract, in_name, out_name)` -> `add_bend(abstract, in_name, out_name)`
- `SimpleTool` was removed; use `AutoTool` or implement the new `Tool`
primitive-offer interface
- `AutoTool.Straight(...)` -> `add_straight(fn, ptype, in_name)`
- `AutoTool.Bend(...)` -> `add_bend(abstract, in_name, out_name)`
- `AutoTool.SBend(...)` -> `add_sbend(fn, ptype, in_name, out_name)`
- reusable native U-turns can be registered with `add_uturn(...)`
- transitions are registered with `add_transition(abstract, external_port, internal_port)`
- transitions are bidirectional by default; pass `one_way=True` to inhibit the reverse adapter
Primitive costs are now explicit and independent of `AutoTool` registration
order. Every `add_*()` method accepts `cost=`, as do the concrete offer
factories. A numeric value scales the default geometric cost; for example,
`cost=2` makes a primitive twice as expensive. A callable receives the
canonical primitive parameter and local endpoint and returns the complete
cost:
```python
tool.add_straight(make_straight, 'm1wire', 'input', cost=1.5)
tool.add_sbend(
make_sbend,
'm1wire',
'input',
'output',
cost=lambda jog, endpoint: abs(jog) + 2 * abs(endpoint.x),
)
```
`PrimitiveOffer.priority_bias` was removed; custom offers should use `cost`
instead. Exact equal-cost candidates still use deterministic discovery order
as the final tie-break, but registration order no longer changes their
reported cost.
For two-port primitives, `AutoTool` can infer omitted port names and, for
generated straight/S-bend primitives, omitted ptype metadata by sampling an
in-domain example. Supply those values explicitly when generation requires
route-specific keyword arguments, because metadata inference does not receive
`tool_options`.
## Custom `Tool` subclasses
If you maintain your own `Tool` subclass, the interface changed:
@ -412,25 +457,268 @@ Check code that calls:
- `Ref.rotate(...)`
- `Ref.mirror(...)`
- `Label.rotate_around(...)` / `Label.mirror(...)`
- `Abstract.mirror_port_offsets(...)` / `Abstract.mirror_ports(...)`
If that code expected offsets or repetition grids to move automatically, it
needs updating. For whole-pattern transforms, prefer calling `Pattern.mirror()`
or `Pattern.rotate_around(...)` at the container level.
`Abstract.mirror_port_offsets()` and `Abstract.mirror_ports()` were removed.
Use `Abstract.mirror(axis)` to mirror both port locations and orientations. If
you intentionally need only one half of that operation, update the individual
ports explicitly with `Port.flip_across(...)` or `Port.mirror(...)`.
## Library hierarchy and graph behavior
`masque/library.py` was split into the `masque.library` package. Imports from
the public module remain stable:
```python
from masque import Library, LazyLibrary
# or
from masque.library import Library, LazyLibrary
```
Code importing the implementation file itself must move to the public package;
do not depend on the new internal `base`, `mapping`, or `lazy` module paths.
Hierarchy helpers now handle dangling references explicitly. The following
methods accept `dangling='error' | 'ignore' | 'include'` and default to
`'error'`:
- `child_graph()`
- `parent_graph()`
- `child_order()`
- `find_refs_local()`
- `find_refs_global()`
- `prune_empty()`
On `master`, graph construction could expose missing targets implicitly or
fail later with a `KeyError`. If dangling refs are intentional, choose the
behavior explicitly, for example:
```python
graph = library.child_graph(dangling='include')
order = library.child_order(dangling='ignore')
```
Graph cycles and invalid hierarchy states are now reported as `LibraryError`
with context. Audit code that caught `KeyError` or `graphlib.CycleError` from
these helpers.
Invalid `dangling=` strings now raise `ValueError`; they no longer fall through
to the `include` behavior. Empty lists stored in `Pattern.refs` are consistently
treated as absent references by hierarchy and geometry traversal. Code that
uses a `defaultdict` lookup such as `pattern.refs[name]` without appending a
`Ref` will therefore not create an edge or force that target to be loaded.
`Library.add()` now resolves the full name plan and remaps references before it
starts inserting cells. Name-resolution and preparation failures no longer
leave partially-added cells behind. A `rename_theirs` callback now receives an
`INameView` containing both existing names and names reserved earlier in the
same addition. Use membership, iteration, `len()`, or `get_name()`; the callback
argument is not the destination object and does not support pattern lookup or
mapping helpers such as `keys()` and `items()`. Update callback annotations from
`ILibraryView` to `INameView`. Custom `_merge()` implementations remain
responsible for their own rollback if they fail during the final commit.
Reference transforms returned by `find_refs_local()` and
`find_refs_global()` are now Nx5 arrays. The fifth column is the cumulative
scale, so rows have the form `(x, y, rotation, mirrored, scale)` rather than the
old Nx4 form.
`BuildReport` mapping fields are now defensively copied and read-only. Copy a
field to a new `dict` before adding or removing report entries.
`PortsLibraryView`, `OverlayLibrary`, and source-backed outputs returned by
`LibraryBuilder.build()` borrow their sources. They do not close source resources, and
`PortsLibraryView` is not a context manager. Keep each lazy source open and
unchanged until every borrowing view or overlay is finished, then close the
owning source explicitly. An eager `build(output='library')` result is detached
and does not need its sources afterward.
Read-only `subtree()` results are now borrowed lazy views rather than eager
`LibraryView` snapshots. Creating one no longer loads its reachable patterns,
and the view preserves source ordering, hierarchy metadata, and lazy GDS
copy-through. Mutable `Library`, `LazyLibrary`, and `OverlayLibrary` subtrees
return the same writable type as their source. Overlay subtrees retain their
source layers and lazy GDS capabilities. Mutable subtree containers are
structurally independent, but already-materialized patterns remain shared.
Keep borrowed sources open and structurally unchanged for the subtree's
lifetime.
`ILibrary` no longer inherits `collections.abc.MutableMapping`. It remains a
readable `Mapping` with explicit insert-only item assignment and deletion, but
generic mutation helpers such as `update()`, `setdefault()`, `pop()`,
`popitem()`, and `clear()` are no longer supplied. Use `add()`, `rename()`,
`delete()`, item insertion, and item deletion so library name and reference
invariants remain explicit.
Library-level `referenced_patterns()` and `dangling_refs()` now report only
named cell targets and return `set[str]`. Populated refs whose target is `None`
are ignored, matching `child_graph()` and recursive geometry behavior;
`Pattern.referenced_patterns()` continues to report `None` locally.
Port-importing views now always process detached patterns, including when the
raw source cell was already cached. Code can safely retain and compare raw and
processed views without port overrides leaking back into the raw pattern.
Once a processed cell is persistently materialized, lazy GDS writers no longer
copy the raw source structure for that cell, so later mutations to the returned
`Pattern` are serialized. Non-persistent materialization does not mark the cell
as changed.
Underscore-prefixed declarations work through the attribute authoring surface:
`builder.cells._helper = pattern` now declares `_helper`. Only the view's exact
internal `_library` attribute is reserved.
Recursive geometry operations now reject cyclic reference hierarchies with a
contextual `PatternError` instead of eventually leaking `RecursionError`. This
applies to bounds calculation, flattened layer polygon extraction, and
visualization as well as the existing flattening checks.
`LibraryBuilder.validate(names=...)` now accepts either one string or a sequence
of strings. Non-string roots raise `TypeError`, and duplicate roots are reduced
to their first occurrence. A recipe may build a different `LibraryBuilder`, but
calling `build()` or `validate()` recursively on its own active builder now
raises `BuildError` before starting another session.
`LibraryBuilder` is an authoring registry, not an `ILibraryView` or mapping.
Use membership, iteration, `keys()`, `get_name()`, assignment, and deletion to
manage declarations, then use the library returned by `build()` for reads and
hierarchy operations. Recipes that need an active library must receive the
builder-owned placeholder as a direct argument:
```python
def make_top(lib: ILibrary) -> Pattern:
return Pather(library=lib, ports='device').pattern
builder.cells.top = cell(make_top)(builder.library)
builder.cells.device = cell(factory)(hole_lib=builder.library)
```
The builder-owned `builder.library` placeholder is read-only. Only direct
positional and keyword values equal to it are substituted; placeholders nested
inside containers are not interpreted. A placeholder from another builder is
rejected when the recipe is assigned.
`IMaterializable` now identifies libraries which support explicit
`materialize()` and `materialize_many()` operations. `LibraryBuilder.add()`
borrows these marked inputs, while ordinary mappings and `ILibraryView`
instances are copied eagerly. Use `add_source()` to force borrowing of an
unmarked view. Wrapping a materializable library in `LibraryView` intentionally
erases the marker.
`IBorrowing` separately identifies composite views which retain direct source
libraries and expose them through `borrowed_sources()`. Keep those sources open
and unchanged for the lifetime of the borrowing view. Owner libraries such as
`LazyLibrary` and the lazy GDS readers are materializable but do not implement
`IBorrowing`. GDS raw-structure copy-through remains a separate,
format-specific capability.
`LibraryBuilder`, `OverlayLibrary`, and `PortsLibraryView` are new additive
library implementations. `LibraryBuilder` supports declarative `@cell` recipes
and dependency-aware builds; `OverlayLibrary` composes source libraries without
eagerly copying all patterns; `PortsLibraryView` overlays port metadata on a
read-only source.
Flattening with `flatten_ports=True` now rejects repeated refs whose target has
ports, because expanding them would create duplicate port names. Resolve the
repetition and assign unique port names before flattening, or use
`flatten_ports=False`.
## GDSII module and lazy-loading changes
`masque.file.gdsii` changed from a module into a package. The eager klamath
API remains available at the old import path, so ordinary `read`, `readfile`,
`write`, and `writefile` calls do not need to change:
```python
from masque.file import gdsii
library, info = gdsii.readfile('layout.gds')
```
The old `gdsii.load_library()` and `gdsii.load_libraryfile()` entry points were
removed. Use the source-backed lazy reader instead:
```python
# old
library, info = gdsii.load_libraryfile('layout.gds')
# new
from masque.file.gdsii import lazy
library, info = lazy.readfile('layout.gds')
try:
pattern = library['TOP']
finally:
library.close()
```
`lazy.read(stream)` and `lazy.readfile(path, use_mmap=...)` return a read-only
`GdsLibrarySource`. It owns file resources when it opens them and also supports
the context-manager protocol. The old `full_load` and `postprocess` arguments
are gone; materialize/copy the desired cells and post-process them explicitly.
An optional Arrow/native backend is available through
`masque.file.gdsii.arrow` and `masque.file.gdsii.lazy_arrow`; install the new
`arrow` extra to use it. These modules are additive and are not a transparent
replacement unless their additional dependencies and native library are
available.
## Shape construction and geometry additions
The public `raw=True` constructor shortcut was removed from `Arc`, `Circle`,
`Ellipse`, `Path`, `Polygon`, `PolyCollection`, and `Text`. Call their normal
constructors without `raw`; `_from_raw()` is an internal fast path and is not a
compatibility API.
```python
# old
polygon = Polygon(vertices, raw=True)
# new
polygon = Polygon(vertices)
```
`Arc` radii must now be strictly positive rather than merely non-negative.
`Arc.angle_ref` is additive and defaults to `Arc.AngleRef.Center`, preserving
the previous center-referenced angle interpretation.
`RectCollection` is a new shape for batches of axis-aligned rectangles and is
exported from both `masque` and `masque.shapes`. `Polygon.boolean()` and the
top-level `masque.boolean()` helper are also new; install the `boolean` extra
for their `pyclipper` dependency.
## Other user-facing changes
### File writers
SVG writing no longer polygonizes or flattens caller-owned patterns in place;
it works from detached copies. `svg.writefile(..., annotate_ports=True)` can
add port arrows. DXF writing now expands shape repetitions into individual DXF
entities, so callers no longer need to wrap repeated shapes solely for DXF
output.
### DXF environments
If you install the DXF extra, the supported `ezdxf` baseline moved from
`~=1.0.2` to `~=1.4`. Any pinned environments should be updated accordingly.
### Optional dependency names
The misspelled `manhatanize_slow` extra was corrected to
`manhattanize_slow`. A separate `manhattanize` extra now installs the
scikit-image implementation. The `arrow` and `boolean` extras are also new.
### New exports
These are additive, but available now from `masque` and `masque.builder`:
- `PortPather`
- `AutoTool`
- `boolean`
- from `masque`: `RectCollection`, `boolean`, `OverlayLibrary`,
`PortsLibraryView`, `IMaterializable`, `IBorrowing`, `LibraryBuilder`,
`BuildReport`, `CellProvenance`, and `cell`
- from `masque.builder`: `CostCallable`, the concrete primitive-offer classes,
and structured route error/status types
## Minimal migration checklist
@ -438,11 +726,15 @@ If your code uses the routing stack, do these first:
1. Replace `path`/`path_to`/`mpath`/`path_into` calls with
`trace`/`trace_to`/multi-port `trace`/`trace_into`.
2. Replace `BasicTool` with `AutoTool`.
2. Replace `SimpleTool` and old `AutoTool` descriptor construction with the
new `AutoTool.add_*()` methods.
3. Fix imports that still reference `masque.builder.builder` or
`masque.builder.renderpather`.
4. Audit any low-level `mirror()` usage, especially on `Port` and `Ref`.
5. Move lazy GDS calls from `gdsii.load_library*()` to
`masque.file.gdsii.lazy`.
6. Remove `raw=True` from public shape constructors.
If your code only uses `Pattern`, `Library`, `place()`, and `plug()` without the
routing helpers, you may not need any changes beyond the transform audit and any
stale imports.
routing helpers, audit transforms, dangling-reference graph calls, raw shape
construction, and any stale imports.