PR for tracking changes for immutability#63
Draft
mjp41 wants to merge 76 commits intoimmutable-basefrom
Draft
Conversation
* Changes made in sprint for PLDI deadline. Co-authored-by: xFrednet <xFrednet@gmail.com> * Addressing code review and CI failures. * Format * Windows build * More CI fixes * More CI fixes * Remove free-threaded build until we do that work. * Fix yaml lint (maybe) * Disable another free threaded build * Disable another free threaded build --------- Co-authored-by: xFrednet <xFrednet@gmail.com>
This commit moves the immutable C module into _immutable, and adds a new Python module that surfaces the immutable module to Python.
xFrednet
approved these changes
Jan 15, 2026
Includes various fixes to get more passing CI too Co-authored-by: Fridtjof Stoldt <xFrednet@gmail.com>
Fix Immutable Proxy Mode for Modules
* Missing clean up code * Fix limited API
Extend the freeze infrastructure with the ability to determine whether an object graph can be viewed as immutable without requiring an explicit freeze call. An object graph can be viewed as immutable if every reachable object is either already frozen or is "shallow immutable" (its own state cannot be mutated, but it may reference other objects). If the graph can be viewed as immutable, it is frozen to set up proper refcount management, and isfrozen() returns True. Key changes: - Add _PyImmutability_CanViewAsImmutable() C API that walks the object graph using an iterative DFS (avoiding stack overflow on deep graphs) and checks that every reachable object is shallow immutable or already frozen. If the check passes, the graph is frozen via the existing _PyImmutability_Freeze(). - Add _PyImmutability_RegisterShallowImmutable() C API for extension modules to register types whose instances are individually immutable. Built-in types (tuple, frozenset, bytes, str, int, float, complex, bool, NoneType, etc.) are registered at init time. - Immutable type objects (Py_TPFLAGS_IMMUTABLETYPE) are recognized as shallow immutable without explicit registration. - Change type_reachable() to visit tp_dict directly rather than via lookup_tp_dict(). For static builtin types, tp_dict is NULL on the struct (stored in per-interpreter state), so the walk skips these internal dicts — avoiding false failures on type internals. - Integrate the check into isfrozen(): it now returns True for objects that are explicitly frozen OR can be viewed as immutable (freezing as a side effect in the latter case). - Add ShallowImmutable test type to _test_reachable C module to exercise RegisterShallowImmutable from C. - Add 24 tests covering: tuples/frozensets of various types, mutable containers (correctly rejected), already-frozen objects, deeply nested structures (10,000 deep — verifies no stack overflow), and C-registered shallow immutable types.
Add set_freezable(obj, status) to control whether individual objects
can be frozen. The status is one of four values:
FREEZABLE_YES - always freezable
FREEZABLE_NO - never freezable
FREEZABLE_EXPLICIT - freezable only when freeze() is called directly
on the object, not when reached as a child
FREEZABLE_PROXY - reserved for future module proxy use
Storage uses a two-tier strategy: first tries setting a __freezable__
attribute on the object (works for any object with a __dict__), falling
back to bits in the header.
During freeze, check_freezable() queries the status by checking the
__freezable__ attribute first, then the weakref dictionary. For
EXPLICIT, the freeze root object is tracked in FreezeState so that
direct vs child freezing can be distinguished.
The old standalone __freezable__ = False check in traverse_freeze is
removed; all __freezable__ handling is now unified through
check_freezable() and _PyImmutability_GetFreezable().
Changes:
- Include/internal/pycore_immutability.h: add _Py_freezable_status
enum, freezable_objects dict, and destroy_objects_cb to state
- Include/cpython/immutability.h: declare SetFreezable/GetFreezable
- Python/immutability.c: implement SetFreezable (attr-first, bits in header
fallback), GetFreezable (attr-first, weakref fallback),
_destroy_dict callback, FreezeState.root, updated check_freezable
- Modules/_immutablemodule.c: expose set_freezable() and constants
- Lib/immutable.py: export set_freezable and FREEZABLE_* constants
- Python/pystate.c: cleanup new state fields in interpreter_clear()
- Lib/test/test_freeze/test_set_freezable.py: 18 tests covering all
status values, storage strategies, edge cases, and constants
Co-authored-by: Fridtjof Stoldt <xFrednet@gmail.com>
When freeze() fails partway through traversal (e.g. an object is marked
FREEZABLE_NO), all objects that were already marked immutable by completed
SCCs must be rolled back to their mutable state. Previously, only the
pending list (incomplete SCCs) was unwound; completed SCCs were leaked as
frozen with corrupted refcounts.
Track completed SCC representatives in an intrusive linked list threaded
through _gc_prev (scc_parent), which is free after complete_scc. On error,
walk the list and reverse each completed SCC via rollback_completed_scc(),
a three-walk algorithm:
Walk 1: Build ring membership set and recover each member's external
refcount (non-root members' ob_refcnt is preserved by
set_indirect_rc which only modifies flag bits).
Walk 2: Re-add internal reference counts via tp_reachable, reversing
the decrements made by add_internal_reference during freeze.
Walk 3: Clear immutability flags and return objects to GC tracking.
Also sweep the visited hashtable to clear immutability flags on non-GC
objects that were marked during traversal.
This enables freeze(a,b) to work, which treats both a and b as roots and enables the freezing in a single step. This is useful if multiple items that form a cycle are each explicitely freezable.
A test from my pre-freeze hook will test this
Correct Segfault introduced by #83
Add pre-freeze hook
PLDI cleanup
Set `tp_reachable` for Python classes
InterpreterLocal provides per-sub-interpreter mutable state behind an immutable indirection. When an object graph is frozen, InterpreterLocal appears immutable to the freezer via tp_reachable (which visits only the type and frozen default/factory fields), while each sub-interpreter independently reads and writes its own value through get()/set() methods. Two construction forms: InterpreterLocal(42) - immutable default (frozen at construction) InterpreterLocal(lambda: []) - factory callable (frozen at construction) The type is a heap type created via PyType_FromModuleAndSpec, with per-interpreter storage in the _immutable module's per-interpreter state. This avoids changes to core interpreter structures. Co-authored-by: Fridtjof Stoldt <xFrednet@gmail.com>
* Add SharedField type as shared escape hatch from freeze system
SharedField provides a mutable field inside a frozen object that only
holds frozen values. Unlike InterpreterLocal (which gives each
sub-interpreter its own independent value), SharedField is truly shared:
writes from one sub-interpreter are visible to all others.
API:
get() - read the current value
set(val) - replace the value (must be frozen)
swap(val) - replace and return the old value (must be frozen)
compare_and_swap(old, new) - CAS: replace only if current value is old
All operations are protected by a PyMutex embedded in the object. A
PyMutex is used rather than Py_BEGIN_CRITICAL_SECTION because the latter
is a no-op on GIL builds, but SharedField can be accessed concurrently
from different sub-interpreters (each with its own GIL).
tp_reachable visits the stored value (unlike InterpreterLocal which hides
its per-interpreter values), since SharedField values are always frozen
and belong to the shared immutable graph.
set() is implemented in terms of swap() to avoid duplication.
Files changed:
Modules/_immutablemodule.c - SharedField type implementation
Lib/immutable.py - Re-export SharedField
Lib/test/test_freeze/test_sharedfield.py - Tests including cross-interpreter
sharing, CAS semantics, and frozen-value enforcement
* Change reachable
* Change reachable
* Don't visit value as could cause issues trying to collect state between interpreters.
Collaborator
|
@copilot This PR implements immutability in Python. I want to get some high-level pointers to interesting parts of the implementation. Examples may be:
I'm looking for interesting parts in the general categories:
We can add more categories or make them more fine-grained |
#89) * Add get_freezable, unset_freezable, and FreezabilityOverride context manager Add C-level get_freezable() and unset_freezable() functions to query and clear per-object freezability status, and a Python-level FreezabilityOverride context manager that temporarily overrides an object's freezability. C API changes (Python/immutability.c, Include/cpython/immutability.h): - _PyImmutability_GetFreezable(obj): returns the effective freezable status (checking object, then ob_flags, then type, then type's ob_flags). Returns -1 if unset, -2 on error. - _PyImmutability_UnsetFreezable(obj): removes any per-object freezable status by deleting __freezable__ and clearing ob_flags. Includes a failing assert on 32-bit for the ob_flags path, matching the pattern in SetFreezable. Module changes (Modules/_immutablemodule.c): - Expose get_freezable() and unset_freezable() via Argument Clinic. Python changes (Lib/immutable.py): - Export get_freezable and unset_freezable. - Add FreezabilityOverride context manager. On enter, snapshots the effective status via get_freezable() and applies the override. On exit, restores the saved status via set_freezable(). Uses a "no-effort unset" design: always restores the effective status that was observed on entry, even if it was inherited from the type. This may leave an explicit per-object status where none existed before, but is simple, predictable, and correct for all object kinds including C types with custom tp_getattro. The alternative "best-effort unset" approach (using a get_direct_freezable that inspects __dict__ directly) was explored and rejected because it cannot reliably distinguish per-object from inherited status for C extension types with custom attribute storage. Tests (Lib/test/test_freeze/): - test_get_freezable.py: tests for get_freezable() (7 tests) and unset_freezable() (5 tests). - test_freezability_override.py: tests for FreezabilityOverride (10 tests) covering basic override/restore, freeze prevention, freeze allowance, exception safety, nested overrides, class-level override, and the no-effort unset behavior. * Lint fix. * Add version from paper.
Immutable artifact markers
* This should be safe * Try changeing the default freezability * Restrict module-proxies to module objects * Change default freezability of _random * Better re-import for modules in proxy mode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a Meta-PR that tracks the overall changes from the CPython 3.15 alpha to enable immutability.