Immutability Implementation Adjustments for PLDI and the PEP

[xFrednet]

• Modules have the Proxy immutability by default (But can opt-out)
  • Use PyModuleDef_Slot to allow modules to opt-in/out of immutability
• Add Py_mod_reachable to module slots

MatP

• Add escape hatches (§6.2):
  • Sub-interpreter Local
  • Cell type
• Split immutable module into a Python module and a C module
• Add new tp_reachable (same signature as tp_traverse) to PyTypeObject (Needed for the PEP)
  • Add dict_reachable since dict_traverse doesn't visit unicode strings.
• Support immutability by construction (§5.3 in the paper)
  • Add register_shallow_freezable(𝑡𝑦𝑝𝑒)
  • Make these objects immutable on first observation
• Add set_freezable() with the values YES, NO, EXPLICIT and PROXY(For modules) to the immutable module
  • Implement the Explicit freezability
  • Check for freezability in freezing code
  • Remove special casing for blocking_on and module_locks which should never be frozen
  • There is a check in traverse_freeze which should be removed in favour of this mechanism. Probably in check_freezable
• Allow freeze() to take multiple arguments
• Make freeze() return the (first) passed in object
• Fix rollback of SCC given failed freeze.
• Remove old code from the immutable module, for example immutable_register_freezable
• Remove not freezable type once explicit freezability is in.
• Add mutable(obj, ...) context manager to temporarily set freezability to No
  • Maybe add a C mechanism which could be used in a similar way
• C-Types should be unfreezable by default? Or the difference should be documented

Tobias

• Add @freezable, @frozen, @explicitlyFreezable and @unfreezable decorator

Fred

• Remove is_freezable_builtin in favour of set_freezable() for all the listed types
• Remove _PyImmutability_RegisterFreezable and state->freezable_types
• Investigate naming convention, should isfrozen be is_frozen?
  • Either adjust paper (which uses is_frozen)
  • Or our implementation which uses isfrozen
• Add pre-freeze hook (and check for it)
  • Change module and functions to use the pre-freeze hook instead of if-statement
• Fix (most) "freeze: type 'XYZ' has no tp_traverse and no tp_reachable" warnings
• Expose mutable module state in sys.mut_modules
• Make shadow_function_globals less smart (Don't freeze values based on strings which might index)
• Some test were failing for module objects in proxy mode

David

• How do we handle weak references??
  • _PyWeakref_GET_REF needs a _Py_IsImmutable branch which requires a TryIncref which is hard for SCCs...
  • Freeing requires GILs of different interpreters

Pending TODOs, but not critical for PLDI or DPO:

• Add ImmutableError to the immutable module (Error name and message is open to change)
• Investigate if _PyOwnership_is_c_wrapper is still needed
• Improve RC overhead for frozen objects
• https://github.com/mjp41/cpython/blob/b6510b9c60b441f3bcdcdc2ce124c78fff44a98e/Objects/typeobject.c#L6599