# embeddings

id: pep-0744.rst
content:
PEP: 744
Title: JIT Compilation
Author: Brandt Bucher <brandt@python.org>,
        Savannah Ostrowski <savannah@python.org>,
Discussions-To: https://discuss.python.org/t/pep-744-jit-compilation/50756
Status: Draft
Type: Informational
Created: 11-Apr-2024
Python-Version: 3.13
Post-History: `11-Apr-2024 <https://discuss.python.org/t/pep-744-jit-compilation/50756>`__

Abstract
========

Earlier this year, an `experimental "just-in-time" compiler
<https://github.com/python/cpython/pull/113465>`__ was merged into CPython's
``main`` development branch. While recent CPython releases have included other
substantial internal changes, this addition represents a particularly
significant departure from the way CPython has traditionally executed Python
code. As such, it deserves wider discussion.

This PEP aims to summarize the design decisions behind this addition, the
current state of the implementation, and future plans for making the JIT a
permanent, non-experimental part of CPython. It does *not* seek to provide a
comprehensive overview of *how* the JIT works, instead focusing on the
particular advantages and disadvantages of the chosen approach, as well as
answering many questions that have been asked about the JIT since its
introduction.

Readers interested in learning more about the new JIT are encouraged to consult
the following resources:

- The `presentation <https://youtu.be/HxSHIpEQRjs>`__ which first introduced the
  JIT at the 2023 CPython Core Developer Sprint. It includes relevant
  background, a light technical introduction to the "copy-and-patch" technique
  used, and an open discussion of its design amongst the core developers
  present. Slides for this talk can be found on `GitHub <https://github.com/brandtbucher/brandtbucher/blob/master/2023/10/10/a_jit_compiler_for_cpython.pdf>`__.

- The `open access paper <https://dl.acm.org/doi/10.1145/3485513>`__ originally
  describing copy-and-patch.

- The `blog post <https://sillycross.github.io/2023/05/12/2023-05-12>`__ by the
  paper's author detailing the implementation of a copy-and-patch JIT compiler
  for Lua. While this is a great low-level explanation of the approach, note
  that it also incorporates other techniques and makes implementation decisions
  that are not particularly relevant to CPython's JIT.

- The `implementation <#reference-implementation>`__ itself.

Motivation
==========

Until this point, CPython has always executed Python code by compiling it to
bytecode, which is interpreted at runtime. This bytecode is a more-or-less
direct translation of the source code: it is untyped, and largely unoptimized.

Since the Python 3.11 release, CPython has used a "specializing adaptive
interpreter" (:pep:`659`), which `rewrites these bytecode instructions in-place
<https://youtu.be/shQtrn1v7sQ>`__ with type-specialized versions as they run.
This new interpreter delivers significant performance improvements, despite the
fact that its optimization potential is limited by the boundaries of individual
bytecode instructions. It also collects a wealth of new profiling information:
the types flowing though a program, the memory layout of particular objects, and
what paths through the program are being executed the most. In other words,
*what* to optimize, and *how* to optimize it.

Since the Python 3.12 release, CPython has generated this interpreter from a
`C-like domain-specific language
<https://github.com/python/cpython/blob/main/Python/bytecodes.c>`__ (DSL). In
addition to taming some of the complexity of the new adaptive interpreter, the
DSL also allows CPython's maintainers to avoid hand-writing tedious boilerplate
code in many parts of the interpreter, compiler, and standard library that must
be kept in sync with the instruction definitions. This ability to generate large
amounts of runtime infrastructure from a single source of truth is not only
convenient for maintenance; it also unlocks many possibilities for expanding
CPython's execution in new ways. For instance, it makes it feasible to
automatically generate tables for translating a sequence of instructions into an
equivalent sequence of smaller "micro-ops", generate an optimizer for sequences
of these micro-ops, and even generate an entire second interpreter for executing
them.

In fact, since early in the Python 3.13 release cycle, all CPython builds have
included this exact micro-op translation, optimization, and execution machinery.
However, it is disabled by default; the overhead of interpreting even optimized
traces of micro-ops is just too large for most code. Heavier optimization
probably won't improve the situation much either, since any efficiency gains
made by new optimizations will likely be offset by the interpretive overhead of
even smaller, more complex micro-ops.

The most obvious strategy to overcome this new bottleneck is to statically
compile these optimized traces. This presents opportunities to avoid several
sources of indirection and overhead introduced by interpretation. In particular,
it allows the removal of dispatch overhead between micro-ops (by replacing a
generic interpreter with a straight-line sequence of hot code), instruction
decoding overhead for individual micro-ops (by "burning" the values or addresses
of arguments, constants, and cached values directly into machine instructions),
and memory traffic (by moving data off of heap-allocated Python frames and into
physical hardware registers).

Since much of this data varies even between identical runs of a program and the
existing optimization pipeline makes heavy use of runtime profiling information,
it doesn't make much sense to compile these traces ahead of time and would be a
substantial redesign of the existing specification and micro-op tracing infrastructure
that has already been implemented. As has been demonstrated for many other dynamic
languages (`and even Python itself <https://www.pypy.org>`__), the most promising
approach is to compile the optimized micro-ops "just in time" for execution.

Rationale
=========

Despite their reputation, JIT compilers are not magic "go faster" machines.
Developing and maintaining any sort of optimizing compiler for even a single
platform, let alone all of CPython's most popular supported platforms, is an
incredibly complicated, expensive task. Using an existing compiler framework
like LLVM can make this task simpler, but only at the cost of introducing heavy
runtime dependencies and significantly higher JIT compilation overhead.

It's clear that successfully compiling Python code at runtime requires not only
high-quality Python-specific optimizations for the code being run, *but also*
quick generation of efficient machine code for the optimized program. The Python
core development team has the necessary skills and experience for the former (a
middle-end tightly coupled to the interpreter), and copy-and-patch compilation
provides an attractive solution for the latter.

In a nutshell, copy-and-patch allows a high-quality template JIT compiler to be
generated from the same DSL used to generate the rest of the interpreter. For a
widely-used, volunteer-driven project like CPython, this benefit cannot be
overstated: CPython's maintainers, by merely editing the bytecode definitions,
will also get the JIT backend updated "for free", for *all* JIT-supported
platforms, at once. This is equally true whether instructions are being added,
modified, or removed.

Like the rest of the interpreter, the JIT compiler is generated at build time,
and has no runtime dependencies. It supports a wide range of platforms (see the
`Support`_ section below), and has comparatively low maintenance burden. In all,
the current implementation is made up of about 900 lines of build-time Python
code and 500 lines of runtime C code.

Specification
=============

The JIT is currently not part of the default build configuration, and it is
likely to remain that way for the foreseeable future (though official binaries
may include it). That sa


id: pep-0750.rst
content:
PEP: 750
Title: Template Strings
Author: Jim Baker <jim.baker@python.org>,
        Guido van Rossum <guido@python.org>,
        Paul Everitt <pauleveritt@me.com>,
        Koudai Aono <koxudaxi@gmail.com>,
        Lysandros Nikolaou <lisandrosnik@gmail.com>,
        Dave Peck <davepeck@davepeck.org>
Discussions-To: https://discuss.python.org/t/71594
Status: Accepted
Type: Standards Track
Created: 08-Jul-2024
Python-Version: 3.14
Post-History: `09-Aug-2024 <https://discuss.python.org/t/60408>`__,
              `17-Oct-2024 <https://discuss.python.org/t/60408/201>`__,
              `21-Oct-2024 <https://discuss.python.org/t/60408/226>`__,
              `18-Nov-2024 <https://discuss.python.org/t/71594>`__,
Resolution: `10-Apr-2025 <https://discuss.python.org/t/71594/130>`__

Abstract
========

This PEP introduces template strings for custom string processing.

Template strings are a generalization of f-strings, using a ``t`` in place of
the ``f`` prefix. Instead of evaluating to ``str``, t-strings evaluate to a new
type, ``Template``:

.. code-block:: python

    template: Template = t"Hello {name}"

Templates provide developers with access to the string and its interpolated
values *before* they are combined. This brings native flexible string
processing to the Python language and enables safety checks, web templating,
domain-specific languages, and more.


Relationship With Other PEPs
============================

Python introduced f-strings in Python 3.6 with :pep:`498`. The grammar was
then formalized in :pep:`701` which also lifted some restrictions. This PEP
is based on PEP 701.

At nearly the same time PEP 498 arrived, :pep:`501` was written to provide
"i-strings" -- that is, "interpolation template strings". The PEP was
deferred pending further experience with f-strings. Work on this PEP was
resumed by a different author in March 2023, introducing "t-strings" as template
literal strings, and built atop PEP 701.

The authors of this PEP consider it to be a generalization and simplification
of the updated work in PEP 501. (That PEP has also recently been updated to
reflect the new ideas in this PEP.)


Motivation
==========

Python f-strings are easy to use and very popular. Over time, however, developers
have encountered limitations that make them
`unsuitable for certain use cases <https://docs.djangoproject.com/en/5.1/ref/utils/#django.utils.html.format_html>`__.
In particular, f-strings provide no way to intercept and transform interpolated
values before they are combined into a final string.

As a result, incautious use of f-strings can lead to security vulnerabilities.
For example, a user executing a SQL query with :mod:`python:sqlite3`
may be tempted to use an f-string to embed values into their SQL expression,
which could lead to a `SQL injection attack <https://en.wikipedia.org/wiki/SQL_injection>`__.
Or, a developer building HTML may include unescaped user input in the string,
leading to a `cross-site scripting (XSS) <https://en.wikipedia.org/wiki/Cross-site_scripting>`__
vulnerability.

More broadly, the inability to transform interpolated values before they are
combined into a final string limits the utility of f-strings in more complex
string processing tasks.

Template strings address these problems by providing
developers with access to the string and its interpolated values.

For example, imagine we want to generate some HTML. Using template strings,
we can define an ``html()`` function that allows us to automatically sanitize
content:

.. code-block:: python

    evil = "<script>alert('evil')</script>"
    template = t"<p>{evil}</p>"
    assert html(template) == "<p>&lt;script&gt;alert('evil')&lt;/script&gt;</p>"

Likewise, our hypothetical ``html()`` function can make it easy for developers
to add attributes to HTML elements using a dictionary:

.. code-block:: python

    attributes = {"src": "shrubbery.jpg", "alt": "looks nice"}
    template = t"<img {attributes} />"
    assert html(template) == '<img src="shrubbery.jpg" alt="looks nice" />'

Neither of these examples is possible with f-strings. By providing a
mechanism to intercept and transform interpolated values, template strings
enable a wide range of string processing use cases.


Specification
=============

Template String Literals
------------------------

This PEP introduces a new string prefix, ``t``, to define template string literals.
These literals resolve to a new type, ``Template``, found in the standard library
module :mod:`!string.templatelib`.

The following code creates a ``Template`` instance:

.. code-block:: python

    from string.templatelib import Template
    template = t"This is a template string."
    assert isinstance(template, Template)

Template string literals support the full syntax of :pep:`701`. This includes
the ability to nest template strings within interpolations, as well as the ability
to use all valid quote marks (``'``, ``"``, ``'''``, and ``"""``). Like other string
prefixes, the ``t`` prefix must immediately precede the quote. Like f-strings,
both lowercase ``t`` and uppercase ``T`` prefixes are supported. Like
f-strings, t-strings may not be combined with ``u`` or the ``b`` prefix.

Additionally, f-strings and t-strings cannot be combined, so the ``ft``
prefix is invalid. t-strings *may* be combined with the ``r`` prefix;
see the `Raw Template Strings`_ section below for more information.


The ``Template`` Type
---------------------

Template strings evaluate to an instance of a new immutable type,
:class:`!string.templatelib.Template`:

.. code-block:: python

    class Template:
        strings: tuple[str, ...]
        """
        A non-empty tuple of the string parts of the template,
        with N+1 items, where N is the number of interpolations
        in the template.
        """

        interpolations: tuple[Interpolation, ...]
        """
        A tuple of the interpolation parts of the template.
        This will be an empty tuple if there are no interpolations.
        """

        def __new__(cls, *args: str | Interpolation):
            """
            Create a new Template instance.

            Arguments can be provided in any order.
            """
            ...

        @property
        def values(self) -> tuple[object, ...]:
            """
            Return a tuple of the `value` attributes of each Interpolation
            in the template.
            This will be an empty tuple if there are no interpolations.
            """
            ...

        def __iter__(self) -> Iterator[str | Interpolation]:
            """
            Iterate over the string parts and interpolations in the template.

            These may appear in any order. Empty strings will not be included.
            """
            ...

The ``strings`` and ``interpolations`` attributes provide access to the string
parts and any interpolations in the literal:

.. code-block:: python

    name = "World"
    template = t"Hello {name}"
    assert template.strings[0] == "Hello "
    assert template.interpolations[0].value == "World"


The ``Interpolation`` Type
--------------------------

The ``Interpolation`` type represents an expression inside a template string.
Like ``Template``, it is a new class found in the :mod:`!string.templatelib` module:

.. code-block:: python

    class Interpolation:
        value: object
        expression: str
        conversion: Literal["a", "r", "s"] | None
        format_spec: str

        __match_args__ = ("value", "expression", "conversion", "format_spec")

        def __new__(
            cls,
            value: object,
            expression: str,
            conversion: Literal["a", "r", "s"] | None = None,
            format_spec: str = "",
        ):
            ...

The ``Interpolation`` type is shallow immutable. Its attributes
cannot be reassigned.

The ``value`` attribute is the evaluated result of the interpolation:

.. code-block:: python

    name = "World"
    template = t"Hello {name}"
    ass


id: pep-0585.rst
content:
PEP: 585
Title: Type Hinting Generics In Standard Collections
Author: Åukasz Langa <lukasz@python.org>
Discussions-To: typing-sig@python.org
Status: Final
Type: Standards Track
Topic: Typing
Created: 03-Mar-2019
Python-Version: 3.9
Resolution: https://mail.python.org/archives/list/python-dev@python.org/thread/HW2NFOEMCVCTAFLBLC3V7MLM6ZNMKP42/

.. canonical-doc:: :ref:`python:types-genericalias`
                   and the documentation for :meth:`~object.__class_getitem__`

Abstract
========

Static typing as defined by PEPs 484, 526, 544, 560, and 563 was built
incrementally on top of the existing Python runtime and constrained by
existing syntax and runtime behavior.  This led to the existence of
a duplicated collection hierarchy in the ``typing`` module due to
generics (for example ``typing.List`` and the built-in ``list``).

This PEP proposes to enable support for the generics syntax in all
standard collections currently available in the ``typing`` module.


Rationale and Goals
===================

This change removes the necessity for a parallel type hierarchy in the
``typing`` module, making it easier for users to annotate their programs
and easier for teachers to teach Python.


Terminology
===========

Generic (n.) -- a type that can be parameterized, typically a container.
Also known as a *parametric type* or a *generic type*.  For example:
``dict``.

parameterized generic -- a specific instance of a generic with the
expected types for container elements provided.  Also known as
a *parameterized type*.  For example: ``dict[str, int]``.


Backwards compatibility
=======================

Tooling, including type checkers and linters, will have to be adapted to
recognize standard collections as generics.

On the source level, the newly described functionality requires
Python 3.9.  For use cases restricted to type annotations, Python files
with the "annotations" future-import (available since Python 3.7) can
parameterize standard collections, including builtins.  To reiterate,
that depends on the external tools understanding that this is valid.

Implementation
==============

Starting with Python 3.7, when ``from __future__ import annotations`` is
used, function and variable annotations can parameterize standard
collections directly.  Example::

    from __future__ import annotations

    def find(haystack: dict[str, list[int]]) -> int:
        ...

Usefulness of this syntax before :pep:`585` is limited as external tooling
like Mypy does not recognize standard collections as generic.  Moreover,
certain features of typing like type aliases or casting require putting
types outside of annotations, in runtime context.  While these are
relatively less common than type annotations, it's important to allow
using the same type syntax in all contexts.  This is why starting with
Python 3.9, the following collections become generic using
``__class_getitem__()`` to parameterize contained types:

* ``tuple``  # typing.Tuple
* ``list``  # typing.List
* ``dict``  # typing.Dict
* ``set``  # typing.Set
* ``frozenset``  # typing.FrozenSet
* ``type``  # typing.Type
* ``collections.deque``
* ``collections.defaultdict``
* ``collections.OrderedDict``
* ``collections.Counter``
* ``collections.ChainMap``
* ``collections.abc.Awaitable``
* ``collections.abc.Coroutine``
* ``collections.abc.AsyncIterable``
* ``collections.abc.AsyncIterator``
* ``collections.abc.AsyncGenerator``
* ``collections.abc.Iterable``
* ``collections.abc.Iterator``
* ``collections.abc.Generator``
* ``collections.abc.Reversible``
* ``collections.abc.Container``
* ``collections.abc.Collection``
* ``collections.abc.Callable``
* ``collections.abc.Set``  # typing.AbstractSet
* ``collections.abc.MutableSet``
* ``collections.abc.Mapping``
* ``collections.abc.MutableMapping``
* ``collections.abc.Sequence``
* ``collections.abc.MutableSequence``
* ``collections.abc.ByteString``
* ``collections.abc.MappingView``
* ``collections.abc.KeysView``
* ``collections.abc.ItemsView``
* ``collections.abc.ValuesView``
* ``contextlib.AbstractContextManager``  # typing.ContextManager
* ``contextlib.AbstractAsyncContextManager``  # typing.AsyncContextManager
* ``re.Pattern``  # typing.Pattern, typing.re.Pattern
* ``re.Match``  # typing.Match, typing.re.Match

Importing those from ``typing`` is deprecated.  Due to :pep:`563` and the
intention to minimize the runtime impact of typing, this deprecation
will not generate DeprecationWarnings.  Instead, type checkers may warn
about such deprecated usage when the target version of the checked
program is signalled to be Python 3.9 or newer.  It's recommended to
allow for those warnings to be silenced on a project-wide basis.

The deprecated functionality may eventually be removed from the ``typing``
module. Removal will occur no sooner than Python 3.9's end of life,
scheduled for October 2025.


Parameters to generics are available at runtime
-----------------------------------------------

Preserving the generic type at runtime enables introspection of the type
which can be used for API generation or runtime type checking.  Such
usage is already present in the wild.

Just like with the ``typing`` module today, the parameterized generic
types listed in the previous section all preserve their type parameters
at runtime::

    >>> list[str]
    list[str]
    >>> tuple[int, ...]
    tuple[int, ...]
    >>> ChainMap[str, list[str]]
    collections.ChainMap[str, list[str]]

This is implemented using a thin proxy type that forwards all method
calls and attribute accesses to the bare origin type with the following
exceptions:

* the ``__repr__`` shows the parameterized type;
* the ``__origin__`` attribute points at the non-parameterized
  generic class;
* the ``__args__`` attribute is a tuple (possibly of length
  1) of generic types passed to the original ``__class_getitem__``;
* the ``__parameters__`` attribute is a lazily computed tuple
  (possibly empty) of unique type variables found in ``__args__``;
* the ``__getitem__`` raises an exception to disallow mistakes
  like ``dict[str][str]``.  However it allows e.g. ``dict[str, T][int]``
  and in that case returns ``dict[str, int]``.

This design means that it is possible to create instances of
parameterized collections, like::

    >>> l = list[str]()
    []
    >>> list is list[str]
    False
    >>> list == list[str]
    False
    >>> list[str] == list[str]
    True
    >>> list[str] == list[int]
    False
    >>> isinstance([1, 2, 3], list[str])
    TypeError: isinstance() arg 2 cannot be a parameterized generic
    >>> issubclass(list, list[str])
    TypeError: issubclass() arg 2 cannot be a parameterized generic
    >>> isinstance(list[str], types.GenericAlias)
    True

Objects created with bare types and parameterized types are exactly the
same.  The generic parameters are not preserved in instances created
with parameterized types, in other words generic types erase type
parameters during object creation.

One important consequence of this is that the interpreter does **not**
attempt to type check operations on the collection created with
a parameterized type.  This provides symmetry between::

    l: list[str] = []

and::

    l = list[str]()

For accessing the proxy type from Python code, it will be exported
from the ``types`` module as ``GenericAlias``.

Pickling or (shallow- or deep-) copying a ``GenericAlias`` instance
will preserve the type, origin, attributes and parameters.


Forward compatibility
---------------------

Future standard collections must implement the same behavior.


Reference implementation
========================

A proof-of-concept or prototype `implementation
<https://bugs.python.org/issue39481>`__ exists.


Rejected alternatives
=====================

Do nothing
----------

Keeping the status quo forces Python programmers to perform book-keeping
of imports from the ``typing`` module for standard collections, making
all but the simplest annotations cumbersome to maintain.  The existence
of parallel t


id: pep-0591.rst
content:
PEP: 591
Title: Adding a final qualifier to typing
Author: Michael J. Sullivan <sully@msully.net>, Ivan Levkivskyi <levkivskyi@gmail.com>
BDFL-Delegate: Guido van Rossum <guido@python.org>
Discussions-To: typing-sig@python.org
Status: Final
Type: Standards Track
Topic: Typing
Created: 15-Mar-2019
Python-Version: 3.8
Post-History:
Resolution: https://mail.python.org/archives/list/typing-sig@python.org/message/FDO4KFYWYQEP3U2HVVBEBR3SXPHQSHYR/

.. canonical-typing-spec:: :ref:`typing:at-final`/:py:func:`@typing.final <typing.final>`
                           and :ref:`typing:uppercase-final`/:py:data:`typing.Final`

Abstract
========

This PEP proposes a "final" qualifier to be added to the ``typing``
module---in the form of a ``final`` decorator and a ``Final`` type
annotation---to serve three related purposes:

* Declaring that a method should not be overridden
* Declaring that a class should not be subclassed
* Declaring that a variable or attribute should not be reassigned


Motivation
==========

The ``final`` decorator
-----------------------
The current ``typing`` module lacks a way to restrict the use of
inheritance or overriding at a typechecker level. This is a common
feature in other object-oriented languages (such as Java), and is
useful for reducing the potential space of behaviors of a class,
easing reasoning.

Some situations where a final class or method may be useful include:

* A class wasnât designed to be subclassed or a method wasn't designed
  to be overridden. Perhaps it would not work as expected, or be
  error-prone.
* Subclassing or overriding would make code harder to understand or
  maintain. For example, you may want to prevent unnecessarily tight
  coupling between base classes and subclasses.
* You want to retain the freedom to arbitrarily change the class
  implementation in the future, and these changes might break
  subclasses.

The ``Final`` annotation
------------------------

The current ``typing`` module lacks a way to indicate that a variable
will not be assigned to. This is a useful feature in several
situations:

* Preventing unintended modification of module and class level
  constants and documenting them as constants in a checkable way.
* Creating a read-only attribute that may not be overridden by
  subclasses. (``@property`` can make an attribute read-only but
  does not prevent overriding)
* Allowing a name to be used in situations where ordinarily a literal
  is expected (for example as a field name for ``NamedTuple``, a tuple
  of types passed to ``isinstance``, or an argument to a function
  with arguments of ``Literal`` type (:pep:`586`)).

Specification
=============

The ``final`` decorator
-----------------------

The ``typing.final`` decorator is used to restrict the use of
inheritance and overriding.

A type checker should prohibit any class decorated with ``@final``
from being subclassed and any method decorated with ``@final`` from
being overridden in a subclass. The method decorator version may be
used with all of instance methods, class methods, static methods, and properties.

For example::

    from typing import final

    @final
    class Base:
        ...

    class Derived(Base):  # Error: Cannot inherit from final class "Base"
        ...

and::

    from typing import final

    class Base:
        @final
        def foo(self) -> None:
            ...

    class Derived(Base):
        def foo(self) -> None:  # Error: Cannot override final attribute "foo"
                                # (previously declared in base class "Base")
            ...


For overloaded methods, ``@final`` should be placed on the
implementation (or on the first overload, for stubs)::

   from typing import Any, overload

   class Base:
       @overload
       def method(self) -> None: ...
       @overload
       def method(self, arg: int) -> int: ...
       @final
       def method(self, x=None):
           ...

It is an error to use ``@final`` on a non-method function.

The ``Final`` annotation
------------------------

The ``typing.Final`` type qualifier is used to indicate that a
variable or attribute should not be reassigned, redefined, or overridden.

Syntax
~~~~~~

``Final`` may be used in one of several forms:

* With an explicit type, using the syntax ``Final[<type>]``. Example::

    ID: Final[float] = 1

* With no type annotation. Example::

    ID: Final = 1

  The typechecker should apply its usual type inference mechanisms to
  determine the type of ``ID`` (here, likely, ``int``). Note that unlike for
  generic classes this is *not* the same as ``Final[Any]``.

* In class bodies and stub files you can omit the right hand side and just write
  ``ID: Final[float]``.  If the right hand side is omitted, there must
  be an explicit type argument to ``Final``.

* Finally, as ``self.id: Final = 1`` (also optionally with a type in
  square brackets). This is allowed *only* in ``__init__`` methods, so
  that the final instance attribute is assigned only once when an
  instance is created.


Semantics and examples
~~~~~~~~~~~~~~~~~~~~~~

The two main rules for defining a final name are:

* There can be *at most one* final declaration per module or class for
  a given attribute. There can't be separate class-level and instance-level
  constants with the same name.

* There must be *exactly one* assignment to a final name.

This means a type checker should prevent further assignments to final
names in type-checked code::

   from typing import Final

   RATE: Final = 3000

   class Base:
       DEFAULT_ID: Final = 0

   RATE = 300  # Error: can't assign to final attribute
   Base.DEFAULT_ID = 1  # Error: can't override a final attribute

Note that a type checker need not allow ``Final`` declarations inside loops
since the runtime will see multiple assignments to the same variable in
subsequent iterations.

Additionally, a type checker should prevent final attributes from
being overridden in a subclass::

   from typing import Final

   class Window:
       BORDER_WIDTH: Final = 2.5
       ...

   class ListView(Window):
       BORDER_WIDTH = 3  # Error: can't override a final attribute

A final attribute declared in a class body without an initializer must
be initialized in the ``__init__`` method (except in stub files)::

   class ImmutablePoint:
       x: Final[int]
       y: Final[int]  # Error: final attribute without an initializer

       def __init__(self) -> None:
           self.x = 1  # Good

Type checkers should infer a final attribute that is initialized in
a class body as being a class variable. Variables should not be annotated
with both ``ClassVar`` and ``Final``.

``Final`` may only be used as the outermost type in assignments or variable
annotations. Using it in any other position is an error. In particular,
``Final`` can't be used in annotations for function arguments::

   x: List[Final[int]] = []  # Error!

   def fun(x: Final[List[int]]) ->  None:  # Error!
       ...

Note that declaring a name as final only guarantees that the name will
not be re-bound to another value, but does not make the value
immutable. Immutable ABCs and containers may be used in combination
with ``Final`` to prevent mutating such values::

   x: Final = ['a', 'b']
   x.append('c')  # OK

   y: Final[Sequence[str]] = ['a', 'b']
   y.append('x')  # Error: "Sequence[str]" has no attribute "append"
   z: Final = ('a', 'b')  # Also works


Type checkers should treat uses of a final name that was initialized
with a literal as if it was replaced by the literal. For example, the
following should be allowed::

   from typing import NamedTuple, Final

   X: Final = "x"
   Y: Final = "y"
   N = NamedTuple("N", [(X, int), (Y, int)])


Reference Implementation
========================

The mypy [#mypy]_ type checker supports ``Final`` and ``final``. A
reference implementation of the runtime component is provided in the
``typing_extensions`` [#typing_extensions]_ module.


Rejected/deferred Ideas
=======================


id: pep-0220.rst
content:
PEP: 220
Title: Coroutines, Generators, Continuations
Author: Gordon McMillan <gmcm@hypernet.com>
Status: Rejected
Type: Informational
Created: 14-Aug-2000
Post-History:

.. rejected::

Abstract
========

Demonstrates why the changes described in the stackless PEP are
desirable.  A low-level continuations module exists.  With it,
coroutines and generators and "green" threads can be written.  A
higher level module that makes coroutines and generators easy to
create is desirable (and being worked on).  The focus of this PEP
is on showing how coroutines, generators, and green threads can
simplify common programming problems.


id: pep-0546.rst
content:
PEP: 546
Title: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
Author: Victor Stinner <vstinner@python.org>,
        Cory Benfield <cory@lukasa.co.uk>,
BDFL-Delegate: Benjamin Peterson <benjamin@python.org>
Status: Rejected
Type: Standards Track
Created: 30-May-2017
Python-Version: 2.7
Post-History: 23-May-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-June/148301.html


Abstract
========

Backport the ssl.MemoryBIO and ssl.SSLObject classes from Python 3 to Python
2.7 to enhance the overall security of Python 2.7.


Rejection Notice
================

This PEP is rejected, see `Withdraw PEP 546? Backport ssl.MemoryBIO and
ssl.SSLObject to Python 2.7
<https://mail.python.org/pipermail/python-dev/2018-May/153760.html>`_
discussion for the rationale.


Rationale
=========

While Python 2.7 is getting closer to its end-of-support date (scheduled for
2020), it is still used on production systems and the Python community is still
responsible for its security. This PEP will help facilitate the future adoption
of :pep:`543` across all supported Python versions, which will improve security
for both Python 2 and Python 3 users.

This PEP does NOT propose a general exception for backporting new
features to Python 2.7 - every new feature proposed for backporting will
still need to be justified independently. In particular, it will need to
be explained why relying on an independently updated backport on the
Python Package Index instead is not an acceptable solution.


PEP 543
-------

:pep:`543` defines a new TLS API for Python which would enhance Python
security by giving Python applications access to the native TLS implementations
on Windows and macOS, instead of using OpenSSL. A side effect is that it gives
access to the system trust store and certificates installed
locally by system administrators, enabling Python applications to use "company
certificates" without having to modify each application and so to correctly
validate TLS certificates (instead of having to ignore or bypass TLS
certificate validation).

For practical reasons, Cory Benfield would like to first implement an
I/O-less class similar to ssl.MemoryBIO and ssl.SSLObject for
:pep:`543`, and to provide a second class based on the first one to use
sockets or file descriptors.  This design would help to structure the code
to support more backends and simplify testing and auditing, as well as
implementation. Later, optimized classes using directly sockets or file
descriptors may be added for performance.

While :pep:`543` defines an API, the PEP would only make sense if it
comes with at least one complete and good implementation. The first
implementation would ideally be based on the ``ssl`` module of the Python
standard library, as this is shipped to all users by default and can be used as
a fallback implementation in the absence of anything more targeted.

If this backport is not performed, the only baseline implementation that could
be used would be pyOpenSSL. This is problematic, however, because of the
interaction with pip, which is shipped with CPython on all supported versions.


requests, pip and ensurepip
---------------------------

There are plans afoot to look at moving Requests to a more event-loop-y
model. The Requests team does not feel at this time it is possible to abandon
support for Python 2.7, so doing so would require using either Twisted or
Tornado, or writing their own asynchronous abstraction.

For asynchronous code, a MemoryBIO provides substantial advantages over using a
wrapped socket. It reduces the amount of buffering that must be done, works on
IOCP-based reactors as well as select/poll based ones, and also greatly
simplifies the reactor and implementation code. For this reason, Requests is
disinclined to use a wrapped-socket-based implementation. In the absence of a
backport to Python 2.7, Requests is required to use the same solution that
Twisted does: namely, a mandatory dependency on  `pyOpenSSL
<https://pypi.python.org/pypi/pyOpenSSL>`_.

The `pip <https://pip.pypa.io/>`_ program has to embed all its
dependencies for practical reasons: namely, that it cannot rely on any other
installation method being present. Since pip depends on requests, it means
that it would have to embed a copy of pyOpenSSL. That would imply substantial
usability pain to install pip. Currently, pip doesn't support embedding
C extensions which must be compiled on each platform and so require a C
compiler.

Since Python 2.7.9, Python embeds a copy of pip both for default
installation and for use in virtual environments via the new ``ensurepip``
module. If pip ends up bundling PyOpenSSL, then CPython will end up
bundling PyOpenSSL. Only backporting ``ssl.MemoryBIO`` and
``ssl.SSLObject`` would avoid the need to embed pyOpenSSL, and would fix the
bootstrap issue (python -> ensurepip -> pip -> requests -> MemoryBIO).

This situation is less problematic than the barrier to adoption of :pep:`543`, as
naturally Requests does not have to move to an event loop model before it drops
support for Python 2.7. However, it does make it painful for Requests (and pip)
to embrace both asyncio and the ``async`` and ``await`` keywords for as long as
it continues to support Python 2.

Other Benefits
--------------

Adopting this PEP would have other smaller ecosystem benefits. For example,
Twisted would be able to reduce its dependency on third-party C extensions.
Additionally, the PyOpenSSL development team would like to sunset the module,
and this backport would free them up to do so in a graceful manner without
leaving their users in the lurch.

Each of these fringe benefits, while small, also provides value to the wider
Python ecosystem.


Concerns
========

There are some concerns that people have about this backport.

What About Old Python 2?
------------------------

A number of the Python 2 users in the world are not keeping pace with Python 2
releases. This is most usually because they are using LTS releases that are not
keeping pace with the minor releases of Python 2. These users would not be able
to use the MemoryBIO, and so projects concerned with Python 2 compatibility may
be unable to rely on the MemoryBIO being present on most of their user's
systems.

This concern is reasonable. How critical it is depends on the likelihood of
current users of Python 2 migrating to Python 3, or just trying to use the most
recent Python 2 release. Put another way, at some point libraries will want to
drop Python 2 support: the question is only whether a significant majority of
their Python 2 users have moved to whatever Python 2 release contains this
backport before they do so.

Ultimately, the authors of this PEP believe that the burden of this backport is
sufficiently minimal to justify backporting despite this concern. If it turns
out that migration to newer 2.7 releases is too slow, then the value of the
work will be minimal, but if the migration to newer 2.7 releases is anything
like reasonable then there will be substantial value gained.


Changes
=======

Add ``MemoryBIO`` and ``SSLObject`` classes to the ``ssl`` module of
Python 2.7.

The code will be backported and adapted from the master branch
(Python 3).

The backport also significantly reduced the size of the Python 2/Python
3 difference of the ``_ssl`` module, which make maintenance easier.


Links
=====

* :pep:`543`
* `[backport] ssl.MemoryBIO
  <https://bugs.python.org/issue22559>`_: Implementation of this PEP
  written by Alex Gaynor (first version written at October 2014)
* :pep:`466`


Discussions
===========

* `[Python-Dev] Backport ssl.MemoryBIO on Python 2.7?
  <https://mail.python.org/pipermail/python-dev/2017-May/147981.html>`_
  (May 2017)


Copyright
=========

This document has been placed in the public domain.


id: pep-0552.rst
content:
PEP: 552
Title: Deterministic pycs
Author: Benjamin Peterson <benjamin@python.org>
Status: Final
Type: Standards Track
Created: 04-Sep-2017
Python-Version: 3.7
Post-History: 07-Sep-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-September/149649.html


Abstract
========

This PEP proposes an extension to the pyc format to make it more deterministic.


Rationale
=========

A `reproducible build`_ is one where the same byte-for-byte output is generated
every time the same sources are builtâeven across different machines (naturally
subject to the requirement that they have rather similar environments
set up). Reproducibility is important for security. It is also a key concept in
content-based build systems such as Bazel_, which are most effective when the
output filesâ contents are a deterministic function of the input filesâ
contents.

The current Python pyc format is the marshaled code object of the module
prefixed by a `magic number`_, the source timestamp, and the source file
size. The presence of a source timestamp means that a pyc is not a deterministic
function of the input fileâs contentsâit also depends on volatile metadata, the
mtime of the source. Thus, pycs are a barrier to proper reproducibility.

Distributors of Python code are currently stuck with the options of

1. not distributing pycs and losing the caching advantages

2. distributing pycs and losing reproducibility

3. carefully giving all Python source files a deterministic timestamp
   (see, for example, https://github.com/python/cpython/pull/296)

4. doing a complicated mixture of 1. and 2. like generating pycs at installation
   time

None of these options are very attractive. This PEP proposes allowing the
timestamp to be replaced with a deterministic hash. The current timestamp
invalidation method will remain the default, though. Despite its nondeterminism,
timestamp invalidation works well for many workflows and usecases. The
hash-based pyc format can impose the cost of reading and hashing every source
file, which is more expensive than simply checking timestamps. Thus, for now, we
expect it to be used mainly by distributors and power use cases.

(Note there are other problems [#frozensets]_ [#interning]_ we do not
address here that can make pycs non-deterministic.)


Specification
=============

The pyc header currently consists of 3 32-bit words. We will expand it to 4. The
first word will continue to be the magic number, versioning the bytecode and pyc
format. The second word, conceptually the new word, will be a bit field. The
interpretation of the rest of the header and invalidation behavior of the pyc
depends on the contents of the bit field.

If the bit field is 0, the pyc is a traditional timestamp-based pyc. I.e., the
third and forth words will be the timestamp and file size respectively, and
invalidation will be done by comparing the metadata of the source file with that
in the header.

If the lowest bit of the bit field is set, the pyc is a hash-based pyc. We call
the second lowest bit the ``check_source`` flag. Following the bit field is a
64-bit hash of the source file. We will use a SipHash_ with a hardcoded key of
the contents of the source file. Another fast hash like MD5 or BLAKE2_ would
also work. We choose SipHash because Python already has a builtin implementation
of it from :pep:`456`, although an interface that allows picking the SipHash key
must be exposed to Python. Security of the hash is not a concern, though we pass
over completely-broken hashes like MD5 to ease auditing of Python in controlled
environments.

When Python encounters a hash-based pyc, its behavior depends on the setting of
the ``check_source`` flag. If the ``check_source`` flag is set, Python will
determine the validity of the pyc by hashing the source file and comparing the
hash with the expected hash in the pyc. If the pyc needs to be regenerated, it
will be regenerated as a hash-based pyc again with the ``check_source`` flag
set.

For hash-based pycs with the ``check_source`` unset, Python will simply load the
pyc without checking the hash of the source file. The expectation in this case
is that some external system (e.g., the local Linux distributionâs package
manager) is responsible for keeping pycs up to date, so Python itself doesnât
have to check. Even when validation is disabled, the hash field should be set
correctly, so out-of-band consistency checkers can verify the up-to-dateness of
the pyc. Note also that the :pep:`3147` edict that pycs without corresponding
source files not be loaded will still be enforced for hash-based pycs.

The programmatic APIs of ``py_compile`` and ``compileall`` will support
generation of hash-based pycs. Principally, ``py_compile`` will define a new
enumeration corresponding to all the available pyc invalidation modules::

  class PycInvalidationMode(Enum):
      TIMESTAMP
      CHECKED_HASH
      UNCHECKED_HASH

``py_compile.compile``, ``compileall.compile_dir``, and
``compileall.compile_file`` will all gain an ``invalidation_mode`` parameter,
which accepts a value of the ``PycInvalidationMode`` enumeration.

The ``compileall`` tool will be extended with a command new option,
``--invalidation-mode`` to generate hash-based pycs with and without the
``check_source`` bit set. ``--invalidation-mode`` will be a tristate option
taking values ``timestamp`` (the default), ``checked-hash``, and
``unchecked-hash`` corresponding to the values of ``PycInvalidationMode``.

``importlib.util`` will be extended with a ``source_hash(source)`` function that
computes the hash used by the pyc writing code for a bytestring **source**.

Runtime configuration of hash-based pyc invalidation will be facilitated by a
new ``--check-hash-based-pycs`` interpreter option. This is a tristate option,
which may take 3 values: ``default``, ``always``, and ``never``. The default
value, ``default``, means the ``check_source`` flag in hash-based pycs
determines invalidation as described above. ``always`` causes the interpreter to
hash the source file for invalidation regardless of value of ``check_source``
bit. ``never`` causes the interpreter to always assume hash-based pycs are
valid. When ``--check-hash-based-pycs=never`` is in effect, unchecked hash-based
pycs will be regenerated as unchecked hash-based pycs. Timestamp-based pycs are
unaffected by ``--check-hash-based-pycs``.


References
==========

.. _reproducible build: https://reproducible-builds.org/
.. _Bazel: https://bazel.build/
.. _BLAKE2: https://blake2.net/
.. _SipHash: https://131002.net/siphash/
.. [#frozensets] http://benno.id.au/blog/2013/01/15/python-determinism
.. [#interning] http://bugzilla.opensuse.org/show_bug.cgi?id=1049186
.. _magic number: https://docs.python.org/3/library/importlib.html#importlib.util.MAGIC_NUMBER


Credits
=======

The author would like to thank Gregory P. Smith, Christian Heimes, and Steve
Dower for useful conversations on the topic of this PEP.


Copyright
=========

This document has been placed in the public domain.


id: pep-0234.rst
content:
PEP: 234
Title: Iterators
Author: Ka-Ping Yee <ping@zesty.ca>, Guido van Rossum <guido@python.org>
Status: Final
Type: Standards Track
Created: 30-Jan-2001
Python-Version: 2.1
Post-History: 30-Apr-2001


Abstract
========

This document proposes an iteration interface that objects can provide to
control the behaviour of ``for`` loops.  Looping is customized by providing a
method that produces an iterator object.  The iterator provides a *get next
value* operation that produces the next item in the sequence each time it is
called, raising an exception when no more items are available.

In addition, specific iterators over the keys of a dictionary and over the
lines of a file are proposed, and a proposal is made to allow spelling
``dict.has_key(key)`` as ``key in dict``.

Note: this is an almost complete rewrite of this PEP by the second author,
describing the actual implementation checked into the trunk of the Python 2.2
CVS tree.  It is still open for discussion.  Some of the more esoteric
proposals in the original version of this PEP have been withdrawn for now;
these may be the subject of a separate PEP in the future.


C API Specification
===================

A new exception is defined, ``StopIteration``, which can be used to signal the
end of an iteration.

A new slot named ``tp_iter`` for requesting an iterator is added to the type
object structure.  This should be a function of one ``PyObject *`` argument
returning a ``PyObject *``, or ``NULL``.  To use this slot, a new C API
function ``PyObject_GetIter()`` is added, with the same signature as the
``tp_iter`` slot function.

Another new slot, named ``tp_iternext``, is added to the type structure, for
obtaining the next value in the iteration.  To use this slot, a new C API
function ``PyIter_Next()`` is added.  The signature for both the slot and the
API function is as follows, although the ``NULL`` return conditions differ:
the argument is a ``PyObject *`` and so is the return value.  When the return
value is non-``NULL``, it is the next value in the iteration.  When it is
``NULL``, then for the ``tp_iternext slot`` there are three possibilities:

- No exception is set; this implies the end of the iteration.

- The ``StopIteration`` exception (or a derived exception class) is set; this
  implies the end of the iteration.

- Some other exception is set; this means that an error occurred that should be
  propagated normally.

The higher-level ``PyIter_Next()`` function clears the ``StopIteration``
exception (or derived exception) when it occurs, so its ``NULL`` return
conditions are simpler:

- No exception is set; this means iteration has ended.

- Some exception is set; this means an error occurred, and should be propagated
  normally.

Iterators implemented in C should *not* implement a ``next()`` method with
similar semantics as the ``tp_iternext`` slot!  When the type's dictionary is
initialized (by ``PyType_Ready()``), the presence of a ``tp_iternext`` slot
causes a method ``next()`` wrapping that slot to be added to the type's
``tp_dict``.  (Exception: if the type doesn't use ``PyObject_GenericGetAttr()``
to access instance attributes, the ``next()`` method in the type's ``tp_dict``
may not be seen.)  (Due to a misunderstanding in the original text of this PEP,
in Python 2.2, all iterator types implemented a ``next()`` method that was
overridden by the wrapper; this has been fixed in Python 2.3.)

To ensure binary backwards compatibility, a new flag ``Py_TPFLAGS_HAVE_ITER``
is added to the set of flags in the ``tp_flags`` field, and to the default
flags macro.  This flag must be tested before accessing the ``tp_iter`` or
``tp_iternext`` slots.  The macro ``PyIter_Check()`` tests whether an object
has the appropriate flag set and has a non-``NULL`` ``tp_iternext`` slot.
There is no such macro for the ``tp_iter`` slot (since the only place where
this slot is referenced should be ``PyObject_GetIter()``, and this can check
for the ``Py_TPFLAGS_HAVE_ITER`` flag directly).

(Note: the ``tp_iter`` slot can be present on any object; the ``tp_iternext``
slot should only be present on objects that act as iterators.)

For backwards compatibility, the ``PyObject_GetIter()`` function implements
fallback semantics when its argument is a sequence that does not implement a
``tp_iter`` function: a lightweight sequence iterator object is constructed in
that case which iterates over the items of the sequence in the natural order.

The Python bytecode generated for ``for`` loops is changed to use new opcodes,
``GET_ITER`` and ``FOR_ITER``, that use the iterator protocol rather than the
sequence protocol to get the next value for the loop variable.  This makes it
possible to use a ``for`` loop to loop over non-sequence objects that support
the ``tp_iter`` slot.  Other places where the interpreter loops over the values
of a sequence should also be changed to use iterators.

Iterators ought to implement the ``tp_iter`` slot as returning a reference to
themselves; this is needed to make it possible to use an iterator (as opposed
to a sequence) in a ``for`` loop.

Iterator implementations (in C or in Python) should guarantee that once the
iterator has signalled its exhaustion, subsequent calls to ``tp_iternext`` or
to the ``next()`` method will continue to do so.  It is not specified whether
an iterator should enter the exhausted state when an exception (other than
``StopIteration``) is raised. Note that Python cannot guarantee that
user-defined or 3rd party iterators implement this requirement correctly.


Python API Specification
========================

The ``StopIteration`` exception is made visible as one of the standard
exceptions.  It is derived from ``Exception``.

A new built-in function is defined, ``iter()``, which can be called in two
ways:

- ``iter(obj)`` calls ``PyObject_GetIter(obj)``.

- ``iter(callable, sentinel)`` returns a special kind of iterator that calls
  the callable to produce a new value, and compares the return value to the
  sentinel value.  If the return value equals the sentinel, this signals the
  end of the iteration and ``StopIteration`` is raised rather than returning
  normal; if the return value does not equal the sentinel, it is returned as
  the next value from the iterator.  If the callable raises an exception, this
  is propagated normally; in particular, the function is allowed to raise
  ``StopIteration`` as an alternative way to end the iteration.  (This
  functionality is available from the C API as
  ``PyCallIter_New(callable, sentinel)``.)

Iterator objects returned by either form of ``iter()`` have a ``next()``
method.  This method either returns the next value in the iteration, or raises
``StopIteration`` (or a derived exception class) to signal the end of the
iteration.  Any other exception should be considered to signify an error and
should be propagated normally, not taken to mean the end of the iteration.

Classes can define how they are iterated over by defining an ``__iter__()``
method; this should take no additional arguments and return a valid iterator
object.  A class that wants to be an iterator should implement two methods: a
``next()`` method that behaves as described above, and an ``__iter__()`` method
that returns ``self``.

The two methods correspond to two distinct protocols:

1. An object can be iterated over with ``for`` if it implements ``__iter__()``
   or ``__getitem__()``.

2. An object can function as an iterator if it implements ``next()``.

Container-like objects usually support protocol 1.  Iterators are currently
required to support both protocols.  The semantics of iteration come only from
protocol 2; protocol 1 is present to make iterators behave like sequences; in
particular so that code receiving an iterator can use a for-loop over the
iterator.


Dictionary Iterators
====================

- Dictionaries implement a ``sq_contains`` slot that implements the same test
  as the ``has_key()`` method.  This means that we can write

  ::


id: pep-0208.rst
content:
PEP: 208
Title: Reworking the Coercion Model
Author: Neil Schemenauer <nas@arctrix.com>, Marc-AndrÃ© Lemburg <mal@lemburg.com>
Status: Final
Type: Standards Track
Created: 04-Dec-2000
Python-Version: 2.1
Post-History:


Abstract
========

Many Python types implement numeric operations.  When the arguments of
a numeric operation are of different types, the interpreter tries to
coerce the arguments into a common type.  The numeric operation is
then performed using this common type.  This PEP proposes a new type
flag to indicate that arguments to a type's numeric operations should
not be coerced.  Operations that do not support the supplied types
indicate it by returning a new singleton object.  Types which do not
set the type flag are handled in a backwards compatible manner.
Allowing operations handle different types is often simpler, more
flexible, and faster than having the interpreter do coercion.


Rationale
=========

When implementing numeric or other related operations, it is often
desirable to provide not only operations between operands of one type
only, e.g. integer + integer, but to generalize the idea behind the
operation to other type combinations as well, e.g. integer + float.

A common approach to this mixed type situation is to provide a method
of "lifting" the operands to a common type (coercion) and then use
that type's operand method as execution mechanism.  Yet, this strategy
has a few drawbacks:

* the "lifting" process creates at least one new (temporary)
  operand object,

* since the coercion method is not being told about the operation
  that is to follow, it is not possible to implement operation
  specific coercion of types,

* there is no elegant way to solve situations were a common type
  is not at hand, and

* the coercion method will always have to be called prior to the
  operation's method itself.

A fix for this situation is obviously needed, since these drawbacks
make implementations of types needing these features very cumbersome,
if not impossible.  As an example, have a look at the ``DateTime`` and
``DateTimeDelta`` [1]_ types, the first being absolute, the second
relative.  You can always add a relative value to an absolute one,
giving a new absolute value.  Yet, there is no common type which the
existing coercion mechanism could use to implement that operation.

Currently, ``PyInstance`` types are treated specially by the interpreter
in that their numeric methods are passed arguments of different types.
Removing this special case simplifies the interpreter and allows other
types to implement numeric methods that behave like instance types.
This is especially useful for extension types like ExtensionClass.


Specification
=============

Instead of using a central coercion method, the process of handling
different operand types is simply left to the operation.  If the
operation finds that it cannot handle the given operand type
combination, it may return a special singleton as indicator.

Note that "numbers" (anything that implements the number protocol, or
part of it) written in Python already use the first part of this
strategy - it is the C level API that we focus on here.

To maintain nearly 100% backward compatibility we have to be very
careful to make numbers that don't know anything about the new
strategy (old style numbers) work just as well as those that expect
the new scheme (new style numbers).  Furthermore, binary compatibility
is a must, meaning that the interpreter may only access and use new
style operations if the number indicates the availability of these.

A new style number is considered by the interpreter as such if and
only if it sets the type flag ``Py_TPFLAGS_CHECKTYPES``.  The main
difference between an old style number and a new style one is that the
numeric slot functions can no longer assume to be passed arguments of
identical type.  New style slots must check all arguments for proper
type and implement the necessary conversions themselves.  This may seem
to cause more work on the behalf of the type implementor, but is in
fact no more difficult than writing the same kind of routines for an
old style coercion slot.

If a new style slot finds that it cannot handle the passed argument
type combination, it may return a new reference of the special
singleton ``Py_NotImplemented`` to the caller.  This will cause the caller
to try the other operands operation slots until it finds a slot that
does implement the operation for the specific type combination.  If
none of the possible slots succeed, it raises a ``TypeError``.

To make the implementation easy to understand (the whole topic is
esoteric enough), a new layer in the handling of numeric operations is
introduced.  This layer takes care of all the different cases that need
to be taken into account when dealing with all the possible
combinations of old and new style numbers.  It is implemented by the
two static functions ``binary_op()`` and ``ternary_op()``, which are both
internal functions that only the functions in Objects/abstract.c
have access to.  The numeric API (``PyNumber_*``) is easy to adapt to
this new layer.

As a side-effect all numeric slots can be NULL-checked (this has to be
done anyway, so the added feature comes at no extra cost).


The scheme used by the layer to execute a binary operation is as
follows:

===  ===  =================================
v    w    Action taken
===  ===  =================================
new  new  v.op(v,w), w.op(v,w)
new  old  v.op(v,w), coerce(v,w), v.op(v,w)
old  new  w.op(v,w), coerce(v,w), v.op(v,w)
old  old  coerce(v,w), v.op(v,w)
===  ===  =================================

The indicated action sequence is executed from left to right until
either the operation succeeds and a valid result (!=
``Py_NotImplemented``) is returned or an exception is raised.  Exceptions
are returned to the calling function as-is.  If a slot returns
``Py_NotImplemented``, the next item in the sequence is executed.

Note that coerce(v,w) will use the old style ``nb_coerce`` slot methods
via a call to ``PyNumber_Coerce()``.

Ternary operations have a few more cases to handle:

===  ===  ===  ====================================================
v    w    z    Action taken
===  ===  ===  ====================================================
new  new  new  v.op(v,w,z), w.op(v,w,z), z.op(v,w,z)
new  old  new  v.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
old  new  new  w.op(v,w,z), z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
old  old  new  z.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
new  new  old  v.op(v,w,z), w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
new  old  old  v.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
old  new  old  w.op(v,w,z), coerce(v,w,z), v.op(v,w,z)
old  old  old  coerce(v,w,z), v.op(v,w,z)
===  ===  ===  ====================================================

The same notes as above, except that coerce(v,w,z) actually does::

    if z != Py_None:
        coerce(v,w), coerce(v,z), coerce(w,z)
    else:
        # treat z as absent variable
        coerce(v,w)


The current implementation uses this scheme already (there's only one
ternary slot: ``nb_pow(a,b,c))``.

Note that the numeric protocol is also used for some other related
tasks, e.g. sequence concatenation.  These can also benefit from the
new mechanism by implementing right-hand operations for type
combinations that would otherwise fail to work.  As an example, take
string concatenation: currently you can only do string + string.  With
the new mechanism, a new string-like type could implement new_type +
string and string + new_type, even though strings don't know anything
about new_type.

Since comparisons also rely on coercion (every time you compare an
integer to a float, the integer is first converted to float and then
compared...), a new slot to handle numeric comparisons is needed::

    PyObject *nb_cmp(PyObject *v, PyObject *w)

This slot should compare the two objects and return an integer object
stating the result.  Currently, this result intege


id: pep-0211.rst
content:
PEP: 211
Title: Adding A New Outer Product Operator
Author: Greg Wilson <gvwilson@ddj.com>
Status: Rejected
Type: Standards Track
Created: 15-Jul-2000
Python-Version: 2.1
Post-History:


.. rejected::

   The approach in the later :pep:`465` was eventually accepted
   in lieu of this PEP. The :pep:`Rejected Ideas
   <465#rejected-alternatives-to-adding-a-new-operator>`
   of that PEP explains the rationale in more detail.


Introduction
============

This PEP describes a proposal to define ``@`` (pronounced "across")
as a new outer product operator in Python 2.2.  When applied to
sequences (or other iterable objects), this operator will combine
their iterators, so that::

    for (i, j) in S @ T:
        pass

will be equivalent to::

    for i in S:
        for j in T:
            pass

Classes will be able to overload this operator using the special
methods ``__across__``, ``__racross__``, and ``__iacross__``.  In
particular, the new Numeric module (:pep:`209`) will overload this
operator for multi-dimensional arrays to implement matrix
multiplication.


Background
==========

Number-crunching is now just a small part of computing, but many
programmers --- including many Python users --- still need to
express complex mathematical operations in code.  Most numerical
languages, such as APL, Fortran-90, MATLAB, IDL, and Mathematica,
therefore provide two forms of the common arithmetic operators.
One form works element-by-element, e.g. multiplies corresponding
elements of its matrix arguments.  The other implements the
"mathematical" definition of that operation, e.g. performs
row-column matrix multiplication.

Zhu and Lielens have :pep:`proposed <225>` doubling up Python's operators in
this way.  Their proposal would create six new binary infix
operators, and six new in-place operators.

The original version of this proposal was much more conservative.
The author consulted the developers of GNU Octave [1]_, an open
source clone of MATLAB.  Its developers agreed that providing an
infix operator for matrix multiplication was important: numerical
programmers really do care whether they have to write ``mmul(A,B)``
instead of ``A op B``.

On the other hand, when asked how important it was to have infix
operators for matrix solution and other operations, Prof. James
Rawlings replied [2]_:

    I DON'T think it's a must have, and I do a lot of matrix
    inversion. I cannot remember if its ``A\b`` or ``b\A`` so I always
    write ``inv(A)*b`` instead. I recommend dropping ``\``.

Based on this discussion, and feedback from students at the US
national laboratories and elsewhere, we recommended adding only
one new operator, for matrix multiplication, to Python.


Iterators
=========

The planned addition of iterators to Python 2.2 opens up a broader
scope for this proposal.  As part of the discussion of :pep:`201`,
Lockstep Iteration, the author of this proposal conducted an
informal usability experiment [3]_.  The results showed that users
are psychologically receptive to "cross-product" loop syntax.  For
example, most users expected::

    S = [10, 20, 30]
    T = [1, 2, 3]
    for x in S; y in T:
        print x+y,

to print ``11 12 13 21 22 23 31 32 33``.  We believe that users will
have the same reaction to::

    for (x, y) in S @ T:
        print x+y

i.e. that they will naturally interpret this as a tidy way to
write loop nests.

This is where iterators come in.  Actually constructing the
cross-product of two (or more) sequences before executing the loop
would be very expensive.  On the other hand, ``@`` could be defined
to get its arguments' iterators, and then create an outer iterator
which returns tuples of the values returned by the inner
iterators.


Discussion
==========

1. Adding a named function "across" would have less impact on
   Python than a new infix operator.  However, this would not make
   Python more appealing to numerical programmers, who really do
   care whether they can write matrix multiplication using an
   operator, or whether they have to write it as a function call.

2. ``@`` would have be chainable in the same way as comparison
   operators, i.e.::

    (1, 2) @ (3, 4) @ (5, 6)

   would have to return ``(1, 3, 5) ... (2, 4, 6)``, and *not*
   ``((1, 3), 5) ... ((2, 4), 6)``.  This should not require special
   support from the parser, as the outer iterator created by the
   first ``@`` could easily be taught how to combine itself with
   ordinary iterators.

3. There would have to be some way to distinguish restartable
   iterators from ones that couldn't be restarted.  For example,
   if ``S`` is an input stream (e.g. a file), and ``L`` is a list, then ``S
   @ L`` is straightforward, but ``L @ S`` is not, since iteration
   through the stream cannot be repeated.  This could be treated
   as an error, or by having the outer iterator detect
   non-restartable inner iterators and cache their values.

4. Whiteboard testing of this proposal in front of three novice
   Python users (all of them experienced programmers) indicates
   that users will expect::

    "ab" @ "cd"

   to return four strings, not four tuples of pairs of
   characters.  Opinion was divided on what::

    ("a", "b") @ "cd"

   ought to return...


Alternatives
============

1. Do nothing --- keep Python simple.

   This is always the default choice.

2. Add a named function instead of an operator.

   Python is not primarily a numerical language; it may not be worth
   complexifying it for this special case.  However, support for real
   matrix multiplication *is* frequently requested, and the proposed
   semantics for ``@`` for built-in sequence types would simplify
   expression of a very common idiom (nested loops).

3. Introduce prefixed forms of all existing operators, such as
   ``~*`` and ``~+``, as proposed in :pep:`225`.

   Our objections to this are that there isn't enough demand to
   justify the additional complexity (see Rawlings' comments [2]_),
   and that the proposed syntax fails the "low toner" readability
   test.


Acknowledgments
===============

I am grateful to Huaiyu Zhu for initiating this discussion, and to
James Rawlings and students in various Python courses for their
discussions of what numerical programmers really care about.


References
==========

.. [1] http://bevo.che.wisc.edu/octave/

.. [2] http://www.egroups.com/message/python-numeric/4

.. [3] https://mail.python.org/pipermail/python-dev/2000-July/006427.html


id: pep-0577.rst
content:
PEP: 577
Title: Augmented Assignment Expressions
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Withdrawn
Type: Standards Track
Created: 14-May-2018
Python-Version: 3.8
Post-History: 22-May-2018


PEP Withdrawal
==============

While working on this PEP, I realised that it didn't really address what was
actually bothering me about :pep:`572`'s proposed scoping rules for previously
unreferenced assignment targets, and also had some significant undesirable
consequences (most notably, allowing ``>>=`` and ``<<=`` as inline augmented
assignment operators that meant something entirely different from the ``>=``
and ``<=`` comparison operators).

I also realised that even without dedicated syntax of their own, :pep:`572`
technically allows inline augmented assignments to be written using the
``operator`` module::

    from operator import iadd
    if (target := iadd(target, value)) < limit:
        ...

The restriction to simple names as inline assignment targets means that the
target expression can always be repeated without side effects, and thus avoids
the ambiguity that would arise from allowing actual embedded augmented
assignments (it's still a bad idea, since it would almost certainly be hard
for humans to read, this note is just about the theoretical limits of language
level expressiveness).

Accordingly, I withdrew this PEP without submitting it for pronouncement. At
the time I also started writing a replacement PEP that focused specifically on
the handling of assignment targets which hadn't already been declared as local
variables in the current scope (for both regular block scopes, and for scoped
expressions), but that draft never even reached a stage where *I* liked it
better than the ultimately accepted proposal in :pep:`572`, so it was never
posted anywhere, nor assigned a PEP number.


Abstract
========

This is a proposal to allow augmented assignments such as ``x += 1`` to be
used as expressions, not just statements.

As part of this, ``NAME := EXPR`` is proposed as an inline assignment expression
that uses the new augmented assignment scoping rules, rather than implicitly
defining a new local variable name the way that existing name binding
statements do. The question of allowing expression level local variable
declarations at function scope is deliberately separated from the question of
allowing expression level name bindings, and deferred to a later PEP.

This PEP is a direct competitor to :pep:`572` (although it borrows heavily from that
PEP's motivation, and even shares the proposed syntax for inline assignments).
See `Relationship with PEP 572`_ for more details on the connections between
the two PEPs.

To improve the usability of the new expressions, a semantic split is proposed
between the handling of augmented assignments in regular block scopes (modules,
classes, and functions), and the handling of augmented assignments in scoped
expressions (lambda expressions, generator expressions, and comprehensions),
such that all inline assignments default to targeting the nearest containing
block scope.

A new compile time ``TargetNameError`` is added as a subclass of ``SyntaxError``
to handle cases where it is deemed to be currently unclear which target is
expected to be rebound by an inline assignment, or else the target scope
for the inline assignment is considered invalid for another reason.


Syntax and semantics
====================

Augmented assignment expressions
--------------------------------

The language grammar would be adjusted to allow augmented assignments to
appear as expressions, where the result of the augmented assignment
expression is the same post-calculation reference as is being bound to the
given target.

For example::

    >>> n = 0
    >>> n += 5
    5
    >>> n -= 2
    3
    >>> n *= 3
    9
    >>> n
    9

For mutable targets, this means the result is always just the original object::

    >>> seq = []
    >>> seq_id = id(seq)
    >>> seq += range(3)
    [0, 1, 2]
    >>> seq_id == id(seq)
    True

Augmented assignments to attributes and container subscripts will be permitted,
with the result being the post-calculation reference being bound to the target,
just as it is for simple name targets::

    def increment(self, step=1):
        return self._value += step

In these cases, ``__getitem__`` and ``__getattribute__`` will *not* be called
after the assignment has already taken place (they will only be called as
needed to evaluate the in-place operation).


Adding an inline assignment operator
------------------------------------

Given only the addition of augmented assignment expressions, it would be
possible to abuse a symbol like ``|=`` as a general purpose assignment
operator by defining a ``Target`` wrapper type that worked as follows::

    >>> class Target:
    ...     def __init__(self, value):
    ...         self.value = value
    ...     def __or__(self, other):
    ...         return Target(other)
    ...
    >>> x = Target(10)
    >>> x.value
    10
    >>> x |= 42
    <__main__.Target object at 0x7f608caa8048>
    >>> x.value
    42

This is similar to the way that storing a single reference in a list was long
used as a workaround for the lack of a ``nonlocal`` keyword, and can still be
used today (in combination with ``operator.itemsetter``) to work around the
lack of expression level assignments.

Rather than requiring such workarounds, this PEP instead proposes that
:pep:`572`'s "NAME := EXPR" syntax be adopted as a new inline assignment
expression that uses the augmented assignment scoping rules described below.

This cleanly handles cases where only the new value is of interest, and the
previously bound value (if any) can just be discarded completely.

Note that for both simple names and complex assignment targets, the inline
assignment operator does *not* read the previous reference before assigning
the new one. However, when used at function scope (either directly or inside
a scoped expression), it does *not* implicitly define a new local variable,
and will instead raise ``TargetNameError`` (as described for augmented
assignments below).


Assignment operator precedence
------------------------------

To preserve the existing semantics of augmented assignment statements,
inline assignment operators will be defined as being of lower precedence
than all other operators, include the comma pseudo-operator. This ensures
that when used as a top level expression the entire right hand side of the
expression is still interpreted as the value to be processed (even when that
value is a tuple without parentheses).

The difference this introduces relative to :pep:`572` is that where
``(n := first, second)`` sets ``n = first`` in :pep:`572`, in this PEP it would set
``n = (first, second)``, and getting the first meaning would require an extra
set of parentheses (``((n := first), second)``).

:pep:`572` quite reasonably notes that this results in ambiguity when assignment
expressions are used as function call arguments. This PEP resolves that concern
a different way by requiring that assignment expressions be parenthesised
when used as arguments to a function call (unless they're the sole argument).

This is a more relaxed version of the restriction placed on generator
expressions (which always require parentheses, except when they're the sole
argument to a function call).


Augmented assignment to names in block scopes
---------------------------------------------

No target name binding changes are proposed for augmented assignments at module
or class scope (this also includes code executed using "exec" or "eval"). These
will continue to implicitly declare a new local variable as the binding target
as they do today, and (if necessary) will be able to resolve the name from an
outer scope before binding it locally.

At function scope, augmented assignments will be changed to require that there
be either a preceding name binding or variable declaration to explicitly
establish the target name as


id: pep-0563.rst
content:
PEP: 563
Title: Postponed Evaluation of Annotations
Author: Åukasz Langa <lukasz@python.org>
Discussions-To: python-dev@python.org
Status: Superseded
Type: Standards Track
Topic: Typing
Created: 08-Sep-2017
Python-Version: 3.7
Post-History: 01-Nov-2017, 21-Nov-2017
Superseded-By: 649, 749
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151042.html


Resolution
==========

The features proposed in this PEP never became the default behaviour,
and have been replaced with deferred evaluation of annotations,
as proposed by :pep:`649` and :pep:`749`.


Abstract
========

:pep:`3107` introduced syntax for function annotations, but the semantics
were deliberately left undefined.  :pep:`484` introduced a standard meaning
to annotations: type hints.  :pep:`526` defined variable annotations,
explicitly tying them with the type hinting use case.

This PEP proposes changing function annotations and variable annotations
so that they are no longer evaluated at function definition time.
Instead, they are preserved in ``__annotations__`` in string form.

This change is being introduced gradually, starting with a
``__future__`` import in Python 3.7.


Rationale and Goals
===================

:pep:`3107` added support for arbitrary annotations on parts of a function
definition.  Just like default values, annotations are evaluated at
function definition time.  This creates a number of issues for the type
hinting use case:

* forward references: when a type hint contains names that have not been
  defined yet, that definition needs to be expressed as a string
  literal;

* type hints are executed at module import time, which is not
  computationally free.

Postponing the evaluation of annotations solves both problems.
NOTE: :pep:`649` proposes an alternative solution to the above issues,
putting this PEP in danger of being superseded.

Non-goals
---------

Just like in :pep:`484` and :pep:`526`, it should be emphasized that **Python
will remain a dynamically typed language, and the authors have no desire
to ever make type hints mandatory, even by convention.**

This PEP is meant to solve the problem of forward references in type
annotations.  There are still cases outside of annotations where
forward references will require usage of string literals.  Those are
listed in a later section of this document.

Annotations without forced evaluation enable opportunities to improve
the syntax of type hints.  This idea will require its own separate PEP
and is not discussed further in this document.

Non-typing usage of annotations
-------------------------------

While annotations are still available for arbitrary use besides type
checking, it is worth mentioning that the design of this PEP, as well
as its precursors (:pep:`484` and :pep:`526`), is predominantly motivated by
the type hinting use case.

In Python 3.8 :pep:`484` will graduate from provisional status.  Other
enhancements to the Python programming language like :pep:`544`, :pep:`557`,
or :pep:`560`, are already being built on this basis as they depend on
type annotations and the ``typing`` module as defined by :pep:`484`.
In fact, the reason :pep:`484` is staying provisional in Python 3.7 is to
enable rapid evolution for another release cycle that some of the
aforementioned enhancements require.

With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.


Implementation
==============

With this PEP, function and variable annotations will no longer be
evaluated at definition time.  Instead, a string form will be preserved
in the respective ``__annotations__`` dictionary.  Static type checkers
will see no difference in behavior, whereas tools using annotations at
runtime will have to perform postponed evaluation.

The string form is obtained from the AST during the compilation step,
which means that the string form might not preserve the exact formatting
of the source.  Note: if an annotation was a string literal already, it
will still be wrapped in a string.

Annotations need to be syntactically valid Python expressions, also when
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
Annotations can only use names present in the module scope as postponed
evaluation using local names is not reliable (with the sole exception of
class-level names resolved by ``typing.get_type_hints()``).

Note that as per :pep:`526`, local variable annotations are not evaluated
at all since they are not accessible outside of the function's closure.

Enabling the future behavior in Python 3.7
------------------------------------------

The functionality described above can be enabled starting from Python
3.7 using the following special import::

    from __future__ import annotations

A reference implementation of this functionality is available
`on GitHub <https://github.com/python/cpython/pull/4390>`_.


Resolving Type Hints at Runtime
===============================

To resolve an annotation at runtime from its string form to the result
of the enclosed expression, user code needs to evaluate the string.

For code that uses type hints, the
``typing.get_type_hints(obj, globalns=None, localns=None)`` function
correctly evaluates expressions back from its string form.  Note that
all valid code currently using ``__annotations__`` should already be
doing that since a type annotation can be expressed as a string literal.

For code which uses annotations for other purposes, a regular
``eval(ann, globals, locals)`` call is enough to resolve the
annotation.

In both cases it's important to consider how globals and locals affect
the postponed evaluation.  An annotation is no longer evaluated at the
time of definition and, more importantly, *in the same scope* where it
was defined.  Consequently, using local state in annotations is no
longer possible in general.  As for globals, the module where the
annotation was defined is the correct context for postponed evaluation.

The ``get_type_hints()`` function automatically resolves the correct
value of ``globalns`` for functions and classes.  It also automatically
provides the correct ``localns`` for classes.

When running ``eval()``,
the value of globals can be gathered in the following way:

* function objects hold a reference to their respective globals in an
  attribute called ``__globals__``;

* classes hold the name of the module they were defined in, this can be
  used to retrieve the respective globals::

    cls_globals = vars(sys.modules[SomeClass.__module__])

  Note that this needs to be repeated for base classes to evaluate all
  ``__annotations__``.

* modules should use their own ``__dict__``.

The value of ``localns`` cannot be reliably retrieved for functions
because in all likelihood the stack frame at the time of the call no
longer exists.

For classes, ``localns`` can be composed by chaining vars of the given
class and its base classes (in the method resolution order).  Since slots
can only be filled after the class was defined, we don't need to consult
them for this purpose.

Runtime annotation resolution and class decorators
--------------------------------------------------

Metaclasses and class decorators that need to resolve annotations for
the current class will fail for annotations that use the name of the
current class.  Example::

    def class_decorator(cls):
        annotations = get_type_hints(cls)  # raises NameError on 'C'
        print(f'Annotations for {cls}: {annotations}')
        return cls

    @class_decorator
    class C:
        singleton: 'C' = None

This was already true before this PEP.  The class decorator acts on
the class before it's assigned a name in the current definition scope.

Runtime annotation resolution and ``TYPE_CHECKING``
---------------------------------------------------

Sometimes there's code that must be seen by a type checker but should
not be executed.  For such situations the ``typing`` module defines a
constant, ``TYPE_CHECKING``, that is


id: pep-0205.rst
content:
PEP: 205
Title: Weak References
Author: Fred L. Drake, Jr. <fred@fdrake.net>
Status: Final
Type: Standards Track
Created: 14-Jul-2000
Python-Version: 2.1
Post-History: 11-Jan-2001

Motivation
==========

There are two basic applications for weak references which have
been noted by Python programmers: object caches and reduction of
pain from circular references.

Caches (weak dictionaries)
--------------------------

There is a need to allow objects to be maintained that represent
external state, mapping a single instance to the external
reality, where allowing multiple instances to be mapped to the
same external resource would create unnecessary difficulty
maintaining synchronization among instances.  In these cases,
a common idiom is to support a cache of instances; a factory
function is used to return either a new or existing instance.

The difficulty in this approach is that one of two things must
be tolerated: either the cache grows without bound, or there
needs to be explicit management of the cache elsewhere in the
application.  The later can be very tedious and leads to more
code than is really necessary to solve the problem at hand,
and the former can be unacceptable for long-running processes
or even relatively short processes with substantial memory
requirements.

- External objects that need to be represented by a single
  instance, no matter how many internal users there are.  This
  can be useful for representing files that need to be written
  back to disk in whole rather than locked & modified for
  every use.

- Objects that are expensive to create, but may be needed by
  multiple internal consumers.  Similar to the first case, but
  not necessarily bound to external resources, and possibly
  not an issue for shared state.  Weak references are only
  useful in this case if there is some flavor of "soft"
  references or if there is a high likelihood that users of
  individual objects will overlap in lifespan.

Circular references
-------------------

- DOMs require a huge amount of circular (to parent & document
  nodes) references, but these could be eliminated using a weak
  dictionary mapping from each node to its parent.  This
  might be especially useful in the context of something like
  ``xml.dom.pulldom``, allowing the ``.unlink()`` operation to become
  a no-op.

This proposal is divided into the following sections:

- Proposed Solution
- Implementation Strategy
- Possible Applications
- Previous Weak Reference Work in Python
- Weak References in Java

The full text of one early proposal is included as an appendix
since it does not appear to be available on the net.


Aspects of the Solution Space
=============================

There are two distinct aspects to the weak references problem:

- Invalidation of weak references
- Presentation of weak references to Python code

Invalidation
------------

Past approaches to weak reference invalidation have often hinged
on storing a strong reference and being able to examine all the
instances of weak reference objects, and invalidating them when
the reference count of their referent goes to one (indicating that
the reference stored by the weak reference is the last remaining
reference).  This has the advantage that the memory management
machinery in Python need not change, and that any type can be
weakly referenced.

The disadvantage of this approach to invalidation is that it
assumes that the management of the weak references is called
sufficiently frequently that weakly-referenced objects are noticed
within a reasonably short time frame; since this means a scan over
some data structure to invalidate references, an operation which
is O(N) on the number of weakly referenced objects, this is not
effectively amortized for any single object which is weakly
referenced.  This also assumes that the application is calling
into code which handles weakly-referenced objects with some
frequency, which makes weak-references less attractive for library
code.

An alternate approach to invalidation is that the de-allocation
code to be aware of the possibility of weak references and make a
specific call into the weak-reference management code to all
invalidation whenever an object is deallocated.  This requires a
change in the tp_dealloc handler for weakly-referencable objects;
an additional call is needed at the "top" of the handler for
objects which support weak-referencing, and an efficient way to
map from an object to a chain of weak references for that object
is needed as well.

Presentation
------------

Two ways that weak references are presented to the Python layer
have been as explicit reference objects upon which some operation
is required in order to retrieve a usable reference to the
underlying object, and proxy objects which masquerade as the
original objects as much as possible.

Reference objects are easy to work with when some additional layer
of object management is being added in Python; references can be
checked for liveness explicitly, without having to invoke
operations on the referents and catching some special exception
raised when an invalid weak reference is used.

However, a number of users favor the proxy approach simply because
the weak reference looks so much like the original object.


Proposed Solution
=================

Weak references should be able to point to any Python object that
may have substantial memory size (directly or indirectly), or hold
references to external resources (database connections, open
files, etc.).

A new module, weakref, will contain new functions used to create
weak references.  ``weakref.ref()`` will create a "weak reference
object" and optionally attach a callback which will be called when
the object is about to be finalized.  ``weakref.mapping()`` will
create a "weak dictionary".  A third function, ``weakref.proxy()``,
will create a proxy object that behaves somewhat like the original
object.

A weak reference object will allow access to the referenced object
if it hasn't been collected and to determine if the object still
exists in memory.  Retrieving the referent is done by calling the
reference object.  If the referent is no longer alive, this will
return None instead.

A weak dictionary maps arbitrary keys to values, but does not own
a reference to the values.  When the values are finalized, the
(key, value) pairs for which it is a value are removed from all
the mappings containing such pairs.  Like dictionaries, weak
dictionaries are not hashable.

Proxy objects are weak references that attempt to behave like the
object they proxy, as much as they can.  Regardless of the
underlying type, proxies are not hashable since their ability to
act as a weak reference relies on a fundamental mutability that
will cause failures when used as dictionary keys -- even if the
proper hash value is computed before the referent dies, the
resulting proxy cannot be used as a dictionary key since it cannot
be compared once the referent has expired, and comparability is
necessary for dictionary keys.  Operations on proxy objects after
the referent dies cause weakref.ReferenceError to be raised in
most cases.  "is" comparisons, ``type()``, and ``id()`` will continue to
work, but always refer to the proxy and not the referent.

The callbacks registered with weak references must accept a single
parameter, which will be the weak reference or proxy object
itself.  The object cannot be accessed or resurrected in the
callback.


Implementation Strategy
=======================

The implementation of weak references will include a list of
reference containers that must be cleared for each weakly-referencable
object.  If the reference is from a weak dictionary,
the dictionary entry is cleared first.  Then, any associated
callback is called with the object passed as a parameter.  Once
all callbacks have been called, the object is finalized and
deallocated.

Many built-in types will participate in the weak-reference
management, and any extension type can elect to d


id: pep-8103.rst
content:
PEP: 8103
Title: 2022 Term Steering Council election
Author: Ewa Jodlowska <ewa@python.org>, Ee Durbin <ee@python.org>, Joe Carey <joe@python.org>
Sponsor: Barry Warsaw <barry@python.org>
Status: Final
Type: Informational
Topic: Governance
Created: 04-Oct-2021


Abstract
========

This document describes the schedule and other details of the December
2021 election for the Python steering council, as specified in
:pep:`13`. This is the steering council election for the 2022 term
(i.e. Python 3.11).


Election Administration
=======================

The steering council appointed the
`Python Software Foundation <https://www.python.org/psf-landing/>`__
Director of Infrastructure, Ee Durbin
and Accounting Manager, Joe Carey, to coadminister the election.


Schedule
========

There will be a two-week nomination period, followed by a two-week
vote.

The nomination period was: November 1, 2021 through November 16, 2021 12:00 UTC
(The end of November 15, 2021 `Anywhere on Earth
<https://www.ieee802.org/16/aoe.html>`_). As announced on `python-committers
<https://mail.python.org/archives/list/python-committers@python.org/thread/6WDCGUKBOVG2FIPOGTPYXD3OA2KDJLLO/>`_,
and took place on `discuss.python.org <https://discuss.python.org/c/core-dev/steering-council-nominations>`_.

The voting period is: December 1, 2021 12:00 UTC through December 16, 2021
12:00 UTC (The end of December 15, 2021 `Anywhere on Earth
<https://www.ieee802.org/16/aoe.html>`_).


Candidates
==========

Candidates must be nominated by a core team member. If the candidate
is a core team member, they may nominate themselves.

Nominees (in alphabetical order):

- `Brett Cannon <https://discuss.python.org/t/steering-council-nomination-brett-cannon-2022-term/11839>`_
- `Mariatta <https://discuss.python.org/t/steering-council-nomination-mariatta-2022-term/11979>`_
- `David Mertz <https://discuss.python.org/t/steering-council-nomination-david-mertz-2022-term/12000>`_
- `Dong-hee Na <https://discuss.python.org/t/steering-council-nomination-dong-hee-na-2022-term/12001>`_
- `Pablo Galindo Salgado <https://discuss.python.org/t/steering-council-nomination-pablo-galindo-2022-term/11947>`_
- `Gregory P. Smith <https://discuss.python.org/t/steering-council-nomination-gregory-p-smith-2022-term/11999>`_
- `Victor Stinner <https://discuss.python.org/t/steering-council-nomination-victor-stinner-2022-term/11981>`_
- `Petr Viktorin <https://discuss.python.org/t/steering-council-nomination-petr-viktorin-2022-term/12003>`_
- `Barry Warsaw <https://discuss.python.org/t/steering-council-nomination-barry-warsaw-2022-term/11911>`_
- `Thomas Wouters <https://discuss.python.org/t/steering-council-nomination-thomas-wouters-2022-term/11946>`_

Withdrawn nominations:

- None

Voter Roll
==========

All active Python core team members are eligible to vote. Active status
is determined as described in :pep:`PEP 13 <13#membership>`
and implemented via the software at `python/voters <https://github.com/python/voters>`_ [1]_.

Ballots will be distributed based on the `The Python Voter Roll for this
election
<https://github.com/python/voters/blob/master/voter-files/>`_
[1]_.

While this file is not public as it contains private email addresses, the
`Complete Voter Roll`_ by name will be made available when the roll is
created.

Election Implementation
=======================

The election will be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2022-python-steering-council``

Name: ``2022 Python Steering Council Election``

Description: ``Election for the Python steering council, as specified in PEP 13. This is steering council election for the 2022 term.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``psf-election@python.org``

Voting starts at: ``December 1, 2021 00:00 UTC``

Voting ends at: ``December 16, 2021 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``- (approval)`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``



Results
=======

Of 85 eligible voters, 67 cast ballots.

The top five vote-getters are:

* Pablo Galindo Salgado
* Petr Viktorin
* Thomas Wouters
* Gregory P. Smith
* Brett Cannon

No conflict of interest as defined in :pep:`13` were observed.

The full vote counts are as follows:

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Pablo Galindo Salgado | 61             |
+-----------------------+----------------+
| Petr Viktorin         | 48             |
+-----------------------+----------------+
| Thomas Wouters        | 48             |
+-----------------------+----------------+
| Gregory P. Smith      | 44             |
+-----------------------+----------------+
| Brett Cannon          | 42             |
+-----------------------+----------------+
| Barry Warsaw          | 39             |
+-----------------------+----------------+
| Victor Stinner        | 35             |
+-----------------------+----------------+
| Mariatta              | 34             |
+-----------------------+----------------+
| Dong-hee Na           | 26             |
+-----------------------+----------------+
| David Mertz           | 24             |
+-----------------------+----------------+


Copyright
=========

This document has been placed in the public domain.


Complete Voter Roll
===================

Active Python core developers
-----------------------------

::

    Abhilash Raj
    Alex Gaynor
    Ammar Askar
    Andrew Kuchling
    Andrew Svetlov
    Antoine Pitrou
    Barry Warsaw
    Batuhan Taskaya
    Benjamin Peterson
    Berker PeksaÄ
    Brandt Bucher
    Brett Cannon
    Brian Curtin
    Brian Quinlan
    Carol Willing
    Cheryl Sabella
    Chris Jerdonek
    Chris Withers
    Christian Heimes
    Dino Viehland
    Dong-hee Na
    Ãric Araujo
    Eric Snow
    Eric V. Smith
    Ethan Furman
    Facundo Batista
    Fred Drake
    Giampaolo RodolÃ 
    Gregory P. Smith
    Guido van Rossum
    Hynek Schlawack
    Inada Naoki
    Irit Katriel
    Ivan Levkivskyi
    Jason R. Coombs
    Jeremy Kloth
    JesÃºs Cea
    Joannah Nanjekye
    Julien Palard
    Karthikeyan Singaravelan
    Ken Jin
    Kushal Das
    Kyle Stanley
    Larry Hastings
    Lisa Roach
    Åukasz Langa
    Lysandros Nikolaou
    Marc-AndrÃ© Lemburg
    Mariatta
    Mark Dickinson
    Mark Shannon
    Nathaniel J. Smith
    Ned Deily
    Neil Schemenauer
    Alyssa Coghlan
    Pablo Galindo
    Paul Ganssle
    Paul Moore
    Petr Viktorin
    Raymond Hettinger
    Ronald Oussoren
    Senthil Kumaran
    Serhiy Storchaka
    Stefan Behnel
    StÃ©phane Wirtel
    Steve Dower
    Tal Einat
    Terry Jan Reedy
    Thomas Wouters
    Tim Golden
    Tim Peters
    Victor Stinner
    Vinay Sajip
    Xiang Zhang
    Yury Selivanov
    Zachary Ware


.. [1] This repository is private and accessible only to Python Core
   Developers, administrators, and Python Software Foundation Staff as it
   contains personal email addresses.


id: pep-0239.rst
content:
PEP: 239
Title: Adding a Rational Type to Python
Author: Christopher A. Craig <python-pep@ccraig.org>, Moshe Zadka <moshez@zadka.site.co.il>
Status: Rejected
Type: Standards Track
Created: 11-Mar-2001
Python-Version: 2.2
Post-History: 16-Mar-2001

.. rejected::

   The needs outlined in the rationale section
   have been addressed to some extent by the acceptance of :pep:`327`
   for decimal arithmetic.  Guido also noted, "Rational arithmetic
   was the default 'exact' arithmetic in ABC and it did not work out as
   expected".  See the python-dev discussion on 17 June 2005 [1]_.

   *Postscript:* With the acceptance of :pep:`3141`, "A Type Hierarchy
   for Numbers", a 'Rational' numeric abstract base class was added
   with a concrete implementation in the 'fractions' module.



Abstract
========

Python has no numeric type with the semantics of an unboundedly
precise rational number.  This proposal explains the semantics of
such a type, and suggests builtin functions and literals to
support such a type.  This PEP suggests no literals for rational
numbers; that is left for :pep:`another PEP <240>`.


Rationale
=========

While sometimes slower and more memory intensive (in general,
unboundedly so) rational arithmetic captures more closely the
mathematical ideal of numbers, and tends to have behavior which is
less surprising to newbies.  Though many Python implementations of
rational numbers have been written, none of these exist in the
core, or are documented in any way.  This has made them much less
accessible to people who are less Python-savvy.


RationalType
============

There will be a new numeric type added called ``RationalType``.  Its
unary operators will do the obvious thing.  Binary operators will
coerce integers and long integers to rationals, and rationals to
floats and complexes.

The following attributes will be supported: ``.numerator`` and
``.denominator``.  The language definition will promise that::

    r.denominator * r == r.numerator

that the GCD of the numerator and the denominator is 1 and that
the denominator is positive.

The method ``r.trim(max_denominator)`` will return the closest
rational ``s`` to ``r`` such that ``abs(s.denominator) <= max_denominator``.


The rational() Builtin
======================

This function will have the signature ``rational(n, d=1)``.  ``n`` and ``d``
must both be integers, long integers or rationals.  A guarantee is
made that::

    rational(n, d) * d == n


Open Issues
===========

- Maybe the type should be called rat instead of rational.
  Somebody proposed that we have "abstract" pure mathematical
  types named complex, real, rational, integer, and "concrete"
  representation types with names like float, rat, long, int.

- Should a rational number with an integer value be allowed as a
  sequence index?  For example, should ``s[5/3 - 2/3]`` be equivalent
  to ``s[1]``?

- Should ``shift`` and ``mask`` operators be allowed for rational numbers?
  For rational numbers with integer values?

- Marcin 'Qrczak' Kowalczyk summarized the arguments for and
  against unifying ints with rationals nicely on c.l.py

  Arguments for unifying ints with rationals:

  - Since ``2 == 2/1`` and maybe ``str(2/1) == '2'``, it reduces surprises
    where objects seem equal but behave differently.

  - ``/`` can be freely used for integer division when I *know* that
    there is no remainder (if I am wrong and there is a remainder,
    there will probably be some exception later).

  Arguments against:

  - When I use the result of ``/`` as a sequence index, it's usually
    an error which should not be hidden by making the program
    working for some data, since it will break for other data.

  - (this assumes that after unification int and rational would be
    different types:) Types should rarely depend on values. It's
    easier to reason when the type of a variable is known: I know
    how I can use it. I can determine that something is an int and
    expect that other objects used in this place will be ints too.

  - (this assumes the same type for them:) Int is a good type in
    itself, not to be mixed with rationals.  The fact that
    something is an integer should be expressible as a statement
    about its type. Many operations require ints and don't accept
    rationals. It's natural to think about them as about different
    types.


References
==========

.. [1] Raymond Hettinger, Propose rejection of PEPs 239 and 240 -- a builtin
       rational type and rational literals
       https://mail.python.org/pipermail/python-dev/2005-June/054281.html

Copyright
=========

This document has been placed in the public domain.


id: pep-0588.rst
content:
PEP: 588
Title: GitHub Issues Migration Plan
Author: Mariatta <mariatta@python.org>
BDFL-Delegate: Barry Warsaw <barry@python.org>
Discussions-To: https://discuss.python.org/t/13791
Status: Final
Type: Informational
Created: 27-Mar-2019

.. canonical-doc:: `psf/gh-migration#13 <https://github.com/psf/gh-migration/issues/13>`__

   The migration was carried out in April 2022.


Abstract
========

This PEP describes the detailed plan for migrating from Python's issue
tracker on Roundup to GitHub issues.  See :pep:`581` for rationale and
background.  :pep:`588` also describes the detailed timeline for the
migration.


Migration Plan
==============

Here we outline the tasks, steps, and core decisions we need to make
in order to migrate bug tracking to GitHub, with the least impact on
CPython developer productivity.

Hire a professional project manager
-----------------------------------

Having a professional project manager to handle the migration, similar to how
the Warehouse project was managed, would help ensure the success of this project.

Create a playground CPython issue tracker on GitHub
---------------------------------------------------

We should create a playground issue tracker on GitHub where we can experiment
and test out the new workflow.

Backup of GitHub data
---------------------

This effort has been started and is being tracked as an issue in
core-workflow [#]_. We're using GitHub's Migrations API [#]_ to
download GitHub data for CPython on a daily basis. The archives will
be dropped in a S3 bucket.

Thanks to Ee Durbin for working on this.


Update the CLA host
-------------------

At the moment, the CLA is hosted within bpo. It needs to be updated such that
signing the CLA does not require a bpo account, and it should be hosted outside
of the bpo.

The current CLA process itself is not ideal. Currently, contributors to
devguide, peps, and core-workflow need to sign a CLA, and it requires a bpo
account. A bpo account should not be required for those projects.

There is an ongoing effort to start using our own instance of CLA
assistant instead of the current CLA process in place. Discussion
about this has been started in `core-workflow mailing list
<https://mail.python.org/archives/list/core-workflow@python.org/thread/JBV3XJVD2DLDX5DY7TZEA6CO5YPNHJ2C/>`_
as well as on `Discourse
<https://discuss.python.org/t/using-cla-assistant-for-python/990>`_.

This effort `is currently stalled <https://discuss.python.org/t/cla-assistant-is-no-go/2066/>`_
because cla-assistant does not yet support CLA signed on behalf of organization.


Create "Python Triage" team on GitHub
-------------------------------------

The bug triagers on bpo are valuable to the core Python workflow, and we
definitely would need even more help with triaging issues on GitHub.

It has been `proposed on Discourse
<https://discuss.python.org/t/proposal-create-bug-triage-team-on-github/992/5>`_
for us to create a "bug triage" team on GitHub to help with closing
issues, notifying the appropriate parties, as well as applying labels
to issues and pull requests.

The new Triage role on GitHub is currently in beta, and the Python organization
has been granted access to this role, and we can begin taking advantage of it.

The "Python Triage" team has been created. A description and expectations of the
triage role `have been added to Devguide <https://devguide.python.org/triaging/#python-triage-team>`_.

Progress of this project can be
tracked in `"Adding Triagers" project board
<https://github.com/python/core-workflow/projects/3>`_.

Create labels for issue triage
------------------------------

In bpo, we currently have the following fields for each issue:

Types: ``behavior``, ``crash``, ``compile error``, ``resource
usage``, ``security``, ``performance``, ``enhancement``.

Components: ``2to3``, ``Argument Clinic``, ``asyncio``, ``Build``,
``Cross-build``, ``ctypes``, ...

Priority: ``release blocker``, ``deferred blocker``, ``critical``,
``high``, ``normal``, ``low``

We will create the corresponding labels::

   type-behavior, type-crash, type-compile error, type-resource usage, ...

   components-2to3, components-argument clinic, components-asyncio, ...

   priority-release blocker, priority-deferred blocker, priority-critical, ...

In addition, we'll create a ``needs triage`` label.

The final "labels" to be created can be decided at a later time when
it is time to start switching to GitHub issues.

A test repository containing all possible labels and color schema has been
created by Carol Willing and can be reviewed at https://github.com/willingc/test-581/labels.

Create issue templates
----------------------

We will create an issue template and add the following headers::

   ---
   Type: behavior | crash | compile error | resource usage (choose one)
   Components: 2to3 | Argument Clinic | asyncio | Build | ... (can select more than one)
   Priority: release blocker | deferred blocker | critical | ...
   Needs backport to: 2.7 | 3.6 | 3.7
   ---

The idea is to allow the issue creator to help us triage the issue.
The above values are pre-filled in the template. The issue creator will remove
texts that do not apply to their issue.

Based on the above headers, bedevere-bot can apply the necessary
labels to the issue. If the issue creator did not supply the above
headers, the bot will apply the ``needs triage`` label. At that point,
it will require a core developer to properly label the issue.

We can also take advantage of GitHub's multiple issue template
feature, and the ability to automatically set issue assignee and
labels by using issue templates.


Updates to bedevere
-------------------

Bedevere-bot will need to be updated to recognize the issue headers described
above and apply the proper labels.

Bedevere-bot can also copy over the labels to pull requests that correspond to
the issue.


Update the devguide
-------------------

Devguide should be updated with information about the new workflow of using GitHub
issues. It can be done as a separate branch, and it should be done ahead of the
migration, not after.

Add a button in bpo to migrate the issue to GitHub
--------------------------------------------------

This will require the bpo to be updated. But I believe the effort needed for
this is much less than a complete overhaul.

We will create a button in bpo, that will copy over the issue description
and associated comments into a GitHub issue.

We need to add a new status: "moved" with the url of the GitHub issue.

We should not be moving all open issues to GitHub. Only when someone
is interested in continuing work or discussion about the issue, that
the issue should be "moved" to GitHub.


Migrated issues
---------------

When an issue is marked as "moved", this issue should be in read-only mode. bpo
should forbid the edition of the issue.


Make bpo read-only
------------------

This should be the final step. Once we start using GitHub issues, make bpo
read-only, instead of shutting it down.
Do not accept new registrations. Do not allow comments or issues to be created.


Mapping between issues from bpo and GitHub
------------------------------------------

Usually when we reference an issue from bpo, we use bpo-XYZ but with
GitHub, we will have a new reference with this format
``https://github.com/python/cpython/issue/XYZ``.

Because we will migrate the issues from bpo to GitHub, we need to have a new
field on bpo for the reference to the issues on GitHub, and the same thing on
GitHub for the 'eventual' reference from bpo.

For GitHub, we need to add ``origin: https://bugs.python.org/issueXYZ``.
For bpo, add a new field ``moved to:
https://github.com/python/cpython/issue/XYZ``.

Nosy-ing the expert
-------------------

A current functionality in bpo is to automatically nosy people who are listed
as an expert of certain area. Several Python core developers have expressed that
they prefer not having to subscribe to everything on GitHub, but only getting noti


id: pep-0775.rst
content:
PEP: 775
Title: Make zlib required to build CPython
Author: Gregory P. Smith <greg@krypto.org>,
        Stan Ulbrych <stanulbrych@gmail.com>,
        Petr Viktorin <encukou@gmail.com>
Discussions-To: https://discuss.python.org/t/82672
Status: Withdrawn
Type: Standards Track
Created: 24-Feb-2025
Python-Version: 3.14
Post-History: `23-Jan-2023 <https://discuss.python.org/t/23062>`__,
              `01-Mar-2025 <https://discuss.python.org/t/82672>`__,


Abstract
========

Building CPython without the `zlib <https://zlib.net>`_ compression library
will no be longer supported, and the :mod:`zlib` module will be required in
the standard library.
The only exception is `WASI <https://wasi.dev>`_, as zlib is not currently
supported in CPython on WASI.
Building the interpreter without zlib may still be possible,
but formally unsupported.


PEP Withdrawal
==============

This PEP was withdrawn by the authors on 2025-05-07. After further reflection,
we realized it did not provide sufficient value and was met with significant
negative feedback, particularly regarding the proposed exception for WASI.


Motivation
==========

The zlib library, which powers the :mod:`!zlib` Python module,
is available on all supported systems except WASI.

Many wheels on PyPI, including the :pypi:`pip` installer, require zlib.
Users of pip would consider CPython without zlib to be broken,
but mostly don't notice because all major builds of CPython include zlib.

CPython developers don't really notice either. It turns out that at the time
of writing, at least one CPython test fails without zlib (the "skip"
decorator in ``test_peg_generator.test_c_parser`` is applied too late),
but our CI didn't catch this.

This PEP treats this as an issue in documentation and messaging.
In practice, we already don't support building CPython without zlib; we
should just say so.


Rationale
=========

There are possible use cases for zlib-less builds, such as embedding and
bootstrapping, as well as unforeseen ones.
Therefore, we don't *remove* support for zlib-less systems; we mark them
unsupported and invite affected users to do their own testing, or to share
use cases that can make us reconsider this decision.

zlib is not yet used by default on the WASI platform -- mostly because
adding it hasn't yet been a priority there. (Note that `Pyodide`_, the main
"real-world" CPython distribution for WASI, does include zlib.)
We take this as an opportunity to  continue testing a platform without
zlib, so that we don't unintentionally break unsupported builds yet.

.. _Pyodide: https://pyodide.org


Specification
=============

In standard library modules that use zlib for optional functionality,
that functionality will raise ``ImportError`` when used.
Code to generate more "friendly" error messages, or to pre-check whether
zlib is available, will be removed.
All functionality unrelated to zlib will still be usable if zlib is
missing.

This affects the following modules, and more that depend on these
transitively:

* :mod:`shutil` (``gztar`` and ``zip`` archive formats)
* :mod:`tarfile`, :mod:`zipfile`, :mod:`zipimport`,
  :mod:`zipapp` (archive compression)
* :mod:`codecs` (``zlib_codec``)

``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar``
as registered formats, even if they are unusable due to missing zlib.

The ``configure`` script will issue a warning when zlib is not found on
platforms other than WASI.

``test_zlib`` will fail on platforms other than WASI.
All other tests will continue to be skipped -- that is, uses of
``@test.support.requires_zlib`` will be kept in place -- for the benefit
of WASI, unsupported builds, and any possible reverts.

:pep:`11` will be adjusted to mark "Systems without zlib, except WASI" as
unsupported.


Backwards Compatibility
=======================

In practice, nothing major changes, except in error cases -- for example,
attempts to use tar compression without zlib available will raise
``ImportError`` and not ``CompressionError``.


Security Implications
=====================

None known.


How to Teach This
=================

We don't expect that any instructions will need to change, as zlib is
already available in all relevant contexts.


Reference Implementation
========================

A reference implementation may be found in a pull request to the CPython
repository, `python/cpython#130297
<https://github.com/python/cpython/pull/130297>`_


Future work
===========

In the future, if no use cases for zlib-less builds are found,
zlib may be made fully required.
The main changes needed for that would be making the ``configure`` script
raise a hard error, and removing ``@test.support.requires_zlib``.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


id: pep-0013.rst
content:
PEP: 13
Title: Python Language Governance
Author: The Python core team and community
Status: Active
Type: Process
Topic: Governance
Created: 16-Dec-2018


Abstract
========

This PEP defines the formal governance process for Python, and records
how this has changed over time. Currently, governance is based around
a steering council. The council has broad authority, which they seek
to exercise as rarely as possible.


Current steering council
========================

The 2025 term steering council consists of:

* Barry Warsaw
* Donghee Na
* Emily Morehouse
* Gregory P. Smith
* Pablo Galindo Salgado

Per the results of the vote tracked in :pep:`8106`.

The core team consists of those listed in the private
https://github.com/python/voters/ repository which is publicly
shared via https://devguide.python.org/developers/.


Specification
=============

The steering council
--------------------

Composition
~~~~~~~~~~~

The steering council is a 5-person committee.


Mandate
~~~~~~~

The steering council shall work to:

* Maintain the quality and stability of the Python language and
  CPython interpreter,
* Make contributing as accessible, inclusive, and sustainable as
  possible,
* Formalize and maintain the relationship between the core team and
  the PSF,
* Establish appropriate decision-making processes for PEPs,
* Seek consensus among contributors and the core team before acting in
  a formal capacity,
* Act as a "court of final appeal" for decisions where all other
  methods have failed.


Powers
~~~~~~

The council has broad authority to make decisions about the project.
For example, they can:

* Accept or reject PEPs
* Enforce or update the project's code of conduct
* Work with the PSF to manage any project assets
* Delegate parts of their authority to other subcommittees or
  processes

However, they cannot modify this PEP, or affect the membership of the
core team, except via the mechanisms specified in this PEP.

The council should look for ways to use these powers as little as
possible. Instead of voting, it's better to seek consensus. Instead of
ruling on individual PEPs, it's better to define a standard process
for PEP decision making (for example, by accepting one of the other
801x series of PEPs). It's better to establish a Code of Conduct
committee than to rule on individual cases. And so on.

To use its powers, the council votes. Every council member must either
vote or explicitly abstain. Members with conflicts of interest on a
particular vote must abstain. Passing requires a strict majority of
non-abstaining council members.

Whenever possible, the council's deliberations and votes shall be held
in public.


Electing the council
~~~~~~~~~~~~~~~~~~~~

A council election consists of two phases:

* Phase 1: Candidates advertise their interest in serving. Candidates
  must be nominated by a core team member. Self-nominations are
  allowed.

* Phase 2: Each core team member can assign zero to five stars to each
  candidate. Voting is performed anonymously. The outcome of the vote
  is determined using the `STAR voting system <https://www.starvoting.org/>`__,
  modified to use the `Multi-winner Bloc STAR <https://www.starvoting.org/multi_winner>`__)
  approach. If a tie occurs, it may
  be resolved by mutual agreement among the candidates, or else the
  winner will be chosen at random.

Each phase lasts one to two weeks, at the outgoing council's discretion.
For the initial election, both phases will last two weeks.

The election process is managed by a returns officer nominated by the
outgoing steering council. For the initial election, the returns
officer will be nominated by the PSF Executive Director.

The council should ideally reflect the diversity of Python
contributors and users, and core team members are encouraged to vote
accordingly.


Term
~~~~

A new council is elected after each feature release. Each council's
term runs from when their election results are finalized until the
next council's term starts. There are no term limits.


Vacancies
~~~~~~~~~

Council members may resign their position at any time.

Whenever there is a vacancy during the regular council term, the
council may vote to appoint a replacement to serve out the rest of the
term.

If a council member drops out of touch and cannot be contacted for a
month or longer, then the rest of the council may vote to replace
them.


Conflicts of interest
~~~~~~~~~~~~~~~~~~~~~

While we trust council members to act in the best interests of Python
rather than themselves or their employers, the mere appearance of any
one company dominating Python development could itself be harmful and
erode trust. In order to avoid any appearance of conflict of interest,
at most 2 members of the council can work for any single employer.

In a council election, if 3 of the top 5 vote-getters work for the
same employer, then whichever of them ranked lowest is disqualified
and the 6th-ranking candidate moves up into 5th place; this is
repeated until a valid council is formed.

During a council term, if changing circumstances cause this rule to be
broken (for instance, due to a council member changing employment),
then one or more council members must resign to remedy the issue, and
the resulting vacancies can then be filled as normal.


Ejecting core team members
~~~~~~~~~~~~~~~~~~~~~~~~~~

In exceptional circumstances, it may be necessary to remove someone
from the core team against their will. (For example: egregious and
ongoing code of conduct violations.) This can be accomplished by a
steering council vote, but unlike other steering council votes, this
requires at least a two-thirds majority. With 5 members voting, this
means that a 3:2 vote is insufficient; 4:1 in favor is the minimum
required for such a vote to succeed. In addition, this is the one
power of the steering council which cannot be delegated, and this
power cannot be used while a vote of no confidence is in process.

If the ejected core team member is also on the steering council, then
they are removed from the steering council as well.


Vote of no confidence
~~~~~~~~~~~~~~~~~~~~~

In exceptional circumstances, the core team may remove a sitting
council member, or the entire council, via a vote of no confidence.

A no-confidence vote is triggered when a core team member calls for
one publicly on an appropriate project communication channel, and
another core team member seconds the proposal within one week.

The vote lasts for two weeks. Core team members vote for or against.
If at least two thirds of voters express a lack of confidence, then
the vote succeeds.

There are two forms of no-confidence votes: those targeting a single
member, and those targeting the council as a whole. The initial call
for a no-confidence vote must specify which type is intended. If a
single-member vote succeeds, then that member is removed from the
council and the resulting vacancy can be handled in the usual way. If
a whole-council vote succeeds, the council is dissolved and a new
council election is triggered immediately.


The core team
-------------

Role
~~~~

The core team is the group of trusted volunteers who manage Python.
They assume many roles required to achieve the project's goals,
especially those that require a high level of trust. They make the
decisions that shape the future of the project.

Core team members are expected to act as role models for the community
and custodians of the project, on behalf of the community and all
those who rely on Python.

They will intervene, where necessary, in online discussions or at
official Python events on the rare occasions that a situation arises
that requires intervention.

They have authority over the Python Project infrastructure, including
the Python Project website itself, the Python GitHub organization and
repositories, the bug tracker, the mailing lists, IRC channels, etc.


Prerogatives
~~~~~~~~~~~~

Core team members may participate in formal votes, typically to nominate new
team mem


id: pep-0007.rst
content:
PEP: 7
Title: Style Guide for C Code
Author: Guido van Rossum <guido@python.org>, Barry Warsaw <barry@python.org>
Status: Active
Type: Process
Created: 05-Jul-2001
Post-History:

.. highlight:: c

Introduction
============

This document gives coding conventions for the C code comprising the C
implementation of Python.  Please see the companion informational PEP
describing :pep:`style guidelines for Python code <8>`.

Note, rules are there to be broken.  Two good reasons to break a
particular rule:

1. When applying the rule would make the code less readable, even for
   someone who is used to reading code that follows the rules.

2. To be consistent with surrounding code that also breaks it (maybe
   for historic reasons) -- although this is also an opportunity to
   clean up someone else's mess (in true XP style).


C dialect
=========

* Python 3.11 and newer versions use C11 without `optional features
  <https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features>`_.
  The public C API should be compatible with C++.

* Python 3.6 to 3.10 use C89 with several select C99 features:

  - Standard integer types in ``<stdint.h>`` and ``<inttypes.h>``. We
    require the fixed width integer types.
  - ``static inline`` functions
  - designated initializers (especially nice for type declarations)
  - intermingled declarations
  - booleans
  - C++-style line comments

* Python versions before 3.6 used ANSI/ISO standard C (the 1989 version
  of the standard).  This meant (amongst many other things) that all
  declarations must be at the top of a block (not necessarily at the
  top of function).

* Don't use compiler-specific extensions, such as those of GCC or MSVC
  (e.g. don't write multi-line strings without trailing backslashes).

* All function declarations and definitions must use full prototypes
  (i.e. specify the types of all arguments).

* No compiler warnings with major compilers (gcc, VC++, a few others).

* ``static inline`` functions should be preferred over macros in new
  code.


Code lay-out
============

* Use 4-space indents and no tabs at all.

* No line should be longer than 79 characters.  If this and the
  previous rule together don't give you enough room to code, your code
  is too complicated -- consider using subroutines.

* No line should end in whitespace.  If you think you need significant
  trailing whitespace, think again -- somebody's editor might delete
  it as a matter of routine.

* Function definition style: function name in column 1, outermost
  curly braces in column 1, blank line after local variable
  declarations.

  .. code-block::
     :class: good

     static int
     extra_ivars(PyTypeObject *type, PyTypeObject *base)
     {
         int t_size = PyType_BASICSIZE(type);
         int b_size = PyType_BASICSIZE(base);

         assert(t_size >= b_size); /* type smaller than base! */
         ...
         return 1;
     }

* Code structure: one space between keywords like ``if``, ``for`` and
  the following left paren; no spaces inside the paren; braces are
  required everywhere, even where C permits them to be omitted, but do
  not add them to code you are not otherwise modifying.  All new C
  code requires braces.  Braces should be formatted as shown:

  .. code-block::
     :class: good

     if (mro != NULL) {
         ...
     }
     else {
         ...
     }

* The return statement should *not* get redundant parentheses:

  .. code-block::
     :class: bad

     return(albatross); /* incorrect */

  Instead:

  .. code-block::
     :class: good

     return albatross; /* correct */

* Function and macro call style: ``foo(a, b, c)`` -- no space before
  the open paren, no spaces inside the parens, no spaces before
  commas, one space after each comma.

* Always put spaces around assignment, Boolean and comparison
  operators.  In expressions using a lot of operators, add spaces
  around the outermost (lowest-priority) operators.

* Breaking long lines: if you can, break after commas in the outermost
  argument list.  Always indent continuation lines appropriately,
  e.g.:

  .. code-block::
     :class: good

     PyErr_Format(PyExc_TypeError,
                  "cannot create '%.100s' instances",
                  type->tp_name);

* When you break a long expression at a binary operator, braces
  should be formatted as shown:

  .. code-block::
     :class: good

     if (type->tp_dictoffset != 0
         && base->tp_dictoffset == 0
         && type->tp_dictoffset == b_size
         && (size_t)t_size == b_size + sizeof(PyObject *))
     {
         return 0; /* "Forgive" adding a __dict__ only */
     }

  It's OK to put operators at ends of lines, especially to be
  consistent with surrounding code.
  (See :ref:`PEP 8 <pep8-operator-linebreak>` for a longer discussion.)

* Vertically align line continuation characters in multi-line macros.

* Macros intended to be used as a statement should use the
  ``do { ... } while (0)`` macro idiom,
  without a final semicolon.
  Example:

  .. code-block::
     :class: good

     #define ADD_INT_MACRO(MOD, INT)                                   \
         do {                                                          \
             if (PyModule_AddIntConstant((MOD), (#INT), (INT)) < 0) {  \
                 goto error;                                           \
             }                                                         \
         } while (0)

     // To be used like a statement with a semicolon:
     ADD_INT_MACRO(m, SOME_CONSTANT);

* ``#undef`` file local macros after use.

* Put blank lines around functions, structure definitions, and major
  sections inside functions.

* Comments go before the code they describe.

* All functions and global variables should be declared static unless
  they are to be part of a published interface.

* For external functions and variables, we always have a declaration
  in an appropriate header file in the "Include" directory, which uses
  the ``PyAPI_FUNC()`` macro and ``PyAPI_DATA()`` macro, like this:

  .. code-block::
     :class: good

     PyAPI_FUNC(PyObject *) PyObject_Repr(PyObject *);

     PyAPI_DATA(PyTypeObject) PySuper_Type;


Naming conventions
==================

* Use a ``Py`` prefix for public functions; never for static
  functions.  The ``Py_`` prefix is reserved for global service
  routines like ``Py_FatalError``; specific groups of routines
  (e.g. specific object type APIs) use a longer prefix,
  e.g. ``PyString_`` for string functions.

* Public functions and variables use MixedCase with underscores, like
  this: ``PyObject_GetAttr``, ``Py_BuildValue``, ``PyExc_TypeError``.

* Occasionally an "internal" function has to be visible to the loader;
  we use the ``_Py`` prefix for this, e.g.: ``_PyObject_Dump``.

* Macros should have a MixedCase prefix and then use upper case, for
  example: ``PyString_AS_STRING``, ``Py_PRINT_RAW``.

* Macro parameters should use ``ALL_CAPS`` style,
  so they are easily distinguishable from C variables and struct members.


Documentation Strings
=====================

* Use the ``PyDoc_STR()`` or ``PyDoc_STRVAR()`` macro for docstrings
  to support building Python without docstrings (``./configure
  --without-doc-strings``).

* The first line of each function docstring should be a "signature
  line" that gives a brief synopsis of the arguments and return value.
  For example:

  .. code-block::
     :class: good

     PyDoc_STRVAR(myfunction__doc__,
     "myfunction(name, value) -> bool\n\n\
     Determine whether name and value make a valid pair.");

  Always include a blank line between the signature line and the text
  of the description.

  If the return value for the function is always ``None`` (because there
  is no meaningful return value), do not include the indication of the
  return type.

* When writing multi-line docstrings, be sure to always use backslash
  continuations, as in the example above, or string literal
  concatenation:

  .. c


id: pep-0761.rst
content:
PEP: 761
Title: Deprecating PGP signatures for CPython artifacts
Author: Seth Michael Larson <seth@python.org>
Sponsor: Hugo van Kemenade
Discussions-To: https://discuss.python.org/t/pep-761-deprecating-pgp-signatures-for-cpython-artifacts/67180
Status: Active
Type: Process
Created: 08-Oct-2024
Python-Version: 3.14
Post-History: `25-Sep-2024 <https://discuss.python.org/t/pre-pep-discussion-stop-providing-gpg-signatures-for-cpython-artifacts/65058>`__, `09-Oct-2024 <https://discuss.python.org/t/pep-761-deprecating-pgp-signatures-for-cpython-artifacts/67180>`__
Resolution: `06-Nov-2024 <https://discuss.python.org/t/pep-761-deprecating-pgp-signatures-for-cpython-artifacts/67180/39>`__

Abstract
========

Since Python 3.11.0, CPython has provided two verifiable digital signatures
for all CPython artifacts: PGP and Sigstore.

PGP's design requires the maintenance and protection of `long-lived private
keys <https://words.filippo.io/giving-up-on-long-term-pgp/>`_ by trusted
parties. PGP's `security and ergonomics have been criticized by security
practitioners <https://www.latacora.com/blog/2019/07/16/the-pgp-problem/>`_
for many years now, with the biggest issue being that there were few
alternatives for "artifact signing" being proposed or adopted.

`Sigstore's <https://docs.sigstore.dev>`_ design philosophy has focused on the
ergonomics of signing and verifying and `uses short-lived keys with
strongly-bound human-readable identities via OpenID Connect <https://docs.sigstore.dev/#how-sigstore-works>`_.
Sigstore has both development and adoption momentum, seeing adoption by PyPI,
NPM, Homebrew, and GitHub, among other ecosystems.

This PEP proposes to move CPython to using Sigstore exclusively for signing
artifacts through a deprecation and eventual discontinuance of providing PGP
signatures with new release managers.

Motivation
==========

CPython's releases are release-manager-centric, where a single person
maintains multiple CPython releases from pre-release to end-of-life over the
course of many years.

Requiring release managers to maintain and protect PGP private keys for seven
or more years is an unnecessary burden in the new age of ergonomic and
ephemeral signing keys. Comparatively, Sigstore only requires release managers
to click a button during the release process to OAuth sign-on to their
identity provider. Maintaining the integrity of accounts on identity providers
like GitHub is already an expectation of being a Python release manager or
core team member, such as through multi-factor authentication and strong
unique passwords.

Rationale
=========

Preserve expectations across a Python release
---------------------------------------------

To avoid breaking downstream verifiers, the expectations for verification
materials availability SHOULD NOT be changed during a feature release's
lifecycle.

Release managers, not releases
------------------------------

The discontinuation of PGP signatures doesn't necessarily have to happen
on a "release manager boundary"; a new Python release could be a potential
boundary.

Because the primary motivation for deprecating PGP is ergonomics, deciding
to drop PGP for one release while a release manager still has obligations to
provide PGP signatures for other releases for multiple years isn't much
savings of effort.

A new release manager also represents a new PGP public key that downstream
verifiers need to adopt. By choosing to make the change during this period,
this minimizes the breakage to a place in downstream maintenance where a
change will already be necessary.

Gordian knot of signing methods and verifiers
---------------------------------------------

CPython providing both PGP and Sigstore signatures concurrently creates a
"`Gordian knot <https://en.wikipedia.org/wiki/Gordian_Knot>`_" where
verifiers are disincentivized to migrate to a new signature method due to the
*continued and expected availability* of an existing signature method, thus
propagating the *apparent demand* for maintaining the existing signature
method.

This situation slows down the adoption of new signature methods like Sigstore for
both signature-producing projects and signature-verifying ecosystems by not
creating a "need" to automate and integrate the signature method into verifier
tooling.

By changing the expectation of what future signature methods will be
available, the incentive-knot can be broken by `spurring the adoption of the
new signature method in downstream tooling <https://lists.debian.org/debian-devel/2024/10/msg00025.html>`_.
This change to verifier tooling also makes other upstream projects able to
migrate to publishing only Sigstore signatures, resulting in a positive
feedback loop of adoption.

Specification
=============

Because PGP keys are tied to a release manager identity, the change to
availability of PGP signatures will be tied to release managers instead of
individual releases (3.13, 3.14, etc). This PEP both deprecates and proposes
a discontinuation timeline for PGP signatures.

Deprecation and discontinuation of PGP signatures
-------------------------------------------------

This PEP deprecates PGP signatures for future CPython releases and recommends
verifiers to adopt Sigstore to verify CPython artifacts as an alternative to
PGP.

This PEP also removes the expectation that PGP signatures be published by
future release managers that don't already maintain a stable Python release.
At the time of writing this would be Hugo van Kemenade, as 3.14 is the next
Python version without a stable release.

Releases which already have a stable release (3.13, 3.12, 3.11, etc) are not
affected and will continue to provide PGP signatures for artifacts until they
are end-of-life. All existing PGP signatures will continue to work as
expected.

Delaying discontinuation of PGP signatures
------------------------------------------

This PEP provides a mechanism to delay the *discontinuation* of PGP signatures
from active and upcoming CPython releases in case of extraordinary
circumstances. *Deprecation* of PGP signatures can't be changed without a
superseding PEP.

The Steering Council MAY at a future date after this PEP's acceptance decide
to delay the discontinuation of PGP signatures to a future CPython release.
If the Steering Council decides to delay the discontinuation of PGP signatures
then all active release managers MUST provide PGP signatures for their covered
CPython artifacts for the remainder of their tenure as a release manager. This
includes all steps required to do so, such as generating a new PGP key and
publishing their identity to python.org.

The discontinuation of PGP signatures then is automatically scheduled for the
next release manager without a stable release, to be highlighted in the
Steering Council decision.

Backwards Compatibility
=======================

This proposal would remove the ability to verify future CPython artifacts
using PGP. Any downstream verifiers using PGP for CPython artifacts would
need to either start using Sigstore, verify their source code of CPython
through other means, or stop verification altogether for future CPython
releases.

Security Implications
=====================

PGP and Sigstore have different security models, so by removing PGP
signatures this means that all users only have the option to rely on the
security model provided by Sigstore.

In general, the security model required for artifact signatures is being
able to detect whether a given artifact is from the expected source and
hasn't been modified, regardless of the security or integrity of the hosting
service (in CPython's case: python.org/downloads).

`Sigstore's security model <https://docs.sigstore.dev/about/security/>`_
depends more on centralized infrastructure compared to PGP, such as the
"public good" signature transparency log (Rekor), certificate authority and
transparency log (Fulcio), and the security of OpenID Connect identity
providers like Google and GitHub.

CPython's developm


id: pep-0749.rst
content:
PEP: 749
Title: Implementing PEP 649
Author: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-749-implementing-pep-649/54974
Status: Accepted
Type: Standards Track
Topic: Typing
Requires: 649
Created: 28-May-2024
Python-Version: 3.14
Post-History: `04-Jun-2024 <https://discuss.python.org/t/pep-749-implementing-pep-649/54974>`__
Resolution: `05-May-2025 <https://discuss.python.org/t/pep-749-implementing-pep-649/54974/66>`__


Abstract
========

This PEP supplements :pep:`649` by providing various tweaks and additions to its
specification:

* ``from __future__ import annotations`` (:pep:`563`) will continue to exist with
  its current behavior at least until Python 3.13 reaches its end-of-life. Subsequently,
  it will be deprecated and eventually removed.
* A new standard library module, ``annotationlib``, is added to provide tooling for
  annotations. It will include the ``get_annotations()`` function, an enum for annotation
  formats, a ``ForwardRef`` class, and a helper function for calling ``__annotate__`` functions.
* Annotations in the REPL are lazily evaluated, just like other module-level annotations.
* We specify the behavior of wrapper objects that provide annotations, such as :py:func:`classmethod`
  and code that uses :py:func:`functools.wraps`.
* There will not be a code flag for marking ``__annotate__`` functions
  that can be run in a "fake globals" environment. Instead, we add a fourth format,
  ``VALUE_WITH_FAKE_GLOBALS``, to allow third-party implementors of annotate functions to
  indicate what formats they support.
* Deleting the ``__annotations__`` attribute directly will also clear ``__annotate__``.
* We add functionality to allow evaluating type alias values and type parameter bounds and defaults
  (which were added by :pep:`695` and :pep:`696`) using PEP 649-like semantics.
* The ``SOURCE`` format is renamed to ``STRING`` to improve clarity and reduce the risk of
  user confusion.
* Conditionally defined class and module annotations are handled correctly.
* If annotations are accessed on a partially executed module, the annotations executed so far
  are returned, but not cached.

Motivation
==========

:pep:`649` provides an excellent framework for creating better semantics for
annotations in Python. It solves a common pain point for users of annotations,
including those using static type hints as well as those using runtime typing,
and it makes the language more elegant and powerful.
The PEP was originally proposed in 2021 for Python 3.10,
and it was accepted in 2023. However, the implementation took longer than anticipated,
and now the PEP is expected to be implemented in Python 3.14.

I have started working on the implementation of the PEP in CPython. I found that
the PEP leaves some areas underspecified, and some
of its decisions in corner cases are questionable. This new PEP proposes several
changes and additions to the specification to address these issues.

This PEP supplements rather than supersedes PEP 649. The changes proposed here
should make the overall user experience better, but they do not change the
general framework of the earlier PEP.


The future of ``from __future__ import annotations``
====================================================

:pep:`563` previously introduced the future import ``from __future__ import annotations``,
which changes all annotations to strings. :pep:`649` proposes an alternative approach
that does not require this future import, and states:

    If this PEP is accepted, PEP 563 will be deprecated and eventually removed.

However, the PEP does not provide a detailed plan for this deprecation.

There is some previous discussion of this topic `on Discourse <https://discuss.python.org/t/pep-649-deferred-evaluation-of-annotations-tentatively-accepted/21331/44>`__
(note that in the linked post I proposed something different from what is proposed here).

Specification
-------------

We suggest the following deprecation plan:

- In Python 3.14, ``from __future__ import annotations`` will continue to work as it
  did before, converting annotations into strings.

  - If the future import is active, the ``__annotate__`` function of objects with
    annotations will return the annotations as strings when called with the ``VALUE``
    format, reflecting the behavior of ``__annotations__``.

- Sometime after the last release that did not support :pep:`649` semantics (expected to be 3.13)
  reaches its end-of-life, ``from __future__ import annotations`` is deprecated. Compiling
  any code that uses the future import will emit a :py:exc:`DeprecationWarning`. This will
  happen no sooner than the first release after Python 3.13 reaches its end-of-life, but
  the community may decide to wait longer.
- After at least two releases, the future import is removed, and annotations are always
  evaluated as per :pep:`649`. Code that continues to use the future import will raise
  a :py:exc:`SyntaxError`, similar to any other undefined future import.

Rejected alternatives
---------------------

*Immediately make the future import a no-op*: We considered applying :pep:`649` semantics
to all code in Python 3.14, making the future import a no-op. However, this would break
code that works in 3.13 under the following set of conditions:

* ``__future__ import annotations`` is active
* There are annotations that rely on forward references
* Annotations are eagerly evaluated at import time, for example by a metaclass or
  class or function decorator. For example, this currently applies to the
  released version of ``typing_extensions.TypedDict``.

This is expected to be a common pattern, so we cannot afford to break such code during
the upgrade from 3.13 to 3.14.

Such code would still break when the future import is eventually removed. However, this
is many years in the future, giving affected libraries plenty of time to update their code.

*Immediately deprecate the future import*: Instead of waiting until Python 3.13 reaches
its end-of-life, we could immediately start emitting warnings when the future import is
used. However, many libraries are already using ``from __future__ import annotations`` as
an elegant way to enable unrestricted forward references in their annotations. If we deprecate
the future import immediately, it would be impossible for these libraries to use unrestricted
forward references on all supported Python versions while avoiding deprecation warnings:
unlike other features deprecated from the standard library, a ``__future__`` import must
be the first statement in a given module, meaning it would be impossible to only
conditionally import ``__future__.annotations`` on Python 3.13 and lower. (The necessary
``sys.version_info`` check would count as a statement preceding the ``__future__`` import.)

*Keep the future import around forever*: We could also decide to keep the future import
indefinitely. However, this would permanently bifurcate the behavior of the Python
language. This is undesirable; the language should have only a single set of semantics,
not two permanently different modes.

*Make the future import a no-op in the future*: Instead of eventually making
``from __future__ import annotations`` a ``SyntaxError``, we could make it do nothing
instead at some point after Python 3.13 reaches its end-of-life. This still has some
of the same issues outlined above around making it a no-op now, although the ecosystem
would have had much longer to adapt. It is better to have users explicitly remove
the future import from their code in the future once they have confirmed they do not
rely on stringized annotations.

New ``annotationlib`` module
============================

:pep:`649` proposes to add tooling related to annotations to the :py:mod:`inspect`
module. However, that module is rather large, has direct or indirect dependencies
on at least 35 other standard library modules, and is so slow to import that other
standard library modules are often d


id: pep-0601.rst
content:
PEP: 601
Title: Forbid return/break/continue breaking out of finally
Author: Damien George, Batuhan Taskaya
Sponsor: Alyssa Coghlan
Discussions-To: https://discuss.python.org/t/pep-601-forbid-return-break-continue-breaking-out-of-finally/2239
Status: Rejected
Type: Standards Track
Created: 26-Aug-2019
Python-Version: 3.8
Post-History: 26-Aug-2019, 23-Sep-2019
Superseded-By: 765
Resolution: https://discuss.python.org/t/pep-601-forbid-return-break-continue-breaking-out-of-finally/2239/32

Rejection Note
==============

This PEP was rejected by the Steering Council by a vote of 4/4.

Guido's arguments for rejecting the PEP are: "it seems to me that most languages
implement this kind of construct but have style guides and/or linters that
reject it.  I would support a proposal to add this to :pep:`8`", and "I note that
the toy examples are somewhat misleading â the functionality that may be useful
is a conditional return (or break etc.) inside a finally block.".

Abstract
========

This PEP proposes to forbid ``return``, ``break`` and ``continue`` statements within
a ``finally`` suite where they would break out of the ``finally``.  Their use in
such a location silently cancels any active exception being raised through
the ``finally``, leading to unclear code and possible bugs.

``continue`` is currently not supported in a ``finally`` in Python 3.7 (due to
implementation issues) and the proposal is to not add support for it in
Python 3.8.  For ``return`` and ``break`` the proposal is to deprecate their use
in Python 3.9, emit a compilation warning in Python 3.10 and then forbid
their use after that.

Motivation
==========

The use of ``return``, ``break`` and ``continue`` within a ``finally`` suite leads to behaviour
which is not at all obvious.  Consider the following function::

    def foo():
        try:
            foo()
        finally:
            return

This will return cleanly (without an exception) even though it has infinite
recursion and raises an exception within the ``try``.  The reason is that the ``return``
within the ``finally`` will silently cancel any exception that propagates through
the ``finally`` suite.  Such behaviour is unexpected and not at all obvious.
This function is equivalent to::

    def foo():
        try:
            foo()
        except:
            pass
        return

``break`` and ``continue`` have similar behaviour (they silence exceptions) if they
jump to code outside the ``finally`` suite.  For example::

    def bar():
        while True:
            try:
                1 / 0
            finally:
                break

This behaviour goes against the following parts of The Zen of Python:

* Explicit is better than implicit - exceptions are implicitly silenced

* Readability counts - the intention of the code is not obvious

* Errors should never pass silently; Unless explicitly silenced - exceptions
  are implicitly silenced

If this behaviour of silencing exceptions is really needed then the explicit
form of a try-except can be used instead, and this makes the code clearer.

Independent to the semantics, implementing ``return``/``break``/``continue`` within a
``finally`` suite is non-trivial as it requires to correctly track any active
exceptions at runtime (an executing ``finally`` suite may or may not have an
active exception) and cancel them as appropriate.  CPython did have a bug in
this for the case of ``continue`` and so originally disallowed it [1]_.  Requiring
correct behaviour for ``return``/``break``/``continue`` within a ``finally`` puts an
unnecessary burden on alternative implementations of Python.

Other languages
===============

Java allows to return from within a ``finally`` block, but its use is discouraged
according to [2]_, [3]_, [4]_.  The Java compiler later on included a linting
option ``-Xlint:finally`` to warn against the use of return within a ``finally`` block.
The Eclipse editor also warns about this use.

Ruby allows return from inside ensure (Python's finally), but it should be an
explicit return.  It is discouraged and handled by linters [5]_, [6]_.

Like Ruby, JavaScript also allows use of ``return``/``break``/``continue`` within a ``finally``
but it is seen as unsafe and it is handled by eslint [7]_.

C# forbids the use of ending statements like ``return``/``goto``/``break`` within a ``finally``
[8]_, [9]_.

Rationale
=========

Since the behaviour of ``return``/``break``/``continue`` within a ``finally`` is unclear, the
pattern is rarely used, and there is a simple alternative to writing equivalent
code (which is more explicit), forbidding the syntax is the most straightforward
approach.

Specification
=============

This is a change to the compiler, not the grammar.  The compiler should
check for the following in a ``finally`` suite:

* A ``return`` in any statement, at any level of nesting.

* A ``break``/``continue`` in any statement, at any level of nesting, that would
  transfer control flow outside the ``finally`` suite.

Upon finding such a case it should emit the appropriate exception:

* For ``continue``, a ``SyntaxError`` (this is the current behaviour of 3.7).

* For ``return``/``break``, a ``SyntaxWarning`` in 3.10, and a ``SyntaxError`` after that.

For example, the following are all forbidden by this proposal::

    def f():
        try:
            pass
        finally:
            return

    def g():
        try:
            pass
        finally:
            try:
                return
            finally:
                pass

    def h():
        try:
            pass
        finally:
            try:
                pass
            finally:
                for x in range(10):
                    return

The following is still allowed because the ``continue`` doesn't escape the
``finally``::

    try:
        pass
    finally:
        for x in range(10):
            continue

Note that yielding from within a ``finally`` remains acceptable by this PEP
because resuming the generator will resume the ``finally`` and eventually
raise any active exceptions (so they are never silenced by yielding).

Backwards Compatibility
=======================

This is a backwards incompatible change, for ``return`` and ``break``.

The following locations in the CPython standard library (at
v3.8.0b1-651-g7fcc2088a5) use ``return`` within ``finally``:

* Lib/subprocess.py:921 - the use here looks like a bug

* Lib/multiprocessing/connection.py:316 - the use here looks legitimate
  but the intention is not clear

* Lib/multiprocessing/connection.py:318 - the use here looks legitimate
  but the intention is not clear

* Lib/test/test_sys_settrace.py:837 - a test for ``return`` within ``finally``

* Lib/test/test_sys_settrace.py:1346 - a test for ``return`` within ``finally``

There are no uses of ``break`` within a ``finally`` (that break out of the ``finally``)
in the standard library.

Security Implications
=====================

This is a simplification of the language, and removal of associated code,
so should not introduce any new paths for a security exploit.

How to Teach This
=================

This feature is very rarely used so forbidding it will likely only impact
advanced users, not beginners and probably not any existing teaching
material.  Since this is the removal of a feature teaching users will be
one by the raising of a ``SyntaxError`` if/when the forbidden feature is used.

Reference Implementation
========================

There is currently no reference implementation, although the way continue
is currently handled in a ``finally`` (raising a ``SyntaxError``) can be extended
to ``return`` and ``break``.

References
==========

.. [1] https://github.com/python/cpython/issues/82011

.. [2] https://stackoverflow.com/questions/48088/returning-from-a-finally-block-in-java

.. [3] https://web.archive.org/web/20070922061412/http://weblogs.java.net/blog/staufferjames/archive/2007/06/_dont_return_in.html

.. [4] https://wiki.sei.cmu.edu/confluence/display/java/ERR04


id: pep-0615.rst
content:
PEP: 615
Title: Support for the IANA Time Zone Database in the Standard Library
Author: Paul Ganssle <paul at ganssle.io>
Discussions-To: https://discuss.python.org/t/3468
Status: Final
Type: Standards Track
Created: 22-Feb-2020
Python-Version: 3.9
Post-History: 25-Feb-2020, 29-Mar-2020
Replaces: 431

.. canonical-doc:: :external+python:mod:`zoneinfo`

Abstract
========

This proposes adding a module, ``zoneinfo``, to provide a concrete time zone
implementation supporting the IANA time zone database.  By default,
``zoneinfo`` will use the system's time zone data if available; if no system
time zone data is available, the library will fall back to using the
first-party package ``tzdata``, deployed on PyPI. [d]_

Motivation
==========

The ``datetime`` library uses a flexible mechanism to handle time zones: all
conversions and time zone information queries are delegated to an instance of a
subclass of the abstract ``datetime.tzinfo`` base class. [#tzinfo]_ This allows
users to implement arbitrarily complex time zone rules, but in practice the
majority of users want support for just three types of time zone: [a]_

1. UTC and fixed offsets thereof
2. The system local time zone
3. IANA time zones

In Python 3.2, the ``datetime.timezone`` class was introduced to support the
first class of time zone (with a special ``datetime.timezone.utc`` singleton
for UTC).

While there is still no "local" time zone, in Python 3.0 the semantics of naÃ¯ve
time zones was changed to support many "local time" operations, and it is now
possible to get a fixed time zone offset from a local time::

    >>> print(datetime(2020, 2, 22, 12, 0).astimezone())
    2020-02-22 12:00:00-05:00
    >>> print(datetime(2020, 2, 22, 12, 0).astimezone()
    ...       .strftime("%Y-%m-%d %H:%M:%S %Z"))
    2020-02-22 12:00:00 EST
    >>> print(datetime(2020, 2, 22, 12, 0).astimezone(timezone.utc))
    2020-02-22 17:00:00+00:00

However, there is still no support for the time zones described in the IANA
time zone database (also called the "tz" database or the Olson database
[#tzdb-wiki]_).  The time zone database is in the public domain and is widely
distributed â it is present by default on many Unix-like operating systems.
Great care goes into the stability of the database: there are IETF RFCs both
for the maintenance procedures (:rfc:`6557`) and for the compiled
binary (TZif) format (:rfc:`8536`).  As such, it is likely that adding
support for the compiled outputs of the IANA database will add great value to
end users even with the relatively long cadence of standard library releases.


Proposal
========

This PEP has three main concerns:

1. The semantics of the ``zoneinfo.ZoneInfo`` class (zoneinfo-class_)
2. Time zone data sources used (data-sources_)
3. Options for configuration of the time zone search path (search-path-config_)

Because of the complexity of the proposal, rather than having separate
"specification" and "rationale" sections the design decisions and rationales
are grouped together by subject.

.. _zoneinfo-class:

The ``zoneinfo.ZoneInfo`` class
-------------------------------

Constructors
############

The initial design of the ``zoneinfo.ZoneInfo`` class has several constructors.

.. code-block::

    ZoneInfo(key: str)

The primary constructor takes a single argument, ``key``, which is a string
indicating the name of a zone file in the system time zone database (e.g.
``"America/New_York"``, ``"Europe/London"``), and returns a ``ZoneInfo``
constructed from the first matching data source on search path (see the
data-sources_ section for more details). All zone information must be eagerly
read from the data source (usually a TZif file) upon construction, and may
not change during the lifetime of the object (this restriction applies to all
``ZoneInfo`` constructors).

In the event that no matching file is found on the search path (either because
the system does not supply time zone data or because the key is invalid), the
constructor will raise a ``zoneinfo.ZoneInfoNotFoundError``, which will be a
subclass of ``KeyError``.

One somewhat unusual guarantee made by this constructor is that calls with
identical arguments must return *identical* objects. Specifically, for all
values of ``key``, the following assertion must always be valid [b]_::

    a = ZoneInfo(key)
    b = ZoneInfo(key)
    assert a is b

The reason for this comes from the fact that the semantics of datetime
operations (e.g. comparison, arithmetic) depend on whether the datetimes
involved represent the same or different zones; two datetimes are in the same
zone only if ``dt1.tzinfo is dt2.tzinfo``. [#nontransitive_comp]_ In addition
to the modest performance benefit from avoiding unnecessary proliferation of
``ZoneInfo`` objects, providing this guarantee should minimize surprising
behavior for end users.

|dateutil.tz.gettz| has provided a similar guarantee since version 2.7.0
(release March 2018). [#dateutil-tz]_

.. |dateutil.tz.gettz| replace:: ``dateutil.tz.gettz``
.. _dateutil.tz.gettz: https://dateutil.readthedocs.io/en/stable/tz.html#dateutil.tz.gettz

.. note::

    The implementation may decide how to implement the cache behavior, but the
    guarantee made here only requires that as long as two references exist to
    the result of identical constructor calls, they must be references to the
    same object. This is consistent with a reference counted cache where
    ``ZoneInfo`` objects are ejected when no references to them exist (for
    example, a cache implemented with a ``weakref.WeakValueDictionary``) â it is
    allowed but not required or recommended to implement this with a "strong"
    cache, where all ``ZoneInfo`` objects are kept alive indefinitely.

.. code-block::

    ZoneInfo.no_cache(key: str)

This is an alternate constructor that bypasses the constructor's cache.  It is
identical to the primary constructor, but returns a new object on each call.
This is likely most useful for testing purposes, or to deliberately induce
"different zone" semantics between datetimes with the same nominal time zone.

Even if an object constructed by this method would have been a cache miss, it
must not be entered into the cache; in other words, the following assertion
should always be true:

.. code-block::

    >>> a = ZoneInfo.no_cache(key)
    >>> b = ZoneInfo(key)
    >>> a is not b

.. code-block::

    ZoneInfo.from_file(fobj: IO[bytes], /, key: str = None)

This is an alternate constructor that allows the construction of a ``ZoneInfo``
object from any TZif byte stream.  This constructor takes an optional
parameter, ``key``, which sets the name of the zone, for the purposes of
``__str__`` and ``__repr__`` (see `Representations <string representation_>`_).

Unlike the primary constructor, this always constructs a new object.  There are
two reasons that this deviates from the primary constructor's caching behavior:
stream objects have mutable state and so determining whether two inputs are
identical is difficult or impossible, and it is likely that users constructing
from a file specifically want to load from that file and not a cache.

As with ``ZoneInfo.no_cache``, objects constructed by this method must not be
added to the cache.

Behavior during data updates
############################

It is important that a given ``ZoneInfo`` object's behavior not change during
its lifetime, because a ``datetime``'s ``utcoffset()`` method is used in both
its equality and hash calculations, and if the result were to change during the
``datetime``'s lifetime, it could break the invariant for all hashable objects
[#hashable_def]_ [#hashes_equality]_  that if ``x == y``, it must also be true
that ``hash(x) == hash(y)`` [c]_ .

Considering both the preservation of ``datetime``'s invariants and the
primary constructor's contract to always return the same object when called
with identical arguments, if a source of time zone data is updated during a run
of the interpreter, it must not invalidate any caches o


id: pep-0629.rst
content:
PEP: 629
Title: Versioning PyPI's Simple API
Author: Donald Stufft <donald@stufft.io>
BDFL-Delegate: Brett Cannon <brett@python.org>
Discussions-To: https://discuss.python.org/t/pep-629-versioning-pypis-simple-api/4720
Status: Final
Type: Standards Track
Topic: Packaging
Created: 16-Jul-2020
Post-History: 16-Jul-2020


.. note::
   This PEP was
   `accepted on 2020-08-20 <https://discuss.python.org/t/pep-629-versioning-pypis-simple-api/4720/15>`__.
   PyPI `merged an implementation <2bfa5a8c75e3af218494fe8de1eb809a43e3bbb9>`__
   on 2020-01-28, marking this PEP as "Final".


Abstract
========

This PEP proposes adding a method for versioning the simple API so
that clients can determine which features of the simple API that a
specific repository supports.


Rationale
=========

When evolving the simple API, clients wish to be able to determine
which features the repository supports. Currently there is no
mechanism to do this, except by attempting to detect new features
by looking at the data in the responses and see if it appears like
a particular feature is in use.

This works reasonably well for a modern version of a client to determine
if the repository supports all of the features it wants to implement,
however it does not do anything to tell an older version the client that
the repository supports features that it might not understand and to
allow messaging to indicate that it might not be correctly understanding
the output of the repository.

An example of a scenario where this happened was the phasing in of
python-requires metadata, while existing clients could still successfully
use the repository, they were lacking the ability to understand this new
piece of data which would have informed their behavior to select a better
file for end users.


Overview
========

This PEP proposes the inclusion of a meta tag on the responses of every
successful request to a simple API page, which contains a name attribute
of "pypi:repository-version", and a content that is a :pep:`440` compatible
version number, which is further constrained to ONLY be Major.Minor, and
none of the additional features supported by :pep:`440`.

This would end up looking like::

  <meta name="pypi:repository-version" content="1.0">

When interpreting the repository version:

* Incrementing the major version is used to signal a backwards
  incompatible change such that existing clients would no longer be
  expected to be able to meaningfully use the API.
* Incrementing the minor version is used to signal a backwards
  compatible change such that existing clients would still be
  expected to be able to meaningfully use the API.

It is left up to the discretion of any future PEPs as to what
specifically constitutes a backwards incompatible vs compatible change
beyond the broad suggestion that existing clients will be able to
"meaningfully" continue to use the API, and can include adding,
modifying, or removing existing features.

It is expectation of this PEP that the major version will never be
incremented, and any future major API evolutions would utilize a
different mechanism for API evolution. However the major version
is included to disambiguate with future versions (e.g. a hypothetical
simple api v2 that lived at /v2/, but which would be confusing if the
repository-version was set to a version >= 2).

This PEP sets the current API version to "1.0", and expects that
future PEPs that further evolve the simple API will increment the
minor version number.


Clients
-------

Clients interacting with the simple API **SHOULD** introspect each
response for the repository version, and if that data does not exist
**MUST** assume that it is version 1.0.

When encountering a major version greater than expected, clients
**MUST** hard fail with an appropriate error message for the user.

When encountering a minor version greater than expected, clients
**SHOULD** warn users with an appropriate message.

Clients **MAY** still continue to use feature detection in order to
determine what features a repository uses.


Rejected Ideas
==============

Using a Header
--------------

Instead of baking this information into the actual HTML, an
alternative would be to use a HTTP header. This idea was
considered and ultimately was rejected because it would make
mirrors have to start modifying headers instead of being able
to operate as a "dumb" HTTP server of files.


Using an URL
------------

Another traditional mechanism for versioning APIs is to bake it
into the URL, something like ``/1.0/simple/`` or so. This works
well for major version changes where olders clients are not
expected to be capable of continuing to use it, but it is not
well suited to minor version bumps, particularly when the version
numbers can be viewed as largely advisory for end users.




Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


id: pep-0365.rst
content:
PEP: 365
Title: Adding the pkg_resources module
Author: Phillip J. Eby <pje@telecommunity.com>
Status: Rejected
Type: Standards Track
Topic: Packaging
Created: 30-Apr-2007
Post-History: 30-Apr-2007


Abstract
========

This PEP proposes adding an enhanced version of the ``pkg_resources``
module to the standard library.

``pkg_resources`` is a module used to find and manage Python
package/version dependencies and access bundled files and resources,
including those inside of zipped ``.egg`` files.  Currently,
``pkg_resources`` is only available through installing the entire
``setuptools`` distribution, but it does not depend on any other part
of setuptools; in effect, it comprises the entire runtime support
library for Python Eggs, and is independently useful.

In addition, with one feature addition, this module could support
easy bootstrap installation of several Python package management
tools, including ``setuptools``, ``workingenv``, and ``zc.buildout``.


Proposal
========

Rather than proposing to include ``setuptools`` in the standard
library, this PEP proposes only that ``pkg_resources`` be added to the
standard library for Python 2.6 and 3.0.  ``pkg_resources`` is
considerably more stable than the rest of setuptools, with virtually
no new features being added in the last 12 months.

However, this PEP also proposes that a new feature be added to
``pkg_resources``, before being added to the stdlib.  Specifically, it
should be possible to do something like::

    python -m pkg_resources SomePackage==1.2

to request downloading and installation of ``SomePackage`` from PyPI.
This feature would *not* be a replacement for ``easy_install``;
instead, it would rely on ``SomePackage`` having pure-Python ``.egg``
files listed for download via the PyPI XML-RPC API, and the eggs would
be placed in the ``$PYTHON_EGG_CACHE`` directory, where they would
**not** be importable by default.  (And no scripts would be installed.)
However, if the download egg contains installation bootstrap code, it
will be given a chance to run.

These restrictions would allow the code to be extremely simple, yet
still powerful enough to support users downloading package management
tools such as ``setuptools``, ``workingenv`` and ``zc.buildout``,
simply by supplying the tool's name on the command line.


Rationale
=========

Many users have requested that ``setuptools`` be included in the
standard library, to save users needing to go through the awkward
process of bootstrapping it.  However, most of the bootstrapping
complexity comes from the fact that setuptools-installed code cannot
use the ``pkg_resources`` runtime module unless setuptools is already
installed. Thus, installing setuptools requires (in a sense) that
setuptools already be installed.

Other Python package management tools, such as ``workingenv`` and
``zc.buildout``, have similar bootstrapping issues, since they both
make use of setuptools, but also want to provide users with something
approaching a "one-step install".  The complexity of creating bootstrap
utilities for these and any other such tools that arise in future, is
greatly reduced if ``pkg_resources`` is already present, and is also
able to download pre-packaged eggs from PyPI.

(It would also mean that setuptools would not need to be installed
in order to simply *use* eggs, as opposed to building them.)

Finally, in addition to providing access to eggs built via setuptools
or other packaging tools, it should be noted that since Python 2.5,
the distutils install package metadata (aka ``PKG-INFO``) files that
can be read by ``pkg_resources`` to identify what distributions are
already on ``sys.path``.  In environments where Python packages are
installed using system package tools (like RPM), the ``pkg_resources``
module provides an API for detecting what versions of what packages
are installed, even if those packages were installed via the distutils
instead of setuptools.


Implementation and Documentation
================================

The ``pkg_resources`` implementation is maintained in the Python
SVN repository under ``/sandbox/trunk/setuptools/``; see
``pkg_resources.py`` and ``pkg_resources.txt``.  Documentation for the
egg format(s) supported by ``pkg_resources`` can be found in
``doc/formats.txt``.  HTML versions of these documents are available
at:

* http://peak.telecommunity.com/DevCenter/PkgResources and

* http://peak.telecommunity.com/DevCenter/EggFormats

(These HTML versions are for setuptools 0.6; they may not reflect all
of the changes found in the Subversion trunk's ``.txt`` versions.)


Copyright
=========

This document has been placed in the public domain.


id: pep-0403.rst
content:
PEP: 403
Title: General purpose decorator clause (aka "@in" clause)
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Deferred
Type: Standards Track
Created: 13-Oct-2011
Python-Version: 3.4
Post-History: 13-Oct-2011


Abstract
========

This PEP proposes the addition of a new ``@in`` decorator clause that makes
it possible to override the name binding step of a function or class
definition.

The new clause accepts a single simple statement that can make a forward
reference to decorated function or class definition.

This new clause is designed to be used whenever a "one-shot" function or
class is needed, and placing the function or class definition before the
statement that uses it actually makes the code harder to read. It also
avoids any name shadowing concerns by making sure the new name is visible
only to the statement in the ``@in`` clause.

This PEP is based heavily on many of the ideas in :pep:`3150` (Statement Local
Namespaces) so some elements of the rationale will be familiar to readers of
that PEP. Both PEPs remain deferred for the time being, primarily due to the
lack of compelling real world use cases in either PEP.


Basic Examples
==============

Before diving into the long history of this problem and the detailed
rationale for this specific proposed solution, here are a few simple
examples of the kind of code it is designed to simplify.

As a trivial example, a weakref callback could be defined as follows::

    @in x = weakref.ref(target, report_destruction)
    def report_destruction(obj):
        print("{} is being destroyed".format(obj))

This contrasts with the current (conceptually) "out of order" syntax for
this operation::

    def report_destruction(obj):
        print("{} is being destroyed".format(obj))

    x = weakref.ref(target, report_destruction)

That structure is OK when you're using the callable multiple times, but
it's irritating to be forced into it for one-off operations.

If the repetition of the name seems especially annoying, then a throwaway
name like ``f`` can be used instead::

    @in x = weakref.ref(target, f)
    def f(obj):
        print("{} is being destroyed".format(obj))


Similarly, a sorted operation on a particularly poorly defined type could
now be defined as::

    @in sorted_list = sorted(original, key=f)
    def f(item):
        try:
            return item.calc_sort_order()
        except NotSortableError:
            return float('inf')

Rather than::

    def force_sort(item):
        try:
            return item.calc_sort_order()
        except NotSortableError:
            return float('inf')

    sorted_list = sorted(original, key=force_sort)

And early binding semantics in a list comprehension could be attained via::

    @in funcs = [adder(i) for i in range(10)]
    def adder(i):
        return lambda x: x + i


Proposal
========

This PEP proposes the addition of a new ``@in`` clause that is a variant
of the existing class and function decorator syntax.

The new ``@in`` clause precedes the decorator lines, and allows forward
references to the trailing function or class definition.

The trailing function or class definition is always named - the name of
the trailing definition is then used to make the forward reference from the
``@in`` clause.

The ``@in`` clause is allowed to contain any simple statement (including
those that don't make any sense in that context, such as ``pass`` - while
such code would be legal, there wouldn't be any point in writing it). This
permissive structure is easier to define and easier to explain, but a more
restrictive approach that only permits operations that "make sense" would
also be possible (see :pep:`3150` for a list of possible candidates).

The ``@in`` clause will not create a new scope - all name binding
operations aside from the trailing function or class definition will affect
the containing scope.

The name used in the trailing function or class definition is only visible
from the associated ``@in`` clause, and behaves as if it was an ordinary
variable defined in that scope. If any nested scopes are created in either
the ``@in`` clause or the trailing function or class definition, those scopes
will see the trailing function or class definition rather than any other
bindings for that name in the containing scope.

In a very real sense, this proposal is about making it possible to override
the implicit "name = <defined function or class>" name binding operation
that is part of every function or class definition, specifically in those
cases where the local name binding isn't actually needed.

Under this PEP, an ordinary class or function definition::

    @deco2
    @deco1
    def name():
        ...

can be explained as being roughly equivalent to::

    @in name = deco2(deco1(name))
    def name():
        ...

Syntax Change
-------------

Syntactically, only one new grammar rule is needed::

    in_stmt: '@in' simple_stmt decorated

Grammar: http://hg.python.org/cpython/file/default/Grammar/Grammar


Design Discussion
=================

Background
----------

The question of "multi-line lambdas" has been a vexing one for many
Python users for a very long time, and it took an exploration of Ruby's
block functionality for me to finally understand why this bugs people
so much: Python's demand that the function be named and introduced
before the operation that needs it breaks the developer's flow of thought.
They get to a point where they go "I need a one-shot operation that does
<X>", and instead of being able to just *say* that directly, they instead
have to back up, name a function to do <X>, then call that function from
the operation they actually wanted to do in the first place. Lambda
expressions can help sometimes, but they're no substitute for being able to
use a full suite.

Ruby's block syntax also heavily inspired the style of the solution in this
PEP, by making it clear that even when limited to *one* anonymous function per
statement, anonymous functions could still be incredibly useful. Consider how
many constructs Python has where one expression is responsible for the bulk of
the heavy lifting:

* comprehensions, generator expressions, map(), filter()
* key arguments to sorted(), min(), max()
* partial function application
* provision of callbacks (e.g. for weak references or asynchronous IO)
* array broadcast operations in NumPy

However, adopting Ruby's block syntax directly won't work for Python, since
the effectiveness of Ruby's blocks relies heavily on various conventions in
the way functions are *defined* (specifically, using Ruby's ``yield`` syntax
to call blocks directly and the ``&arg`` mechanism to accept a block as a
function's final argument).

Since Python has relied on named functions for so long, the signatures of
APIs that accept callbacks are far more diverse, thus requiring a solution
that allows one-shot functions to be slotted in at the appropriate location.

The approach taken in this PEP is to retain the requirement to name the
function explicitly, but allow the relative order of the definition and the
statement that references it to be changed to match the developer's flow of
thought. The rationale is essentially the same as that used when introducing
decorators, but covering a broader set of applications.


Relation to PEP 3150
--------------------

:pep:`3150` (Statement Local Namespaces) describes its primary motivation
as being to elevate ordinary assignment statements to be on par with ``class``
and ``def`` statements where the name of the item to be defined is presented
to the reader in advance of the details of how the value of that item is
calculated. This PEP achieves the same goal in a different way, by allowing
the simple name binding of a standard function definition to be replaced
with something else (like assigning the result of the function to a value).

Despite having the same author, the two PEPs are in direct competition with
each other. :pep:`403` represents a minimalist approach that atte


id: pep-0417.rst
content:
PEP: 417
Title: Including mock in the Standard Library
Author: Michael Foord <michael@python.org>
Status: Final
Type: Standards Track
Created: 12-Mar-2012
Python-Version: 3.3
Post-History: 12-Mar-2012
Resolution: https://mail.python.org/pipermail/python-dev/2012-March/117507.html


Abstract
========

This PEP proposes adding the mock [1]_ testing library
to the Python standard library as ``unittest.mock``.


Rationale
=========

Creating mock objects for testing is a common need in Python.
Many developers create ad-hoc mocks, as needed, in their test
suites. This is currently what we do in the Python test suite,
where a standardised mock object library would be helpful.

There are many mock object libraries available for Python [2]_.
Of these, mock is overwhelmingly the most popular, with as many
downloads on PyPI as the other mocking libraries combined.

An advantage of mock is that it is a mocking library and not a
framework. It provides a configurable and flexible mock object,
without being opinionated about how you write your tests. The
mock api is now well battle-tested and stable.

mock also handles safely monkeypatching and unmonkeypatching
objects during the scope of a test. This is hard to do safely
and many developers / projects mimic this functionality
(often incorrectly). A standardised way to do this, handling
the complexity of patching in the presence of the descriptor
protocol (etc) is useful. People are asking for a "patch" [3]_
feature to unittest. Doing this via mock.patch is preferable
to re-implementing part of this functionality in unittest.


Background
==========

Addition of mock to the Python standard library was discussed
and agreed to at the Python Language Summit 2012.


Open Issues
===========

As of release 0.8, which is current at the time of writing,
mock is compatible with Python 2.4-3.2. Moving into the Python
standard library will allow for the removal of some Python 2
specific "compatibility hacks".

mock 0.8 introduced a new feature, "auto-speccing", obsoletes
an older mock feature called "mocksignature". The
"mocksignature" functionality can be removed from mock
altogether prior to inclusion.


References
==========

.. [1] `mock library on PyPI <http://pypi.python.org/pypi/mock>`_
.. [2] http://pypi.python.org/pypi?%3Aaction=search&term=mock&submit=search
.. [3] http://bugs.python.org/issue11664


Copyright
=========

This document has been placed in the public domain.


id: pep-0371.rst
content:
PEP: 371
Title: Addition of the multiprocessing package to the standard library
Author: Jesse Noller <jnoller@gmail.com>,
        Richard Oudkerk <r.m.oudkerk@googlemail.com>
Status: Final
Type: Standards Track
Created: 06-May-2008
Python-Version: 2.6, 3.0
Post-History: `03-Jun-2008 <https://mail.python.org/pipermail/python-dev/2008-June/080011.html>`__


Abstract
========

This PEP proposes the inclusion of the ``pyProcessing`` [1]_ package
into the Python standard library, renamed to "multiprocessing".

The ``processing`` package mimics the standard library ``threading``
module functionality to provide a process-based approach to
threaded programming allowing end-users to dispatch multiple
tasks that effectively side-step the global interpreter lock.

The package also provides server and client functionality
(``processing.Manager``) to provide remote sharing and management of
objects and tasks so that applications may not only leverage
multiple cores on the local machine, but also distribute objects
and tasks across a cluster of networked machines.

While the distributed capabilities of the package are beneficial,
the primary focus of this PEP is the core threading-like API and
capabilities of the package.

Rationale
=========

The current CPython interpreter implements the Global Interpreter
Lock (GIL) and barring work in Python 3000 or other versions
currently planned [2]_, the GIL will remain as-is within the
CPython interpreter for the foreseeable future.  While the GIL
itself enables clean and easy to maintain C code for the
interpreter and extensions base, it is frequently an issue for
those Python programmers who are leveraging multi-core machines.

The GIL itself prevents more than a single thread from running
within the interpreter at any given point in time, effectively
removing Python's ability to take advantage of multi-processor
systems.

The pyprocessing package offers a method to side-step the GIL
allowing applications within CPython to take advantage of
multi-core architectures without asking users to completely change
their programming paradigm (i.e.: dropping threaded programming
for another "concurrent" approach - Twisted, Actors, etc).

The Processing package offers CPython a "known API" which mirrors
albeit in a :pep:`8` compliant manner, that of the threading API,
with known semantics and easy scalability.

In the future, the package might not be as relevant should the
CPython interpreter enable "true" threading, however for some
applications, forking an OS process may sometimes be more
desirable than using lightweight threads, especially on those
platforms where process creation is fast and optimized.

For example, a simple threaded application::

    from threading import Thread as worker

    def afunc(number):
        print number * 3

    t = worker(target=afunc, args=(4,))
    t.start()
    t.join()

The pyprocessing package mirrored the API so well, that with a
simple change of the import to::

    from processing import process as worker

The code would now execute through the processing.process class.
Obviously, with the renaming of the API to :pep:`8` compliance there
would be additional renaming which would need to occur within
user applications, however minor.

This type of compatibility means that, with a minor (in most cases)
change in code, users' applications will be able to leverage all
cores and processors on a given machine for parallel execution.
In many cases the pyprocessing package is even faster than the
normal threading approach for I/O bound programs.  This of course,
takes into account that the pyprocessing package is in optimized C
code, while the threading module is not.

The "Distributed" Problem
=========================

In the discussion on Python-Dev about the inclusion of this
package [3]_ there was confusion about the intentions this PEP with
an attempt to solve the "Distributed" problem - frequently
comparing the functionality of this package with other solutions
like MPI-based communication [4]_, CORBA, or other distributed
object approaches [5]_.

The "distributed" problem is large and varied.  Each programmer
working within this domain has either very strong opinions about
their favorite module/method or a highly customized problem for
which no existing solution works.

The acceptance of this package does not preclude or recommend that
programmers working on the "distributed" problem not examine other
solutions for their problem domain.  The intent of including this
package is to provide entry-level capabilities for local
concurrency and the basic support to spread that concurrency
across a network of machines - although the two are not tightly
coupled, the pyprocessing package could in fact, be used in
conjunction with any of the other solutions including MPI/etc.

If necessary - it is possible to completely decouple the local
concurrency abilities of the package from the
network-capable/shared aspects of the package.  Without serious
concerns or cause however, the author of this PEP does not
recommend that approach.

Performance Comparison
======================

As we all know - there are "lies, damned lies, and benchmarks".
These speed comparisons, while aimed at showcasing the performance
of the pyprocessing package, are by no means comprehensive or
applicable to all possible use cases or environments.  Especially
for those platforms with sluggish process forking timing.

All benchmarks were run using the following:

* 4 Core Intel Xeon CPU @ 3.00GHz
* 16 GB of RAM
* Python 2.5.2 compiled on Gentoo Linux (kernel 2.6.18.6)
* pyProcessing 0.52

All of the code for this can be downloaded from
http://jessenoller.com/code/bench-src.tgz

The basic method of execution for these benchmarks is in the
run_benchmarks.py [6]_ script, which is simply a wrapper to execute a
target function through a single threaded (linear), multi-threaded
(via threading), and multi-process (via pyprocessing) function for
a static number of iterations with increasing numbers of execution
loops and/or threads.

The run_benchmarks.py script executes each function 100 times,
picking the best run of that 100 iterations via the timeit module.

First, to identify the overhead of the spawning of the workers, we
execute a function which is simply a pass statement (empty)::

    cmd: python run_benchmarks.py empty_func.py
    Importing empty_func
    Starting tests ...
    non_threaded (1 iters)  0.000001 seconds
    threaded (1 threads)    0.000796 seconds
    processes (1 procs)     0.000714 seconds

    non_threaded (2 iters)  0.000002 seconds
    threaded (2 threads)    0.001963 seconds
    processes (2 procs)     0.001466 seconds

    non_threaded (4 iters)  0.000002 seconds
    threaded (4 threads)    0.003986 seconds
    processes (4 procs)     0.002701 seconds

    non_threaded (8 iters)  0.000003 seconds
    threaded (8 threads)    0.007990 seconds
    processes (8 procs)     0.005512 seconds

As you can see, process forking via the pyprocessing package is
faster than the speed of building and then executing the threaded
version of the code.

The second test calculates 50000 Fibonacci numbers inside of each
thread (isolated and shared nothing)::

    cmd: python run_benchmarks.py fibonacci.py
    Importing fibonacci
    Starting tests ...
    non_threaded (1 iters)  0.195548 seconds
    threaded (1 threads)    0.197909 seconds
    processes (1 procs)     0.201175 seconds

    non_threaded (2 iters)  0.397540 seconds
    threaded (2 threads)    0.397637 seconds
    processes (2 procs)     0.204265 seconds

    non_threaded (4 iters)  0.795333 seconds
    threaded (4 threads)    0.797262 seconds
    processes (4 procs)     0.206990 seconds

    non_threaded (8 iters)  1.591680 seconds
    threaded (8 threads)    1.596824 seconds
    processes (8 procs)     0.417899 seconds

The third test calculates the sum of all primes below 100000,
again sharing nothing::

    cmd: run_benchmarks.py crunch_primes.py
    Import


id: pep-0359.rst
content:
PEP: 359
Title: The "make" Statement
Author: Steven Bethard <steven.bethard@gmail.com>
Status: Withdrawn
Type: Standards Track
Created: 05-Apr-2006
Python-Version: 2.6
Post-History: 05-Apr-2006, 06-Apr-2006, 13-Apr-2006


Abstract
========

This PEP proposes a generalization of the class-declaration syntax,
the ``make`` statement.  The proposed syntax and semantics parallel
the syntax for class definition, and so::

   make <callable> <name> <tuple>:
       <block>

is translated into the assignment::

   <name> = <callable>("<name>", <tuple>, <namespace>)

where ``<namespace>`` is the dict created by executing ``<block>``.
This is mostly syntactic sugar for::

   class <name> <tuple>:
       __metaclass__ = <callable>
       <block>

and is intended to help more clearly express the intent of the
statement when something other than a class is being created.  Of
course, other syntax for such a statement is possible, but it is hoped
that by keeping a strong parallel to the class statement, an
understanding of how classes and metaclasses work will translate into
an understanding of how the make-statement works as well.

The PEP is based on a suggestion [1]_ from Michele Simionato on the
python-dev list.


Withdrawal Notice
=================

This PEP was withdrawn at Guido's request [2]_. Guido didn't like it,
and in particular didn't like how the property use-case puts the
instance methods of a property at a different level than other
instance methods and requires fixed names for the property functions.


Motivation
==========

Class statements provide two nice facilities to Python:

(1) They execute a block of statements and provide the resulting
    bindings as a dict to the metaclass.

(2) They encourage DRY (don't repeat yourself) by allowing the class
    being created to know the name it is being assigned.

Thus in a simple class statement like::

     class C(object):
         x = 1
         def foo(self):
             return 'bar'

the metaclass (``type``) gets called with something like::

    C = type('C', (object,), {'x':1, 'foo':<function foo at ...>})

The class statement is just syntactic sugar for the above assignment
statement, but clearly a very useful sort of syntactic sugar.  It
avoids not only the repetition of ``C``, but also simplifies the
creation of the dict by allowing it to be expressed as a series of
statements.

Historically, type instances (a.k.a. class objects) have been the
only objects blessed with this sort of syntactic support.  The make
statement aims to extend this support to other sorts of objects where
such syntax would also be useful.


Example: simple namespaces
--------------------------

Let's say I have some attributes in a module that I access like::

    mod.thematic_roletype
    mod.opinion_roletype

    mod.text_format
    mod.html_format

and since "Namespaces are one honking great idea", I'd like to be able
to access these attributes instead as::

    mod.roletypes.thematic
    mod.roletypes.opinion

    mod.format.text
    mod.format.html

I currently have two main options:

(1) Turn the module into a package, turn ``roletypes`` and ``format``
    into submodules, and move the attributes to the submodules.

(2) Create ``roletypes`` and ``format`` classes, and move the
    attributes to the classes.

The former is a fair chunk of refactoring work, and produces two tiny
modules without much content.  The latter keeps the attributes local
to the module, but creates classes when there is no intention of ever
creating instances of those classes.

In situations like this, it would be nice to simply be able to declare
a "namespace" to hold the few attributes.  With the new make
statement, I could introduce my new namespaces with something like::

    make namespace roletypes:
        thematic = ...
        opinion = ...

    make namespace format:
        text = ...
        html = ...

and keep my attributes local to the module without making classes that
are never intended to be instantiated.  One definition of namespace
that would make this work is::

    class namespace(object):
        def __init__(self, name, args, kwargs):
            self.__dict__.update(kwargs)

Given this definition, at the end of the make-statements above,
``roletypes`` and ``format`` would be namespace instances.


Example: GUI objects
--------------------

In GUI toolkits, objects like frames and panels are often associated
with attributes and functions.  With the make-statement, code that
looks something like::

    root = Tkinter.Tk()
    frame = Tkinter.Frame(root)
    frame.pack()
    def say_hi():
        print "hi there, everyone!"
    hi_there = Tkinter.Button(frame, text="Hello", command=say_hi)
    hi_there.pack(side=Tkinter.LEFT)
    root.mainloop()

could be rewritten to group the Button's function with its
declaration::

    root = Tkinter.Tk()
    frame = Tkinter.Frame(root)
    frame.pack()
    make Tkinter.Button hi_there(frame):
        text = "Hello"
        def command():
            print "hi there, everyone!"
    hi_there.pack(side=Tkinter.LEFT)
    root.mainloop()


Example: custom descriptors
---------------------------

Since descriptors are used to customize access to an attribute, it's
often useful to know the name of that attribute.  Current Python
doesn't give an easy way to find this name and so a lot of custom
descriptors, like Ian Bicking's setonce descriptor [3]_, have to hack
around this somehow.  With the make-statement, you could create a
``setonce`` attribute like::

    class A(object):
        ...
        make setonce x:
            "A's x attribute"
        ...

where the ``setonce`` descriptor would be defined like::

    class setonce(object):

        def __init__(self, name, args, kwargs):
            self._name = '_setonce_attr_%s' % name
            self.__doc__ = kwargs.pop('__doc__', None)

        def __get__(self, obj, type=None):
            if obj is None:
                return self
            return getattr(obj, self._name)

        def __set__(self, obj, value):
            try:
                getattr(obj, self._name)
            except AttributeError:
                setattr(obj, self._name, value)
            else:
                raise AttributeError("Attribute already set")

        def set(self, obj, value):
            setattr(obj, self._name, value)

        def __delete__(self, obj):
            delattr(obj, self._name)

Note that unlike the original implementation, the private attribute
name is stable since it uses the name of the descriptor, and therefore
instances of class A are pickleable.


Example: property namespaces
----------------------------

Python's property type takes three function arguments and a docstring
argument which, though relevant only to the property, must be declared
before it and then passed as arguments to the property call, e.g.::

    class C(object):
        ...
        def get_x(self):
            ...
        def set_x(self):
            ...
        x = property(get_x, set_x, "the x of the frobulation")

This issue has been brought up before, and Guido [4]_ and others [5]_
have briefly mused over alternate property syntaxes to make declaring
properties easier.  With the make-statement, the following syntax
could be supported::

    class C(object):
        ...
        make block_property x:
            '''The x of the frobulation'''
            def fget(self):
                ...
            def fset(self):
                ...

with the following definition of ``block_property``::

    def block_property(name, args, block_dict):
        fget = block_dict.pop('fget', None)
        fset = block_dict.pop('fset', None)
        fdel = block_dict.pop('fdel', None)
        doc = block_dict.pop('__doc__', None)
        assert not block_dict
        return property(fget, fset, fdel, doc)


Example: interfaces
-------------------

Guido [6]_ and others have occasionally suggested introducing
interfaces into python.  Most suggestions have offered s


id: pep-0358.rst
content:
PEP: 358
Title: The "bytes" Object
Author: Neil Schemenauer <nas@arctrix.com>, Guido van Rossum <guido@python.org>
Status: Final
Type: Standards Track
Created: 15-Feb-2006
Python-Version: 2.6, 3.0
Post-History:


Update
======

This PEP has partially been superseded by :pep:`3137`.


Abstract
========

This PEP outlines the introduction of a raw bytes sequence type.
Adding the bytes type is one step in the transition to
Unicode-based str objects which will be introduced in Python 3.0.

The PEP describes how the bytes type should work in Python 2.6, as
well as how it should work in Python 3.0.  (Occasionally there are
differences because in Python 2.6, we have two string types, str
and unicode, while in Python 3.0 we will only have one string
type, whose name will be str but whose semantics will be like the
2.6 unicode type.)


Motivation
==========

Python's current string objects are overloaded.  They serve to hold
both sequences of characters and sequences of bytes.  This
overloading of purpose leads to confusion and bugs.  In future
versions of Python, string objects will be used for holding
character data.  The bytes object will fulfil the role of a byte
container.  Eventually the unicode type will be renamed to str
and the old str type will be removed.


Specification
=============

A bytes object stores a mutable sequence of integers that are in
the range 0 to 255.  Unlike string objects, indexing a bytes
object returns an integer.  Assigning or comparing an object that
is not an integer to an element causes a ``TypeError`` exception.
Assigning an element to a value outside the range 0 to 255 causes
a ``ValueError`` exception.  The ``.__len__()`` method of bytes returns
the number of integers stored in the sequence (i.e. the number of
bytes).

The constructor of the bytes object has the following signature::

    bytes([initializer[, encoding]])

If no arguments are provided then a bytes object containing zero
elements is created and returned.  The initializer argument can be
a string (in 2.6, either str or unicode), an iterable of integers,
or a single integer.  The pseudo-code for the constructor
(optimized for clear semantics, not for speed) is::

    def bytes(initializer=0, encoding=None):
        if isinstance(initializer, int): # In 2.6, int -> (int, long)
            initializer = [0]*initializer
        elif isinstance(initializer, basestring):
            if isinstance(initializer, unicode): # In 3.0, "if True"
                if encoding is None:
                    # In 3.0, raise TypeError("explicit encoding required")
                    encoding = sys.getdefaultencoding()
                initializer = initializer.encode(encoding)
            initializer = [ord(c) for c in initializer]
        else:
            if encoding is not None:
                raise TypeError("no encoding allowed for this initializer")
            tmp = []
            for c in initializer:
                if not isinstance(c, int):
                    raise TypeError("initializer must be iterable of ints")
                if not 0 <= c < 256:
                    raise ValueError("initializer element out of range")
                tmp.append(c)
            initializer = tmp
        new = <new bytes object of length len(initializer)>
        for i, c in enumerate(initializer):
            new[i] = c
        return new

The ``.__repr__()`` method returns a string that can be evaluated to
generate a new bytes object containing a bytes literal::

    >>> bytes([10, 20, 30])
    b'\n\x14\x1e'

The object has a ``.decode()`` method equivalent to the ``.decode()``
method of the str object.  The object has a classmethod ``.fromhex()``
that takes a string of characters from the set ``[0-9a-fA-F ]`` and
returns a bytes object (similar to ``binascii.unhexlify``).  For
example::

    >>> bytes.fromhex('5c5350ff')
    b'\\SP\xff'
    >>> bytes.fromhex('5c 53 50 ff')
    b'\\SP\xff'

The object has a ``.hex()`` method that does the reverse conversion
(similar to ``binascii.hexlify``)::

    >> bytes([92, 83, 80, 255]).hex()
    '5c5350ff'

The bytes object has some methods similar to list methods, and
others similar to str methods.  Here is a complete list of
methods, with their approximate signatures::

    .__add__(bytes) -> bytes
    .__contains__(int | bytes) -> bool
    .__delitem__(int | slice) -> None
    .__delslice__(int, int) -> None
    .__eq__(bytes) -> bool
    .__ge__(bytes) -> bool
    .__getitem__(int | slice) -> int | bytes
    .__getslice__(int, int) -> bytes
    .__gt__(bytes) -> bool
    .__iadd__(bytes) -> bytes
    .__imul__(int) -> bytes
    .__iter__() -> iterator
    .__le__(bytes) -> bool
    .__len__() -> int
    .__lt__(bytes) -> bool
    .__mul__(int) -> bytes
    .__ne__(bytes) -> bool
    .__reduce__(...) -> ...
    .__reduce_ex__(...) -> ...
    .__repr__() -> str
    .__reversed__() -> bytes
    .__rmul__(int) -> bytes
    .__setitem__(int | slice, int | iterable[int]) -> None
    .__setslice__(int, int, iterable[int]) -> Bote
    .append(int) -> None
    .count(int) -> int
    .decode(str) -> str | unicode # in 3.0, only str
    .endswith(bytes) -> bool
    .extend(iterable[int]) -> None
    .find(bytes) -> int
    .index(bytes | int) -> int
    .insert(int, int) -> None
    .join(iterable[bytes]) -> bytes
    .partition(bytes) -> (bytes, bytes, bytes)
    .pop([int]) -> int
    .remove(int) -> None
    .replace(bytes, bytes) -> bytes
    .rindex(bytes | int) -> int
    .rpartition(bytes) -> (bytes, bytes, bytes)
    .split(bytes) -> list[bytes]
    .startswith(bytes) -> bool
    .reverse() -> None
    .rfind(bytes) -> int
    .rindex(bytes | int) -> int
    .rsplit(bytes) -> list[bytes]
    .translate(bytes, [bytes]) -> bytes

Note the conspicuous absence of ``.isupper()``, ``.upper()``, and friends.
(But see "Open Issues" below.)  There is no ``.__hash__()`` because
the object is mutable.  There is no use case for a ``.sort()`` method.

The bytes type also supports the buffer interface, supporting
reading and writing binary (but not character) data.


Out of Scope Issues
===================

* Python 3k will have a much different I/O subsystem.  Deciding
  how that I/O subsystem will work and interact with the bytes
  object is out of the scope of this PEP.  The expectation however
  is that binary I/O will read and write bytes, while text I/O
  will read strings.  Since the bytes type supports the buffer
  interface, the existing binary I/O operations in Python 2.6 will
  support bytes objects.

* It has been suggested that a special method named ``.__bytes__()``
  be added to the language to allow objects to be converted into
  byte arrays.  This decision is out of scope.

* A bytes literal of the form ``b"..."`` is also proposed.  This is
  the subject of :pep:`3112`.


Open Issues
===========

* The ``.decode()`` method is redundant since a bytes object ``b`` can
  also be decoded by calling ``unicode(b, <encoding>)`` (in 2.6) or
  ``str(b, <encoding>)`` (in 3.0).  Do we need encode/decode methods
  at all?  In a sense the spelling using a constructor is cleaner.

* Need to specify the methods still more carefully.

* Pickling and marshalling support need to be specified.

* Should all those list methods really be implemented?

* A case could be made for supporting ``.ljust()``, ``.rjust()``,
  ``.center()`` with a mandatory second argument.

* A case could be made for supporting ``.split()`` with a mandatory
  argument.

* A case could even be made for supporting ``.islower()``, ``.isupper()``,
  ``.isspace()``, ``.isalpha()``, ``.isalnum()``, ``.isdigit()`` and the
  corresponding conversions (``.lower()`` etc.), using the ASCII
  definitions for letters, digits and whitespace.  If this is
  accepted, the cases for ``.ljust()``, ``.rjust()``, ``.center()`` and
  ``.split()`` become much stronger, and they should have default
  arguments as well, using an ASCII space or all ASCII whitespace
  (for ``.split()``).


Frequently Ask


id: pep-0416.rst
content:
PEP: 416
Title: Add a frozendict builtin type
Author: Victor Stinner <vstinner@python.org>
Status: Rejected
Type: Standards Track
Created: 29-Feb-2012
Python-Version: 3.3


Rejection Notice
================

I'm rejecting this PEP.  A number of reasons (not exhaustive):

* According to Raymond Hettinger, use of frozendict is low.  Those
  that do use it tend to use it as a hint only, such as declaring
  global or class-level "constants": they aren't really immutable,
  since anyone can still assign to the name.
* There are existing idioms for avoiding mutable default values.
* The potential of optimizing code using frozendict in PyPy is
  unsure; a lot of other things would have to change first.  The same
  holds for compile-time lookups in general.
* Multiple threads can agree by convention not to mutate a shared
  dict, there's no great need for enforcement.  Multiple processes
  can't share dicts.
* Adding a security sandbox written in Python, even with a limited
  scope, is frowned upon by many, due to the inherent difficulty with
  ever proving that the sandbox is actually secure.  Because of this
  we won't be adding one to the stdlib any time soon, so this use
  case falls outside the scope of a PEP.

On the other hand, exposing the existing read-only dict proxy as a
built-in type sounds good to me.  (It would need to be changed to
allow calling the constructor.)  GvR.

**Update** (2012-04-15): A new ``MappingProxyType`` type was added to the types
module of Python 3.3.


Abstract
========

Add a new frozendict builtin type.


Rationale
=========

A frozendict is a read-only mapping: a key cannot be added nor removed, and a
key is always mapped to the same value. However, frozendict values can be not
hashable. A frozendict is hashable if and only if all values are hashable.

Use cases:

* Immutable global variable like a default configuration.
* Default value of a function parameter. Avoid the issue of mutable default
  arguments.
* Implement a cache: frozendict can be used to store function keywords.
  frozendict can be used as a key of a mapping or as a member of set.
* frozendict avoids the need of a lock when the frozendict is shared
  by multiple threads or processes, especially hashable frozendict. It would
  also help to prohibe coroutines (generators + greenlets) to modify the
  global state.
* frozendict lookup can be done at compile time instead of runtime because the
  mapping is read-only. frozendict can be used instead of a preprocessor to
  remove conditional code at compilation, like code specific to a debug build.
* frozendict helps to implement read-only object proxies for security modules.
  For example, it would be possible to use frozendict type for __builtins__
  mapping or type.__dict__. This is possible because frozendict is compatible
  with the PyDict C API.
* frozendict avoids the need of a read-only proxy in some cases. frozendict is
  faster than a proxy because getting an item in a frozendict is a fast lookup
  whereas a proxy requires a function call.


Constraints
===========

* frozendict has to implement the Mapping abstract base class
* frozendict keys and values can be unorderable
* a frozendict is hashable if all keys and values are hashable
* frozendict hash does not depend on the items creation order


Implementation
==============

* Add a PyFrozenDictObject structure based on PyDictObject with an extra
  "Py_hash_t hash;" field
* frozendict.__hash__() is implemented using hash(frozenset(self.items())) and
  caches the result in its private hash attribute
* Register frozendict as a collections.abc.Mapping
* frozendict can be used with PyDict_GetItem(), but PyDict_SetItem() and
  PyDict_DelItem() raise a TypeError


Recipe: hashable dict
======================

To ensure that a frozendict is hashable, values can be checked
before creating the frozendict::

    import itertools

    def hashabledict(*args, **kw):
        # ensure that all values are hashable
        for key, value in itertools.chain(args, kw.items()):
            if isinstance(value, (int, str, bytes, float, frozenset, complex)):
                # avoid the compute the hash (which may be slow) for builtin
                # types known to be hashable for any value
                continue
            hash(value)
            # don't check the key: frozendict already checks the key
        return frozendict.__new__(cls, *args, **kw)


Objections
==========

*namedtuple may fit the requirements of a frozendict.*

A namedtuple is not a mapping, it does not implement the Mapping abstract base
class.

*frozendict can be implemented in Python using descriptors" and "frozendict
just need to be practically constant.*

If frozendict is used to harden Python (security purpose), it must be
implemented in C. A type implemented in C is also faster.

*The* :pep:`351` *was rejected.*

The :pep:`351` tries to freeze an object and so may convert a mutable object to an
immutable object (using a different type). frozendict doesn't convert anything:
hash(frozendict) raises a TypeError if a value is not hashable. Freezing an
object is not the purpose of this PEP.


Alternative: dictproxy
======================

Python has a builtin dictproxy type used by type.__dict__ getter descriptor.
This type is not public. dictproxy is a read-only view of a dictionary, but it
is not read-only mapping.  If a dictionary is modified, the dictproxy is also
modified.

dictproxy can be used using ctypes and the Python C API, see for example the
`make dictproxy object via ctypes.pythonapi and type() (Python recipe 576540)`_
by Ikkei Shimomura. The recipe contains a test checking that a dictproxy is
"mutable" (modify the dictionary linked to the dictproxy).

However dictproxy can be useful in some cases, where its mutable property is
not an issue, to avoid a copy of the dictionary.


Existing implementations
========================

Whitelist approach.

* `Implementing an Immutable Dictionary (Python recipe 498072)
  <http://code.activestate.com/recipes/498072/>`_ by Aristotelis Mikropoulos.
  Similar to frozendict except that it is not truly read-only: it is possible
  to access to this private internal dict.  It does not implement __hash__ and
  has an implementation issue: it is possible to call again __init__() to
  modify the mapping.
* PyWebmail contains an ImmutableDict type: `webmail.utils.ImmutableDict
  <http://pywebmail.cvs.sourceforge.net/viewvc/pywebmail/webmail/webmail/utils/ImmutableDict.py?revision=1.2&view=markup>`_.
  It is hashable if keys and values are hashable. It is not truly read-only:
  its internal dict is a public attribute.
* remember project: `remember.dicts.FrozenDict
  <https://bitbucket.org/mikegraham/remember/src/tip/remember/dicts.py>`_.
  It is used to implement a cache: FrozenDict is used to store function callbacks.
  FrozenDict may be hashable. It has an extra supply_dict() class method to
  create a FrozenDict from a dict without copying the dict: store the dict as
  the internal dict. Implementation issue: __init__() can be called to modify
  the mapping and the hash may differ depending on item creation order. The
  mapping is not truly read-only: the internal dict is accessible in Python.


Blacklist approach: inherit from dict and override write methods to raise an
exception. It is not truly read-only: it is still possible to call dict methods
on such "frozen dictionary" to modify it.

* brownie: `brownie.datastructures.ImmutableDict
  <https://github.com/DasIch/brownie/blob/HEAD/brownie/datastructures/mappings.py>`_.
  It is hashable if keys and values are hashable. werkzeug project has the
  same code: `werkzeug.datastructures.ImmutableDict
  <https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/datastructures.py>`_.
  ImmutableDict is used for global constant (configuration options). The Flask
  project uses ImmutableDict of werkzeug for its default configuration.
* SQLAlchemy project: `sqlalchemy.util.immutabledict
  <http://hg


id: pep-0370.rst
content:
PEP: 370
Title: Per user site-packages directory
Author: Christian Heimes <christian@python.org>
Status: Final
Type: Standards Track
Created: 11-Jan-2008
Python-Version: 2.6, 3.0
Post-History:


Abstract
========

This PEP proposes a new a per user site-packages directory to allow
users the local installation of Python packages in their home directory.


Rationale
=========

Current Python versions don't have a unified way to install packages
into the home directory of a user (except for Mac Framework
builds). Users are either forced to ask the system administrator to
install or update a package for them or to use one of the many
workarounds like Virtual Python [1]_, Working Env [2]_ or
Virtual Env [3]_.

It's not the goal of the PEP to replace the tools or to implement
isolated installations of Python. It only implements the most common
use case of an additional site-packages directory for each user.

The feature can't be implemented using the environment variable
*PYTHONPATH*. The env var just inserts a new directory to the beginning
of *sys.path* but it doesn't parse the pth files in the directory. A
full blown site-packages path is required for several applications
and Python eggs.


Specification
=============

site directory (site-packages)

   A directory in ``sys.path``. In contrast to ordinary directories the pth
   files in the directory are processed, too.

user site directory

   A site directory inside the users' home directory. A user site
   directory is specific to a Python version. The path contains
   the version number (major and minor only).

   Unix (including Mac OS X)
      ``~/.local/lib/python2.6/site-packages``
   Windows
      ``%APPDATA%/Python/Python26/site-packages``

user data directory

   Usually the parent directory of the user site directory. It's meant
   for Python version specific data like config files, docs, images
   and translations.

   Unix (including Mac)
      ``~/.local/lib/python2.6``
   Windows
      ``%APPDATA%/Python/Python26``

user base directory

   It's located inside the user's home directory. The user site and
   use config directory are inside the base directory. On some systems
   the directory may be shared with 3rd party apps.

   Unix (including Mac)
      ``~/.local``
   Windows
      ``%APPDATA%/Python``

user script directory

   A directory for binaries and scripts. [10]_ It's shared across Python
   versions and the destination directory for scripts.

   Unix (including Mac)
      ``~/.local/bin``
   Windows
      ``%APPDATA%/Python/Scripts``


Windows Notes
-------------

On Windows the *Application Data* directory (aka ``APPDATA``) was chosen
because it is the most designated place for application data. Microsoft
recommends that software doesn't write to ``USERPROFILE`` [5]_ and
``My Documents`` is not suited for application data, either. [8]_ The code
doesn't query the Win32 API, instead it uses the environment variable
%APPDATA%.

The application data directory is part of the roaming profile. In networks
with domain logins the application data may be copied from and to the a
central server. This can slow down log-in and log-off. Users can keep
the data on the server by e.g. setting PYTHONUSERBASE to the value
"%HOMEDRIVE%%HOMEPATH%\Applicata Data". Users should consult their local
administrator for more information. [13]_


Unix Notes
----------

On Unix ``~/.local`` was chosen in favor over ``~/.python`` because the
directory is already used by several other programs in analogy to
``/usr/local``. [7]_ [11]_


Mac OS X Notes
--------------

On Mac OS X Python uses ~/.local directory as well. [12]_ Framework builds
of Python include ``~/Library/Python/2.6/site-packages`` as an additional
search path.


Implementation
==============

The site module gets a new method ``adduserpackage()`` which adds the
appropriate directory to the search path. The directory is not added if
it doesn't exist when Python is started. However the location of the
user site directory and user base directory is stored in an internal
variable for distutils.

The user site directory is added before the system site directories
but after Python's search paths and ``PYTHONPATH``. This setup allows
the user to install a different version of a package than the system
administrator but it prevents the user from accidentally overwriting a
stdlib module. Stdlib modules can still be overwritten with
``PYTHONPATH``.

For security reasons the user site directory is *not* added to
``sys.path`` when the effective user id or group id is not equal to the
process uid / gid [9]_. It's an additional barrier against code injection
into suid apps.  However Python suid scripts *must* always use the -E
and -s option or users can sneak in their own code.

The user site directory can be suppressed with a new option ``-s`` or
the environment variable ``PYTHONNOUSERSITE``. The feature can be
disabled globally by setting ``site.ENABLE_USER_SITE`` to the value
``False``. It must be set by editing ``site.py``. It can't be altered
``in sitecustomize.py`` or later.

The path to the user base directory can be overwritten with the
environment variable ``PYTHONUSERBASE``. The default location is used
when ``PYTHONUSERBASE`` is not set or empty.

``distutils.command.install`` (setup.py install) gets a new argument
``--user`` to install packages in the user site directory. The required
directories are created on demand.

``distutils.command.build_ext`` (setup.py build_ext) gets a new argument
``--user`` which adds the include/ and lib/ directories in the user base
directory to the search paths for header files and libraries. It also
adds the lib/ directory to rpath.

The ``site`` module gets two arguments ``--user-base`` and ``--user-site``
to print the path to the user base or user site directory to the standard
output. The feature is intended for scripting, e.g.
``./configure --prefix $(python2.5 -m site --user-base)``

``distutils.sysconfig`` will get methods to access the private variables
of site. (not yet implemented)

The Windows updater needs to be updated, too. It should create a menu
item which opens the user site directory in a new explorer windows.


Backwards Compatibility
=======================

TBD


Reference Implementation
========================

A reference implementation is available in the bug tracker. [4]_


Copyright
=========

This document has been placed in the public domain.


References
==========

.. [1] Virtual Python
   http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python

.. [2]  Working Env
   https://pypi.org/project/workingenv.py/
   https://ianbicking.org/archive/workingenv-revisited.html

.. [3] Virtual Env
   https://pypi.org/project/virtualenv/

.. [4] reference implementation
   https://github.com/python/cpython/issues/46132
   http://svn.python.org/view/sandbox/trunk/pep370

.. [5] MSDN: CSIDL
   https://learn.microsoft.com/en/windows/win32/shell/csidl

[6] Initial suggestion for a per user site-packages directory
\   https://mail.python.org/archives/list/python-dev@python.org/message/V23CUKRH3VCHFLV33ADMHJSM53STPA7I/

.. [7] Suggestion of ~/.local/
   https://mail.python.org/pipermail/python-dev/2008-January/075985.html

.. [8] APPDATA discussion
   https://mail.python.org/pipermail/python-dev/2008-January/075993.html

.. [9] Security concerns and -s option
   https://mail.python.org/pipermail/python-dev/2008-January/076130.html

.. [10] Discussion about the bin directory
   https://mail.python.org/pipermail/python-dev/2008-January/076162.html

.. [11] freedesktop.org XGD basedir specs mentions ~/.local
   https://www.freedesktop.org/wiki/Specifications/basedir-spec/

.. [12] ~/.local for Mac and usercustomize file
   https://mail.python.org/pipermail/python-dev/2008-January/076236.html

.. [13] Roaming profile on Windows
   https://mail.python.org/pipermail/python-dev/2008-January/076256.html


id: pep-0364.rst
content:
PEP: 364
Title: Transitioning to the Py3K Standard Library
Author: Barry Warsaw <barry@python.org>
Status: Withdrawn
Type: Standards Track
Created: 01-Mar-2007
Python-Version: 2.6
Post-History:


Abstract
========

:pep:`3108` describes the reorganization of the Python standard library
for the Python 3.0 release.  This PEP describes a
mechanism for transitioning from the Python 2.x standard library to
the Python 3.0 standard library.  This transition will allow and
encourage Python programmers to use the new Python 3.0 library names
starting with Python 2.6, while maintaining the old names for backward
compatibility.  In this way, a Python programmer will be able to write
forward compatible code without sacrificing interoperability with
existing Python programs.


Rationale
=========

:pep:`3108` presents a rationale for Python standard library (stdlib)
reorganization.  The reader is encouraged to consult that PEP for
details about why and how the library will be reorganized.  Should
:pep:`3108` be accepted in part or in whole, then it is advantageous to
allow Python programmers to begin the transition to the new stdlib
module names in Python 2.x, so that they can write forward compatible
code starting with Python 2.6.

Note that :pep:`3108` proposes to remove some "silly old stuff",
i.e. modules that are no longer useful or necessary.  The PEP you are
reading does not address this because there are no forward
compatibility issues for modules that are to be removed, except to
stop using such modules.

This PEP concerns only the mechanism by which mappings from old stdlib
names to new stdlib names are maintained.  Please consult :pep:`3108` for
all specific module renaming proposals.  Specifically see the section
titled ``Modules to Rename`` for guidelines on the old name to new
name mappings.  The few examples in this PEP are given for
illustrative purposes only and should not be used for specific
renaming recommendations.


Supported Renamings
===================

There are at least 4 use cases explicitly supported by this PEP:

- Simple top-level package name renamings, such as ``StringIO`` to
  ``stringio``;

- Sub-package renamings where the package name may or may not be
  renamed, such as ``email.MIMEText`` to ``email.mime.text``;

- Extension module renaming, such as ``cStringIO`` to ``cstringio``;

- Third party renaming of any of the above.

Two use cases supported by this PEP include renaming simple top-level
modules, such as ``StringIO``, as well as modules within packages,
such as ``email.MIMEText``.

In the former case, :pep:`3108` currently recommends ``StringIO`` be
renamed to ``stringio``, following :pep:`8` recommendations.

In the latter case, the email 4.0 package distributed with Python 2.5
already renamed ``email.MIMEText`` to ``email.mime.text``, although it
did so in a one-off, uniquely hackish way inside the email package.
The mechanism described in this PEP is general enough to handle all
module renamings, obviating the need for the Python 2.5 hack (except
for backward compatibility with earlier Python versions).

An additional use case is to support the renaming of C extension
modules.  As long as the new name for the C module is importable, it
can be remapped to the new name.  E.g. ``cStringIO`` renamed to
``cstringio``.

Third party package renaming is also supported, via several public
interfaces accessible by any Python module.

Remappings are not performed recursively.


.mv files
=========

Remapping files are called ``.mv`` files; the suffix was chosen to be
evocative of the Unix mv(1) command.  An ``.mv`` file is a simple
line-oriented text file.  All blank lines and lines that start with a
# are ignored.  All other lines must contain two whitespace separated
fields.  The first field is the old module name, and the second field
is the new module name.  Both module names must be specified using
their full dotted-path names.  Here is an example ``.mv`` file from
Python 2.6::

    # Map the various string i/o libraries to their new names
    StringIO    stringio
    cStringIO   cstringio

``.mv`` files can appear anywhere in the file system, and there is a
programmatic interface provided to parse them, and register the
remappings inside them.  By default, when Python starts up, all the
``.mv`` files in the ``oldlib`` package are read, and their remappings
are automatically registered.  This is where all the module remappings
should be specified for top-level Python 2.x standard library modules.


Implementation Specification
============================

This section provides the full specification for how module renamings
in Python 2.x are implemented.  The central mechanism relies on
various import hooks as described in :pep:`302`.  Specifically
``sys.path_importer_cache``, ``sys.path``, and ``sys.meta_path`` are
all employed to provide the necessary functionality.

When Python's import machinery is initialized, the oldlib package is
imported.  Inside oldlib there is a class called ``OldStdlibLoader``.
This class implements the :pep:`302` interface and is automatically
instantiated, with zero arguments.  The constructor reads all the
``.mv`` files from the oldlib package directory, automatically
registering all the remappings found in those ``.mv`` files.  This is
how the Python 2.x standard library is remapped.

The OldStdlibLoader class should not be instantiated by other Python
modules.  Instead, you can access the global OldStdlibLoader instance
via the ``sys.stdlib_remapper`` instance.  Use this instance if you want
programmatic access to the remapping machinery.

One important implementation detail: as needed by the :pep:`302` API, a
magic string is added to sys.path, and module __path__ attributes in
order to hook in our remapping loader.  This magic string is currently
``<oldlib>`` and some changes were necessary to Python's site.py file
in order to treat all sys.path entries starting with ``<`` as
special.  Specifically, no attempt is made to make them absolute file
names (since they aren't file names at all).

In order for the remapping import hooks to work, the module or package
must be physically located under its new name.  This is because the
import hooks catch only modules that are not already imported, and
cannot be imported by Python's built-in import rules.  Thus, if a
module has been moved, say from Lib/StringIO.py to Lib/stringio.py,
and the former's ``.pyc`` file has been removed, then without the
remapper, this would fail::

    import StringIO

Instead, with the remapper, this failing import will be caught, the
old name will be looked up in the registered remappings, and in this
case, the new name ``stringio`` will be found.  The remapper then
attempts to import the new name, and if that succeeds, it binds the
resulting module into sys.modules, under both the old and new names.
Thus, the above import will result in entries in sys.modules for
'StringIO' and 'stringio', and both will point to the exact same
module object.

Note that no way to disable the remapping machinery is proposed, short
of moving all the ``.mv`` files away or programmatically removing them
in some custom start up code.  In Python 3.0, the remappings will be
eliminated, leaving only the "new" names.


Programmatic Interface
======================

Several methods are added to the ``sys.stdlib_remapper`` object, which
third party packages can use to register their own remappings.  Note
however that in all cases, there is one and only one mapping from an
old name to a new name.  If two ``.mv`` files contain different
mappings for an old name, or if a programmatic call is made with an
old name that is already remapped, the previous mapping is lost.  This
will not affect any already imported modules.

The following methods are available on the ``sys.stdlib_remapper``
object:

- ``read_mv_file(filename)`` -- Read the given file and register all
  remappings found in the file.

- ``read_directory_mv_files(dirname, suffix='.mv')``


id: pep-0402.rst
content:
PEP: 402
Title: Simplified Package Layout and Partitioning
Author: Phillip J. Eby
Status: Rejected
Type: Standards Track
Topic: Packaging
Created: 12-Jul-2011
Python-Version: 3.3
Post-History: 20-Jul-2011
Replaces: 382

Rejection Notice
================

On the first day of sprints at US PyCon 2012 we had a long and
fruitful discussion about :pep:`382` and :pep:`402`.  We ended up rejecting
both but a new PEP will be written to carry on in the spirit of PEP
402.  Martin von LÃ¶wis wrote up a summary: [3]_.

Abstract
========

This PEP proposes an enhancement to Python's package importing
to:

* Surprise users of other languages less,
* Make it easier to convert a module into a package, and
* Support dividing packages into separately installed components
  (ala "namespace packages", as described in :pep:`382`)

The proposed enhancements do not change the semantics of any
currently-importable directory layouts, but make it possible for
packages to use a simplified directory layout (that is not importable
currently).

However, the proposed changes do NOT add any performance overhead to
the importing of existing modules or packages, and performance for the
new directory layout should be about the same as that of previous
"namespace package" solutions (such as ``pkgutil.extend_path()``).


The Problem
===========

.. epigraph::

    "Most packages are like modules.  Their contents are highly
    interdependent and can't be pulled apart.  [However,] some
    packages exist to provide a separate namespace. ...  It should
    be possible to distribute sub-packages or submodules of these
    [namespace packages] independently."

    -- Jim Fulton, shortly before the release of Python 2.3 [1]_


When new users come to Python from other languages, they are often
confused by Python's package import semantics.  At Google, for example,
Guido received complaints from "a large crowd with pitchforks" [2]_
that the requirement for packages to contain an ``__init__`` module
was a "misfeature", and should be dropped.

In addition, users coming from languages like Java or Perl are
sometimes confused by a difference in Python's import path searching.

In most other languages that have a similar path mechanism to Python's
``sys.path``, a package is merely a namespace that contains modules
or classes, and can thus be spread across multiple directories in
the language's path.  In Perl, for instance, a ``Foo::Bar`` module
will be searched for in ``Foo/`` subdirectories all along the module
include path, not just in the first such subdirectory found.

Worse, this is not just a problem for new users: it prevents *anyone*
from easily splitting a package into separately-installable
components.  In Perl terms, it would be as if every possible ``Net::``
module on CPAN had to be bundled up and shipped in a single tarball!

For that reason, various workarounds for this latter limitation exist,
circulated under the term "namespace packages".  The Python standard
library has provided one such workaround since Python 2.3 (via the
``pkgutil.extend_path()`` function), and the "setuptools" package
provides another (via ``pkg_resources.declare_namespace()``).

The workarounds themselves, however, fall prey to a *third* issue with
Python's way of laying out packages in the filesystem.

Because a package *must* contain an ``__init__`` module, any attempt
to distribute modules for that package must necessarily include that
``__init__`` module, if those modules are to be importable.

However, the very fact that each distribution of modules for a package
must contain this (duplicated) ``__init__`` module, means that OS
vendors who package up these module distributions must somehow handle
the conflict caused by several module distributions installing that
``__init__`` module to the same location in the filesystem.

This led to the proposing of :pep:`382` ("Namespace Packages") - a way
to signal to Python's import machinery that a directory was
importable, using unique filenames per module distribution.

However, there was more than one downside to this approach.
Performance for all import operations would be affected, and the
process of designating a package became even more complex.  New
terminology had to be invented to explain the solution, and so on.

As terminology discussions continued on the Import-SIG, it soon became
apparent that the main reason it was so difficult to explain the
concepts related to "namespace packages" was because Python's
current way of handling packages is somewhat underpowered, when
compared to other languages.

That is, in other popular languages with package systems, no special
term is needed to describe "namespace packages", because *all*
packages generally behave in the desired fashion.

Rather than being an isolated single directory with a special marker
module (as in Python), packages in other languages are typically just
the *union* of appropriately-named directories across the *entire*
import or inclusion path.

In Perl, for example, the module ``Foo`` is always found in a
``Foo.pm`` file, and a module ``Foo::Bar`` is always found in a
``Foo/Bar.pm`` file.  (In other words, there is One Obvious Way to
find the location of a particular module.)

This is because Perl considers a module to be *different* from a
package: the package is purely a *namespace* in which other modules
may reside, and is only *coincidentally* the name of a module as well.

In current versions of Python, however, the module and the package are
more tightly bound together.  ``Foo`` is always a module -- whether it
is found in ``Foo.py`` or ``Foo/__init__.py`` -- and it is tightly
linked to its submodules (if any), which *must* reside in the exact
same directory where the ``__init__.py`` was found.

On the positive side, this design choice means that a package is quite
self-contained, and can be installed, copied, etc. as a unit just by
performing an operation on the package's root directory.

On the negative side, however, it is non-intuitive for beginners, and
requires a more complex step to turn a module into a package.  If
``Foo`` begins its life as ``Foo.py``, then it must be moved and
renamed to ``Foo/__init__.py``.

Conversely, if you intend to create a ``Foo.Bar`` module from the
start, but have no particular module contents to put in ``Foo``
itself, then you have to create an empty and seemingly-irrelevant
``Foo/__init__.py`` file, just so that ``Foo.Bar`` can be imported.

(And these issues don't just confuse newcomers to the language,
either: they annoy many experienced developers as well.)

So, after some discussion on the Import-SIG, this PEP was created
as an alternative to PEP \382, in an attempt to solve *all* of the
above problems, not just the "namespace package" use cases.

And, as a delightful side effect, the solution proposed in this PEP
does not affect the import performance of ordinary modules or
self-contained (i.e. ``__init__``-based) packages.


The Solution
============

In the past, various proposals have been made to allow more intuitive
approaches to package directory layout.  However, most of them failed
because of an apparent backward-compatibility problem.

That is, if the requirement for an ``__init__`` module were simply
dropped, it would open up the possibility for a directory named, say,
``string`` on ``sys.path``, to block importing of the standard library
``string`` module.

Paradoxically, however, the failure of this approach does *not* arise
from the elimination of the ``__init__`` requirement!

Rather, the failure arises because the underlying approach takes for
granted that a package is just ONE thing, instead of two.

In truth, a package comprises two separate, but related entities: a
module (with its own, optional contents), and a *namespace* where
*other* modules or packages can be found.

In current versions of Python, however, the module part (found in
``__init__``) and the namespace for submodule imports (represented
by the ``__path__`` a


id: pep-0628.rst
content:
PEP: 628
Title: Add ``math.tau``
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Final
Type: Standards Track
Created: 28-Jun-2011
Python-Version: 3.6
Post-History: 28-Jun-2011


Abstract
========

In honour of Tau Day 2011, this PEP proposes the addition of the circle
constant ``math.tau`` to the Python standard library.

The concept of ``tau`` (``Ï``) is based on the observation that the ratio of a
circle's circumference to its radius is far more fundamental and interesting
than the ratio between its circumference and diameter. It is simply a matter
of assigning a name to the value ``2 * pi`` (``2Ï``).


PEP Acceptance
==============

This PEP is now `accepted`_ and ``math.tau`` will be a part of Python 3.6.
Happy birthday Alyssa!

The idea in this PEP has been implemented in the auspiciously named
`issue 12345`_.

.. _accepted: https://bugs.python.org/issue12345#msg272287
.. _issue 12345: http://bugs.python.org/issue12345


The Rationale for Tau
=====================

``pi`` is defined as the ratio of a circle's circumference to its diameter.
However, a circle is defined by its centre point and its *radius*. This is
shown clearly when we note that the parameter of integration to go from a
circle's circumference to its area is the radius, not the diameter. If we
use the diameter instead we have to divide by four to get rid of the
extraneous multiplier.

When working with radians, it is trivial to convert any given fraction of a
circle to a value in radians in terms of ``tau``. A quarter circle is
``tau/4``, a half circle is ``tau/2``, seven 25ths is ``7*tau/25``, etc. In
contrast with the equivalent expressions in terms of ``pi`` (``pi/2``, ``pi``,
``14*pi/25``), the unnecessary and needlessly confusing multiplication by
two is gone.


Other Resources
===============

I've barely skimmed the surface of the many examples put forward to point out
just how much *easier* and more *sensible* many aspects of mathematics become
when conceived in terms of ``tau`` rather than ``pi``. If you don't find my
specific examples sufficiently persuasive, here are some more resources that
may be of interest:

* Michael Hartl is the primary instigator of Tau Day in his `Tau Manifesto`_
* Bob Palais, the author of the original mathematics journal article
  highlighting the problems with ``pi`` has `a page of resources`_ on the
  topic
* For those that prefer videos to written text, `Pi is wrong!`_ and
  `Pi is (still) wrong`_ are available on YouTube

.. _Tau Manifesto: http://tauday.com/
.. _Pi is (still) wrong: http://www.youtube.com/watch?v=jG7vhMMXagQ
.. _Pi is wrong!: http://www.youtube.com/watch?v=IF1zcRoOVN0
.. _a page of resources: http://www.math.utah.edu/~palais/pi.html


Copyright
=========

This document has been placed in the public domain.


id: pep-0614.rst
content:
PEP: 614
Title: Relaxing Grammar Restrictions On Decorators
Author: Brandt Bucher <brandt@python.org>
Sponsor: Guido van Rossum <guido@python.org>
Status: Final
Type: Standards Track
Created: 10-Feb-2020
Python-Version: 3.9
Post-History: 11-Feb-2020, 18-Feb-2020, 03-Mar-2020
Resolution: https://mail.python.org/archives/list/python-dev@python.org/thread/VSR66MOTCDCY7ZFH4IG7QVFI2JXQQZQ5


Abstract
========

Python currently requires that all decorators consist of a dotted
name, optionally followed by a single call. This PEP proposes removing
these limitations and allowing decorators to be any valid expression.


Motivation
==========

When decorators were first being introduced, `Guido described
<https://mail.python.org/archives/list/python-dev@python.org/message/P3JD24UFFPZUUDANOAI6GZAPIGY4CVK7>`_
the motivation to limit their syntax as a preference, not a technical
requirement:

    I have a gut feeling about this one.  I'm not sure where it comes
    from, but I have it...  So while it would be quite easy to change
    the syntax to ``@test`` in the future, I'd like to stick to with
    the more restricted form unless a real use case is presented where
    allowing ``@test`` would increase readability.

While these limitations were rarely encountered in practice, `BPO
issues <https://bugs.python.org/issue19660>`_ and `mailing list posts
<https://mail.python.org/archives/list/python-ideas@python.org/thread/UQOCJH3KOPBP7P3AVNS3OYBGZPR3V2WO/#CAOXYF4GV76AFJNCYSYMQTBM7CIPPH5M>`_
have consistently surfaced over the years requesting that they be
removed. The `most recent one
<https://mail.python.org/archives/list/python-ideas@python.org/thread/WOWD4P323DYDIGUQVWMESDWUG6QOW4MP>`_
(which `prompted this proposal
<https://mail.python.org/archives/list/python-ideas@python.org/message/FKE7ZFGUDCU5WVOE2QTD5XGMCNCOMETV>`_)
contained a good example of code using the ``PyQt5`` library that
would become more readable, idiomatic, and maintainable if the
existing restrictions were relaxed.  Slightly modified::

    buttons = [QPushButton(f'Button {i}') for i in range(10)]

    # Do stuff with the list of buttons...

    @buttons[0].clicked.connect
    def spam():
        ...

    @buttons[1].clicked.connect
    def eggs():
        ...

    # Do stuff with the list of buttons...

Currently, these decorations must be rewritten as something like::

    button_0 = buttons[0]

    @button_0.clicked.connect
    def spam():
        ...

    button_1 = buttons[1]

    @button_1.clicked.connect
    def eggs():
        ...

Further, the current grammar is already loose enough that it's trivial
to hack more complicated decorator expressions together. So rather
than disallow arbitrarily complex expressions, as intended, the
current restrictions only make them uglier and less efficient::

    # Identity function hack:

    def _(x):
        return x

    @_(buttons[0].clicked.connect)
    def spam():
        ...

    # eval hack:

    @eval("buttons[1].clicked.connect")
    def eggs():
        ...


Rationale
=========

Allowing Any Expression
-----------------------

The decision to allow *any* valid expression (and not just relaxing
the current restrictions to allow, for example, subscripting) has
been considered as the next logical step in the evolution of decorator
grammar for quite some time.  As `Guido noted
<https://mail.python.org/archives/list/python-ideas@python.org/message/CAOXYF4GV76AFJNCYSYMQTBM7CIPPH5M>`_,
during yet another mailing list thread:

    I don't think it's reasonable to constrain it less than it
    currently is but more than a general expression.

Special-casing the grammar to allow *some* useful cases would only
complicate the current situation, and all but guarantee that the
process would repeat itself sometime in the future.  Further, one
purpose of this grammatical change is to discourage the temptation to
use hacks like the ``eval`` and identity-function anti-patterns shown
above.

In short: if we're removing somewhat arbitrary restrictions, we should
remove *all* of them.


What Counts As An "Expression"
------------------------------

Throughout this document, the word "expression" is used as defined in
the `Python Language Reference
<https://docs.python.org/3.9/reference/expressions.html#grammar-token-expression>`_.
This can be summarized as "anything that's valid as a test in ``if``,
``elif``, and ``while`` blocks".  This differs subtly from a perhaps
more popular `definition
<https://docs.python.org/3/glossary.html#term-expression>`_, which can
be summarized as "anything that's valid as string input to ``eval``".

This definition of "expression" is convenient in that it fits our
needs well, and reuses the allowed grammar of existing language
constructs.  It has two subtle differences from the other definition:


Tuple Displays *Must* Be Parenthesized
''''''''''''''''''''''''''''''''''''''

This is based on an observation Guido made in the same email.
Continued immediately from above:

    Though I wouldn't allow commas-- there's no way that

    .. code::

        @f, g
        def pooh(): ...

    can make sense.

Indeed, it may even lead inexperienced readers to conclude that
several decorators are being applied, as if they were stacked.
Requiring parentheses here makes the (admittedly nonsensical) intent
clear without imposing further restrictions and grammar complications.


Named Expressions *Need Not* Be Parenthesized
'''''''''''''''''''''''''''''''''''''''''''''

Here, the choice of syntax is unambiguous.  :pep:`572` explains
why it requires parentheses around top-level expression statements:

    This rule is included to simplify the choice for the user between
    an assignment statement and an assignment expression -- there is
    no syntactic position where both are valid.

Since an assignment statement is not valid here, assignment
expressions should not be unnecessarily burdened with parentheses.


Specification
=============

The grammar for decorators is currently::

    decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE

This PEP proposes that it be simplified to::

    decorator: '@' namedexpr_test NEWLINE


Backwards Compatibility
=======================

This new grammar is fully backward-compatible with the existing
grammar.


How To Teach This
=================

Decorators can continue to be taught as they always have; the average
Python programmer is likely unaware that the current restriction even
exists.


Reference Implementation
========================

The author has written a `CPython implementation
<https://github.com/python/cpython/pull/18570>`_.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


id: pep-0600.rst
content:
PEP: 600
Title: Future 'manylinux' Platform Tags for Portable Linux Built Distributions
Author: Nathaniel J. Smith <njs@pobox.com>,
        Thomas Kluyver <thomas@kluyver.me.uk>
Sponsor: Paul Moore <p.f.moore@gmail.com>
BDFL-Delegate: Paul Moore <p.f.moore@gmail.com>
Discussions-To: https://discuss.python.org/t/the-next-manylinux-specification/1043
Status: Final
Type: Standards Track
Topic: Packaging
Created: 03-May-2019
Post-History: 03-May-2019
Replaces: 513, 571, 599
Resolution: https://discuss.python.org/t/pep-600-future-manylinux-platform-tags-for-portable-linux-built-distributions/2414/27

Abstract
========

This PEP proposes a scheme for new 'manylinux' wheel tags to be
defined without requiring a PEP for every specific tag, similar to how
Windows and macOS tags already work. This will allow package
maintainers to take advantage of new tags more quickly, while making
better use of limited volunteer time.

Non-goals include: handling non-glibc-based platforms; integrating
with external package managers or handling external dependencies such
as CUDA; making manylinux tags more sophisticated than their
Windows/macOS equivalents; doing anything besides taking our existing
tried-and-tested approach and streamlining it. These are important
issues and other PEPs may address them in the future, but for this PEP
they're out of scope.


Rationale
=========

Python users appreciate it when PyPI has pre-compiled packages for
their platform, because it makes installation fast and simple. But
distributing pre-compiled binaries on Linux is challenging because of
the diversity of Linux-based platforms. For example, Debian, Android,
and Alpine all use the Linux kernel, but with radically different
userspace libraries, which makes it difficult or impossible to create
a single wheel that works on all three. This complexity has caused
many previous discussions of Linux wheels to stall out.

The "manylinux" project succeeded by adopting a strategy of ruthless
pragmatism. We chose a large but tractable set of Linux platforms â
specifically, mainstream glibc-based distributions like Debian,
OpenSuSE, Ubuntu, RHEL, etc. â and then we did whatever it takes to
make wheels that work across all these platforms.

This approach requires many compromises. Manylinux wheels can only
rely on external libraries that maintain a consistent ABI and are
universally available across all these distributions, which in
practice restricts them to a small set of core libraries like glibc
and a few others. Wheels have to be built on carefully-chosen
platforms of the oldest possible vintage, using a Python that is
itself built in a carefully-chosen configuration. Other shared library
dependencies have to be bundled into the wheel, which requires a
complex process to avoid collisions between unrelated wheels. And
finally, the details of these requirements change over time, as new
distro versions are released, and old ones fall out of use.

It turns out that these requirements are not too onerous: they're
essentially equivalent to what you have to do to ship Windows or macOS
wheels, and the manylinux approach has achieved substantial uptake
among both package maintainers and end-users. But any manylinux PEP
needs some way to address these complexities.

In previous manylinux PEPs (:pep:`513`, :pep:`571`, :pep:`599`), we've
done this by attempting to write down in the PEP the exact set of
libraries, symbol versions, Python configuration, etc. that we
believed would lead to wheels that work on all mainstream glibc-based
Linux systems. But this created several problems:

First, PEPs are generally supposed to be normative references: if
software doesn't match the PEP, then we fix the software. But in this
case, the PEPs are attempting to describe Linux distributions, which
are a moving target, and do not consider our PEPs to constrain their
behavior. This means that we've been taking on an unbounded commitment
to keep updating every manylinux PEP whenever the Linux distro
landscape changes. This is a substantial commitment for unfunded
volunteers to take on, and it's not clear that this work produces
value for our users.

And second, every time we move manylinux forward to a newer range of
supported platforms, or add support for a new architecture, we have to
go through a fairly elaborate process: writing a new PEP, updating the
PyPI and pip codebases to recognize the new tag, waiting for the new
pip to percolate to users, etc. None of this happens on Windows/macOS;
it's only a tax on Linux maintainers. This slows deployment of new
manylinux versions, and consumes part of our community's limited PEP
review bandwidth, thus slowing progress of the Python packaging
ecosystem as a whole. This is especially problematic for less-popular
architectures, who have less volunteer resources to overcome these
barriers.

How can we fix it?

A manylinux PEP has to address three main audiences:

- **Package installers**, like pip, need to be able to determine which
  wheel tags are compatible with the system they find themselves
  running on. This requires some automated process to introspect the
  system and match it up with wheel tags.

- **Package indexes**, like PyPI, need to be able to validate which
  wheel tags are valid. Generally, this just requires something like a
  list of valid tags, or regex they match, with no need to know
  anything about the actual semantics for individual tags. (But see
  the discussion of upload verification below.)

- **Package maintainers** need to be able to build wheels that meet
  the requirements for a given wheel tag.

Here's the key insight behind this new PEP: it's crucial that
different **package installers** and **package indexes** all agree on
which manylinux tags are valid and which systems they install on, so
we need a PEP to specify these â but, these are straightforward, and
don't really change between manylinux versions. The complicated part
that keeps changing is the process of actually **building the wheels**
â but, if there are multiple competing build environments, it *doesn't
matter* whether they use exactly the same rules as each other, as long
as they all produce wheels that work on end-user systems. Therefore,
we don't need an interoperability standard for building wheels, so we
don't need to write the details into a PEP.

To further convince ourselves that this approach will work, let's look
again at how we handle wheels on Windows and macOS: the PEPs describe
which tags are valid, and which systems they're supposed to work on,
but not how to actually build wheels for those platforms. And in
practice, if you want to distribute Windows or macOS wheels, you might
have to jump through some complicated and poorly documented hoops in
order to bundle dependencies, target the right range of OS versions,
etc. But the system works, and the way to improve it is to write
better docs and build better tooling; no-one thinks that the way to
make Windows wheels work better is to publish a PEP describing
which symbols we think Microsoft should be including in their
libraries and how their linker ought to work. This PEP extends that
philosophy to manylinux as well.


Specification
=============

Core definition
---------------

Tags using the new scheme will look like::

    manylinux_2_17_x86_64

Or more generally::

    manylinux_${GLIBCMAJOR}_${GLIBCMINOR}_${ARCH}

This tag is a promise: the wheel's creator promises that the wheel
will work on any mainstream Linux distro that uses glibc version
``${GLIBCMAJOR}.${GLIBCMINOR}`` or later, and where the ``${ARCH}``
matches the return value from ``distutils.util.get_platform()``. (For
more detail about architecture tags, see :pep:`425`.)

If a user installs this wheel into an environment that matches these
requirements and it doesn't work, then that wheel does not comply with
this specification. This should be considered a bug in the wheel, and
it's the wheel creator's responsibility to look for a fix (possib


id: pep-0748.rst
content:
PEP: 748
Title: A Unified TLS API for Python
Author: Joop van de Pol <joop.vandepol@trailofbits.com>,
        William Woodruff <william@yossarian.net>
Sponsor: Alyssa Coghlan <ncoghlan@gmail.com>
Discussions-To: https://discuss.python.org/t/pre-pep-discussion-revival-of-pep-543/51263
Status: Draft
Type: Standards Track
Created: 27-Jun-2024
Python-Version: 3.14
Post-History: `17-Apr-2024 <https://discuss.python.org/t/pre-pep-discussion-revival-of-pep-543/51263>`__
Replaces: 543

Abstract
========

This PEP defines a standard TLS interface in the form of a collection of
protocol classes. This interface will allow Python implementations and
third-party libraries to provide bindings to TLS libraries other than OpenSSL.

These bindings can be used by tools that expect the interface provided by the
Python standard library, with the goal of reducing the dependence of the Python
ecosystem on OpenSSL.

Rationale
=========

It has become increasingly clear that robust and user-friendly TLS support is an
extremely important part of the ecosystem of any popular programming language.
For most of its lifetime, this role in the Python ecosystem has primarily been
served by the :mod:`ssl` module, which provides a Python API to the `OpenSSL
library <https://www.openssl.org/>`_.

Because the :mod:`ssl` module is distributed with the Python standard library,
it has become the overwhelmingly most popular method for handling TLS in Python.
A majority of Python libraries, both in the standard library and
on the Python Package Index, rely on the :mod:`ssl` module for their TLS
connectivity.

Unfortunately, the preeminence of the :mod:`ssl` module has had a number of
tied the entire Python
ecosystem tightly to OpenSSL. This has forced Python users to use OpenSSL even
in situations where it may provide a worse user experience than alternative TLS
implementations, which imposes a cognitive burden and makes it hard to provide
âplatform-nativeâ experiences.

Problems
--------

The fact that the :mod:`ssl` module is built into the standard library has meant
that all standard-library Python networking libraries are entirely reliant on
the OpenSSL that the Python implementation has been linked against. This leads
to the following issues:

* It is difficult to take advantage of new, higher-security TLS without
  recompiling Python to get a new OpenSSL. While there are third-party bindings
  to OpenSSL (e.g. `pyOpenSSL <https://pypi.org/project/pyOpenSSL/>`_), these
  need to be shimmed into a format that the standard library understands,
  forcing projects that want to use them to maintain substantial compatibility
  layers.

* Windows distributions of Python need to be shipped with a copy of
  OpenSSL. This puts the CPython development team in the position of being
  OpenSSL redistributors, potentially needing to ship security updates to the
  Windows Python distributions when OpenSSL vulnerabilities are released.

* macOS distributions of Python need either to be shipped with a copy
  of OpenSSL or linked against the system OpenSSL library. Apple has formally
  deprecated linking against the system OpenSSL library, and even if they had
  not, that library version has been unsupported by upstream for nearly one year
  as of the time of writing. The CPython development team has started shipping
  newer OpenSSLs with the Python available from python.org, but this has the
  same problem as with Windows.

* Users may wish to integrate with TLS libraries other than OpenSSL for other
  reasons, such as maintenance burden versus a system-provided implementation,
  or because OpenSSL is simply too large and unwieldy for their platform (e.g.
  for embedded Python). Those users are left with the requirement to use
  third-party networking libraries that can interact with their preferred TLS
  library or to shim their preferred library into the OpenSSL-specific
  :mod:`ssl` module API.

Additionally, the :mod:`ssl` module as implemented today limits the ability of
CPython itself to add support for alternative TLS implementations, or remove
OpenSSL support entirely, should either of these become necessary or useful. The
:mod:`ssl` module exposes too many OpenSSL-specific function calls and features
to easily map to an alternative TLS implementation.

Proposal
========

This PEP proposes to introduce a few new Protocol Classes in Python 3.14 to
provide TLS functionality that is not so strongly tied to OpenSSL. It also
proposes to update standard library modules to use only the interface exposed by
these protocol classes wherever possible. There are three goals here:

#. To provide a common API surface for both core and third-party developers to
   target their TLS implementations to. This allows TLS developers to provide
   interfaces that can be used by most Python code, and allows network
   developers to have an interface that they can target that will work with a
   wide range of TLS implementations.

#. To provide an API that has few or no OpenSSL-specific concepts leak through.
   The :mod:`ssl` module today has a number of warts caused by leaking OpenSSL
   concepts through to the API: the new protocol classes would remove those
   specific concepts.

#. To provide a path for the core development team to make OpenSSL one of many
   possible TLS implementations, rather than requiring that it be present on a
   system in order for Python to have TLS support.

The proposed interface is laid out below.

Interfaces
----------

There are several interfaces that require standardization. Those interfaces are:

#. Configuring TLS, currently implemented by the :class:`~ssl.SSLContext` class
   in the :mod:`ssl` module.

#. Providing an in-memory buffer for doing in-memory encryption or decryption
   with no actual I/O (necessary for asynchronous I/O models), currently
   implemented by the :class:`~ssl.SSLObject` class in the :mod:`ssl` module.

#. Wrapping a socket object, currently implemented by the
   :class:`~ssl.SSLSocket` class in the :mod:`ssl` module.

#. Applying TLS configuration to the wrapping objects in (2) and (3). Currently
   this is also implemented by the SSLContext class in the :mod:`ssl` module.

#. Specifying TLS cipher suites. There is currently no code for doing this in
   the standard library: instead, the standard library uses OpenSSL cipher suite
   strings.

#. Specifying application-layer protocols that can be negotiated during the TLS
   handshake.

#. Specifying TLS versions.

#. Reporting errors to the caller, currently implemented by the
   :class:`~ssl.SSLError` class in the :mod:`ssl` module.

#. Specifying certificates to load, either as client or server certificates.

#. Specifying which trust database should be used to validate certificates
   presented by a remote peer.

#. Finding a way to get hold of these interfaces at run time.

For the sake of simplicity, this PEP proposes to remove interfaces (3) and (4),
and replace them by a simpler interface that returns a socket which ensures that
all communication through the socket is protected by TLS. In other words, this
interface treats concepts such as socket initialization, the TLS handshake,
Server Name Indication (SNI), etc., as an atomic part of creating a client or
server connection. However, in-memory buffers are still supported, as they are
useful for asynchronous communication.

Obviously, (5) doesn't require a protocol class: instead, it requires a richer
API for configuring supported cipher suites that can be easily updated with
supported cipher suites for different implementations.

(9) is a thorny problem, because in an ideal world the private keys associated
with these certificates would never end up in-memory in the Python process
(that is, the TLS library would collaborate with a Hardware Security Module
(HSM) to provide the private key in such a way that it cannot be extracted
from process memory). Thus, we need to provide an extensible model of
providing certificates that allows concret


id: pep-0006.rst
content:
PEP: 6
Title: Bug Fix Releases
Author: Aahz <aahz@pythoncraft.com>, Anthony Baxter <anthony@interlink.com.au>
Status: Superseded
Type: Process
Created: 15-Mar-2001
Post-History: 15-Mar-2001, 18-Apr-2001, 19-Aug-2004

.. note:: This PEP is obsolete.
   The current release policy is documented in `the devguide
   <https://devguide.python.org/developer-workflow/development-cycle/>`__.
   See also :pep:`101` for mechanics of the release process.


Abstract
========

Python has historically had only a single fork of development, with
releases having the combined purpose of adding new features and
delivering bug fixes (these kinds of releases will be referred to as
"major releases").  This PEP describes how to fork off maintenance, or
bug fix, releases of old versions for the primary purpose of fixing
bugs.

This PEP is not, repeat NOT, a guarantee of the existence of bug fix
releases; it only specifies a procedure to be followed if bug fix
releases are desired by enough of the Python community willing to do
the work.


Motivation
==========

With the move to SourceForge, Python development has accelerated.
There is a sentiment among part of the community that there was too
much acceleration, and many people are uncomfortable with upgrading to
new versions to get bug fixes when so many features have been added,
sometimes late in the development cycle.

One solution for this issue is to maintain the previous major release,
providing bug fixes until the next major release.  This should make
Python more attractive for enterprise development, where Python may
need to be installed on hundreds or thousands of machines.


Prohibitions
============

Bug fix releases are required to adhere to the following restrictions:

1. There must be zero syntax changes.  All ``.pyc`` and ``.pyo`` files must
   work (no regeneration needed) with all bugfix releases forked off
   from a major release.

2. There must be zero pickle changes.

3. There must be no incompatible C API changes.  All extensions must
   continue to work without recompiling in all bugfix releases in the
   same fork as a major release.

Breaking any of these prohibitions requires a BDFL proclamation (and a
prominent warning in the release notes).


Not-Quite-Prohibitions
======================

Where possible, bug fix releases should also:

1. Have no new features. The purpose of a bug fix release is to fix
   bugs, not add the latest and greatest whizzo feature from the HEAD
   of the CVS root.

2. Be a painless upgrade. Users should feel confident that an upgrade
   from 2.x.y to 2.x.(y+1) will not break their running systems. This
   means that, unless it is necessary to fix a bug, the standard
   library should not change behavior, or worse yet, APIs.


Applicability of Prohibitions
=============================

The above prohibitions and not-quite-prohibitions apply both for a
final release to a bugfix release (for instance, 2.4 to 2.4.1) and for
one bugfix release to the next in a series (for instance 2.4.1 to
2.4.2).

Following the prohibitions listed in this PEP should help keep the
community happy that a bug fix release is a painless and safe upgrade.


Helping the Bug Fix Releases Happen
===================================

Here's a few pointers on helping the bug fix release process along.

1. Backport bug fixes. If you fix a bug, and it seems appropriate,
   port it to the CVS branch for the current bug fix release. If
   you're unwilling or unable to backport it yourself, make a note in
   the commit message, with words like 'Bugfix candidate' or
   'Backport candidate'.

2. If you're not sure, ask. Ask the person managing the current bug
   fix releases if they think a particular fix is appropriate.

3. If there's a particular bug you'd particularly like fixed in a bug
   fix release, jump up and down and try to get it done. Do not wait
   until 48 hours before a bug fix release is due, and then start
   asking for bug fixes to be included.


Version Numbers
===============

Starting with Python 2.0, all major releases are required to have a
version number of the form X.Y; bugfix releases will always be of the
form X.Y.Z.

The current major release under development is referred to as release
N; the just-released major version is referred to as N-1.

In CVS, the bug fix releases happen on a branch. For release 2.x, the
branch is named 'release2x-maint'. For example, the branch for the 2.3
maintenance releases is release23-maint


Procedure
=========

The process for managing bugfix releases is modeled in part on the Tcl
system [1]_.

The Patch Czar is the counterpart to the BDFL for bugfix releases.
However, the BDFL and designated appointees retain veto power over
individual patches. A Patch Czar might only be looking after a single
branch of development - it's quite possible that a different person
might be maintaining the 2.3.x and the 2.4.x releases.

As individual patches get contributed to the current trunk of CVS,
each patch committer is requested to consider whether the patch is a
bug fix suitable for inclusion in a bugfix release. If the patch is
considered suitable, the committer can either commit the release to
the maintenance branch, or else mark the patch in the commit message.

In addition, anyone from the Python community is free to suggest
patches for inclusion. Patches may be submitted specifically for
bugfix releases; they should follow the guidelines in :pep:`3`. In
general, though, it's probably better that a bug in a specific release
also be fixed on the HEAD as well as the branch.

The Patch Czar decides when there are a sufficient number of patches
to warrant a release. The release gets packaged up, including a
Windows installer, and made public. If any new bugs are found, they
must be fixed immediately and a new bugfix release publicized (with an
incremented version number). For the 2.3.x cycle, the Patch Czar
(Anthony) has been trying for a release approximately every six
months, but this should not be considered binding in any way on any
future releases.

Bug fix releases are expected to occur at an interval of roughly six
months. This is only a guideline, however - obviously, if a major bug
is found, a bugfix release may be appropriate sooner. In general, only
the N-1 release will be under active maintenance at any time. That is,
during Python 2.4's development, Python 2.3 gets bugfix releases. If,
however, someone qualified wishes to continue the work to maintain an
older release, they should be encouraged.


Patch Czar History
==================

Anthony Baxter is the Patch Czar for 2.3.1 through 2.3.4.

Barry Warsaw is the Patch Czar for 2.2.3.

Guido van Rossum is the Patch Czar for 2.2.2.

Michael Hudson is the Patch Czar for 2.2.1.

Anthony Baxter is the Patch Czar for 2.1.2 and 2.1.3.

Thomas Wouters is the Patch Czar for 2.1.1.

Moshe Zadka is the Patch Czar for 2.0.1.


History
=======

This PEP started life as a proposal on comp.lang.python.  The original
version suggested a single patch for the N-1 release to be released
concurrently with the N release.  The original version also argued for
sticking with a strict bug fix policy.

Following feedback from the BDFL and others, the draft PEP was written
containing an expanded bugfix release cycle that permitted any
previous major release to obtain patches and also relaxed the strict
bug fix requirement (mainly due to the example of :pep:`235`, which
could be argued as either a bug fix or a feature).

Discussion then mostly moved to python-dev, where BDFL finally issued
a proclamation basing the Python bugfix release process on Tcl's,
which essentially returned to the original proposal in terms of being
only the N-1 release and only bug fixes, but allowing multiple bugfix
releases until release N is published.

Anthony Baxter then took this PEP and revised it, based on lessons
from the 2.3 release cycle.


References
==========

.. [1] http://www.tcl.tk/cgi-bin/tct/tip/28.html


Copyright
=========

This do


id: pep-0760.rst
content:
PEP: 760
Title: No More Bare Excepts
Author: Pablo Galindo <pablogsal@python.org>, Brett Cannon <brett@python.org>
Status: Withdrawn
Type: Standards Track
Created: 02-Oct-2024
Python-Version: 3.14
Post-History: `09-Oct-2024 <https://discuss.python.org/t/pep-760-no-more-bare-excepts/67182>`__


Abstract
========

This PEP proposes disallowing bare ``except:`` clauses in Python's
exception-handling syntax. Currently, Python allows catching all exceptions
with a bare ``except:`` clause, which can lead to overly broad exception
handling and mask important errors. This PEP suggests requiring explicit
exception types in all except clauses, promoting more precise and intentional
error handling.

Motivation
==========

The current syntax allows for catching all exceptions with a bare ``except:`` clause:

.. code-block:: python

   try:
       risky_operation()
   except:
       handle_any_error()

While this syntax can be convenient for a "catch all" handler, it often leads
to poor coding practices:

1. It can mask important errors that should be propagated.
2. It makes debugging more difficult by catching and potentially hiding
   unexpected exceptions.
3. It goes against the Python principle of explicit over implicit.

Various linters [1]_ [2]_ [3]_ and style guides (including :pep:`8`) [4]_ [5]_
[6]_ [7]_ discourage bare ``except`` clauses.

By requiring explicit exception types, we can encourage more thoughtful and
precise error handling:

.. code-block:: python

   try:
       risky_operation()
   except Exception as e:
       handle_expected_error(e)

Another view of this problem is that bare except handlers are ambiguous
regarding the intended handling of terminating exceptions, as the intention
could have been either:

* Only catch non-terminating exceptions (``except Exception:``). If this was the
  intention, using a bare ``except:`` is an outright bug, since that isn't what it
  means.
* Catch all exceptions, including terminating ones (``except BaseException:``).
  using bare ``except:`` here it is at least correct, but readers need to check
  to be sure it isn't an instance of the first case.

Since both possible intentions have available unambiguous spellings, the
ambiguous form is redundant and that's why we propose to disallow it.

Rationale
=========

The decision to disallow bare except clauses is based on the following
considerations:

1. Requiring specific exception types makes the programmer's intentions clear
   and encourages thinking about what exceptions might occur.

2. Catching only specific exceptions makes identifying and debugging unexpected
   errors easier.

3. Preventing overly broad exception handling reduces the risk of silently
   ignoring critical errors.

4. Many style guides and linters already discourage the use of bare except
   clauses.


Specification
=============

The syntax for the except clause will be modified to require an exception type.
The grammar will be updated to remove the possibility of adding an empty
expression in except clauses.

This change disallows the bare ``except:`` syntax. All except clauses must
specify at least one exception type:

.. code-block:: python

   try:
       ...
   except ValueError:
       ...
   except (TypeError, RuntimeError):
       ...
   except Exception:
       ...  # Still allowed, but catches all exceptions explicitly

The semantics of exception handling remain unchanged, except that it will no
longer be possible to catch all exceptions without explicitly specifying
``BaseException`` or a similarly broad exception type.


Backwards Compatibility
=======================

This change is not backwards compatible. Existing code that uses bare ``except:``
clauses will need to be modified. To ease the transition:

1. A deprecation warning will be issued for bare except clauses in Python 3.14.
2. The syntax will be fully disallowed in Python 3.17.
3. A ``from __future__ import strict_excepts`` will be provided to invalidate bare
   except handlers in earlier versions of Python.

A tool will be provided to automatically update code to replace bare ``except:``
with ``except BaseException:``.


Security Implications
=====================

This change has no security implications.


How to Teach This
=================

For new Python users, exception handling should be taught with explicit
exception types from the start:

.. code-block:: python

   try:
       result = risky_operation()
   except ValueError:
       handle_value_error()
   except TypeError:
       handle_type_error()
   except Exception as e:
       handle_unexpected_error(e)

For experienced users, the change can be introduced as a best practice that is
now enforced by the language. The following points should be emphasized:

1. Always catch specific exceptions when possible.
2. Use ``except Exception:`` as a last resort for truly unexpected errors.
3. Never silence exceptions without careful consideration.

Documentation should guide common exception hierarchies and how to choose
appropriate exception types to catch.

Rejected ideas
==============

* There are genuine cases where the use of bare ``except:`` handlers are correct. one
  of the examples that have been raised from Mailman [8]_ involves handling transactions
  in the face of any exception:

   .. code-block:: python

       @contextmanager
       def transaction():
           """Context manager for ensuring the transaction is complete."""
           try:
               yield
           except:
               config.db.abort()
               raise
           else:
               config.db.commit()

  This code guarantees that no matter what exception occurs, any open
  transaction will be aborted, while in the successful condition, the
  transaction will be committed.

  We do believe that although there are cases such like this one where
  bare ``except:`` handlers are correct, it would be better to actually
  be explicit and use ``except BaseException:`` for the reasons indicated
  in the "Motivation" section.


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

.. [1] https://pylint.pycqa.org/en/latest/user_guide/messages/warning/bare-except.html
.. [2] https://www.flake8rules.com/rules/E722.html
.. [3] https://docs.astral.sh/ruff/rules/bare-except/
.. [4] https://google.github.io/styleguide/pyguide.html#24-exceptions
.. [5] https://chromium.googlesource.com/chromiumos/platform/factory/+/HEAD/CODING_STYLE.md#Avoid-bare_except
.. [6] https://4.docs.plone.org/develop/plone-coredev/style.html#concrete-rules
.. [7] https://docs.openedx.org/en/latest/developers/references/developer_guide/style_guides/python-guidelines.html
.. [8] https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/database/transaction.py#L27


id: pep-0774.rst
content:
PEP: 774
Title: Removing the LLVM requirement for JIT builds
Author: Savannah Ostrowski <savannah@python.org>
Discussions-To: https://discuss.python.org/t/78548/
Status: Draft
Type: Standards Track
Created: 27-Jan-2025
Python-Version: 3.14
Post-History: `27-Jan-2025 <https://discuss.python.org/t/78548/>`__

Abstract
========

Since Python 3.13, CPython has been able to be configured and built with an
experimental just-in-time (JIT) compiler via the ``--enable-experimental-jit``
flag on Linux and Mac and ``--experimental-jit`` on Windows. To build CPython with
the JIT enabled, users are required to have LLVM installed on their machine
(initially, with LLVM 16 but more recently, with LLVM 19). LLVM is responsible
for generating stencils that are essential to our copy-and-patch JIT (see :pep:`744`).
These stencils are predefined, architecture-specific templates that are used
to generate machine code at runtime.

This PEP proposes removing the LLVM build-time dependency for JIT-enabled builds
by hosting the generated stencils in the CPython repository. This approach
allows us to leverage the checked-in stencils for supported platforms at build
time, simplifying the contributor experience and address concerns raised at the
Python Core Developer Sprint in September 2024. That said, there is a clear
tradeoff to consider, as improved developer experience does come at the cost of
increased repository size.

It is important to note that this PEP is not a proposal to accept or reject the
JIT itself but rather to determine whether the build-time dependency on LLVM is
acceptable for JIT builds moving forward. If this PEP is rejected, we will
proceed with the status quo, retaining the LLVM build-time requirement. While
this dependency has served the JIT development process effectively thus far, it
introduces setup complexity and additional challenges that this PEP seeks to
alleviate.

Motivation
==========

At the Python Core Developer Sprint that took place in September 2024, there was
discussion about the next steps for the JIT - a related discussion also took
place on `GitHub <https://github.com/python/cpython/issues/115869>`__. As part
of that discussion, there was also a clear appetite for removing the LLVM
requirement for JIT builds in preparation for shipping the JIT off by default in
3.14. The consensus at the sprint was that it would be sufficient to provide
pre-generated stencils for non-debug builds for Tier 1 platforms and that
checking these files into the CPython repo would be adequate for the limited
number of platforms (though more options have been explored; see `Rejected
Ideas`_).

Currently, building CPython with `the JIT requires LLVM
<https://github.com/python/cpython/tree/main/Tools/jit#installing-llvm>`__ as a
build-time dependency. Despite not being exposed to end users, this dependency
is suboptimal. Requiring LLVM adds a setup burden for developers and those who
wish to build CPython with the JIT enabled. Depending on the operating system,
the version of LLVM shipped with the OS may differ from that required by our JIT
builds, which introduces additional complexity to troubleshoot and resolve. With
few core developers currently contributing to and maintaining the JIT, we also
want to make sure that the friction to work on JIT-related code is minimized as
much as possible.

With the proposed approach, hosting pre-compiled stencils for supported
architectures can be generated in advance, stored in a central location, and
automatically used during builds. This approach ensures reproducible builds,
making the JIT a more stable and sustainable part of CPython's future.

Rationale
=========

This PEP proposes checking JIT stencils directly into the CPython repo as the
best path forward for eliminating our build-time dependency on LLVM.

This approach:

* Provides the best end-to-end experience for those looking to build CPython
  with the JIT
* Lessens the barrier to entry for those looking to contribute to the JIT
* Ensures builds remain reproducible and consistent across platforms without
  relying on external infrastructure or download mechanisms
* Eliminates variability introduced by network conditions or potential
  discrepancies between hosted files and the CPython repository state, and
* Subjects stencils to the same review processes we have for all other JIT-related
  code

However, this approach does result in a slight increase in overall
repository size. Comparing repo growth on commits over the past 90 days, the
difference between the actual commits and the same commits with stencils added
amounts to a difference of 0.03 MB per stencil file. This is a small increase in
the context of the overall repository size, which has grown by 2.55 MB in the
same time period. For six stencil files, this amounts to an upper bound of 0.18 MB.
The current total size of the stencil files for all six platforms is 7.2 MB. [#stencil_calc]_

These stencils could become larger in the future with changes to register
allocation, which would introduce 5-6 variants per instruction in each stencil
file (5-6x larger). However, if we ended up going this route, there are
additional modifications we could make to stencil files that could help
counteract this size increase (e.g., stripping comments, minimizing the
stencils).

Specification
=============

This specification outlines the proposed changes to remove the build-time
dependency on LLVM and the contributor experience if this PEP is accepted.

Repository changes
------------------

The CPython repository would now host the pre-compiled JIT stencils in a new
subdirectory in ``Tools/jit`` called ``stencils/``. At present, the JIT is tested
and built for six platforms, so to start, we'd check in six stencil files. In
the future, we may check in additional stencil files if support for additional
platforms is desired or relevant.

.. code-block:: text

    cpython/
        Tools/
            jit/
                stencils/
                    aarch64-apple-darwin.h
                    aarch64-unknown-linux-gnu.h
                    i686-pc-windows-msvc.h
                    x86_64-apple-darwin.h
                    x86_64-pc-windows-msvc.h
                    x86_64-pc-linux-gnu.h

Workflow
--------

The workflow changes can be split into two parts, namely building CPython with
the JIT enabled and working on the JIT's implementation.

Building CPython with the JIT
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Precompiled JIT stencil files will be stored in the ``Tools/jit/stencils``
directory, with each file name corresponding to its target triple as outlined
above. At build time, we determine whether to use the checked in stencils or to
generate a new stencil for the user's platform. Specifically, for contributors
with LLVM installed, the ``build.py`` script in ``Tools/jit/stencils`` will allow
them to regenerate the stencil for their platform. Those without LLVM can rely
on the precompiled stencil files directly from the repository.

Working on the JIT's implementation (or touching JIT files)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In continuous integration (CI), stencil files will be automatically validated and updated when changes
are made to JIT-related files. When a pull request is opened that touches these
files, the ``jit.yml`` workflow, which builds and tests our builds, will run as
usual.

However, as part of this, we will introduce a new step that diffs the current
stencils in the repo against those generated in CI. If there is a diff for a
platform's stencil file, a patch file for the updated stencil is generated and
the step will fail. Each patch is uploaded to GitHub Actions. After CI is
finished running across all platforms, the patches are aggregated into a single
patch file for convenience. You can download this aggregated patch, apply it
locally, and commit the updated stencils back to your branch. Then, the
subsequent CI run will pass.

Reference Implementation
========================


id: pep-0012.rst
content:
PEP: 12
Title: Sample reStructuredText PEP Template
Author: David Goodger <goodger@python.org>,
        Barry Warsaw <barry@python.org>,
        Brett Cannon <brett@python.org>
Status: Active
Type: Process
Created: 05-Aug-2002
Post-History: `30-Aug-2002 <https://mail.python.org/archives/list/python-dev@python.org/thread/KX3AS7QAY26QH3WIUAEOCCNXQ4V2TGGV/>`__


.. highlight:: rst

.. note::
    For those who have written a PEP before, there is a template_
    (which is included as a file in the `PEPs repository`_).

Abstract
========

This PEP provides a boilerplate or sample template for creating your
own reStructuredText PEPs.  In conjunction with the content guidelines
in :pep:`1`, this should make it easy for you to conform your own
PEPs to the format outlined below.

Note: if you are reading this PEP via the web, you should first grab
the text (reStructuredText) source of this PEP in order to complete
the steps below.  **DO NOT USE THE HTML FILE AS YOUR TEMPLATE!**

The source for this (or any) PEP can be found in the
`PEPs repository <https://github.com/python/peps/>`_,
as well as via a link at the bottom of each PEP.


Rationale
=========

If you intend to submit a PEP, you MUST use this template, in
conjunction with the format guidelines below, to ensure that your PEP
submission won't get automatically rejected because of form.

ReStructuredText provides PEP authors with useful functionality and
expressivity, while maintaining easy readability in the source text.
The processed HTML form makes the functionality accessible to readers:
live hyperlinks, styled text, tables, images, and automatic tables of
contents, among other advantages.


How to Use This Template
========================

To use this template you must first decide whether your PEP is going
to be an Informational or Standards Track PEP.  Most PEPs are
Standards Track because they propose a new feature for the Python
language or standard library.  When in doubt, read :pep:`1` for details,
or open a tracker issue on the PEPs repo to ask for assistance.

Once you've decided which type of PEP yours is going to be, follow the
directions below.

- Make a copy of this file (the ``.rst`` file, **not** the HTML!) and
  perform the following edits. Name the new file :file:`pep-{NNNN}.rst`, using
  the next available number (not used by a published or in-PR PEP).

- Replace the "PEP: 12" header with "PEP: NNNN",
  matching the file name. Note that the file name should be padded with
  zeros (eg ``pep-0012.rst``), but the header should not (``PEP: 12``).

- Change the Title header to the title of your PEP.

- Change the Author header to include your name, and optionally your
  email address.  Be sure to follow the format carefully: your name
  must appear first, and it must not be contained in parentheses.
  Your email address may appear second (or it can be omitted) and if
  it appears, it must appear in angle brackets.  It is okay to
  obfuscate your email address.

- If none of the authors are Python core developers, include a Sponsor
  header with the name of the core developer sponsoring your PEP.

- Add the direct URL of the PEP's canonical discussion thread
  (on e.g. Python-Dev, Discourse, etc) under the Discussions-To header.
  If the thread will be created after the PEP is submitted as an official
  draft, it is okay to just put "Pending" initially, but remember to
  update the PEP with the URL as soon as the PEP is successfully merged
  to the PEPs repository and you create the corresponding discussion thread.
  See :pep:`PEP 1 <1#discussing-a-pep>` for more details.

- Change the Status header to "Draft".

- For Standards Track PEPs, change the Type header to "Standards
  Track".

- For Informational PEPs, change the Type header to "Informational".

- For Standards Track PEPs, if your feature depends on the acceptance
  of some other currently in-development PEP, add a Requires header
  right after the Type header.  The value should be the PEP number of
  the PEP yours depends on.  Don't add this header if your dependent
  feature is described in a Final PEP.

- Change the Created header to today's date.  Be sure to follow the
  format carefully: it must be in ``dd-mmm-yyyy`` format, where the
  ``mmm`` is the 3 English letter month abbreviation, i.e. one of Jan,
  Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec.

- For Standards Track PEPs, after the Created header, add a
  Python-Version header and set the value to the next planned version
  of Python, i.e. the one your new feature will hopefully make its
  first appearance in.  Do not use an alpha or beta release
  designation here.  Thus, if the last version of Python was 2.2 alpha
  1 and you're hoping to get your new feature into Python 2.2, set the
  header to:

  .. code-block:: email

      Python-Version: 2.2

- Add a Topic header if the PEP belongs under one shown at the :ref:`topic-index`.
  Most PEPs don't.

- Leave Post-History alone for now; you'll add dates and corresponding links
  to this header each time you post your PEP to the designated discussion forum
  (and update the Discussions-To header with said link, as above).
  For each thread, use the date (in the ``dd-mmm-yyy`` format) as the
  linked text, and insert the URLs inline as anonymous reST `hyperlinks`_,
  with commas in between each posting.

  If you posted threads for your PEP on August 14, 2001 and September 3, 2001,
  the Post-History header would look like, e.g.:

  .. code-block:: email

      Post-History: `14-Aug-2001 <https://www.example.com/thread_1>`__,
                    `03-Sept-2001 <https://www.example.com/thread_2>`__

  You should add the new dates/links here as soon as you post a
  new discussion thread.

- Add a Replaces header if your PEP obsoletes an earlier PEP.  The
  value of this header is the number of the PEP that your new PEP is
  replacing.  Only add this header if the older PEP is in "final"
  form, i.e. is either Accepted, Final, or Rejected.  You aren't
  replacing an older open PEP if you're submitting a competing idea.

- Now write your Abstract, Rationale, and other content for your PEP,
  replacing all this gobbledygook with your own text. Be sure to
  adhere to the format guidelines below, specifically on the
  prohibition of tab characters and the indentation requirements.
  See "Suggested Sections" below for a template of sections to include.

- Update your Footnotes section, listing any footnotes and
  non-inline link targets referenced by the text.

- Run ``./build.py`` to ensure the PEP is rendered without errors,
  and check that the output in :file:`build/pep-{NNNN}.html` looks as you intend.

- Create a pull request against the `PEPs repository`_.

For reference, here are all of the possible header fields (everything
in brackets should either be replaced or have the field removed if
it has a leading ``*`` marking it as optional and it does not apply to
your PEP):

.. code-block:: email

  PEP: [NNN]
  Title: [...]
  Author: [Full Name <email at example.com>]
  Sponsor: *[Full Name <email at example.com>]
  PEP-Delegate:
  Discussions-To: [URL]
  Status: Draft
  Type: [Standards Track | Informational | Process]
  Topic: *[Governance | Packaging | Release | Typing]
  Requires: *[NNN]
  Created: [DD-MMM-YYYY]
  Python-Version: *[M.N]
  Post-History: [`DD-MMM-YYYY <URL>`__]
  Replaces: *[NNN]
  Superseded-By: *[NNN]
  Resolution:


ReStructuredText PEP Formatting Requirements
============================================

The following is a PEP-specific summary of reStructuredText syntax.
For the sake of simplicity and brevity, much detail is omitted.  For
more detail, see `Resources`_ below.  `Literal blocks`_ (in which no
markup processing is done) are used for examples throughout, to
illustrate the plaintext markup.


General
-------

Lines should usually not extend past column 79,
excepting URLs and similar circumstances.
Tab characters must never appear in the document at all.


Section Headings
--


id: pep-0589.rst
content:
PEP: 589
Title: TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys
Author: Jukka Lehtosalo <jukka.lehtosalo@iki.fi>
Sponsor: Guido van Rossum <guido@python.org>
BDFL-Delegate: Guido van Rossum <guido@python.org>
Discussions-To: typing-sig@python.org
Status: Final
Type: Standards Track
Topic: Typing
Created: 20-Mar-2019
Python-Version: 3.8
Post-History:
Resolution: https://mail.python.org/archives/list/typing-sig@python.org/message/FDO4KFYWYQEP3U2HVVBEBR3SXPHQSHYR/

.. canonical-typing-spec:: :ref:`typing:typeddict` and
                           :py:class:`typing.TypedDict`

Abstract
========

:pep:`484` defines the type ``Dict[K, V]`` for uniform
dictionaries, where each value has the same type, and arbitrary key
values are supported.  It doesn't properly support the common pattern
where the type of a dictionary value depends on the string value of
the key.  This PEP proposes a type constructor ``typing.TypedDict`` to
support the use case where a dictionary object has a specific set of
string keys, each with a value of a specific type.

Here is an example where :pep:`484` doesn't allow us to annotate
satisfactorily::

    movie = {'name': 'Blade Runner',
             'year': 1982}

This PEP proposes the addition of a new type constructor, called
``TypedDict``, to allow the type of ``movie`` to be represented
precisely::

    from typing import TypedDict

    class Movie(TypedDict):
        name: str
        year: int

Now a type checker should accept this code::

    movie: Movie = {'name': 'Blade Runner',
                    'year': 1982}


Motivation
==========

Representing an object or structured data using (potentially nested)
dictionaries with string keys (instead of a user-defined class) is a
common pattern in Python programs.  Representing JSON objects is
perhaps the canonical use case, and this is popular enough that Python
ships with a JSON library.  This PEP proposes a way to allow such code
to be type checked more effectively.

More generally, representing pure data objects using only Python
primitive types such as dictionaries, strings and lists has had
certain appeal.  They are easy to serialize and deserialize even
when not using JSON.  They trivially support various useful operations
with no extra effort, including pretty-printing (through ``str()`` and
the ``pprint`` module), iteration, and equality comparisons.

:pep:`484` doesn't properly support the use cases mentioned above.  Let's
consider a dictionary object that has exactly two valid string keys,
``'name'`` with value type ``str``, and ``'year'`` with value type
``int``.  The :pep:`484` type ``Dict[str, Any]`` would be suitable, but
it is too lenient, as arbitrary string keys can be used, and arbitrary
values are valid.  Similarly, ``Dict[str, Union[str, int]]`` is too
general, as the value for key ``'name'`` could be an ``int``, and
arbitrary string keys are allowed.  Also, the type of a subscription
expression such as ``d['name']`` (assuming ``d`` to be a dictionary of
this type) would be ``Union[str, int]``, which is too wide.

Dataclasses are a more recent alternative to solve this use case, but
there is still a lot of existing code that was written before
dataclasses became available, especially in large existing codebases
where type hinting and checking has proven to be helpful.  Unlike
dictionary objects, dataclasses don't directly support JSON
serialization, though there is a third-party package that implements
it [#dataclasses-json]_.


Specification
=============

A TypedDict type represents dictionary objects with a specific set of
string keys, and with specific value types for each valid key.  Each
string key can be either required (it must be present) or
non-required (it doesn't need to exist).

This PEP proposes two ways of defining TypedDict types.  The first uses
a class-based syntax.  The second is an alternative
assignment-based syntax that is provided for backwards compatibility,
to allow the feature to be backported to older Python versions.  The
rationale is similar to why :pep:`484` supports a comment-based
annotation syntax for Python 2.7: type hinting is particularly useful
for large existing codebases, and these often need to run on older
Python versions.  The two syntax options parallel the syntax variants
supported by ``typing.NamedTuple``.  Other proposed features include
TypedDict inheritance and totality (specifying whether keys are
required or not).

This PEP also provides a sketch of how a type checker is expected
to support type checking operations involving TypedDict objects.
Similar to :pep:`484`, this discussion is left somewhat vague on purpose,
to allow experimentation with a wide variety of different type
checking approaches.  In particular, type compatibility should be
based on structural compatibility: a more specific TypedDict type can
be compatible with a smaller (more general) TypedDict type.


Class-based Syntax
------------------

A TypedDict type can be defined using the class definition syntax with
``typing.TypedDict`` as the sole base class::

    from typing import TypedDict

    class Movie(TypedDict):
        name: str
        year: int

``Movie`` is a TypedDict type with two items: ``'name'`` (with type
``str``) and ``'year'`` (with type ``int``).

A type checker should validate that the body of a class-based
TypedDict definition conforms to the following rules:

* The class body should only contain lines with item definitions of the
  form ``key: value_type``, optionally preceded by a docstring.  The
  syntax for item definitions is identical to attribute annotations,
  but there must be no initializer, and the key name actually refers
  to the string value of the key instead of an attribute name.

* Type comments cannot be used with the class-based syntax, for
  consistency with the class-based ``NamedTuple`` syntax.  (Note that
  it would not be sufficient to support type comments for backwards
  compatibility with Python 2.7, since the class definition may have a
  ``total`` keyword argument, as discussed below, and this isn't valid
  syntax in Python 2.7.)  Instead, this PEP provides an alternative,
  assignment-based syntax for backwards compatibility, discussed in
  `Alternative Syntax`_.

* String literal forward references are valid in the value types.

* Methods are not allowed, since the runtime type of a TypedDict
  object will always be just ``dict`` (it is never a subclass of
  ``dict``).

* Specifying a metaclass is not allowed.

An empty TypedDict can be created by only including ``pass`` in the
body (if there is a docstring, ``pass`` can be omitted)::

    class EmptyDict(TypedDict):
        pass


Using TypedDict Types
---------------------

Here is an example of how the type ``Movie`` can be used::

    movie: Movie = {'name': 'Blade Runner',
                    'year': 1982}

An explicit ``Movie`` type annotation is generally needed, as
otherwise an ordinary dictionary type could be assumed by a type
checker, for backwards compatibility.  When a type checker can infer
that a constructed dictionary object should be a TypedDict, an
explicit annotation can be omitted.  A typical example is a dictionary
object as a function argument.  In this example, a type checker is
expected to infer that the dictionary argument should be understood as
a TypedDict::

    def record_movie(movie: Movie) -> None: ...

    record_movie({'name': 'Blade Runner', 'year': 1982})

Another example where a type checker should treat a dictionary display
as a TypedDict is in an assignment to a variable with a previously
declared TypedDict type::

    movie: Movie
    ...
    movie = {'name': 'Blade Runner', 'year': 1982}

Operations on ``movie`` can be checked by a static type checker::

    movie['director'] = 'Ridley Scott'  # Error: invalid key 'director'
    movie['year'] = '1982'  # Error: invalid value type ("int" expected)

The code below should be rejected, since ``'title'`` is not a valid
key, and the ``'name'`


id: pep-NNNN.rst
content:
PEP: <REQUIRED: pep number>
Title: <REQUIRED: pep title>
Author: <REQUIRED: list of authors' names and optionally, email addrs>
Sponsor: <name of sponsor>
PEP-Delegate: <PEP delegate's name>
Discussions-To: Pending
Status: <REQUIRED: Draft | Active | Accepted | Provisional | Deferred | Rejected | Withdrawn | Final | Superseded>
Type: <REQUIRED: Standards Track | Informational | Process>
Topic: <Governance | Packaging | Release | Typing>
Requires: <pep numbers>
Created: <date created on, in dd-mmm-yyyy format>
Python-Version: <version number>
Post-History: <REQUIRED: dates, in dd-mmm-yyyy format, and corresponding links to PEP discussion threads>
Replaces: <pep number>
Superseded-By: <pep number>
Resolution: <url>


Abstract
========

[A short (~200 word) description of the technical issue being addressed.]


Motivation
==========

[Clearly explain why the existing language specification is inadequate to address the problem that the PEP solves.]


Rationale
=========

[Describe why particular design decisions were made.]


Specification
=============

[Describe the syntax and semantics of any new language feature.]


Backwards Compatibility
=======================

[Describe potential impact and severity on pre-existing code.]


Security Implications
=====================

[How could a malicious user take advantage of this new feature?]


How to Teach This
=================

[How to teach users, new and experienced, how to apply the PEP to their work.]


Reference Implementation
========================

[Link to any existing implementation and details about its state, e.g. proof-of-concept.]


Rejected Ideas
==============

[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.]


Open Issues
===========

[Any points that are still being decided/discussed.]


Footnotes
=========

[A collection of footnotes cited in the PEP, and a place to list non-inline hyperlink targets.]


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


id: pep-8102.rst
content:
PEP: 8102
Title: 2021 Term Steering Council election
Author: Ewa Jodlowska <ewa@python.org>, Ee Durbin <ee@python.org>, Joe Carey <joe@python.org>
Sponsor: Brett Cannon <brett@python.org>
Status: Final
Type: Informational
Topic: Governance
Created: 29-Oct-2020


Abstract
========

This document describes the schedule and other details of the December
2020 election for the Python steering council, as specified in
:pep:`13`. This is the steering council election for the 2021 term.


Election Administration
=======================

The steering council appointed the
`Python Software Foundation <https://www.python.org/psf-landing/>`__
Director of Infrastructure, Ee Durbin,
and Accounting Manager, Joe Carey, to coadminister the election.

`Python Software Foundation <https://www.python.org/psf-landing/>`__
Executive Director, Ewa Jodlowska, will communicate announcements
regarding the election.


Schedule
========

There will be a two-week nomination period, followed by a two-week
vote.

The nomination period was: November 1, 2020 through November 16, 2020 12:00 UTC
(The end of November 15, 2020 `Anywhere on Earth
<https://www.ieee802.org/16/aoe.html>`_). As announced on `python-committers
<https://mail.python.org/archives/list/python-committers@python.org/thread/JHYSU6FEYM3A5AZXSICO5OE3VAWDPGEJ/>`_,
and took place on `discuss.python.org <https://discuss.python.org/c/core-dev/steering-council-nominations>`_.

The voting period is: December 1, 2020 12:00 UTC through December 16, 2020
12:00 UTC (The end of December 15, 2020 `Anywhere on Earth
<https://www.ieee802.org/16/aoe.html>`_).


Candidates
==========

Candidates must be nominated by a core team member. If the candidate
is a core team member, they may nominate themselves.

Nominees (in alphabetical order):

- `Brett Cannon <https://discuss.python.org/t/steering-council-nomination-brett-cannon-2021-term/5633>`_
- `Matt Harrison <https://discuss.python.org/t/steering-council-nomination-matt-harrison-2021-term/5645>`_
- `Christian Heimes <https://discuss.python.org/t/steering-council-nomination-christian-heimes-2021-term/5628>`_
- `Raymond Hettinger <https://discuss.python.org/t/steering-council-nomination-raymond-hettinger-2021-term/5731>`_
- `David Mertz <https://discuss.python.org/t/steering-council-nomination-david-mertz-2021-term/5718>`_
- `Pablo Galindo Salgado <https://discuss.python.org/t/steering-council-nomination-pablo-galindo-salgado-2021-term/5720>`_
- `Barry Warsaw <https://discuss.python.org/t/steering-council-nomination-barry-warsaw-2021-term/5742>`_
- `Carol Willing <https://discuss.python.org/t/steering-council-nomination-carol-willing-2021-term/5763>`_
- `Thomas Wouters <https://discuss.python.org/t/steering-council-nomination-thomas-wouters-2021-term/5678>`_
- `Mariatta <https://discuss.python.org/t/steering-council-nomination-mariatta-2021-term/5765>`_

Withdrawn nominations:

- None

Voter Roll
==========

All active Python core team members are eligible to vote. Active status
is determined as described in :pep:`PEP 13 <13#membership>`
and implemented via the software at `python/voters <https://github.com/python/voters>`_ [1]_.

Ballots will be distributed based on the `The Python Voter Roll for this
election
<https://github.com/python/voters/blob/master/voter-files/>`_
[1]_.

While this file is not public as it contains private email addresses, the
`Complete Voter Roll`_ by name will be made available when the roll is
created.

Election Implementation
=======================

The election will be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2021-python-steering-council``

Name: ``2021 Python Steering Council Election``

Description: ``Election for the Python steering council, as specified in PEP 13. This is steering council election for the 2021 term.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``psf-election@python.org``

Voting starts at: ``December 1, 2020 00:00 UTC``

Voting ends at: ``December 16, 2020 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``- (approval)`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``



Results
=======

Of 91 eligible voters, 74 cast ballots.

The top five vote-getters are:

* Barry Warsaw
* Brett Cannon
* Carol Willing
* Pablo Galindo Salgado
* Thomas Wouters

No conflict of interest as defined in :pep:`13` were observed.

The full vote counts are as follows:

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Pablo Galindo Salgado | 59             |
+-----------------------+----------------+
| Carol Willing         | 57             |
+-----------------------+----------------+
| Barry Warsaw          | 56             |
+-----------------------+----------------+
| Brett Cannon          | 54             |
+-----------------------+----------------+
| Thomas Wouters        | 52             |
+-----------------------+----------------+
| Christian Heimes      | 43             |
+-----------------------+----------------+
| Mariatta              | 42             |
+-----------------------+----------------+
| Raymond Hettinger     | 31             |
+-----------------------+----------------+
| David Mertz           | 28             |
+-----------------------+----------------+
| Matt Harrison         | 15             |
+-----------------------+----------------+


Copyright
=========

This document has been placed in the public domain.


Complete Voter Roll
===================

Active Python core developers
-----------------------------

::

    Abhilash Raj
    Alex Gaynor
    Alex Martelli
    Alexander Belopolsky
    Andrew Kuchling
    Andrew Svetlov
    Antoine Pitrou
    Barry Warsaw
    Batuhan Taskaya
    Benjamin Peterson
    Berker PeksaÄ
    Brandt Bucher
    Brett Cannon
    Brian Curtin
    Brian Quinlan
    Carol Willing
    Cheryl Sabella
    Chris Jerdonek
    Chris Withers
    Christian Heimes
    Christian Tismer
    Davin Potts
    Dino Viehland
    Donald Stufft
    Dong-hee Na
    Emily Morehouse
    Ãric Araujo
    Eric Snow
    Eric V. Smith
    Ethan Furman
    Ezio Melotti
    Facundo Batista
    Fred Drake
    Georg Brandl
    Giampaolo RodolÃ 
    Gregory P. Smith
    Guido van Rossum
    Hynek Schlawack
    Inada Naoki
    Ivan Levkivskyi
    Jack Jansen
    Jason R. Coombs
    Jeremy Kloth
    JesÃºs Cea
    Joannah Nanjekye
    Julien Palard
    Karthikeyan Singaravelan
    Kurt B. Kaiser
    Kushal Das
    Kyle Stanley
    Larry Hastings
    Lisa Roach
    Åukasz Langa
    Lysandros Nikolaou
    Marc-AndrÃ© Lemburg
    Mariatta
    Mark Dickinson
    Mark Hammond
    Mark Shannon
    Matthias Klose
    Michael Foord
    Nathaniel J. Smith
    Ned Deily
    Neil Schemenauer
    Alyssa Coghlan
    Pablo Galindo
    Paul Ganssle
    Paul Moore
    Petr Viktorin
    R. David Murray
    Raymond Hettinger
    Robert Collins
    Ronald Oussoren
    Sandro Tosi
    Senthil Kumaran
    Serhiy Storchaka
    Stefan Behnel
    Steve Dower
    Steven D'Aprano
    StÃ©phane Wirtel
    Tal Einat
    Terry Jan Reedy
    Thomas Wouters
    Tim Golden
    Tim Peters
    Victor Stinner
    Vinay Sajip
    Walter DÃ¶rwald
    Xiang Zhang
    Yury Selivanov
    Zachary Ware


.. [1] This repository is private and accessible only to Python Core
   Developers, administrators, and Python Software Foundation Staff as it
   contains personal email addresses.


id: pep-0238.rst
content:
PEP: 238
Title: Changing the Division Operator
Author: Moshe Zadka <moshez@zadka.site.co.il>,
        Guido van Rossum <guido@python.org>
Status: Final
Type: Standards Track
Created: 11-Mar-2001
Python-Version: 2.2
Post-History: 16-Mar-2001, 26-Jul-2001, 27-Jul-2001


Abstract
========

The current division (``/``) operator has an ambiguous meaning for numerical
arguments: it returns the floor of the mathematical result of division if the
arguments are ints or longs, but it returns a reasonable approximation of the
division result if the arguments are floats or complex.  This makes
expressions expecting float or complex results error-prone when integers are
not expected but possible as inputs.

We propose to fix this by introducing different operators for different
operations: ``x/y`` to return a reasonable approximation of the mathematical
result of the division ("true division"), ``x//y`` to return the floor
("floor division").  We call the current, mixed meaning of x/y
"classic division".

Because of severe backwards compatibility issues, not to mention a major
flamewar on c.l.py, we propose the following transitional measures (starting
with Python 2.2):

- Classic division will remain the default in the Python 2.x series; true
  division will be standard in Python 3.0.

- The ``//`` operator will be available to request floor division
  unambiguously.

- The future division statement, spelled ``from __future__ import division``,
  will change the ``/`` operator to mean true division throughout the module.

- A command line option will enable run-time warnings for classic division
  applied to int or long arguments; another command line option will make true
  division the default.

- The standard library will use the future division statement and the ``//``
  operator when appropriate, so as to completely avoid classic division.


Motivation
==========

The classic division operator makes it hard to write numerical expressions
that are supposed to give correct results from arbitrary numerical inputs.
For all other operators, one can write down a formula such as ``x*y**2 + z``,
and the calculated result will be close to the mathematical result (within the
limits of numerical accuracy, of course) for any numerical input type (int,
long, float, or complex).  But division poses a problem: if the expressions
for both arguments happen to have an integral type, it implements floor
division rather than true division.

The problem is unique to dynamically typed languages: in a statically typed
language like C, the inputs, typically function arguments, would be declared
as double or float, and when a call passes an integer argument, it is
converted to double or float at the time of the call.  Python doesn't have
argument type declarations, so integer arguments can easily find their way
into an expression.

The problem is particularly pernicious since ints are perfect substitutes for
floats in all other circumstances: ``math.sqrt(2)`` returns the same value as
``math.sqrt(2.0)``, ``3.14*100`` and ``3.14*100.0`` return the same value, and
so on.  Thus, the author of a numerical routine may only use floating point
numbers to test his code, and believe that it works correctly, and a user may
accidentally pass in an integer input value and get incorrect results.

Another way to look at this is that classic division makes it difficult to
write polymorphic functions that work well with either float or int arguments;
all other operators already do the right thing.  No algorithm that works for
both ints and floats has a need for truncating division in one case and true
division in the other.

The correct work-around is subtle: casting an argument to float() is wrong if
it could be a complex number; adding 0.0 to an argument doesn't preserve the
sign of the argument if it was minus zero.  The only solution without either
downside is multiplying an argument (typically the first) by 1.0.  This leaves
the value and sign unchanged for float and complex, and turns int and long
into a float with the corresponding value.

It is the opinion of the authors that this is a real design bug in Python, and
that it should be fixed sooner rather than later. Assuming Python usage will
continue to grow, the cost of leaving this bug in the language will eventually
outweigh the cost of fixing old code -- there is an upper bound to the amount
of code to be fixed, but the amount of code that might be affected by the bug
in the future is unbounded.

Another reason for this change is the desire to ultimately unify Python's
numeric model.  This is the subject of :pep:`228` (which is currently
incomplete).  A unified numeric model removes most of the user's need to be
aware of different numerical types.  This is good for beginners, but also
takes away concerns about different numeric behavior for advanced programmers.
(Of course, it won't remove concerns about numerical stability and accuracy.)

In a unified numeric model, the different types (int, long, float, complex,
and possibly others, such as a new rational type) serve mostly as storage
optimizations, and to some extent to indicate orthogonal properties such as
inexactness or complexity.  In a unified model, the integer 1 should be
indistinguishable from the floating point number 1.0 (except for its
inexactness), and both should behave the same in all numeric contexts.
Clearly, in a unified numeric model, if ``a==b`` and ``c==d``, ``a/c`` should
equal ``b/d`` (taking some liberties due to rounding for inexact numbers), and
since everybody agrees that ``1.0/2.0`` equals 0.5, ``1/2`` should also equal
0.5.  Likewise, since ``1//2`` equals zero, ``1.0//2.0`` should also equal
zero.


Variations
==========

Aesthetically, ``x//y`` doesn't please everyone, and hence several variations
have been proposed.  They are addressed here:

- ``x div y``.  This would introduce a new keyword.  Since ``div`` is a
  popular identifier, this would break a fair amount of existing code, unless
  the new keyword was only recognized under a future division statement.
  Since it is expected that the majority of code that needs to be converted is
  dividing integers, this would greatly increase the need for the future
  division statement.  Even with a future statement, the general sentiment
  against adding new keywords unless absolutely necessary argues against this.

- ``div(x, y)``.  This makes the conversion of old code much harder.
  Replacing ``x/y`` with ``x//y`` or ``x div y`` can be done with a simple
  query replace; in most cases the programmer can easily verify that a
  particular module only works with integers so all occurrences of ``x/y`` can
  be replaced.  (The query replace is still needed to weed out slashes
  occurring in comments or string literals.)  Replacing ``x/y`` with
  ``div(x, y)`` would require a much more intelligent tool, since the extent
  of the expressions to the left and right of the ``/`` must be analyzed
  before the placement of the ``div(`` and ``)`` part can be decided.

- ``x \ y``.  The backslash is already a token, meaning line continuation, and
  in general it suggests an *escape* to Unix eyes.  In addition (this due to
  Terry Reedy) this would make things like ``eval("x\y")`` harder to get
  right.


Alternatives
============

In order to reduce the amount of old code that needs to be converted, several
alternative proposals have been put forth. Here is a brief discussion of each
proposal (or category of proposals).  If you know of an alternative that was
discussed on c.l.py that isn't mentioned here, please mail the second author.

- Let ``/`` keep its classic semantics; introduce ``//`` for true division.
  This still leaves a broken operator in the language, and invites to use the
  broken behavior.  It also shuts off the road to a unified numeric model a la
  :pep:`228`.

- Let int division return a special "portmanteau" type that behaves as an
  integer in integer context, but like a float in a float conte


id: pep-0562.rst
content:
PEP: 562
Title: Module __getattr__ and __dir__
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
Status: Final
Type: Standards Track
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151033.html


.. canonical-doc:: `Customizing Module Attribute Access <https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access>`_


Abstract
========

It is proposed to support ``__getattr__`` and ``__dir__`` function defined
on modules to provide basic customization of module attribute access.



Rationale
=========

It is sometimes convenient to customize or otherwise have control over
access to module attributes. A typical example is managing deprecation
warnings. Typical workarounds are assigning ``__class__`` of a module object
to a custom subclass of ``types.ModuleType`` or replacing the ``sys.modules``
item with a custom wrapper instance. It would be convenient to simplify this
procedure by recognizing ``__getattr__`` defined directly in a module that
would act like a normal ``__getattr__`` method, except that it will be defined
on module *instances*. For example::

  # lib.py

  from warnings import warn

  deprecated_names = ["old_function", ...]

  def _deprecated_old_function(arg, other):
      ...

  def __getattr__(name):
      if name in deprecated_names:
          warn(f"{name} is deprecated", DeprecationWarning)
          return globals()[f"_deprecated_{name}"]
      raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

  # main.py

  from lib import old_function  # Works, but emits the warning

Another widespread use case for ``__getattr__`` would be lazy submodule
imports. Consider a simple example::

  # lib/__init__.py

  import importlib

  __all__ = ['submod', ...]

  def __getattr__(name):
      if name in __all__:
          return importlib.import_module("." + name, __name__)
      raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

  # lib/submod.py

  print("Submodule loaded")
  class HeavyClass:
      ...

  # main.py

  import lib
  lib.submod.HeavyClass  # prints "Submodule loaded"

There is a related proposal :pep:`549` that proposes to support instance
properties for a similar functionality. The difference is this PEP proposes
a faster and simpler mechanism, but provides more basic customization.
An additional motivation for this proposal is that :pep:`484` already defines
the use of module ``__getattr__`` for this purpose in Python stub files,
see :pep:`484#stub-files`.

In addition, to allow modifying result of a ``dir()`` call on a module
to show deprecated and other dynamically generated attributes, it is
proposed to support module level ``__dir__`` function. For example::

  # lib.py

  deprecated_names = ["old_function", ...]
  __all__ = ["new_function_one", "new_function_two", ...]

  def new_function_one(arg, other):
     ...
  def new_function_two(arg, other):
      ...

  def __dir__():
      return sorted(__all__ + deprecated_names)

  # main.py

  import lib

  dir(lib)  # prints ["new_function_one", "new_function_two", "old_function", ...]


Specification
=============

The ``__getattr__`` function at the module level should accept one argument
which is the name of an attribute and return the computed value or raise
an ``AttributeError``::

  def __getattr__(name: str) -> Any: ...

If an attribute is not found on a module object through the normal lookup
(i.e. ``object.__getattribute__``), then ``__getattr__`` is searched in
the module ``__dict__`` before raising an ``AttributeError``. If found, it is
called with the attribute name and the result is returned. Looking up a name
as a module global will bypass module ``__getattr__``. This is intentional,
otherwise calling ``__getattr__`` for builtins will significantly harm
performance.

The ``__dir__`` function should accept no arguments, and return
a list of strings that represents the names accessible on module::

  def __dir__() -> List[str]: ...

If present, this function overrides the standard ``dir()`` search on
a module.

The reference implementation for this PEP can be found in [2]_.


Backwards compatibility and impact on performance
=================================================

This PEP may break code that uses module level (global) names ``__getattr__``
and ``__dir__``.  (But the language reference explicitly reserves *all*
undocumented dunder names, and allows "breakage without warning"; see [3]_.)
The performance implications of this PEP are minimal, since ``__getattr__``
is called only for missing attributes.

Some tools that perform module attributes discovery might not expect
``__getattr__``. This problem is not new however, since it is already possible
to replace a module with a module subclass with overridden ``__getattr__`` and
``__dir__``, but with this PEP such problems can occur more often.


Discussion
==========

Note that the use of module ``__getattr__`` requires care to keep the referred
objects pickleable. For example, the ``__name__`` attribute of a function
should correspond to the name with which it is accessible via
``__getattr__``::

  def keep_pickleable(func):
      func.__name__ = func.__name__.replace('_deprecated_', '')
      func.__qualname__ = func.__qualname__.replace('_deprecated_', '')
      return func

  @keep_pickleable
  def _deprecated_old_function(arg, other):
      ...

One should be also careful to avoid recursion as one would do with
a class level ``__getattr__``.

To use a module global with triggering ``__getattr__`` (for example if one
wants to use a lazy loaded submodule) one can access it as::

    sys.modules[__name__].some_global

or as::

    from . import some_global

Note that the latter sets the module attribute, thus ``__getattr__`` will be
called only once.


References
==========

.. [2] The reference implementation
   (https://github.com/ilevkivskyi/cpython/pull/3/files)

.. [3] Reserved classes of identifiers
   (https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers)


Copyright
=========

This document has been placed in the public domain.


id: pep-0204.rst
content:
PEP: 204
Title: Range Literals
Author: Thomas Wouters <thomas@python.org>
Status: Rejected
Type: Standards Track
Created: 14-Jul-2000
Python-Version: 2.0
Post-History:

.. rejected::

   After careful consideration, and a period of meditation, this
   proposal has been rejected. The open issues, as well as some
   confusion between ranges and slice syntax, raised enough questions
   for Guido not to accept it for Python 2.0, and later to reject the
   proposal altogether. The new syntax and its intentions were deemed
   not obvious enough.

   [ TBD: Guido, amend/confirm this, please. Preferably both; this
   is a PEP, it should contain *all* the reasons for rejection
   and/or reconsideration, for future reference. ]

Introduction
============

This PEP describes the "range literal" proposal for Python 2.0.
This PEP tracks the status and ownership of this feature, slated
for introduction in Python 2.0.  It contains a description of the
feature and outlines changes necessary to support the feature.
This PEP summarizes discussions held in mailing list forums, and
provides URLs for further information, where appropriate.  The CVS
revision history of this file contains the definitive historical
record.


List ranges
===========

Ranges are sequences of numbers of a fixed stepping, often used in
for-loops.  The Python for-loop is designed to iterate over a
sequence directly::

   >>> l = ['a', 'b', 'c', 'd']
   >>> for item in l:
   ...     print item
   a
   b
   c
   d

However, this solution is not always prudent.  Firstly, problems
arise when altering the sequence in the body of the for-loop,
resulting in the for-loop skipping items.  Secondly, it is not
possible to iterate over, say, every second element of the
sequence.  And thirdly, it is sometimes necessary to process an
element based on its index, which is not readily available in the
above construct.

For these instances, and others where a range of numbers is
desired, Python provides the ``range`` builtin function, which
creates a list of numbers.  The ``range`` function takes three
arguments, *start*, *end* and *step*.  *start* and *step* are
optional, and default to 0 and 1, respectively.

The ``range`` function creates a list of numbers, starting at
*start*, with a step of *step*, up to, but not including *end*, so
that ``range(10)`` produces a list that has exactly 10 items, the
numbers 0 through 9.

Using the ``range`` function, the above example would look like
this::

   >>> for i in range(len(l)):
   ...     print l[i]
   a
   b
   c
   d

Or, to start at the second element of ``l`` and processing only
every second element from then on::

   >>> for i in range(1, len(l), 2):
   ...     print l[i]
   b
   d

There are several disadvantages with this approach:

- Clarity of purpose: Adding another function call, possibly with
  extra arithmetic to determine the desired length and step of the
  list, does not improve readability of the code.  Also, it is
  possible to "shadow" the builtin ``range`` function by supplying a
  local or global variable with the same name, effectively
  replacing it.  This may or may not be a desired effect.

- Efficiency: because the ``range`` function can be overridden, the
  Python compiler cannot make assumptions about the for-loop, and
  has to maintain a separate loop counter.

- Consistency: There already is a syntax that is used to denote
  ranges, as shown below.  This syntax uses the exact same
  arguments, though all optional, in the exact same way.  It seems
  logical to extend this syntax to ranges, to form "range
  literals".


Slice Indices
=============

In Python, a sequence can be indexed in one of two ways:
retrieving a single item, or retrieving a range of items.
Retrieving a range of items results in a new object of the same
type as the original sequence, containing zero or more items from
the original sequence.  This is done using a "range notation"::

   >>> l[2:4]
   ['c', 'd']

This range notation consists of zero, one or two indices separated
by a colon.  The first index is the *start* index, the second the
*end*.  When either is left out, they default to respectively the
start and the end of the sequence.

There is also an extended range notation, which incorporates
*step* as well.  Though this notation is not currently supported
by most builtin types, if it were, it would work as follows::

   >>> l[1:4:2]
   ['b', 'd']

The third "argument" to the slice syntax is exactly the same as
the *step* argument to ``range()``.  The underlying mechanisms of the
standard, and these extended slices, are sufficiently different
and inconsistent that many classes and extensions outside of
mathematical packages do not implement support for the extended
variant.  While this should be resolved, it is beyond the scope of
this PEP.

Extended slices do show, however, that there is already a
perfectly valid and applicable syntax to denote ranges in a way
that solve all of the earlier stated disadvantages of the use of
the ``range()`` function:

- It is clearer, more concise syntax, which has already proven to
  be both intuitive and easy to learn.

- It is consistent with the other use of ranges in Python
  (e.g. slices).

- Because it is built-in syntax, instead of a builtin function, it
  cannot be overridden.  This means both that a viewer can be
  certain about what the code does, and that an optimizer will not
  have to worry about ``range()`` being "shadowed".


The Proposed Solution
=====================

The proposed implementation of range-literals combines the syntax
for list literals with the syntax for (extended) slices, to form
range literals::

   >>> [1:10]
   [1, 2, 3, 4, 5, 6, 7, 8, 9]
   >>> [:5]
   [0, 1, 2, 3, 4]
   >>> [5:1:-1]
   [5, 4, 3, 2]

There is one minor difference between range literals and the slice
syntax: though it is possible to omit all of *start*, *end* and
*step* in slices, it does not make sense to omit *end* in range
literals.  In slices, *end* would default to the end of the list,
but this has no meaning in range literals.


Reference Implementation
========================

The proposed implementation can be found on SourceForge [1]_.  It
adds a new bytecode, ``BUILD_RANGE``, that takes three arguments from
the stack and builds a list on the bases of those.  The list is
pushed back on the stack.

The use of a new bytecode is necessary to be able to build ranges
based on other calculations, whose outcome is not known at compile
time.

The code introduces two new functions to ``listobject.c``, which are
currently hovering between private functions and full-fledged API
calls.

``PyList_FromRange()`` builds a list from start, end and step,
returning NULL if an error occurs.  Its prototype is::

    PyObject * PyList_FromRange(long start, long end, long step)

``PyList_GetLenOfRange()`` is a helper function used to determine the
length of a range.  Previously, it was a static function in
``bltinmodule.c``, but is now necessary in both ``listobject.c`` and
``bltinmodule.c`` (for ``xrange``).  It is made non-static solely to avoid
code duplication.  Its prototype is::

    long PyList_GetLenOfRange(long start, long end, long step)


Open issues
===========

- One possible solution to the discrepancy of requiring the *end*
  argument in range literals is to allow the range syntax to
  create a "generator", rather than a list, such as the ``xrange``
  builtin function does.  However, a generator would not be a
  list, and it would be impossible, for instance, to assign to
  items in the generator, or append to it.

  The range syntax could conceivably be extended to include tuples
  (i.e. immutable lists), which could then be safely implemented
  as generators.  This may be a desirable solution, especially for
  large number arrays: generators require very little in the way
  of storage and initialization, and there is only a small
  performance impact in calculating and creating the appropriate
  number on request.  (


id: pep-0210.rst
content:
PEP: 210
Title: Decoupling the Interpreter Loop
Author: David Ascher <davida@activestate.com>
Status: Rejected
Type: Standards Track
Created: 15-Jul-2000
Python-Version: 2.1
Post-History:

.. rejected::


id: pep-0576.rst
content:
PEP: 576
Title: Rationalize Built-in function classes
Author: Mark Shannon <mark@hotpy.org>
BDFL-Delegate: Petr Viktorin
Status: Withdrawn
Type: Standards Track
Created: 10-May-2018
Python-Version: 3.8
Post-History: 17-May-2018,
              23-Jun-2018,
              08-Jul-2018,
              29-Mar-2019

Abstract
========

Expose the "FastcallKeywords" convention used internally by CPython to third-party code, and make the ``inspect`` module use duck-typing.
In combination this will allow third-party C extensions and tools like Cython to create objects that use the same calling conventions as built-in and Python functions, thus gaining performance parity with built-in functions like ``len`` or ``print``.

A small improvement in the performance of existing code is expected.

Motivation
==========

Currently third-party module authors face a dilemma when implementing
functions in C. Either they can use one of the pre-existing built-in function
or method classes or implement their own custom class in C.
The first choice causes them to lose the ability to access the internals of the callable object.
The second choice is an additional maintenance burden and, more importantly,
has a significant negative impact on performance.

This PEP aims to allow authors of third-party C modules, and tools like to Cython, to utilize the faster calling convention used internally by CPython for built-in functions and methods, and to do so without a loss of capabilities relative to a function implemented in Python.

Introspection
-------------

The inspect module will fully support duck-typing when introspecting callables.

The ``inspect.Signature.from_callable()`` function computes the signature of a callable. If an object has a ``__signature__``
property, then ``inspect.Signature.from_callable()`` simply returns that. To further support duck-typing, if a callable has a ``__text_signature__``
then the ``__signature__`` will be created from that.

This means that 3rd party builtin-functions can implement ``__text_signature__`` if sufficient,
and the more expensive ``__signature__`` if necessary.

Efficient calls to third-party callables
----------------------------------------

Currently the majority of calls are dispatched to ``function``\s and ``method_descriptor``\s in custom code, using the "FastcallKeywords" internal calling convention. This PEP proposes that this calling convention is implemented via a C function pointer. Third-party callables which implement this binary interface will have the potential to be called as fast as a built-in function.

Continued prohibition of callable classes as base classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Currently any attempt to use ``function``, ``method`` or ``method_descriptor`` as a base class for a new class will fail with a ``TypeError``. This behaviour is desirable as it prevents errors when a subclass overrides the ``__call__`` method. If callables could be sub-classed then any call to a ``function`` or a ``method_descriptor`` would need an additional check that the ``__call__`` method had not been overridden. By exposing an additional call mechanism, the potential for errors  becomes greater. As a consequence, any third-party class implementing the addition call interface will not be usable as a base class.


New classes and changes to existing classes
===========================================

Python visible changes
----------------------

#. A new built-in class, ``builtin_function``, will be added.

#. ``types.BuiltinFunctionType`` will refer to ``builtin_function`` not ``builtin_function_or_method``.

#. Instances of the ``builtin_function`` class will retain the ``__module__`` property of ``builtin_function_or_method`` and gain the ``func_module`` and ``func_globals`` properties. The ``func_module`` allows access to the module to which the function belongs. Note that this is different from the ``__module__`` property which merely returns the name of the module. The ``func_globals`` property is equivalent to ``func_module.__dict__`` and is provided to mimic the Python function property of the same name.

#. When binding a ``method_descriptor`` instance to an instance of its owning class, a ``bound_method`` will be created instead of a ``builtin_function_or_method``. This means that the ``method_descriptors`` now mimic the behaviour of Python functions more closely. In other words, ``[].append`` becomes a ``bound_method`` instead of a ``builtin_function_or_method``.


C API changes
-------------

#. A new function ``PyBuiltinFunction_New(PyMethodDef *ml, PyObject *module)`` is added to create built-in functions.

#. ``PyCFunction_NewEx()`` and ``PyCFunction_New()`` are deprecated and will return a ``PyBuiltinFunction`` if able, otherwise a ``builtin_function_or_method``.

Retaining backwards compatibility in the C API and ABI
======================================================

The proposed changes are fully backwards and forwards compatible at both the API and ABI level.


Internal C changes
------------------

Two new flags will be allowed for the ``typeobject.tp_flags`` field.
These are ``Py_TPFLAGS_EXTENDED_CALL`` and ``Py_TPFLAGS_FUNCTION_DESCRIPTOR``

Py_TPFLAGS_EXTENDED_CALL
~~~~~~~~~~~~~~~~~~~~~~~~

For any built-in class that sets ``Py_TPFLAGS_EXTENDED_CALL``
The C struct corresponding to this built-in class must begin with the struct ``PyExtendedCallable`` which is defined as follows::

    typedef PyObject *(*extended_call_ptr)(PyObject *callable, PyObject** args,
                       int positional_argcount, PyTupleObject* kwnames);

    typedef struct {
        PyObject_HEAD
        extended_call_ptr ext_call;
    } PyExtendedCallable;

Any class that sets the ``Py_TPFLAGS_EXTENDED_CALL`` cannot be used as a base class and a TypeError will be raised if any Python code tries to use it a base class.


Py_TPFLAGS_FUNCTION_DESCRIPTOR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If this flag is set for a built-in class ``F``, then instances of that class are expected to behave the same as a Python function when used as a class attribute.
Specifically, this mean that the value of ``c.m`` where ``C.m`` is an instanceof the built-in class ``F`` (and ``c`` is an instance of ``C``) must be a bound-method binding ``C.m`` and ``c``.
Without this flag, it would be impossible for custom callables to behave like Python functions *and* be efficient as Python or built-in functions.



Changes to existing C structs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``function``, ``method_descriptor`` and ``method`` classes will have their corresponding structs changed to
start with the ``PyExtendedCallable`` struct.

Third-party built-in classes using the new extended call interface
------------------------------------------------------------------

To enable call performance on a par with Python functions and built-in functions, third-party callables should set the ``Py_TPFLAGS_EXTENDED_CALL`` bit of ``tp_flags`` and ensure that the corresponding C struct starts with the ``PyExtendedCallable``.
Any built-in class that has the ``Py_TPFLAGS_EXTENDED_CALL`` bit set must also implement the ``tp_call`` function and make sure its behaviour is consistent with the ``ext_call`` function.

Performance implications of these changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Adding a function pointer to each callable, rather than each class of callable, enables the choice of dispatching function (the code to shuffle arguments about and do error checking) to be made when the callable object is created rather than when it is called. This should reduce the number of instructions executed between the call-site in the interpreter and the execution of the callee.


Alternative Suggestions
=======================

:pep:`580` is an alternative approach to solving the same problem as this PEP.



Reference implementation
========================

A draft implementation can be found at https://github.com/markshannon/cpython/tree/pep-576


id: pep-0206.rst
content:
PEP: 206
Title: Python Advanced Library
Author: A.M. Kuchling <amk@amk.ca>
Status: Withdrawn
Type: Informational
Created: 14-Jul-2000
Post-History:

.. withdrawn::

Introduction
============

This PEP describes the Python Advanced Library, a collection of
high-quality and frequently-used third party extension modules.


Batteries Included Philosophy
=============================

The Python source distribution has long maintained the philosophy
of "batteries included" -- having a rich and versatile standard
library which is immediately available, without making the user
download separate packages.  This gives the Python language a head
start in many projects.

However, the standard library modules aren't always the best
choices for a job.  Some library modules were quick hacks
(e.g. ``calendar``, ``commands``), some were designed poorly and are now
near-impossible to fix (``cgi``), and some have been rendered obsolete
by other, more complete modules (``binascii`` offers the same features
as the ``binhex``, ``uu``, ``base64`` modules).  This PEP describes a list of
third-party modules that make Python more competitive for various
application domains, forming the Python Advanced Library.

The deliverable is a set of scripts that will retrieve, build, and
install the packages for a particular application domain.  The
Python Package Index now contains enough information to let
software automatically find packages and download them, so the
time is ripe to implement this.

Currently this document doesn't suggest *removing* modules from
the standard library that are superseded by a third-party module.
That's difficult to do because it entails many backward-compatibility
problems, so it's not worth bothering with now.

Please suggest additional domains of interest.


Domain: Web tasks
=================

XML parsing: ElementTree + SAX.

URL retrieval: libcurl? other possibilities?

HTML parsing: mxTidy? HTMLParser?

Async network I/O: Twisted

RDF parser: ???

HTTP serving: ???

HTTP cookie processing: ???

Web framework: A WSGI gateway, perhaps?  Paste?

Graphics: PIL, Chaco.


Domain: Scientific Programming
==============================

Numeric: Numeric, SciPy

Graphics: PIL, Chaco.


Domain: Application Development
===============================

GUI toolkit: ???

Graphics: Reportlab for PDF generation.


Domain: Education
=================

Graphics: PyGame


Software covered by the GNU General Public License
==================================================

Some of these third-party modules are covered by the GNU General
Public License and the GNU Lesser General Public License.
Providing a script to download and install such packages, or even
assembling all these packages into a single tarball or CD-ROM,
shouldn't cause any difficulties with the GPL, under the "mere
aggregation" clause of the license.


Open Issues
===========

What other application domains are important?

Should this just be a set of Ubuntu or Debian packages?  Compiling
things such as PyGame can be very complicated and may be too
difficult to automate.


Acknowledgements
================

The PEP is based on an earlier draft PEP by Moshe Zadka, titled
"2.0 Batteries Included."


id: pep-0560.rst
content:
PEP: 560
Title: Core support for typing module and generic types
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
Status: Final
Type: Standards Track
Topic: Typing
Created: 03-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017, 14-Nov-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151038.html

.. canonical-doc:: the documentation for
                   :external+python:meth:`~object.__class_getitem__` and
                   :external+python:meth:`~object.__mro_entries__`

Abstract
========

Initially :pep:`484` was designed in such way that it would not introduce
*any* changes to the core CPython interpreter. Now type hints and
the ``typing`` module are extensively used by the community, e.g. :pep:`526`
and :pep:`557` extend the usage of type hints, and the backport of ``typing``
on PyPI has 1M downloads/month. Therefore, this restriction can be removed.
It is proposed to add two special methods ``__class_getitem__`` and
``__mro_entries__`` to the core CPython for better support of
generic types.


Rationale
=========

The restriction to not modify the core CPython interpreter led to some
design decisions that became questionable when the ``typing`` module started
to be widely used. There are three main points of concern:
performance of the ``typing`` module, metaclass conflicts, and the large
number of hacks currently used in ``typing``.


Performance
-----------

The ``typing`` module is one of the heaviest and slowest modules in
the standard library even with all the optimizations made. Mainly this is
because subscripted generic types (see :pep:`484` for definition of terms used
in this PEP) are class objects (see also [1]_). There are three main ways how
the performance can be improved with the help of the proposed special methods:

- Creation of generic classes is slow since the ``GenericMeta.__new__`` is
  very slow; we will not need it anymore.

- Very long method resolution orders (MROs) for generic classes will be
  half as long; they are present because we duplicate the ``collections.abc``
  inheritance chain in ``typing``.

- Instantiation of generic classes will be faster (this is minor however).


Metaclass conflicts
-------------------

All generic types are instances of ``GenericMeta``, so if a user uses
a custom metaclass, then it is hard to make a corresponding class generic.
This is particularly hard for library classes that a user doesn't control.
A workaround is to always mix-in ``GenericMeta``::

  class AdHocMeta(GenericMeta, LibraryMeta):
      pass

  class UserClass(LibraryBase, Generic[T], metaclass=AdHocMeta):
      ...

but this is not always practical or even possible. With the help of the
proposed special attributes the ``GenericMeta`` metaclass will not be needed.


Hacks and bugs that will be removed by this proposal
----------------------------------------------------

- ``_generic_new`` hack that exists because ``__init__`` is not called on
  instances with a type differing from the type whose ``__new__`` was called,
  ``C[int]().__class__ is C``.

- ``_next_in_mro`` speed hack will be not necessary since subscription will
  not create new classes.

- Ugly ``sys._getframe`` hack. This one is particularly nasty since it looks
  like we can't remove it without changes outside ``typing``.

- Currently generics do dangerous things with private ABC caches
  to fix large memory consumption that grows at least as O(N\ :sup:`2`),
  see [2]_. This point is also important because it was recently proposed to
  re-implement ``ABCMeta`` in C.

- Problems with sharing attributes between subscripted generics,
  see [3]_. The current solution already uses ``__getattr__`` and ``__setattr__``,
  but it is still incomplete, and solving this without the current proposal
  will be hard and will need ``__getattribute__``.

- ``_no_slots_copy`` hack, where we clean up the class dictionary on every
  subscription thus allowing generics with ``__slots__``.

- General complexity of the ``typing`` module. The new proposal will not
  only allow to remove the above-mentioned hacks/bugs, but also simplify
  the implementation, so that it will be easier to maintain.


Specification
=============

``__class_getitem__``
---------------------

The idea of ``__class_getitem__`` is simple: it is an exact analog of
``__getitem__`` with an exception that it is called on a class that
defines it, not on its instances. This allows us to avoid
``GenericMeta.__getitem__`` for things like ``Iterable[int]``.
The ``__class_getitem__`` is automatically a class method and
does not require ``@classmethod`` decorator (similar to
``__init_subclass__``) and is inherited like normal attributes.
For example::

  class MyList:
      def __getitem__(self, index):
          return index + 1
      def __class_getitem__(cls, item):
          return f"{cls.__name__}[{item.__name__}]"

  class MyOtherList(MyList):
      pass

  assert MyList()[0] == 1
  assert MyList[int] == "MyList[int]"

  assert MyOtherList()[0] == 1
  assert MyOtherList[int] == "MyOtherList[int]"

Note that this method is used as a fallback, so if a metaclass defines
``__getitem__``, then that will have the priority.


``__mro_entries__``
-------------------

If an object that is not a class object appears in the tuple of bases of
a class definition, then method ``__mro_entries__`` is searched on it.
If found, it is called with the original tuple of bases as an argument.
The result of the call must be a tuple, that is unpacked in the base classes
in place of this object. (If the tuple is empty, this means that the original
bases is simply discarded.) If there are more than one object with
``__mro_entries__``, then all of them are called with the same original tuple
of bases. This step happens first in the process of creation of a class,
all other steps, including checks for duplicate bases and MRO calculation,
happen normally with the updated bases.

Using the method API instead of just an attribute is necessary to avoid
inconsistent MRO errors, and perform other manipulations that are currently
done by ``GenericMeta.__new__``. The original bases are stored as
``__orig_bases__`` in the class namespace (currently this is also done by
the metaclass). For example::

  class GenericAlias:
      def __init__(self, origin, item):
          self.origin = origin
          self.item = item
      def __mro_entries__(self, bases):
          return (self.origin,)

  class NewList:
      def __class_getitem__(cls, item):
          return GenericAlias(cls, item)

  class Tokens(NewList[int]):
      ...

  assert Tokens.__bases__ == (NewList,)
  assert Tokens.__orig_bases__ == (NewList[int],)
  assert Tokens.__mro__ == (Tokens, NewList, object)

Resolution using ``__mro_entries__`` happens *only* in bases of a class
definition statement. In all other situations where a class object is
expected, no such resolution will happen, this includes ``isinstance``
and ``issubclass`` built-in functions.

NOTE: These two method names are reserved for use by the ``typing`` module
and the generic types machinery, and any other use is discouraged.
The reference implementation (with tests) can be found in [4]_, and
the proposal was originally posted and discussed on the ``typing`` tracker,
see [5]_.


Dynamic class creation and ``types.resolve_bases``
--------------------------------------------------

``type.__new__`` will not perform any MRO entry resolution. So that a direct
call ``type('Tokens', (List[int],), {})`` will fail. This is done for
performance reasons and to minimize the number of implicit transformations.
Instead, a helper function ``resolve_bases`` will be added to
the ``types`` module to allow an explicit ``__mro_entries__`` resolution in
the context of dynamic class creation. Correspondingly, ``types.new_class``
will be updated to reflect the new class creation steps while maintaining
the backwards compatibility::

  def new_class(name, bases=(), kwds=None, exec_body=None):
      resol


id: pep-0574.rst
content:
PEP: 574
Title: Pickle protocol 5 with out-of-band data
Author: Antoine Pitrou <solipsis@pitrou.net>
BDFL-Delegate: Alyssa Coghlan
Status: Final
Type: Standards Track
Created: 23-Mar-2018
Python-Version: 3.8
Post-History: 28-Mar-2018, 30-Apr-2019
Resolution: https://mail.python.org/pipermail/python-dev/2019-May/157284.html


Abstract
========

This PEP proposes to standardize a new pickle protocol version, and
accompanying APIs to take full advantage of it:

1. A new pickle protocol version (5) to cover the extra metadata needed
   for out-of-band data buffers.
2. A new ``PickleBuffer`` type for ``__reduce_ex__`` implementations
   to return out-of-band data buffers.
3. A new ``buffer_callback`` parameter when pickling, to handle out-of-band
   data buffers.
4. A new ``buffers`` parameter when unpickling to provide out-of-band data
   buffers.

The PEP guarantees unchanged behaviour for anyone not using the new APIs.


Rationale
=========

The pickle protocol was originally designed in 1995 for on-disk persistency
of arbitrary Python objects.  The performance of a 1995-era storage medium
probably made it irrelevant to focus on performance metrics such as
use of RAM bandwidth when copying temporary data before writing it to disk.

Nowadays the pickle protocol sees a growing use in applications where most
of the data isn't ever persisted to disk (or, when it is, it uses a portable
format instead of Python-specific).  Instead, pickle is being used to transmit
data and commands from one process to another, either on the same machine
or on multiple machines.  Those applications will sometimes deal with very
large data (such as Numpy arrays or Pandas dataframes) that need to be
transferred around.  For those applications, pickle is currently
wasteful as it imposes spurious memory copies of the data being serialized.

As a matter of fact, the standard ``multiprocessing`` module uses pickle
for serialization, and therefore also suffers from this problem when
sending large data to another process.

Third-party Python libraries, such as Dask [#dask]_, PyArrow [#pyarrow]_
and IPyParallel [#ipyparallel]_, have started implementing alternative
serialization schemes with the explicit goal of avoiding copies on large
data.  Implementing a new serialization scheme is difficult and often
leads to reduced generality (since many Python objects support pickle
but not the new serialization scheme).  Falling back on pickle for
unsupported types is an option, but then you get back the spurious
memory copies you wanted to avoid in the first place.  For example,
``dask`` is able to avoid memory copies for Numpy arrays and
built-in containers thereof (such as lists or dicts containing Numpy
arrays), but if a large Numpy array is an attribute of a user-defined
object, ``dask`` will serialize the user-defined object as a pickle
stream, leading to memory copies.

The common theme of these third-party serialization efforts is to generate
a stream of object metadata (which contains pickle-like information about
the objects being serialized) and a separate stream of zero-copy buffer
objects for the payloads of large objects.  Note that, in this scheme,
small objects such as ints, etc. can be dumped together with the metadata
stream.  Refinements can include opportunistic compression of large data
depending on its type and layout, like ``dask`` does.

This PEP aims to make ``pickle`` usable in a way where large data is handled
as a separate stream of zero-copy buffers, letting the application handle
those buffers optimally.


Example
=======

To keep the example simple and avoid requiring knowledge of third-party
libraries, we will focus here on a bytearray object (but the issue is
conceptually the same with more sophisticated objects such as Numpy arrays).
Like most objects, the bytearray object isn't immediately understood by
the pickle module and must therefore specify its decomposition scheme.

Here is how a bytearray object currently decomposes for pickling::

   >>> b.__reduce_ex__(4)
   (<class 'bytearray'>, (b'abc',), None)

This is because the ``bytearray.__reduce_ex__`` implementation reads
morally as follows::

   class bytearray:

      def __reduce_ex__(self, protocol):
         if protocol == 4:
            return type(self), bytes(self), None
         # Legacy code for earlier protocols omitted

In turn it produces the following pickle code::

   >>> pickletools.dis(pickletools.optimize(pickle.dumps(b, protocol=4)))
       0: \x80 PROTO      4
       2: \x95 FRAME      30
      11: \x8c SHORT_BINUNICODE 'builtins'
      21: \x8c SHORT_BINUNICODE 'bytearray'
      32: \x93 STACK_GLOBAL
      33: C    SHORT_BINBYTES b'abc'
      38: \x85 TUPLE1
      39: R    REDUCE
      40: .    STOP

(the call to ``pickletools.optimize`` above is only meant to make the
pickle stream more readable by removing the MEMOIZE opcodes)

We can notice several things about the bytearray's payload (the sequence
of bytes ``b'abc'``):

* ``bytearray.__reduce_ex__`` produces a first copy by instantiating a
  new bytes object from the bytearray's data.
* ``pickle.dumps`` produces a second copy when inserting the contents of
  that bytes object into the pickle stream, after the SHORT_BINBYTES opcode.
* Furthermore, when deserializing the pickle stream, a temporary bytes
  object is created when the SHORT_BINBYTES opcode is encountered (inducing
  a data copy).

What we really want is something like the following:

* ``bytearray.__reduce_ex__`` produces a *view* of the bytearray's data.
* ``pickle.dumps`` doesn't try to copy that data into the pickle stream
  but instead passes the buffer view to its caller (which can decide on the
  most efficient handling of that buffer).
* When deserializing, ``pickle.loads`` takes the pickle stream and the
  buffer view separately, and passes the buffer view directly to the
  bytearray constructor.

We see that several conditions are required for the above to work:

* ``__reduce__`` or ``__reduce_ex__`` must be able to return *something*
  that indicates a serializable no-copy buffer view.
* The pickle protocol must be able to represent references to such buffer
  views, instructing the unpickler that it may have to get the actual buffer
  out of band.
* The ``pickle.Pickler`` API must provide its caller with a way
  to receive such buffer views while serializing.
* The ``pickle.Unpickler`` API must similarly allow its caller to provide
  the buffer views required for deserialization.
* For compatibility, the pickle protocol must also be able to contain direct
  serializations of such buffer views, such that current uses of the ``pickle``
  API don't have to be modified if they are not concerned with memory copies.


Producer API
============

We are introducing a new type ``pickle.PickleBuffer`` which can be
instantiated from any buffer-supporting object, and is specifically meant
to be returned from ``__reduce__`` implementations::

   class bytearray:

      def __reduce_ex__(self, protocol):
         if protocol >= 5:
            return type(self), (PickleBuffer(self),), None
         # Legacy code for earlier protocols omitted

``PickleBuffer`` is a simple wrapper that doesn't have all the memoryview
semantics and functionality, but is specifically recognized by the ``pickle``
module if protocol 5 or higher is enabled.  It is an error to try to
serialize a ``PickleBuffer`` with pickle protocol version 4 or earlier.

Only the raw *data* of the ``PickleBuffer`` will be considered by the
``pickle`` module.  Any type-specific *metadata* (such as shapes or
datatype) must be returned separately by the type's ``__reduce__``
implementation, as is already the case.


PickleBuffer objects
--------------------

The ``PickleBuffer`` class supports a very simple Python API.  Its constructor
takes a single :pep:`3118`-compatible object.  ``PickleBuffer``
objects themselves support the buffer protocol, so consumers can
call ``memoryview(...)`` on them to get additional 


id: pep-0212.rst
content:
PEP: 212
Title: Loop Counter Iteration
Author: Peter Schneider-Kamp <nowonder@nowonder.de>
Status: Rejected
Type: Standards Track
Created: 22-Aug-2000
Python-Version: 2.1
Post-History:


Rejection Notice
================

This PEP has been rejected.  ``enumerate()``, introduced in :pep:`279`,
covers the use-case proposed in this PEP, and the PEP author has been
unreachable.


Introduction
============

This PEP describes the often proposed feature of exposing the loop
counter in for-loops.  This PEP tracks the status and ownership of
this feature.  It contains a description of the feature and
outlines changes necessary to support the feature.  This PEP
summarizes discussions held in mailing list forums, and provides
URLs for further information, where appropriate.  The CVS revision
history of this file contains the definitive historical record.


Motivation
==========

Standard for-loops in Python iterate over the elements of a
sequence [1]_.  Often it is desirable to loop over the indices or
both the elements and the indices instead.

The common idioms used to accomplish this are unintuitive.  This
PEP proposes two different ways of exposing the indices.


Loop counter iteration
======================

The current idiom for looping over the indices makes use of the
built-in ``range`` function::

    for i in range(len(sequence)):
        # work with index i

Looping over both elements and indices can be achieved either by the
old idiom or by using the new ``zip`` built-in function [2]_::

    for i in range(len(sequence)):
        e = sequence[i]
        # work with index i and element e

or::

   for i, e in zip(range(len(sequence)), sequence):
      # work with index i and element e


The Proposed Solutions
======================

There are three solutions that have been discussed.  One adds a
non-reserved keyword, the other adds two built-in functions.
A third solution adds methods to sequence objects.


Non-reserved keyword ``indexing``
=================================

This solution would extend the syntax of the for-loop by adding
an optional ``<variable> indexing`` clause which can also be used
instead of the ``<variable> in`` clause.

Looping over the indices of a sequence would thus become::

    for i indexing sequence:
        # work with index i

Looping over both indices and elements would similarly be::

    for i indexing e in sequence:
        # work with index i and element e


Built-in functions ``indices`` and ``irange``
=============================================

This solution adds two built-in functions ``indices`` and ``irange``.
The semantics of these can be described as follows::

    def indices(sequence):
        return range(len(sequence))

    def irange(sequence):
        return zip(range(len(sequence)), sequence)

These functions could be implemented either eagerly or lazily and
should be easy to extend in order to accept more than one sequence
argument.

The use of these functions would simplify the idioms for looping
over the indices and over both elements and indices::

    for i in indices(sequence):
        # work with index i

    for i, e in irange(sequence):
        # work with index i and element e


Methods for sequence objects
============================

This solution proposes the addition of ``indices``, ``items``
and ``values`` methods to sequences, which enable looping over
indices only, both indices and elements, and elements only
respectively.

This would immensely simplify the idioms for looping over indices
and for looping over both elements and indices::

    for i in sequence.indices():
        # work with index i

    for i, e in sequence.items():
        # work with index i and element e

Additionally it would allow to do looping over the elements
of sequences and dictionaries in a consistent way::

    for e in sequence_or_dict.values():
        # do something with element e


Implementations
===============

For all three solutions some more or less rough patches exist
as patches at SourceForge:

- ``for i indexing a in l``: exposing the for-loop counter [3]_
- add ``indices()`` and ``irange()`` to built-ins [4]_
- add ``items()`` method to listobject [5]_

All of them have been pronounced on and rejected by the BDFL.

Note that the ``indexing`` keyword is only a ``NAME`` in the
grammar and so does not hinder the general use of ``indexing``.


Backward Compatibility Issues
=============================

As no keywords are added and the semantics of existing code
remains unchanged, all three solutions can be implemented
without breaking existing code.


Copyright
=========

This document has been placed in the public domain.


References
==========

.. [1] http://docs.python.org/reference/compound_stmts.html#for

.. [2] Lockstep Iteration, :pep:`201`

.. [3] http://sourceforge.net/patch/download.php?id=101138

.. [4] http://sourceforge.net/patch/download.php?id=101129

.. [5] http://sourceforge.net/patch/download.php?id=101178


id: pep-8100.rst
content:
PEP: 8100
Title: January 2019 Steering Council election
Author: Nathaniel J. Smith <njs@pobox.com>, Ee Durbin <ee@python.org>
Status: Final
Type: Informational
Topic: Governance
Created: 03-Jan-2019


Abstract
========

This document describes the schedule and other details of the January
2019 election for the Python steering council, as specified in
:pep:`13`. This is the first steering council election.


Returns officer
===============

In future elections, the returns officer will be appointed by the
outgoing steering council. Since this is the first election, we have
no outgoing steering council, and :pep:`13` says that the returns officer
is instead appointed by the PSF Executive Director, Ewa Jodlowska.
`She appointed Ee Durbin
<https://discuss.python.org/t/officially-appointing-the-returns-officer-for-the-steering-council-election/603>`__.


Schedule
========

There will be a two-week nomination period, followed by a two-week
vote.

The nomination period is: January 7, 2019 through January 20, 2019

The voting period is: January 21, 2019 12:00 UTC through February 4, 2019 12:00
UTC (The end of February 3, 2019 `Anywhere on Earth
<http://www.ieee802.org/16/aoe.html>`_)


Candidates
==========

Candidates must be nominated by a core team member. If the candidate
is a core team member, they may nominate themselves.

Once the nomination period opens, candidates will be listed here:

1. `Brett Cannon <https://discuss.python.org/t/steering-council-nomination-brett-cannon/620>`_
2. `Alyssa (Nick) Coghlan <https://discuss.python.org/t/steering-council-nomination-nick-coghlan/624>`_
3. `Barry Warsaw <https://discuss.python.org/t/steering-council-nomination-barry-warsaw/629>`__
4. `Guido van Rossum <https://discuss.python.org/t/steering-council-nomination-guido-van-rossum/628>`__
5. `Victor Stinner <https://discuss.python.org/t/steering-council-nomination-victor-stinner/635>`_
6. `Yury Selivanov <https://discuss.python.org/t/steering-council-nomination-yury-selivanov/645>`_
7. `David Mertz <https://discuss.python.org/t/steering-council-nomination-david-mertz/647>`_
8. `Åukasz Langa <https://discuss.python.org/t/steering-council-nomination-lukasz-langa/660>`_
9. `Benjamin Peterson <https://discuss.python.org/t/steering-council-nomination-benjamin-peterson/665>`_
10. `Mariatta <https://discuss.python.org/t/steering-council-nomination-mariatta/670>`_
11. `Carol Willing <https://discuss.python.org/t/steering-council-nomination-carol-willing/674>`_
12. `Emily Morehouse <https://discuss.python.org/t/steering-council-nomination-emily-morehouse/686>`_
13. `Peter Wang <https://discuss.python.org/t/steering-council-nomination-peter-wang/694>`_
14. `Donald Stufft <https://discuss.python.org/t/steering-council-nomination-donald-stufft/700>`_
15. `Travis Oliphant <https://discuss.python.org/t/steering-council-nomination-travis-oliphant/705>`_
16. `Kushal Das <https://discuss.python.org/t/steering-council-nomination-kushal-das/702>`_
17. `Gregory P. Smith <https://discuss.python.org/t/steering-council-nomination-gregory-p-smith/737>`_

Voter Roll
==========

All active Python core team members are eligible to vote.

Ballots will be distributed based on the `The Python Voter Roll for this
election
<https://github.com/python/voters/blob/master/2019-01-21-2019-python-steering-committee-election-election.csv>`__
[1]_.

While this file is not public as it contains private email addresses, the
`Complete Voter Roll`_ by name is available.

Election Implementation
=======================

The election will be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2019-python-steering-committee``

Name: ``2019 Python Steering Committee Election``

Description: ``Election for the Python steering council, as specified in PEP 13. This is the first steering council election.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``ee@python.org``

Voting starts at: ``January 21, 2019 12:00 UTC``

Voting ends at: ``February 4, 2019 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``5`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``



Results
=======

Of the 96 eligible voters, 69 cast ballots.

The top five vote-getters are:

* Barry Warsaw
* Brett Cannon
* Carol Willing
* Guido van Rossum
* Alyssa (Nick) Coghlan

No conflict of interest as defined in :pep:`13` were observed.

The full vote counts are as follows:

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Guido van Rossum      | 45             |
+-----------------------+----------------+
| Brett Cannon          | 44             |
+-----------------------+----------------+
| Carol Willing         | 33             |
+-----------------------+----------------+
| Barry Warsaw          | 31             |
+-----------------------+----------------+
| Alyssa (Nick) Coghlan | 25             |
+-----------------------+----------------+
| Benjamin Peterson     | 22             |
+-----------------------+----------------+
| Åukasz Langa          | 21             |
+-----------------------+----------------+
| Victor Stinner        | 21             |
+-----------------------+----------------+
| Mariatta              | 20             |
+-----------------------+----------------+
| Emily Morehouse       | 18             |
+-----------------------+----------------+
| Yury Selivanov        | 15             |
+-----------------------+----------------+
| Donald Stufft         | 11             |
+-----------------------+----------------+
| Peter Wang            | 10             |
+-----------------------+----------------+
| Travis Oliphant       | 8              |
+-----------------------+----------------+
| Kushal Das            | 7              |
+-----------------------+----------------+
| Gregory P. Smith      | 6              |
+-----------------------+----------------+
| David Mertz           | 3              |
+-----------------------+----------------+

Copyright
=========

This document has been placed in the public domain.


Complete Voter Roll
===================

Active Python core developers
-----------------------------

::

    Alex Gaynor
    Alex Martelli
    Alexander Belopolsky
    Alexandre Vassalotti
    Amaury Forgeot d'Arc
    Andrew Kuchling
    Andrew Svetlov
    Antoine Pitrou
    Armin Ronacher
    Barry Warsaw
    Benjamin Peterson
    Berker Peksag
    Brett Cannon
    Brian Curtin
    Carol Willing
    Chris Jerdonek
    Chris Withers
    Christian Heimes
    David Malcolm
    David Wolever
    Davin Potts
    Dino Viehland
    Donald Stufft
    Doug Hellmann
    Eli Bendersky
    Emily Morehouse
    Ãric Araujo
    Eric Snow
    Eric V. Smith
    Ethan Furman
    Ezio Melotti
    Facundo Batista
    Fred Drake
    Georg Brandl
    Giampaolo Rodola'
    Gregory P. Smith
    Guido van Rossum
    Hyeshik Chang
    Hynek Schlawack
    INADA Naoki
    Ivan Levkivskyi
    Jack Diederich
    Jack Jansen
    Jason R. Coombs
    Jeff Hardy
    Jeremy Hylton
    JesÃºs Cea
    Julien Palard
    Kurt B. Kaiser
    Kushal Das
    Larry Hastings
    Lars GustÃ¤bel
    Lisa Roach
    Åukasz Langa
    Marc-Andre Lemburg
    Mariatta
    Mark Dickinson
    Mark Hammond
    Mark Shannon
    Martin Panter
    Matthias Klose
    Meador Inge
    Michael Hudson-Doyle
    Nathaniel J. Smith
    Ned Deily
    Neil Schemenauer
    Alyssa Coghlan
    Pablo Galindo
    Paul Moor


id: pep-0548.rst
content:
PEP: 548
Title: More Flexible Loop Control
Author: R David Murray
Status: Rejected
Type: Standards Track
Created: 05-Sep-2017
Python-Version: 3.7
Post-History: 05-Aug-2017


Rejection Note
==============

Rejection by Guido:
https://mail.python.org/pipermail/python-dev/2017-September/149232.html


Abstract
========

This PEP proposes enhancing the ``break`` and ``continue`` statements
with an optional boolean expression that controls whether or not
they execute.  This allows the flow of control in loops to be
expressed more clearly and compactly.


Motivation
==========

Quoting from the rejected :pep:`315`:

    It is often necessary for some code to be executed before each
    evaluation of the while loop condition.  This code is often
    duplicated outside the loop, as setup code that executes once
    before entering the loop::

        <setup code>
        while <condition>:
            <loop body>
            <setup code>

That PEP was rejected because no syntax was found that was superior
to the following form::

    while True:
        <setup code>
        if not <condition>:
            break
        <loop body>

This PEP proposes a superior form, one that also has application to
for loops. It is superior because it makes the flow of control in
loops more explicit, while preserving Python's indentation aesthetic.


Syntax
======

The syntax of the break and continue statements are extended
as follows::

    break_stmt : "break" ["if" expression]
    continue_stmt : "continue" ["if" expression]

In addition, the syntax of the while statement is modified as follows::

    while_stmt : while1_stmt|while2_stmt
    while1_stmt : "while" expression ":" suite
                  ["else" ":" suite]
    while2_stmt : "while" ":" suite


Semantics
=========

A ``break if`` or ``continue if`` is executed if and only if
``expression`` evaluates to true.

A ``while`` statement with no expression loops until a break or return
is executed (or an error is raised), as if it were a ``while True``
statement.  Given that the loop can never terminate except in a
way that would not cause an ``else`` suite to execute, no ``else``
suite is allowed in the expressionless form.  If practical, it
should also be an error if the body of an expressionless ``while``
does not contain at least one ``break`` or ``return`` statement.


Justification and Examples
==========================

The previous "best possible" form::

    while True:
        <setup code>
        if not <condition>:
            break
        <loop body>

could be formatted as::

    while True:
        <setup code>
        if not <condition>: break
        <loop body>

This is superficially almost identical to the form proposed by this
PEP::

    while:
        <setup code>
        break if not <condition>
        <loop body>

The significant difference here is that the loop flow control
keyword appears *first* in the line of code.  This makes it easier
to comprehend the flow of control in the loop at a glance, especially
when reading colorized code.

For example, this is a common code pattern, taken in this case
from the tarfile module::

    while True:
        buf = self._read(self.bufsize)
        if not buf:
            break
        t.append(buf)

Reading this, we either see the break and possibly need to think about
where the while is that it applies to, since the break is indented
under the if, and then track backward to read the condition that
triggers it; or, we read the condition and only afterward discover
that this condition changes the flow of the loop.

With the new syntax this becomes::

    while:
        buf = self._read(self.bufsize)
        break if not buf
        t.append(buf)

Reading this we first see the ``break``, which obviously applies to
the while since it is at the same level of indentation as the loop
body, and then we read the condition that causes the flow of control
to change.

Further, consider a more complex example from sre_parse::

    while True:
        c = self.next
        self.__next()
        if c is None:
            if not result:
                raise self.error("missing group name")
            raise self.error("missing %s, unterminated name" % terminator,
                             len(result))
        if c == terminator:
            if not result:
                raise self.error("missing group name", 1)
            break
        result += c
    return result

This is the natural way to write this code given current Python
loop control syntax.  However, given ``break if``, it would be more
natural to write this as follows::

    while:
        c = self.next
        self.__next()
        break if c is None or c == terminator
        result += c
    if not result:
        raise self.error("missing group name")
    elif c is None:
        raise self.error("missing %s, unterminated name" % terminator,
                         len(result))
    return result

This form moves the error handling out of the loop body, leaving the
loop logic much more understandable.  While it would certainly be
possible to write the code this way using the current syntax, the
proposed syntax makes it more natural to write it in the clearer form.

The proposed syntax also provides a natural, Pythonic spelling of
the classic  ``repeat ... until <expression>`` construct found in
other languages, and for which no good syntax has previously been
found for Python::

    while:
        ...
        break if <expression>

The tarfile module, for example, has a couple of "read until" loops like
the following::

    while True:
        s = self.__read(1)
        if not s or s == NUL:
            break

With the new syntax this would read more clearly::

    while:
        s = self.__read(1)
        break if not s or s == NUL

The case for extending this syntax to ``continue`` is less strong,
but buttressed by the value of consistency.

It is much more common for a ``continue`` statement to be at the
end of a multiline if suite, such as this example from zipfile ::

    while True:
        try:
            self.fp = io.open(file, filemode)
        except OSError:
            if filemode in modeDict:
                filemode = modeDict[filemode]
                continue
            raise
        break

The only opportunity for improvement the new syntax would offer for
this loop would be the omission of the ``True`` token.

On the other hand, consider this example from uuid.py::

    for i in range(adapters.length):
        ncb.Reset()
        ncb.Command = netbios.NCBRESET
        ncb.Lana_num = ord(adapters.lana[i])
        if win32wnet.Netbios(ncb) != 0:
            continue
        ncb.Reset()
        ncb.Command = netbios.NCBASTAT
        ncb.Lana_num = ord(adapters.lana[i])
        ncb.Callname = '*'.ljust(16)
        ncb.Buffer = status = netbios.ADAPTER_STATUS()
        if win32wnet.Netbios(ncb) != 0:
            continue
        status._unpack()
        bytes = status.adapter_address[:6]
        if len(bytes) != 6:
            continue
        return int.from_bytes(bytes, 'big')

This becomes::

    for i in range(adapters.length):
        ncb.Reset()
        ncb.Command = netbios.NCBRESET
        ncb.Lana_num = ord(adapters.lana[i])
        continue if win32wnet.Netbios(ncb) != 0
        ncb.Reset()
        ncb.Command = netbios.NCBASTAT
        ncb.Lana_num = ord(adapters.lana[i])
        ncb.Callname = '*'.ljust(16)
        ncb.Buffer = status = netbios.ADAPTER_STATUS()
        continue if win32wnet.Netbios(ncb) != 0
        status._unpack()
        bytes = status.adapter_address[:6]
        continue if len(bytes) != 6
        return int.from_bytes(bytes, 'big')

This example indicates that there are non-trivial use cases where
``continue if`` also improves the readability of the loop code.

It is probably significant to note that all of the examples selected
for this PEP were found by grepping the standard library for ``while
True`` and ``continue``, and the relevant examples 


id: pep-0762.rst
content:
PEP: 762
Title: REPL-acing the default REPL
Author: Pablo Galindo Salgado <pablogsal@python.org>, Åukasz Langa <lukasz@python.org>, Lysandros Nikolaou <lisandrosnik@gmail.com>, Emily Morehouse-Valcarcel <emily@python.org>
Sponsor: Pablo Galindo Salgado
Status: Final
Type: Informational
Created: 11-Oct-2024
Python-Version: 3.13

Abstract
========

One of Pythonâs core strengths is its interactive mode, also known as the
Read-Eval-Print Loop (REPL), or the Python console, or the Python shell. This
PEP describes the new implementation of this functionality written in Python.
The new REPL released in Python 3.13 aims to provide modern features expected by
today's users, such as multi-line editing, syntax highlighting, custom commands,
and an overall improved interactive experience.

Motivation
==========

Up to Python 3.12, the interactive shell of CPython was written in C as a
special mode of the parser. It was therefore difficult to maintain and extend.
It relied on the existence of GNU readline (or an equivalent) for basic
functionality like cursor movement and history tracking. Python compiled without
this library provided an interactive mode of very limited capability. On the
other hand, Python compiled with readline outsourced decisions and configuration
around user input in ways that made extending it difficult.

This complexity has deterred contributions and has made it challenging to
implement new features. As a result, the CPython interactive shell has seen
minimal changes, falling behind user expectations for modern equivalents.

Many features that users have come to expect from modern REPLs were absent in
the previous version. Some examples of these features include multi-line editing
and history, custom commands, syntax highlighting, or ergonomic handling of copy
and paste. The lack of these features greatly impacts the user experience of
many groups of users of CPython, in particular in environments where users donât
control dependencies and cannot install their own packages. This is especially
common for users learning the language and educators.

Addressing such issues with the C implementation would require complex
workarounds, such as AST matching of commands, which would add prohibitive
complexity to the codebase.

With the new REPL written in Python, we are addressing these limitations while
also bringing CPython's interactive experience more in line with modern
expectations and capabilities.

Rationale
=========

Implementing the new REPL in Python, rather than C, has significantly lowered
the barrier to entry for contributors. This change has made it easier to test,
validate, and modify the REPL, leading to increased community involvement and
faster feature development. The improved accessibility of the codebase is
expected to result in a more rapidly evolving and user-responsive REPL.

Instead of writing a Python REPL from scratch, we decided to base the
implementation of the new REPL on `PyREPL <https://github.com/pypy/pypy/tree/d102094b863ce49b7af030dcb0cecaac515d97c6/lib_pypy/pyrepl>`_.
This decision was driven by several key factors. First and foremost,
developing a terminal application that works consistently across different
operating systems and terminal emulators is a complex undertaking.
By adopting PyREPL, which has been battle-tested in the PyPy project,
we can leverage existing, proven code rather than starting from scratch.

Sharing a codebase with PyPy for the REPL implementation offers mutual benefits
to both projects. It allows for shared maintenance efforts, faster bug fixes,
and feature improvements that can benefit users of both CPython and PyPy. This
collaboration can lead to a more robust and feature-rich REPL for the entire
Python ecosystem.

The previous REPL written in C leveraged the âreadlineâ or âeditlineâ libraries
as a backend to allow certain features such as navigation, history preservation
and recall, autocompletion, and configurable keyboard behavior. PyREPL does not
use those libraries, implementing most of the other functionality directly as
part of the shell. The main missing functionality (configurability of input) is
outweighed by the benefits of the new architecture. The configuration files for
these libraries (e.g. inputrc) are complex and include features that PyREPL
doesnât plan to implement, making it infeasible to transparently add support for
them in the new shell. Using âreadlineâ or âeditlineâ in PyREPL would be
prohibitively complex due to multi-line editing handling and multiplatform
support.

Although this means that existing readline/editline configurations will not be
compatible with the PyREPL, we believe the enhanced features and improved
extensibility are an overall win. See âBackwards Compatibilityâ for discussion
of continued support for customized workflows.

The previous REPL made it challenging to properly implement custom commands,
which is a very common feature of interactive shells. For instance, the ``exit``
command was implemented as a method call of a custom object injected in the
global namespace, leading to unintuitive behavior that often confuses users when
they simply type ``exit``, as the interpreter prompts them to the supposedly
correct usage ``exit()``.

Specification
=============

PyREPL is implemented as a new private Python module called :mod:`!_pyrepl`, existing
alongside the current C implementation. In its first implementation, it
introduces the following key features:

1. Multi-line History and Editing: Users are able to navigate and edit their
   command history across multiple lines, improving the ability to refine and reuse
   complex blocks of code.

   Editing multi-line blocks provides automatic indentation using four spaces, which
   is consistent with :pep:`8` recommendations. When a line ending with a colon is
   encountered, the following line is automatically indented utilizing the
   indentation pattern that is inferred from the first line that contains
   indentation. Lines are indented with four spaces, and tabs are converted into
   spaces.

   Users can access history of commands by using :kbd:`up` and :kbd:`down` arrows. Within
   a multi-line entry, the arrow keys navigate line-by-line within the block before
   moving to the next history entry. The :kbd:`down` arrow works in reverse, navigating
   from older entries to the most recent.

   History can be searched forward (using :kbd:`Ctrl+S`) and in reverse (using :kbd:`Ctrl+R`)
   using a custom-specified substring query. It can also be searched with a prefix
   query by entering the prefix into a shell line and using PageUp and PageDown
   keys.

2. Copying and Pasting: in supported terminal emulators, bracketed pasting
   capability is discovered and used by PyREPL. This allows for transparent pasting
   of blocks of code without immediate execution or invalid automatic indentation.

   For terminal emulators that donât support this mode, a dedicated paste mode is
   implemented to allow for easy insertion of multi-line code snippets without
   triggering immediate execution or indentation issues.

   Users enter manual paste mode by hitting the :kbd:`F3` key. The prompt changes from
   ``>>>`` to ``(paste)`` where users can paste contents from their clipboard or
   manually type as desired. Once the content is ready, hitting :kbd:`F3` exits paste
   mode. Then, pressing Enter executes the block.

   Users can enter multiple commands on a single input when using paste mode, which
   will help paste code from other sources.

   To copy blocks of code without the leading command prompts and without the
   output of the commands, users can enter the history view via the :kbd:`F2` key. This
   mode uses a pager to display history of executed commands without the prompts
   and output.

3. Help via :kbd:`F1`.

   Access to the standard Help module is accessible via a Custom Command ``help``
   (see below) or via the :kbd:`F1` key. Hit :kbd:`F1` to enter 


id: pep-0004.rst
content:
PEP: 4
Title: Deprecation of Standard Modules
Author: Brett Cannon <brett@python.org>, Martin von LÃ¶wis <martin@v.loewis.de>
Status: Active
Type: Process
Created: 01-Oct-2000
Post-History:


Introduction
============

When new modules were added to the standard Python library in the
past, it was not possible to foresee whether they would still be
useful in the future.  Even though Python "Comes With Batteries
Included", batteries may discharge over time.  Carrying old modules
around is a burden on the maintainer, especially when there is no
interest in the module anymore.

At the same time, removing a module from the distribution is
difficult, as it is not known in general whether anybody is still
using it.  This PEP defines a procedure for removing modules from the
standard Python library.  Usage of a module may be 'deprecated', which
means that it may be removed from a future Python release.


Procedure for declaring a module deprecated
===========================================

To remove a top-level module/package from the standard library, a PEP
is required. The deprecation process is outlined in :pep:`387`.

For removing a submodule of a package in the standard library,
:pep:`387` must be followed, but a PEP is not required.


Copyright
=========

This document has been placed in the public domain.


id: pep-0010.rst
content:
PEP: 10
Title: Voting Guidelines
Author: Barry Warsaw <barry@python.org>
Status: Active
Type: Process
Created: 07-Mar-2002
Post-History: 07-Mar-2002



Abstract
========

This PEP outlines the python-dev voting guidelines.  These guidelines
serve to provide feedback or gauge the "wind direction" on a
particular proposal, idea, or feature.  They don't have a binding
force.


Rationale
=========

When a new idea, feature, patch, etc. is floated in the Python
community, either through a PEP or on the mailing lists (most likely
on python-dev [1]_), it is sometimes helpful to gauge the community's
general sentiment.  Sometimes people just want to register their
opinion of an idea.  Sometimes the BDFL wants to take a straw poll.
Whatever the reason, these guidelines have been adopted so as to
provide a common language for developers.

While opinions are (sometimes) useful, but they are never binding.
Opinions that are accompanied by rationales are always valued higher
than bare scores (this is especially true with -1 votes).


Voting Scores
=============

The scoring guidelines are loosely derived from the Apache voting
procedure [2]_, with of course our own spin on things.  There are 4
possible vote scores:

- ``+1`` I like it

- ``+0`` I don't care, but go ahead

- ``-0`` I don't care, so why bother?

- ``-1`` I hate it

You may occasionally see wild flashes of enthusiasm (either for or
against) with vote scores like +2, +1000, or -1000.  These aren't
really valued much beyond the above scores, but it's nice to see
people get excited about such geeky stuff.


References
==========

.. [1] Python Developer's Guide,
   (http://www.python.org/dev/)

.. [2] Apache Project Guidelines and Voting Rules
   (http://httpd.apache.org/dev/guidelines.html)


Copyright
=========

This document has been placed in the public domain.


