# embeddings

id: pep-0776.rst
content:
PEP: 776
Title: Emscripten Support
Author: Hood Chatham <roberthoodchatham at gmail.com>
Sponsor: Åukasz Langa <lukasz at python.org>
Discussions-To: https://discuss.python.org/t/86276
Status: Draft
Type: Informational
Created: 18-Mar-2025
Python-Version: 3.14
Post-History: `18-Mar-2025 <https://discuss.python.org/t/84996>`__,
              `28-Mar-2025 <https://discuss.python.org/t/86276>`__,

Abstract
========

`Emscripten <https://emscripten.org/>`__ is a complete open source compiler
toolchain. It compiles C/C++ code into WebAssembly/JavaScript executables, for
use in JavaScript runtimes, including browsers and Node.js. The Rust language
also maintains an Emscripten target.

This PEP formalizes the addition of Tier 3 for Emscripten support in Python 3.14
which `was approved by the Steering Council on October 25, 2024
<https://github.com/python/steering-council/issues/256>`__. The goals are:

1. To describe the current state of the CPython Emscripten runtime
2. To describe the current state of the Pyodide runtime
3. To identify minor features to be upstreamed from the Pyodide runtime into the
   CPython Emscripten runtime

The minor features identified here are all features that could be implemented
without a PEP. We discuss more significant runtime features that we would like
to implement but we defer decisions on those features to subsequent PEPs.


Motivation
==========

A web browser is a universal computing platform, available on Windows, macOS,
Linux, and every smartphone.

`The Pyodide project <https://pyodide.org/>`__ has supported Emscripten Python
since 2018. Hundreds of thousands of students have learned Python through
Pyodide via projects like `Capytale
<https://web.archive.org/web/20241211090946/https://cfp.jupytercon.com/2023/talk/TJ9YEV/>`__
and `PyodideU <https://stanford.edu/~cpiech/bio/papers/pyodideU.pdf>`__. Pyodide
is also increasingly being used by Python packages to provide interactive
documentation. This demonstrates both the importance and the maturity of the
Emscripten platform.

Emscripten and WASI are also the only supported platforms that offer any
meaningful sandboxing.


Emscripten Platform Information
===============================

"Pyodide" vs "Emscripten Python"
--------------------------------

For the sake of this document, we use the term "Emscripten Python" to refer to
the Emscripten Python maintained in the ``python/cpython`` repository, without
any downstream additions. We contrast the features present in Emscripten Python
to the features present in Pyodide.

Pyodide is maintained `on GitHub <https://github.com/pyodide/pyodide>`__ and
distributed via `jsDelivr <https://www.jsdelivr.com/oss-cdn/pyodide>`__, `npm
<https://www.npmjs.com/package/pyodide>`__, and `GitHub releases
<https://github.com/pyodide/pyodide/releases>`__.

Emscripten Python is not distributed, but it is possible to build `by following
the instructions in the devguide
<https://devguide.python.org/getting-started/setup-building/#emscripten>`__

Background on Emscripten
------------------------

`Emscripten
<https://emscripten.org/docs/introducing_emscripten/about_emscripten.html>`__
consists of a C and C++ compiler and linker based on LLVM__, together with a
runtime based on a mildly patched musl libc.

__ https://llvm.org/

Emscripten is a POSIX-based platform. It uses the `WebAssembly binary format`_,
and the `WebAssembly dynamic linking section`_.

.. _WebAssembly binary format: https://webassembly.github.io/spec/core/binary/index.html
.. _WebAssembly dynamic linking section: https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md

The ``emcc`` compiler is a wrapper around ``clang``. The ``emcc`` linker is a
wrapper around ``wasm-ld`` (`also part of the LLVM toolchain
<https://lld.llvm.org/WebAssembly.html>`__).

Emscripten support for portable C/C++ code source compatibility with Linux is
fairly comprehensive, with certain expected exceptions to be spelled out.
CPython already supports compilation to Emscripten, and it only requires a very
modest number of modifications to the normal Linux target.

POSIX Compliance
----------------

Emscripten is a POSIX platform. However, there are POSIX APIs that exist but
always fail when called and POSIX APIs that don't exist at all. In particular,
there are problems with networking APIs and blocking I/O, and there is no
support for ``fork()``. See `Emscripten Portability Guidelines
<https://emscripten.org/docs/porting/guidelines/portability_guidelines.html>`__.

Emscripten executables can be linked with threading support, but it comes
with several limitations:

* Enabling threading requires websites to be served with special security headers
  that indicate acceptance of the possibility of Spectre_-style information
  leakage. These headers are a usability hazard for users who are not intimately
  familiar with the web platform.

  .. _Spectre: https://en.wikipedia.org/wiki/Spectre_(security_vulnerability)

* If an executable is linked with both threading and a dynamic loader, Emscripten
  prints a warning that using dynamic loading and pthreads together is
  experimental. It may cause performance problems or crashes. These problems may
  require WebAssembly standards work to resolve.

Because of these limitations, Pyodide standardizes a no-pthreads build of
Python. If there is sufficient demand, a pthreads build with no dynamic loader
could be added later.


Development Tools
-----------------

Emscripten development tools are equally well supported on Linux, Windows, and
macOS. The upstream tools include:

* The Emscripten Software Developer Kit (:program:`emsdk`) which can be used to
  install the Emscripten compiler toolchain (:program:`emcc`).
* :program:`emcc` is a C and C++ compiler, linker, and a sysroot with headers
  for the system libraries. The system libraries themselves are generated on
  the fly based on the ABI requested.
* Node.js can be used as an "emulator" to run Emscripten programs from the
  command line. This emulation behaves best on Linux with macOS as a runner up.
  Node.js is the most convenient way to test Emscripten programs.
* It is possible to run Emscripten programs inside of any web browser. Browser
  automation tools like Selenium, Playwright, or Puppeteer can be used to test
  features that are browser-only.

Pyodide's tools:

* ``pyodide build`` can be used to cross compile Python packages to run on
  Emscripten. Cross compilation works best on Linux, there is experimental
  support on macOS, and it is entirely unsupported on Windows.
* ``pyodide venv`` can make a virtual environment that runs in Pyodide.
* ``pytest-pyodide`` can test Python code against various JavaScript runtimes.

cibuildwheel__ supports building wheels to target Emscripten using ``pyodide build``.

__ https://cibuildwheel.pypa.io/

In the short term, Pyodide's packaging tooling will stay in the Pyodide
repository. It is an open question where Pyodide's packaging tooling should live
in the long term. Two sensible options would be for it to remain under the
``pyodide`` organization or be moved into the ``pypa`` organization on GitHub.


Emscripten Application Lifecycle
--------------------------------

An Emscripten "binary" consists of a pair of files, an ``.mjs`` file and a
``.wasm`` file. The ``.wasm`` file contains all of the compiled C/C++/Rust code.
The ``.mjs`` file contains the lifecycle code to set up the runtime, locate the
``.wasm`` file, compile it, instantiate it, call the ``main()`` function, and to
shut down the runtime on exit. It also includes an implementation for all of the
system calls, including the file system, the dynamic loader, and any logic to
expose additional functionality from the JavaScript runtime to C code.

The ``.mjs`` file exports a single ``bootstrapEmscriptenExecutable()``
JavaScript function that bootstraps the runtime, calls the ``main()`` function,
and returns an API object that can be used to call C functions. Each time it is
call


id: pep-0789.rst
content:
PEP: 789
Title: Preventing task-cancellation bugs by limiting yield in async generators
Author: Zac Hatfield-Dodds <zac@zhd.dev>,
        Nathaniel J. Smith <njs@pobox.com>
PEP-Delegate:
Discussions-To: https://discuss.python.org/t/preventing-yield-inside-certain-context-managers/1091
Status: Draft
Type: Standards Track
Created: 14-May-2024
Python-Version: 3.14


Abstract
========

`Structured concurrency`_ is increasingly popular in Python.  Interfaces such as
the ``asyncio.TaskGroup`` and ``asyncio.timeout`` context managers support
compositional reasoning, and allow developers to clearly scope the lifetimes of
concurrent tasks. However, using ``yield`` to suspend a frame inside such a
context leads to situations where the wrong task is canceled, timeouts are
ignored, and exceptions are mishandled. More fundamentally, suspending a frame
inside a ``TaskGroup`` violates the structured concurrency design principle that
child tasks are encapsulated within their parent frame.

To address these issues, this PEP proposes a new ``sys.prevent_yields()`` context
manager. When syntactically inside this context, attempting to ``yield`` will
raise a RuntimeError, preventing the task from yielding. Additionally, a
mechanism will be provided for decorators such as ``@contextmanager`` to allow
yields inside the decorated function.  ``sys.prevent_yields()`` will be used by
asyncio and downstream libraries to implement task groups, timeouts, and
cancellation; and a related mechanism by ``contextlib`` etc. to convert
generators into context managers which allow safe yields.


Background
==========

Structured concurrency is increasingly popular in Python, in the form of newer
:py:mod:`asyncio` interfaces and third-party libraries such as Trio and anyio.
These interfaces support compositional reasoning, *so long as* users never write
a ``yield`` which suspends a frame while inside a cancel scope.

A cancel scope is a context manager which can... cancel... whatever work occurs
within that context (...scope).  In asyncio, this is implicit in the design of
``with asyncio.timeout():`` or ``async with asyncio.TaskGroup() as tg:``, which
respectively cancel the contained work after the specified duration, or cancel
sibling tasks when one of them raises an exception.  The core functionality of
a cancel scope is synchronous, but the user-facing context managers may be
either sync or async. [#trio-cancel-scope]_ [#tg-cs]_

This structured approach works beautifully, unless you hit one specific sharp
edge: breaking the nesting structure by ``yield``\ ing inside a cancel scope.
This has much the same effect on structured control flow as adding just a few
cross-function ``goto``\ s, and the effects are truly dire:

- The wrong task can be canceled, whether due to a timeout, an error in a
  sibling task, or an explicit request to cancel some other task
- Exceptions, including ``CancelledError``, can be delivered to the wrong task
- Exceptions can go missing entirely, being dropped instead of added to an
  ``ExceptionGroup``


Problem statement
=================

Here's the fundamental issue: yield suspends a call frame. It only makes sense
to yield in a leaf frame -- i.e., if your call stack goes like A -> B -> C, then
you can suspend C, but you can't suspend B while leaving C running.

But, TaskGroup is a kind of "concurrent call" primitive, where a single frame
can have multiple child frames that run concurrently. This means that if we
allow people to mix yield and TaskGroup, then we can end up in exactly this
situation, where B gets suspended but C is actively running. This is
nonsensical, and causes serious practical problems (e.g., if C raises an
exception and A has returned, we have no way to propagate it).

This is a fundamental incompatibility between generator control flow and
structured concurrency control flow, not something we can fix by tweaking our
APIs. The only solution seems to be to forbid yield inside a TaskGroup.

Although timeouts don't leave a child task running, the close analogy and
related problems lead us to conclude that yield should be forbidden inside all
cancel scopes, not only TaskGroups.  See :ref:`just-deliver` for discussion.


Motivating examples
===================

Let's consider three examples, to see what this might look like in practice.


Leaking a timeout to the outer scope
------------------------------------

Suppose that we want to iterate over an async iterator, but wait for at most
``max_time`` seconds for each element.  We might naturally encapsulate the logic
for doing so in an async generator, so that the call site can continue to use a
straightforward ``async for`` loop:

.. code-block:: python

    async def iter_with_timeout(ait, max_time):
        try:
            while True:
                with timeout(max_time):
                    yield await anext(ait)
        except StopAsyncIteration:
            return

    async def fn():
        async for elem in iter_with_timeout(ait, max_time=1.0):
            await do_something_with(elem)

Unfortunately, there's a bug in this version: the timeout might expire after the
generator yields but before it is resumed!  In this case, we'll see a
``CancelledError`` raised in the outer task, where it cannot be caught by the
``with timeout(max_time):`` statement.

The fix is fairly simple: get the next element inside the timeout context, and
then yield *outside* that context.

.. code-block:: python

    async def correct_iter_with_timeout(ait, max_time):
        try:
            while True:
                with timeout(max_time):
                    tmp = await anext(ait)
                yield tmp
        except StopAsyncIteration:
            return

Leaking background tasks (breaks cancellation and exception handling)
---------------------------------------------------------------------

Timeouts are not the only interface which wrap a cancel scope - and if you
need some background worker tasks, you can't simply close the ``TaskGroup``
before yielding.

As an example, let's look at a fan-in generator, which we'll use to merge the
feeds from several "sensors".  We'll also set up our mock sensors with a small
buffer, so that we'll raise an error in the background task while control flow
is outside the ``combined_iterators`` generator.

.. code-block:: python

    import asyncio, itertools

    async def mock_sensor(name):
        for n in itertools.count():
            await asyncio.sleep(0.1)
            if n == 1 and name == "b":  # 'presence detection'
                yield "PRESENT"
            elif n == 3 and name == "a":  # inject a simple bug
                print("oops, raising RuntimeError")
                raise RuntimeError
            else:
                yield f"{name}-{n}"  # non-presence sensor data

    async def move_elements_to_queue(ait, queue):
        async for obj in ait:
            await queue.put(obj)

    async def combined_iterators(*aits):
        """Combine async iterators by starting N tasks, each of
        which move elements from one iterable to a shared queue."""
        q = asyncio.Queue(maxsize=2)
        async with asyncio.TaskGroup() as tg:
            for ait in aits:
                tg.create_task(move_elements_to_queue(ait, q))
            while True:
                yield await q.get()

    async def turn_on_lights_when_someone_gets_home():
        combined = combined_iterators(mock_sensor("a"), mock_sensor("b"))
        async for event in combined:
            print(event)
            if event == "PRESENT":
                break
        print("main task sleeping for a bit")
        await asyncio.sleep(1)  # do some other operation

    asyncio.run(turn_on_lights_when_someone_gets_home())

When we run this code, we see the expected sequence of observations, then a
'detection', and then while the main task is sleeping we trigger that
``RuntimeError`` in the background.  But... we don't actually observe the
``RuntimeError``, not even as the ``__cont


id: pep-0616.rst
content:
PEP: 616
Title: String methods to remove prefixes and suffixes
Author: Dennis Sweeney <sweeney.dennis650@gmail.com>
Sponsor: Eric V. Smith <eric@trueblade.com>
Status: Final
Type: Standards Track
Created: 19-Mar-2020
Python-Version: 3.9
Post-History: 20-Mar-2020


Abstract
========

This is a proposal to add two new methods, ``removeprefix()`` and
``removesuffix()``, to the APIs of Python's various string objects.  These
methods would remove a prefix or suffix (respectively) from a string,
if present, and would be added to Unicode ``str`` objects, binary
``bytes`` and ``bytearray`` objects, and ``collections.UserString``.


Rationale
=========

There have been repeated issues on Python-Ideas [#pyid]_ [3]_,
Python-Dev [4]_ [5]_ [6]_ [7]_, the Bug Tracker, and
StackOverflow [#confusion]_, related to user confusion about the
existing ``str.lstrip`` and ``str.rstrip`` methods.  These users are
typically expecting the behavior of ``removeprefix`` and ``removesuffix``,
but they are surprised that the parameter for ``lstrip`` is
interpreted as a set of characters, not a substring.  This repeated
issue is evidence that these methods are useful.  The new methods
allow a cleaner redirection of users to the desired behavior.

As another testimonial for the usefulness of these methods, several
users on Python-Ideas [#pyid]_ reported frequently including similar
functions in their code for productivity.  The implementation
often contained subtle mistakes regarding the handling of the empty
string, so a well-tested built-in method would be useful.

The existing solutions for creating the desired behavior are to either
implement the methods as in the `Specification`_ below, or to use
regular expressions as in the expression
``re.sub('^' + re.escape(prefix), '', s)``, which is less discoverable,
requires a module import, and results in less readable code.


Specification
=============

The builtin ``str`` class will gain two new methods which will behave
as follows when ``type(self) is type(prefix) is type(suffix) is str``::

    def removeprefix(self: str, prefix: str, /) -> str:
        if self.startswith(prefix):
            return self[len(prefix):]
        else:
            return self[:]

    def removesuffix(self: str, suffix: str, /) -> str:
        # suffix='' should not call self[:-0].
        if suffix and self.endswith(suffix):
            return self[:-len(suffix)]
        else:
            return self[:]

When the arguments are instances of ``str`` subclasses, the methods should
behave as though those arguments were first coerced to base ``str``
objects, and the return value should always be a base ``str``.

Methods with the corresponding semantics will be added to the builtin
``bytes`` and ``bytearray`` objects.  If ``b`` is either a ``bytes``
or ``bytearray`` object, then ``b.removeprefix()`` and ``b.removesuffix()``
will accept any bytes-like object as an argument. The two methods will
also be added to ``collections.UserString``, with similar behavior.


Motivating examples from the Python standard library
====================================================

The examples below demonstrate how the proposed methods can make code
one or more of the following:

1. Less fragile:

   The code will not depend on the user to count the length of a literal.

2. More performant:

   The code does not require a call to the Python built-in ``len``
   function nor to the more expensive ``str.replace()`` method.

3. More descriptive:

   The methods give a higher-level API for code readability as
   opposed to the traditional method of string slicing.


find_recursionlimit.py
----------------------

- Current::

    if test_func_name.startswith("test_"):
        print(test_func_name[5:])
    else:
        print(test_func_name)

- Improved::

    print(test_func_name.removeprefix("test_"))


deccheck.py
-----------

This is an interesting case because the author chose to use the
``str.replace`` method in a situation where only a prefix was
intended to be removed.

- Current::

    if funcname.startswith("context."):
        self.funcname = funcname.replace("context.", "")
        self.contextfunc = True
    else:
        self.funcname = funcname
        self.contextfunc = False

- Improved::

    if funcname.startswith("context."):
        self.funcname = funcname.removeprefix("context.")
        self.contextfunc = True
    else:
        self.funcname = funcname
        self.contextfunc = False

- Arguably further improved::

    self.contextfunc = funcname.startswith("context.")
    self.funcname = funcname.removeprefix("context.")


cookiejar.py
------------

- Current::

    def strip_quotes(text):
        if text.startswith('"'):
            text = text[1:]
        if text.endswith('"'):
            text = text[:-1]
        return text

- Improved::

    def strip_quotes(text):
        return text.removeprefix('"').removesuffix('"')


test_i18n.py
------------

- Current::

    creationDate = header['POT-Creation-Date']

    # peel off the escaped newline at the end of string
    if creationDate.endswith('\\n'):
        creationDate = creationDate[:-len('\\n')]

- Improved::

    creationDate = header['POT-Creation-Date'].removesuffix('\\n')


There were many other such examples in the stdlib.


Rejected Ideas
==============

Expand the lstrip and rstrip APIs
---------------------------------

Because ``lstrip`` takes a string as its argument, it could be viewed
as taking an iterable of length-1 strings.  The API could, therefore, be
generalized to accept any iterable of strings, which would be
successively removed as prefixes.  While this behavior would be
consistent, it would not be obvious for users to have to call
``'foobar'.lstrip(('foo',))`` for the common use case of a
single prefix.


Remove multiple copies of a prefix
----------------------------------

This is the behavior that would be consistent with the aforementioned
expansion of the ``lstrip``/``rstrip`` API -- repeatedly applying the
function until the argument is unchanged.  This behavior is attainable
from the proposed behavior via by the following::

    >>> s = 'Foo' * 100 + 'Bar'
    >>> prefix = 'Foo'
    >>> while s.startswith(prefix): s = s.removeprefix(prefix)
    >>> s
    'Bar'


Raising an exception when not found
-----------------------------------

There was a suggestion that ``s.removeprefix(pre)`` should raise an
exception if ``not s.startswith(pre)``.  However, this does not match
with the behavior and feel of other string methods.  There could be
``required=False`` keyword added, but this violates the KISS
principle.


Accepting a tuple of affixes
----------------------------

It could be convenient to write the ``test_concurrent_futures.py``
example above as ``name.removesuffix(('Mixin', 'Tests', 'Test'))``, so
there was a suggestion that the new methods be able to take a tuple of
strings as an argument, similar to the ``startswith()`` API.  Within
the tuple, only the first matching affix would be removed.  This was
rejected on the following grounds:

* This behavior can be surprising or visually confusing, especially
  when one prefix is empty or is a substring of another prefix, as in
  ``'FooBar'.removeprefix(('', 'Foo')) == 'FooBar'``
  or ``'FooBar text'.removeprefix(('Foo', 'FooBar ')) == 'Bar text'``.

* The API for ``str.replace()`` only accepts a single pair of
  replacement strings, but has stood the test of time by refusing the
  temptation to guess in the face of ambiguous multiple replacements.

* There may be a compelling use case for such a feature in the future,
  but generalization before the basic feature sees real-world use would
  be easy to get permanently wrong.


Alternative Method Names
------------------------

Several alternatives method names have been proposed.  Some are listed
below, along with commentary for why they should be rejected in favor
of ``removeprefix`` (the same arguments hold for ``removesuffix``).

- ``ltrim``, ``trimprefix``, etc


id: pep-0602.rst
content:
PEP: 602
Title: Annual Release Cycle for Python
Author: Åukasz Langa <lukasz@python.org>
PEP-Delegate: Brett Cannon <brett@python.org>
Discussions-To: https://discuss.python.org/t/pep-602-annual-release-cycle-for-python/2296/
Status: Active
Type: Process
Created: 04-Jun-2019
Python-Version: 3.9
Post-History: `09-Oct-2023 <https://discuss.python.org/t/27002>`__
Resolution: https://mail.python.org/archives/list/python-dev@python.org/thread/KE7OS4PZASZMFTW2FP2MWZU5R4Q2QZKU/


Abstract
========

This document describes a change in the release calendar for Python
starting with Python 3.9.  This change accelerates the release cadence
such that feature versions are released predictably every twelve months,
in October every year.


Implementation
==============

Seventeen months to develop a feature version
---------------------------------------------

This PEP proposes that Python 3.X.0 will be developed for around
17 months:

- The first *five months* overlap with Python 3.(X-1).0's beta
  and release candidate stages and are thus unversioned.

- The next *seven months* are spent on versioned alpha releases where
  both new features are incrementally added and bug fixes are included.

- The following *three months* are spent on four versioned beta releases
  where **no new features** can be added but bug fixes are still
  included.

- The final *two months* are spent on two release candidates (or more,
  if necessary) and conclude with the release of the final release of
  Python 3.X.0.

2 years of full support, 3 more years of security fixes
-------------------------------------------------------

After the release of Python 3.X.0, the 3.X series is maintained for
five years:

- During the *first twenty four months* (2 years) it receives bugfix
  updates and full releases (sources and installers for Windows and
  macOS) are made approximately every other month.

- For the next *thirty six months* (3 years) it receives security
  updates and source-only releases are made on an as-needed basis
  (no fixed cadence).

- The final source-only release is made *five years* after 3.X.0.

Note: 2 years of full support start with
:pep:`Python 3.13 <719>`.  Python versions
3.9 - 3.12 operate on a calendar with 1Â½ year of full support, followed
by 3Â½ more years of security fixes.


Annual release cadence
----------------------

Feature development of Python 3.(X+1).0 starts as soon as
Python 3.X.0 Beta 1 is released.  This creates a twelve-month delta
between Python feature versions.


Example
=======

- 3.9 development begins: Tuesday, 2019-06-04
- 3.9.0 alpha 1: Monday, 2019-10-14
- 3.9.0 alpha 2: Monday, 2019-11-18
- 3.9.0 alpha 3: Monday, 2019-12-16
- 3.9.0 alpha 4: Monday, 2020-01-13
- 3.9.0 alpha 5: Monday, 2020-02-17
- 3.9.0 alpha 6: Monday, 2020-03-16
- 3.9.0 alpha 7: Monday, 2020-04-13
- 3.9.0 beta 1: Monday, 2020-05-18
  (No new features beyond this point.)

- 3.9.0 beta 2: Monday, 2020-06-08
- 3.9.0 beta 3: Monday, 2020-06-29
- 3.9.0 beta 4: Monday, 2020-07-20
- 3.9.0 candidate 1: Monday, 2020-08-10
- 3.9.0 candidate 2: Monday, 2020-09-14
- 3.9.0 final: Monday, 2020-10-05

.. figure:: pep-0602-example-release-calendar.png
   :align: center
   :width: 100%

   Figure 1. Consequences of the annual release cycle on the calendar.

In comparison, if this PEP is rejected and Python keeps the current
release schedule:

- 3.9 development begins: Tuesday, 2019-06-04
- 3.9.0 alpha 1: Monday, 2020-08-03  (10 months later)
- 3.9.0 alpha 2: Monday, 2020-09-07
- 3.9.0 alpha 3: Monday, 2020-10-05
- 3.9.0 alpha 4: Monday, 2020-11-02
- 3.9.0 beta 1: Monday, 2020-11-30  (6 months later)
- 3.9.0 beta 2: Monday, 2021-01-04
- 3.9.0 beta 3: Monday, 2021-02-01
- 3.9.0 beta 4: Monday, 2021-03-01
- 3.9.0 candidate 1: Monday, 2021-03-29
- 3.9.0 candidate 2: Monday, 2021-04-05 (if necessary)
- 3.9.0 final: Monday, 2021-04-19  (6 months later)


Dependent Policies
==================

Deprecations
------------

The current policy around breaking changes assumes at least two releases
before a deprecated feature is removed from Python or a ``__future__``
behavior is enabled by default.  This is documented in :pep:`387`.

This PEP proposes to keep this policy of **at least** two releases
before making a breaking change.

The term of the Steering Council
--------------------------------

The current wording of :pep:`13` states that "a new council is elected
after each feature release".  This PEP proposes to keep this policy
as it will lead to a consistent election schedule.

The term of the Release Manager
-------------------------------

The current undocumented convention is for a single Release Manager to
handle two feature releases of Python.  This PEP proposes to keep this
policy, allowing for the term to be extended to more releases with
approval from the Steering Council and the Cabal of Release Managers.

In particular, since this PEP is authored by the active Release Manager
and its effect would shorten the term of the Release Manager, the author
is open to managing the release of a third feature release to compensate
for the disruption.


Rationale and Goals
===================

This change provides the following advantages:

- makes releases smaller: since doubling the cadence doesn't double our
  available development resources, consecutive releases are going to be
  smaller in terms of features;

- puts features and bug fixes in hands of users sooner;

- creates a more gradual upgrade path for users, by decreasing the
  surface of change in any single release;

- creates a predictable calendar for releases where the final release is
  always in October (so after the annual core sprint), and the beta
  phase starts in late May (so after PyCon US sprints), which is
  especially important for core developers who need to plan to include
  Python involvement in their calendar;

- decreases the urge to rush features shortly before "Beta 1" due to
  the risk of them "slipping for 18 months";

- allows for synchronizing the schedule of Python release management
  with external distributors like Fedora who've been historically very
  helpful in finding regressions early not only in core Python but also
  in third-party libraries, helping moving the community forward to
  support the latest version of Python from Day 1;

- increases the explicit alpha release phase, which provides meaningful
  snapshots of progress on new features;

- significantly cuts the implicit "alpha 0" release phase which provides
  limited use for new development anyway (it overlaps with the beta of
  the *currently developed*, still unreleased, version).

Non-goals
---------

Adopting an annual release calendar allows for natural switching to
calendar versioning, for example by calling Python 3.9 "Python 3.20"
since it's released in October '20 and so on ("Python 3.23" would be the
one released in October '23).

While the ease of switching to calendar versioning can be treated as
an advantage of an annual release cycle, this PEP does not advocate for
or against a change in how Python is versioned.  Should the annual
release cycle be adopted, the versioning question will be dealt with in
a separate PEP.

Non-risks
---------

This change does not shorten the currently documented support calendar
for a Python release, both in terms of bugfix releases and security
fixes.

This change does not accelerate the velocity of development.  Python is
not going to become incompatible faster or accrue new features faster.
It's just that features are going to be released more gradually as they
are developed.

Consequently, while this change introduces the ability for users to
upgrade much faster, it does not require them to do so.  Say, if they
upgrade every second release, their experience with Python is going to
be similar to the current situation.

Risks
-----

Python redistribution
~~~~~~~~~~~~~~~~~~~~~

This requires changes to how integrators, like Linux distributions,
release Python within their systems.

The te


id: pep-0372.rst
content:
PEP: 372
Title: Adding an ordered dictionary to collections
Author: Armin Ronacher <armin.ronacher@active-4.com>,
        Raymond Hettinger <python@rcn.com>
Status: Final
Type: Standards Track
Created: 15-Jun-2008
Python-Version: 2.7, 3.1
Post-History:


Abstract
========

This PEP proposes an ordered dictionary as a new data structure for
the ``collections`` module, called "OrderedDict" in this PEP.  The
proposed API incorporates the experiences gained from working with
similar implementations that exist in various real-world applications
and other programming languages.


Patch
=====

A working Py3.1 patch including tests and documentation is at:

    `OrderedDict patch <https://github.com/python/cpython/issues/49647>`_

The check-in was in revisions: 70101 and 70102

Rationale
=========

In current Python versions, the widely used built-in dict type does
not specify an order for the key/value pairs stored.  This makes it
hard to use dictionaries as data storage for some specific use cases.

Some dynamic programming languages like PHP and Ruby 1.9 guarantee a
certain order on iteration.  In those languages, and existing Python
ordered-dict implementations, the ordering of items is defined by the
time of insertion of the key.  New keys are appended at the end, but
keys that are overwritten are not moved to the end.

The following example shows the behavior for simple assignments:

    >>> d = OrderedDict()
    >>> d['parrot'] = 'dead'
    >>> d['penguin'] = 'exploded'
    >>> d.items()
    [('parrot', 'dead'), ('penguin', 'exploded')]

That the ordering is preserved makes an OrderedDict useful for a couple of
situations:

- XML/HTML processing libraries currently drop the ordering of
  attributes, use a list instead of a dict which makes filtering
  cumbersome, or implement their own ordered dictionary.  This affects
  ElementTree, html5lib, Genshi and many more libraries.

- There are many ordered dict implementations in various libraries
  and applications, most of them subtly incompatible with each other.
  Furthermore, subclassing dict is a non-trivial task and many
  implementations don't override all the methods properly which can
  lead to unexpected results.

  Additionally, many ordered dicts are implemented in an inefficient
  way, making many operations more complex then they have to be.

- :pep:`3115` allows metaclasses to change the mapping object used for
  the class body.  An ordered dict could be used to create ordered
  member declarations similar to C structs.  This could be useful, for
  example, for future ``ctypes`` releases as well as ORMs that define
  database tables as classes, like the one the Django framework ships.
  Django currently uses an ugly hack to restore the ordering of
  members in database models.

- The RawConfigParser class accepts a ``dict_type`` argument that
  allows an application to set the type of dictionary used internally.
  The motivation for this addition was expressly to allow users to
  provide an ordered dictionary. [1]_

- Code ported from other programming languages such as PHP often
  depends on an ordered dict.  Having an implementation of an
  ordering-preserving dictionary in the standard library could ease
  the transition and improve the compatibility of different libraries.


Ordered Dict API
================

The ordered dict API would be mostly compatible with dict and existing
ordered dicts.  Note: this PEP refers to the 2.7 and 3.0 dictionary
API as described in collections.Mapping abstract base class.

The constructor and ``update()`` both accept iterables of tuples as
well as mappings like a dict does.  Unlike a regular dictionary,
the insertion order is preserved.

    >>> d = OrderedDict([('a', 'b'), ('c', 'd')])
    >>> d.update({'foo': 'bar'})
    >>> d
    collections.OrderedDict([('a', 'b'), ('c', 'd'), ('foo', 'bar')])

If ordered dicts are updated from regular dicts, the ordering of new
keys is of course undefined.

All iteration methods as well as ``keys()``, ``values()`` and
``items()`` return the values ordered by the time the key was
first inserted:

    >>> d['spam'] = 'eggs'
    >>> d.keys()
    ['a', 'c', 'foo', 'spam']
    >>> d.values()
    ['b', 'd', 'bar', 'eggs']
    >>> d.items()
    [('a', 'b'), ('c', 'd'), ('foo', 'bar'), ('spam', 'eggs')]

New methods not available on dict:

``OrderedDict.__reversed__()``
    Supports reverse iteration by key.


Questions and Answers
=====================

What happens if an existing key is reassigned?

    The key is not moved but assigned a new value in place.  This is
    consistent with existing implementations.

What happens if keys appear multiple times in the list passed to the
constructor?

    The same as for regular dicts -- the latter item overrides the
    former.  This has the side-effect that the position of the first
    key is used because only the value is actually overwritten::

        >>> OrderedDict([('a', 1), ('b', 2), ('a', 3)])
        collections.OrderedDict([('a', 3), ('b', 2)])

    This behavior is consistent with existing implementations in
    Python, the PHP array and the hashmap in Ruby 1.9.

Is the ordered dict a dict subclass?  Why?

    Yes.  Like ``defaultdict``, an ordered dictionary subclasses ``dict``.
    Being a dict subclass make some of the methods faster (like
    ``__getitem__`` and ``__len__``).  More importantly, being a dict
    subclass lets ordered dictionaries be usable with tools like json that
    insist on having dict inputs by testing isinstance(d, dict).

Do any limitations arise from subclassing dict?

    Yes.  Since the API for dicts is different in Py2.x and Py3.x, the
    OrderedDict API must also be different.  So, the Py2.7 version will need
    to override iterkeys, itervalues, and iteritems.

Does ``OrderedDict.popitem()`` return a particular key/value pair?

    Yes.  It pops-off the most recently inserted new key and its
    corresponding value.  This corresponds to the usual LIFO behavior
    exhibited by traditional push/pop pairs.  It is semantically
    equivalent to ``k=list(od)[-1]; v=od[k]; del od[k]; return (k,v)``.
    The actual implementation is more efficient and pops directly
    from a sorted list of keys.

Does OrderedDict support indexing, slicing, and whatnot?

    As a matter of fact, ``OrderedDict`` does not implement the ``Sequence``
    interface.  Rather, it is a ``MutableMapping`` that remembers
    the order of key insertion.  The only sequence-like addition is
    support for ``reversed``.

    A further advantage of not allowing indexing is that it leaves open
    the possibility of a fast C implementation using linked lists.

Does OrderedDict support alternate sort orders such as alphabetical?

   No.  Those wanting different sort orders really need to be using another
   technique.  The OrderedDict is all about recording insertion order.   If any
   other order is of interest, then another structure (like an in-memory
   dbm) is likely a better fit.

How well does OrderedDict work with the json module, PyYAML, and ConfigParser?

   For json, the good news is that json's encoder respects OrderedDict's iteration order::

        >>> items = [('one', 1), ('two', 2), ('three',3), ('four',4), ('five',5)]
        >>> json.dumps(OrderedDict(items))
        '{"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}'

   In Py2.6, the object_hook for json decoders passes-in an already built
   dictionary so order is lost before the object hook sees it.  This
   problem is being fixed for Python 2.7/3.1 by adding a new hook that
   preserves order (see https://github.com/python/cpython/issues/49631 ).
   With the new hook, order can be preserved::

        >>> jtext = '{"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}'
        >>> json.loads(jtext, object_pairs_hook=OrderedDict)
        OrderedDict({'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5})

   For PyYAML, a full round-trip is problem free::

        >>> ytext = yaml.dump(Ord


id: pep-0414.rst
content:
PEP: 414
Title: Explicit Unicode Literal for Python 3.3
Author: Armin Ronacher <armin.ronacher@active-4.com>,
        Alyssa Coghlan <ncoghlan@gmail.com>
Status: Final
Type: Standards Track
Created: 15-Feb-2012
Python-Version: 3.3
Post-History: 28-Feb-2012, 04-Mar-2012
Resolution: https://mail.python.org/pipermail/python-dev/2012-February/116995.html


Abstract
========

This document proposes the reintegration of an explicit unicode literal
from Python 2.x to the Python 3.x language specification, in order to
reduce the volume of changes needed when porting Unicode-aware
Python 2 applications to Python 3.


BDFL Pronouncement
==================

This PEP has been formally accepted for Python 3.3:

    I'm accepting the PEP. It's about as harmless as they come. Make it so.


Proposal
========

This PEP proposes that Python 3.3 restore support for Python 2's Unicode
literal syntax, substantially increasing the number of lines of existing
Python 2 code in Unicode aware applications that will run without modification
on Python 3.

Specifically, the Python 3 definition for string literal prefixes will be
expanded to allow::

    "u" | "U"

in addition to the currently supported::

    "r" | "R"

The following will all denote ordinary Python 3 strings::

    'text'
    "text"
    '''text'''
    """text"""
    u'text'
    u"text"
    u'''text'''
    u"""text"""
    U'text'
    U"text"
    U'''text'''
    U"""text"""

No changes are proposed to Python 3's actual Unicode handling, only to the
acceptable forms for string literals.


Exclusion of "Raw" Unicode Literals
===================================

Python 2 supports a concept of "raw" Unicode literals that don't meet the
conventional definition of a raw string: ``\uXXXX`` and ``\UXXXXXXXX`` escape
sequences are still processed by the compiler and converted to the
appropriate Unicode code points when creating the associated Unicode objects.

Python 3 has no corresponding concept - the compiler performs *no*
preprocessing of the contents of raw string literals. This matches the
behaviour of 8-bit raw string literals in Python 2.

Since such strings are rarely used and would be interpreted differently in
Python 3 if permitted, it was decided that leaving them out entirely was
a better choice. Code which uses them will thus still fail immediately on
Python 3 (with a Syntax Error), rather than potentially producing different
output.

To get equivalent behaviour that will run on both Python 2 and Python 3,
either an ordinary Unicode literal can be used (with appropriate additional
escaping within the string), or else string concatenation or string
formatting can be combine the raw portions of the string with those that
require the use of Unicode escape sequences.

Note that when using ``from __future__ import unicode_literals`` in Python 2,
the nominally "raw" Unicode string literals will process ``\uXXXX`` and
``\UXXXXXXXX`` escape sequences, just like Python 2 strings explicitly marked
with the "raw Unicode" prefix.


Author's Note
=============

This PEP was originally written by Armin Ronacher, and Guido's approval was
given based on that version.

The currently published version has been rewritten by Alyssa Coghlan to
include additional historical details and rationale that were taken into
account when Guido made his decision, but were not explicitly documented in
Armin's version of the PEP.

Readers should be aware that many of the arguments in this PEP are *not*
technical ones. Instead, they relate heavily to the *social* and *personal*
aspects of software development.


Rationale
=========

With the release of a Python 3 compatible version of the Web Services Gateway
Interface (WSGI) specification (:pep:`3333`) for Python 3.2, many parts of the
Python web ecosystem have been making a concerted effort to support Python 3
without adversely affecting their existing developer and user communities.

One major item of feedback from key developers in those communities, including
Chris McDonough (WebOb, Pyramid), Armin Ronacher (Flask, Werkzeug), Jacob
Kaplan-Moss (Django) and Kenneth Reitz (``requests``) is that the requirement
to change the spelling of *every* Unicode literal in an application
(regardless of how that is accomplished) is a key stumbling block for porting
efforts.

In particular, unlike many of the other Python 3 changes, it isn't one that
framework and library authors can easily handle on behalf of their users. Most
of those users couldn't care less about the "purity" of the Python language
specification, they just want their websites and applications to work as well
as possible.

While it is the Python web community that has been most vocal in highlighting
this concern, it is expected that other highly Unicode aware domains (such as
GUI development) may run into similar issues as they (and their communities)
start making concerted efforts to support Python 3.


Common Objections
=================


Complaint: This PEP may harm adoption of Python 3.2
---------------------------------------------------

This complaint is interesting, as it carries within it a tacit admission that
this PEP *will* make it easier to port Unicode aware Python 2 applications to
Python 3.

There are many existing Python communities that are prepared to put up with
the constraints imposed by the existing suite of porting tools, or to update
their Python 2 code bases sufficiently that the problems are minimised.

This PEP is not for those communities. Instead, it is designed specifically to
help people that *don't* want to put up with those difficulties.

However, since the proposal is for a comparatively small tweak to the language
syntax with no semantic changes, it is feasible to support it as a third
party import hook. While such an import hook imposes some import time
overhead, and requires additional steps from each application that needs it
to get the hook in place, it allows applications that target Python 3.2
to use libraries and frameworks that would otherwise only run on Python 3.3+
due to their use of unicode literal prefixes.

One such import hook project is Vinay Sajip's ``uprefix`` [4]_.

For those that prefer to translate their code in advance rather than
converting on the fly at import time, Armin Ronacher is working on a hook
that runs at install time rather than during import [5]_.

Combining the two approaches is of course also possible. For example, the
import hook could be used for rapid edit-test cycles during local
development, but the install hook for continuous integration tasks and
deployment on Python 3.2.

The approaches described in this section may prove useful, for example, for
applications that wish to target Python 3 on the Ubuntu 12.04 LTS release,
which will ship with Python 2.7 and 3.2 as officially supported Python
versions.

Complaint: Python 3 shouldn't be made worse just to support porting from Python 2
---------------------------------------------------------------------------------

This is indeed one of the key design principles of Python 3. However, one of
the key design principles of Python as a whole is that "practicality beats
purity". If we're going to impose a significant burden on third party
developers, we should have a solid rationale for doing so.

In most cases, the rationale for backwards incompatible Python 3 changes are
either to improve code correctness (for example, stricter default separation
of binary and text data and integer division upgrading to floats when
necessary), reduce typical memory usage (for example, increased usage of
iterators and views over concrete lists), or to remove distracting nuisances
that make Python code harder to read without increasing its expressiveness
(for example, the comma based syntax for naming caught exceptions). Changes
backed by such reasoning are *not* going to be reverted, regardless of
objections from Python 2 developers attempting to make the transition to
Python 3.

In many cases, Python 2 offered two ways of doi


id: pep-0400.rst
content:
PEP: 400
Title: Deprecate codecs.StreamReader and codecs.StreamWriter
Author: Victor Stinner <vstinner@python.org>
Status: Deferred
Type: Standards Track
Created: 28-May-2011
Python-Version: 3.3


Abstract
========

io.TextIOWrapper and codecs.StreamReaderWriter offer the same API
[#f1]_. TextIOWrapper has more features and is faster than
StreamReaderWriter. Duplicate code means that bugs should be fixed
twice and that we may have subtle differences between the two
implementations.

The codecs module was introduced in Python 2.0 (see the :pep:`100`).
The io module was
introduced in Python 2.6 and 3.0 (see the :pep:`3116`),
and reimplemented in C in
Python 2.7 and 3.1.

PEP Deferral
============

Further exploration of the concepts covered in this PEP has been deferred
for lack of a current champion interested in promoting the goals of the PEP
and collecting and incorporating feedback, and with sufficient available
time to do so effectively.

Motivation
==========

When the Python I/O model was updated for 3.0, the concept of a
"stream-with-known-encoding" was introduced in the form of
io.TextIOWrapper. As this class is critical to the performance of
text-based I/O in Python 3, this module has an optimised C version
which is used by CPython by default. Many corner cases in handling
buffering, stateful codecs and universal newlines have been dealt with
since the release of Python 3.0.

This new interface overlaps heavily with the legacy
codecs.StreamReader, codecs.StreamWriter and codecs.StreamReaderWriter
interfaces that were part of the original codec interface design in
:pep:`100`. These interfaces are organised around the principle of an
encoding with an associated stream (i.e. the reverse of arrangement in
the io module), so the original :pep:`100` design required that codec
writers provide appropriate StreamReader and StreamWriter
implementations in addition to the core codec encode() and decode()
methods. This places a heavy burden on codec authors providing these
specialised implementations to correctly handle many of the corner
cases (see `Appendix A`_) that have now been dealt with by io.TextIOWrapper. While deeper
integration between the codec and the stream allows for additional
optimisations in theory, these optimisations have in practice either
not been carried out and else the associated code duplication means
that the corner cases that have been fixed in io.TextIOWrapper are
still not handled correctly in the various StreamReader and
StreamWriter implementations.

Accordingly, this PEP proposes that:

* codecs.open() be updated to delegate to the builtin open() in Python
  3.3;
* the legacy codecs.Stream* interfaces, including the streamreader and
  streamwriter attributes of codecs.CodecInfo be deprecated in Python
  3.3.


Rationale
=========

StreamReader and StreamWriter issues
''''''''''''''''''''''''''''''''''''

* StreamReader is unable to translate newlines.
* StreamWriter doesn't support "line buffering" (flush if the input
  text contains a newline).
* StreamReader classes of the CJK encodings (e.g. GB18030) only
  supports UNIX newlines ('\\n').
* StreamReader and StreamWriter are stateful codecs but don't expose
  functions to control their state (getstate() or setstate()). Each
  codec has to handle corner cases, see `Appendix A <PEP 400 Appendix A_>`_.
* StreamReader and StreamWriter are very similar to IncrementalReader
  and IncrementalEncoder, some code is duplicated for stateful codecs
  (e.g. UTF-16).
* Each codec has to reimplement its own StreamReader and StreamWriter
  class, even if it's trivial (just call the encoder/decoder).
* codecs.open(filename, "r") creates an io.TextIOWrapper object.
* No codec implements an optimized method in StreamReader or
  StreamWriter based on the specificities of the codec.

Issues in the bug tracker:

* `Issue #5445 <http://bugs.python.org/issue5445>`_ (2009-03-08):
  codecs.StreamWriter.writelines problem when passed generator
* `Issue #7262: <http://bugs.python.org/issue7262>`_ (2009-11-04):
  codecs.open() + eol (windows)
* `Issue #8260 <http://bugs.python.org/issue8260>`_ (2010-03-29):
  When I use codecs.open(...) and f.readline() follow up by f.read()
  return bad result
* `Issue #8630 <http://bugs.python.org/issue8630>`_ (2010-05-05):
  Keepends param in codec readline(s)
* `Issue #10344 <http://bugs.python.org/issue10344>`_ (2010-11-06):
  codecs.readline doesn't care buffering
* `Issue #11461 <http://bugs.python.org/issue11461>`_ (2011-03-10):
  Reading UTF-16 with codecs.readline() breaks on surrogate pairs
* `Issue #12446 <http://bugs.python.org/issue12446>`_ (2011-06-30):
  StreamReader Readlines behavior odd
* `Issue #12508 <http://bugs.python.org/issue12508>`_ (2011-07-06):
  Codecs Anomaly
* `Issue #12512 <http://bugs.python.org/issue12512>`_ (2011-07-07):
  codecs: StreamWriter issues with stateful codecs after a seek or
  with append mode
* `Issue #12513 <http://bugs.python.org/issue12513>`_ (2011-07-07):
  codec.StreamReaderWriter: issues with interlaced read-write

TextIOWrapper features
''''''''''''''''''''''

* TextIOWrapper supports any kind of newline, including translating
  newlines (to UNIX newlines), to read and write.
* TextIOWrapper reuses codecs incremental encoders and decoders (no
  duplication of code).
* The io module (TextIOWrapper) is faster than the codecs module
  (StreamReader). It is implemented in C, whereas codecs is
  implemented in Python.
* TextIOWrapper has a readahead algorithm which speeds up small
  reads: read character by character or line by line (io is 10x
  through 25x faster than codecs on these operations).
* TextIOWrapper has a write buffer.
* TextIOWrapper.tell() is optimized.
* TextIOWrapper supports random access (read+write) using a single
  class which permit to optimize interlaced read-write (but no such
  optimization is implemented).

TextIOWrapper issues
''''''''''''''''''''

* `Issue #12215 <http://bugs.python.org/issue12215>`_ (2011-05-30):
  TextIOWrapper: issues with interlaced read-write

Possible improvements of StreamReader and StreamWriter
''''''''''''''''''''''''''''''''''''''''''''''''''''''

By adding codec state read/write functions to the StreamReader and
StreamWriter classes, it will become possible to fix issues with
stateful codecs in a base class instead of in each stateful
StreamReader and StreamWriter classes.

It would be possible to change StreamReader and StreamWriter to make
them use IncrementalDecoder and IncrementalEncoder.

A codec can implement variants which are optimized for the specific
encoding or intercept certain stream methods to add functionality or
improve the encoding/decoding performance. TextIOWrapper cannot
implement such optimization, but TextIOWrapper uses incremental
encoders and decoders and uses read and write buffers, so the overhead
of incomplete inputs is low or nul.

A lot more could be done for other variable length encoding codecs,
e.g. UTF-8, since these often have problems near the end of a read due
to missing bytes. The UTF-32-BE/LE codecs could simply multiply the
character position by 4 to get the byte position.


Usage of StreamReader and StreamWriter
''''''''''''''''''''''''''''''''''''''

These classes are rarely used directly, but indirectly using
codecs.open(). They are not used in Python 3 standard library (except
in the codecs module).

Some projects implement their own codec with StreamReader and
StreamWriter, but don't use these classes.


Backwards Compatibility
=======================

Keep the public API, codecs.open
''''''''''''''''''''''''''''''''

codecs.open() can be replaced by the builtin open() function. open()
has a similar API but has also more options. Both functions return
file-like objects (same API).

codecs.open() was the only way to open a text file in Unicode mode
until Python 2.6. Many Python 2 programs uses this function. Removing
codecs.open() implies more work to port programs from Python 2 to
Python 3, especially projects usin


id: pep-0366.rst
content:
PEP: 366
Title: Main module explicit relative imports
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Final
Type: Standards Track
Created: 01-May-2007
Python-Version: 2.6, 3.0
Post-History: 01-May-2007, 04-Jul-2007, 07-Jul-2007, 23-Nov-2007


Abstract
========

This PEP proposes a backwards compatible mechanism that permits
the use of explicit relative imports from executable modules within
packages. Such imports currently fail due to an awkward interaction
between :pep:`328` and :pep:`338`.

By adding a new module level attribute, this PEP allows relative imports
to work automatically if the module is executed using the ``-m`` switch.
A small amount of boilerplate in the module itself will allow the relative
imports to work when the file is executed by name.

Guido accepted the PEP in November 2007 [5]_.

Proposed Change
===============

The major proposed change is the introduction of a new module level
attribute, ``__package__``. When it is present, relative imports will
be based on this attribute rather than the module ``__name__``
attribute.

As with the current ``__name__`` attribute, setting ``__package__`` will
be the responsibility of the :pep:`302` loader used to import a module.
Loaders which use ``imp.new_module()`` to create the module object will
have the new attribute set automatically to ``None``. When the import
system encounters an explicit relative import in a module without
``__package__`` set (or with it set to ``None``), it will calculate and
store the correct value (``__name__.rpartition('.')[0]`` for normal
modules and ``__name__`` for package initialisation modules). If
``__package__`` has already been set then the import system will use
it in preference to recalculating the package name from the
``__name__`` and ``__path__`` attributes.

The ``runpy`` module will explicitly set the new attribute, basing it off
the name used to locate the module to be executed rather than the name
used to set the module's ``__name__`` attribute. This will allow relative
imports to work correctly from main modules executed with the ``-m``
switch.

When the main module is specified by its filename, then the
``__package__`` attribute will be set to ``None``. To allow
relative imports when the module is executed directly, boilerplate
similar to the following would be needed before the first relative
import statement::

  if __name__ == "__main__" and __package__ is None:
      __package__ = "expected.package.name"

Note that this boilerplate is sufficient only if the top level package
is already accessible via ``sys.path``. Additional code that manipulates
``sys.path`` would be needed in order for direct execution to work
without the top level package already being importable.

This approach also has the same disadvantage as the use of absolute
imports of sibling modules - if the script is moved to a different
package or subpackage, the boilerplate will need to be updated
manually. It has the advantage that this change need only be made
once per file, regardless of the number of relative imports.

Note that setting ``__package__`` to the empty string explicitly is
permitted, and has the effect of disabling all relative imports from
that module (since the import machinery will consider it to be a
top level module in that case). This means that tools like ``runpy``
do not need to provide special case handling for top level modules
when setting ``__package__``.

Rationale for Change
====================

The current inability to use explicit relative imports from the main
module is the subject of at least one open SF bug report (#1510172) [1]_,
and has most likely been a factor in at least a few queries on
comp.lang.python (such as Alan Isaac's question in [2]_).

This PEP is intended to provide a solution which permits explicit
relative imports from main modules, without incurring any significant
costs during interpreter startup or normal module import.

The section in :pep:`338` on relative imports and the main module provides
further details and background on this problem.


Reference Implementation
========================

Rev 47142 in SVN implemented an early variant of this proposal
which stored the main module's real module name in the
``__module_name__`` attribute. It was reverted due to the fact
that 2.5 was already in beta by that time.

Patch 1487 [4]_ is the proposed implementation for this PEP.

Alternative Proposals
=====================

:pep:`3122` proposed addressing this problem by changing the way
the main module is identified. That's a significant compatibility cost
to incur to fix something that is a pretty minor bug in the overall
scheme of things, and the PEP was rejected [3]_.

The advantage of the proposal in this PEP is that its only impact on
normal code is the small amount of time needed to set the extra
attribute when importing a module. Relative imports themselves should
be sped up fractionally, as the package name is cached in the module
globals, rather than having to be worked out again for each relative
import.


References
==========

.. [1] Absolute/relative import not working?
   (https://github.com/python/cpython/issues/43535)

.. [2] c.l.p. question about modules and relative imports
   (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/)

.. [3] Guido's rejection of PEP 3122
   (https://mail.python.org/pipermail/python-3000/2007-April/006793.html)

.. [4] PEP 366 implementation patch
   (https://github.com/python/cpython/issues/45828)

.. [5] Acceptance of the PEP
   (https://mail.python.org/pipermail/python-dev/2007-November/075475.html)

Copyright
=========

This document has been placed in the public domain.


id: pep-0428.rst
content:
PEP: 428
Title: The pathlib module -- object-oriented filesystem paths
Author: Antoine Pitrou <solipsis@pitrou.net>
Status: Final
Type: Standards Track
Created: 30-Jul-2012
Python-Version: 3.4
Post-History: `05-Oct-2012 <https://mail.python.org/pipermail/python-ideas/2012-October/016338.html>`__
Resolution: https://mail.python.org/pipermail/python-dev/2013-November/130424.html


Abstract
========

This PEP proposes the inclusion of a third-party module, `pathlib`_, in
the standard library.  The inclusion is proposed under the provisional
label, as described in :pep:`411`.  Therefore, API changes can be done,
either as part of the PEP process, or after acceptance in the standard
library (and until the provisional label is removed).

The aim of this library is to provide a simple hierarchy of classes to
handle filesystem paths and the common operations users do over them.

.. _`pathlib`: http://pypi.python.org/pypi/pathlib/


Related work
============

An object-oriented API for filesystem paths has already been proposed
and rejected in :pep:`355`.  Several third-party implementations of the
idea of object-oriented filesystem paths exist in the wild:

* The historical `path.py module`_ by Jason Orendorff, Jason R. Coombs
  and others, which provides a ``str``-subclassing ``Path`` class;

* Twisted's slightly specialized `FilePath class`_;

* An `AlternativePathClass proposal`_, subclassing ``tuple`` rather than
  ``str``;

* `Unipath`_, a variation on the str-subclassing approach with two public
  classes, an ``AbstractPath`` class for operations which don't do I/O and a
  ``Path`` class for all common operations.

This proposal attempts to learn from these previous attempts and the
rejection of :pep:`355`.


.. _`path.py module`: https://github.com/jaraco/path.py
.. _`FilePath class`: http://twistedmatrix.com/documents/current/api/twisted.python.filepath.FilePath.html
.. _`AlternativePathClass proposal`: http://wiki.python.org/moin/AlternativePathClass
.. _`Unipath`: https://bitbucket.org/sluggo/unipath/overview


Implementation
==============

The implementation of this proposal is tracked in the ``pep428`` branch
of pathlib's `Mercurial repository`_.

.. _`Mercurial repository`: https://bitbucket.org/pitrou/pathlib/


Why an object-oriented API
==========================

The rationale to represent filesystem paths using dedicated classes is the
same as for other kinds of stateless objects, such as dates, times or IP
addresses.  Python has been slowly moving away from strictly replicating
the C language's APIs to providing better, more helpful abstractions around
all kinds of common functionality.  Even if this PEP isn't accepted, it is
likely that another form of filesystem handling abstraction will be adopted
one day into the standard library.

Indeed, many people will prefer handling dates and times using the high-level
objects provided by the ``datetime`` module, rather than using numeric
timestamps and the ``time`` module API.  Moreover, using a dedicated class
allows to enable desirable behaviours by default, for example the case
insensitivity of Windows paths.


Proposal
========

Class hierarchy
---------------

The `pathlib`_ module implements a simple hierarchy of classes::

                           +----------+
                           |          |
                  ---------| PurePath |--------
                  |        |          |       |
                  |        +----------+       |
                  |             |             |
                  |             |             |
                  v             |             v
           +---------------+    |    +-----------------+
           |               |    |    |                 |
           | PurePosixPath |    |    | PureWindowsPath |
           |               |    |    |                 |
           +---------------+    |    +-----------------+
                  |             v             |
                  |          +------+         |
                  |          |      |         |
                  |   -------| Path |------   |
                  |   |      |      |     |   |
                  |   |      +------+     |   |
                  |   |                   |   |
                  |   |                   |   |
                  v   v                   v   v
             +-----------+           +-------------+
             |           |           |             |
             | PosixPath |           | WindowsPath |
             |           |           |             |
             +-----------+           +-------------+


This hierarchy divides path classes along two dimensions:

* a path class can be either pure or concrete: pure classes support only
  operations that don't need to do any actual I/O, which are most path
  manipulation operations; concrete classes support all the operations
  of pure classes, plus operations that do I/O.

* a path class is of a given flavour according to the kind of operating
  system paths it represents.  `pathlib`_ implements two flavours: Windows
  paths for the filesystem semantics embodied in Windows systems, POSIX
  paths for other systems.

Any pure class can be instantiated on any system: for example, you can
manipulate ``PurePosixPath`` objects under Windows, ``PureWindowsPath``
objects under Unix, and so on.  However, concrete classes can only be
instantiated on a matching system: indeed, it would be error-prone to start
doing I/O with ``WindowsPath`` objects under Unix, or vice-versa.

Furthermore, there are two base classes which also act as system-dependent
factories: ``PurePath`` will instantiate either a ``PurePosixPath`` or a
``PureWindowsPath`` depending on the operating system.  Similarly, ``Path``
will instantiate either a ``PosixPath`` or a ``WindowsPath``.

It is expected that, in most uses, using the ``Path`` class is adequate,
which is why it has the shortest name of all.


No confusion with builtins
--------------------------

In this proposal, the path classes do not derive from a builtin type.  This
contrasts with some other Path class proposals which were derived from
``str``.  They also do not pretend to implement the sequence protocol:
if you want a path to act as a sequence, you have to lookup a dedicated
attribute (the ``parts`` attribute).

The key reasoning behind not inheriting from ``str`` is to prevent accidentally
performing operations with a string representing a path and a string that
doesn't, e.g. ``path + an_accident``. Since operations with a string will not
necessarily lead to a valid or expected file system path, "explicit is better
than implicit" by avoiding accidental operations with strings by not
subclassing it. A `blog post`_ by a Python core developer goes into more detail
on the reasons behind this specific design decision.

.. _blog post: http://www.snarky.ca/why-pathlib-path-doesn-t-inherit-from-str


Immutability
------------

Path objects are immutable, which makes them hashable and also prevents a
class of programming errors.


Sane behaviour
--------------

Little of the functionality from os.path is reused.  Many os.path functions
are tied by backwards compatibility to confusing or plain wrong behaviour
(for example, the fact that ``os.path.abspath()`` simplifies ".." path
components without resolving symlinks first).


Comparisons
-----------

Paths of the same flavour are comparable and orderable, whether pure or not::

    >>> PurePosixPath('a') == PurePosixPath('b')
    False
    >>> PurePosixPath('a') < PurePosixPath('b')
    True
    >>> PurePosixPath('a') == PosixPath('a')
    True

Comparing and ordering Windows path objects is case-insensitive::

    >>> PureWindowsPath('a') == PureWindowsPath('A')
    True

Paths of different flavours always compare unequal, and cannot be ordered::

    >>> PurePosixPath('a') == PureWindowsPath('a')
    False
    >>> PurePosixPath('a') < PureWindowsPath('a')
    Traceback (most recent call last):
      File "<stdin>", line 1,


id: pep-0399.rst
content:
PEP: 399
Title: Pure Python/C Accelerator Module Compatibility Requirements
Author: Brett Cannon <brett@python.org>
Status: Final
Type: Informational
Created: 04-Apr-2011
Python-Version: 3.3
Post-History: 04-Apr-2011, 12-Apr-2011, 17-Jul-2011, 15-Aug-2011, 01-Jan-2013

Abstract
========

The Python standard library under CPython contains various instances
of modules implemented in both pure Python and C (either entirely or
partially). This PEP requires that in these instances that the
C code **must** pass the test suite used for the pure Python code
so as to act as much as a drop-in replacement as reasonably possible
(C- and VM-specific tests are exempt). It is also required that new
C-based modules lacking a pure Python equivalent implementation get
special permission to be added to the standard library.


Rationale
=========

Python has grown beyond the CPython virtual machine (VM). IronPython_,
Jython_, and PyPy_ are all currently viable alternatives to the
CPython VM. The VM ecosystem that has sprung up around the Python
programming language has led to Python being used in many different
areas where CPython cannot be used, e.g., Jython allowing Python to be
used in Java applications.

A problem all of the VMs other than CPython face is handling modules
from the standard library that are implemented (to some extent) in C.
Since other VMs do not typically support the entire `C API of CPython`_
they are unable to use the code used to create the module. Oftentimes
this leads these other VMs to either re-implement the modules in pure
Python or in the programming language used to implement the VM itself
(e.g., in C# for IronPython). This duplication of effort between
CPython, PyPy, Jython, and IronPython is extremely unfortunate as
implementing a module **at least** in pure Python would help mitigate
this duplicate effort.

The purpose of this PEP is to minimize this duplicate effort by
mandating that all new modules added to Python's standard library
**must** have a pure Python implementation *unless* special dispensation
is given. This makes sure that a module in the stdlib is available to
all VMs and not just to CPython (pre-existing modules that do not meet
this requirement are exempt, although there is nothing preventing
someone from adding in a pure Python implementation retroactively).

Re-implementing parts (or all) of a module in C (in the case
of CPython) is still allowed for performance reasons, but any such
accelerated code must pass the same test suite (sans VM- or C-specific
tests) to verify semantics and prevent divergence. To accomplish this,
the test suite for the module must have comprehensive coverage of the
pure Python implementation before the acceleration code may be added.


Details
=======

Starting in Python 3.3, any modules added to the standard library must
have a pure Python implementation. This rule can only be ignored if
the Python development team grants a special exemption for the module.
Typically the exemption will be granted only when a module wraps a
specific C-based library (e.g., sqlite3_). In granting an exemption it
will be recognized that the module will be considered exclusive to
CPython and not part of Python's standard library that other VMs are
expected to support. Usage of ``ctypes`` to provide an
API for a C library will continue to be frowned upon as ``ctypes``
lacks compiler guarantees that C code typically relies upon to prevent
certain errors from occurring (e.g., API changes).

Even though a pure Python implementation is mandated by this PEP, it
does not preclude the use of a companion acceleration module. If an
acceleration module is provided it is to be named the same as the
module it is accelerating with an underscore attached as a prefix,
e.g., ``_warnings`` for ``warnings``. The common pattern to access
the accelerated code from the pure Python implementation is to import
it with an ``import *``, e.g., ``from _warnings import *``. This is
typically done at the end of the module to allow it to overwrite
specific Python objects with their accelerated equivalents. This kind
of import can also be done before the end of the module when needed,
e.g., an accelerated base class is provided but is then subclassed by
Python code. This PEP does not mandate that pre-existing modules in
the stdlib that lack a pure Python equivalent gain such a module. But
if people do volunteer to provide and maintain a pure Python
equivalent (e.g., the PyPy team volunteering their pure Python
implementation of the ``csv`` module and maintaining it) then such
code will be accepted. In those instances the C version is considered
the reference implementation in terms of expected semantics.

Any new accelerated code must act as a drop-in replacement as close
to the pure Python implementation as reasonable. Technical details of
the VM providing the accelerated code are allowed to differ as
necessary, e.g., a class being a ``type`` when implemented in C. To
verify that the Python and equivalent C code operate as similarly as
possible, both code bases must be tested using the same tests which
apply to the pure Python code (tests specific to the C code or any VM
do not follow under this requirement). The test suite is expected to
be extensive in order to verify expected semantics.

Acting as a drop-in replacement also dictates that no public API be
provided in accelerated code that does not exist in the pure Python
code.  Without this requirement people could accidentally come to rely
on a detail in the accelerated code which is not made available to
other VMs that use the pure Python implementation. To help verify
that the contract of semantic equivalence is being met, a module must
be tested both with and without its accelerated code as thoroughly as
possible.

As an example, to write tests which exercise both the pure Python and
C accelerated versions of a module, a basic idiom can be followed::

    from test.support import import_fresh_module
    import unittest

    c_heapq = import_fresh_module('heapq', fresh=['_heapq'])
    py_heapq = import_fresh_module('heapq', blocked=['_heapq'])


    class ExampleTest:

        def test_example(self):
            self.assertTrue(hasattr(self.module, 'heapify'))


    class PyExampleTest(ExampleTest, unittest.TestCase):
        module = py_heapq


    @unittest.skipUnless(c_heapq, 'requires the C _heapq module')
    class CExampleTest(ExampleTest, unittest.TestCase):
        module = c_heapq


    if __name__ == '__main__':
        unittest.main()


The test module defines a base class (``ExampleTest``) with test methods
that access the ``heapq`` module through a ``self.heapq`` class attribute,
and two subclasses that set this attribute to either the Python or the C
version of the module.  Note that only the two subclasses inherit from
``unittest.TestCase`` -- this prevents the ``ExampleTest`` class from
being detected as a ``TestCase`` subclass by ``unittest`` test discovery.
A ``skipUnless`` decorator can be added to the class that tests the C code
in order to have these tests skipped when the C module is not available.

If this test were to provide extensive coverage for
``heapq.heappop()`` in the pure Python implementation then the
accelerated C code would be allowed to be added to CPython's standard
library. If it did not, then the test suite would need to be updated
until proper coverage was provided before the accelerated C code
could be added.

To also help with compatibility, C code should use abstract APIs on
objects to prevent accidental dependence on specific types. For
instance, if a function accepts a sequence then the C code should
default to using ``PyObject_GetItem()`` instead of something like
``PyList_GetItem()``. C code is allowed to have a fast path if the
proper ``PyList_CheckExact()`` is used, but otherwise APIs should work
with any object that duck types to the proper interface instead of a
specific type.


Copyright
=========

This document has been placed in t


id: pep-0398.rst
content:
PEP: 398
Title: Python 3.3 Release Schedule
Author: Georg Brandl <georg@python.org>
Status: Final
Type: Informational
Topic: Release
Created: 23-Mar-2011
Python-Version: 3.3


Abstract
========

This document describes the development and release schedule for
Python 3.3.  The schedule primarily concerns itself with PEP-sized
items.

.. Small features may be added up to and including the first beta
   release.  Bugs may be fixed until the final release, which is planned
   for September 2012.


Release Manager and Crew
========================

- 3.3 Release Managers: Georg Brandl, Ned Deily (3.3.7+)
- Windows installers: Martin v. LÃ¶wis
- Mac installers: Ronald Oussoren/Ned Deily
- Documentation: Georg Brandl


3.3 Lifespan
============

3.3 will receive bugfix updates approximately every 4-6 months for
approximately 18 months.  After the release of 3.4.0 final, a final
3.3 bugfix update will be released.  After that, security updates
(source only) will be released until 5 years after the release of 3.3
final, which will be September 2017.

As of 2017-09-29, Python 3.3.x reached end-of-life status.


Release Schedule
================

3.3.0 schedule
--------------

- 3.3.0 alpha 1: March 5, 2012
- 3.3.0 alpha 2: April 2, 2012
- 3.3.0 alpha 3: May 1, 2012
- 3.3.0 alpha 4: May 31, 2012
- 3.3.0 beta 1: June 27, 2012

(No new features beyond this point.)

- 3.3.0 beta 2: August 12, 2012
- 3.3.0 candidate 1: August 24, 2012
- 3.3.0 candidate 2: September 9, 2012
- 3.3.0 candidate 3: September 24, 2012
- 3.3.0 final: September 29, 2012

.. don't forget to update final date above as well

.. Every release day listed here is the prospective day of tagging the release;
   the actual availability of the release for download on python.org depends
   on the schedules of the crew.

3.3.1 schedule
--------------

- 3.3.1 candidate 1: March 23, 2013
- 3.3.1 final: April 6, 2013

3.3.2 schedule
--------------

- 3.3.2 final: May 13, 2013

3.3.3 schedule
--------------

- 3.3.3 candidate 1: October 27, 2013
- 3.3.3 candidate 2: November 9, 2013
- 3.3.3 final: November 16, 2013

3.3.4 schedule
--------------

- 3.3.4 candidate 1: January 26, 2014
- 3.3.4 final: February 9, 2014

3.3.5 schedule
--------------

Python 3.3.5 was the last regular maintenance release before 3.3 entered
security-fix only mode.

- 3.3.5 candidate 1: February 22, 2014
- 3.3.5 candidate 2: March 1, 2014
- 3.3.5 final: March 8, 2014

3.3.6 schedule
--------------

Security fixes only

- 3.3.6 candidate 1 (source-only release): October 4, 2014
- 3.3.6 final (source-only release): October 11, 2014

3.3.7 schedule
--------------

Security fixes only

- 3.3.7 candidate 1 (source-only release): September 6, 2017
- 3.3.7 final (source-only release): September 19, 2017

3.3.x end-of-life
-----------------

- September 29, 2017

Features for 3.3
================

Implemented / Final PEPs:

* :pep:`362`: Function Signature Object
* :pep:`380`: Syntax for Delegating to a Subgenerator
* :pep:`393`: Flexible String Representation
* :pep:`397`: Python launcher for Windows
* :pep:`399`: Pure Python/C Accelerator Module Compatibility Requirements
* :pep:`405`: Python Virtual Environments
* :pep:`409`: Suppressing exception context
* :pep:`412`: Key-Sharing Dictionary
* :pep:`414`: Explicit Unicode Literal for Python 3.3
* :pep:`415`: Implement context suppression with exception attributes
* :pep:`417`: Including mock in the Standard Library
* :pep:`418`: Add monotonic time, performance counter, and process time functions
* :pep:`420`: Implicit Namespace Packages
* :pep:`421`: Adding sys.implementation
* :pep:`3118`: Revising the buffer protocol (protocol semantics finalised)
* :pep:`3144`: IP Address manipulation library
* :pep:`3151`: Reworking the OS and IO exception hierarchy
* :pep:`3155`: Qualified name for classes and functions

Other final large-scale changes:

* Addition of the "faulthandler" module
* Addition of the "lzma" module, and lzma/xz support in tarfile
* Implementing ``__import__`` using importlib
* Addition of the C decimal implementation
* Switch of Windows build toolchain to VS 2010

Candidate PEPs:

* None

.. (Note that these are not accepted yet and even if they are, they might
   not be finished in time for Python 3.3.)

Other planned large-scale changes:

* None

Deferred to post-3.3:

* :pep:`395`: Qualified Names for Modules
* :pep:`3143`: Standard daemon process library
* :pep:`3154`: Pickle protocol version 4
* Breaking out standard library and docs in separate repos
* Addition of the "packaging" module, deprecating "distutils"
* Addition of the "regex" module
* Email version 6
* A standard event-loop interface (PEP by Jim Fulton pending)

Copyright
=========

This document has been placed in the public domain.


id: pep-0429.rst
content:
PEP: 429
Title: Python 3.4 Release Schedule
Author: Larry Hastings <larry@hastings.org>
Status: Final
Type: Informational
Topic: Release
Created: 17-Oct-2012
Python-Version: 3.4


Abstract
========

This document describes the development and release schedule for
Python 3.4.  The schedule primarily concerns itself with PEP-sized
items.

.. Small features may be added up to the first beta
   release.  Bugs may be fixed until the final release,
   which is planned for March 2014.


Release Manager and Crew
========================

- 3.4 Release Manager: Larry Hastings
- Windows installers: Martin v. LÃ¶wis
- Mac installers: Ned Deily
- Documentation: Georg Brandl


Release Schedule
================

Python 3.4 has now reached its end-of-life and has been retired.
No more releases will be made.

These are all the historical releases of Python 3.4,
including their release dates.

- 3.4.0 alpha 1: August 3, 2013
- 3.4.0 alpha 2: September 9, 2013
- 3.4.0 alpha 3: September 29, 2013
- 3.4.0 alpha 4: October 20, 2013
- 3.4.0 beta 1: November 24, 2013
- 3.4.0 beta 2: January 5, 2014
- 3.4.0 beta 3: January 26, 2014
- 3.4.0 candidate 1: February 10, 2014
- 3.4.0 candidate 2: February 23, 2014
- 3.4.0 candidate 3: March 9, 2014
- 3.4.0 final: March 16, 2014
- 3.4.1 candidate 1: May 5, 2014
- 3.4.1 final: May 18, 2014
- 3.4.2 candidate 1: September 22, 2014
- 3.4.2 final: October 6, 2014
- 3.4.3 candidate 1: February 8, 2015
- 3.4.3 final: February 25, 2015
- 3.4.4 candidate 1: December 6, 2015
- 3.4.4 final: December 20, 2015
- 3.4.5 candidate 1: June 12, 2016
- 3.4.5 final: June 26, 2016
- 3.4.6 candidate 1: January 2, 2017
- 3.4.6 final: January 17, 2017
- 3.4.7 candidate 1: July 25, 2017
- 3.4.7 final: August 9, 2017
- 3.4.8 candidate 1: January 23, 2018
- 3.4.8 final: February 4, 2018
- 3.4.9 candidate 1: July 19, 2018
- 3.4.9 final: August 2, 2018
- 3.4.10 candidate 1: March 4, 2019
- 3.4.10 final: March 18, 2019


Features for 3.4
================

Implemented / Final PEPs:

* :pep:`428`, a "pathlib" module providing object-oriented filesystem paths
* :pep:`435`, a standardized "enum" module
* :pep:`436`, a build enhancement that will help generate introspection information for builtins
* :pep:`442`, improved semantics for object finalization
* :pep:`443`, adding single-dispatch generic functions to the standard library
* :pep:`445`, a new C API for implementing custom memory allocators
* :pep:`446`, changing file descriptors to not be inherited by default in subprocesses
* :pep:`450`, a new "statistics" module
* :pep:`451`, standardizing module metadata for Python's module import system
* :pep:`453`, a bundled installer for the *pip* package manager
* :pep:`454`, a new "tracemalloc" module for tracing Python memory allocations
* :pep:`456`, a new hash algorithm for Python strings and binary data
* :pep:`3154`, a new and improved protocol for pickled objects
* :pep:`3156`, a new "asyncio" module, a new framework for asynchronous I/O

Deferred to post-3.4:

* :pep:`431`, improved support for time zone databases
* :pep:`441`, improved Python zip application support
* :pep:`447`, support for __locallookup__ metaclass method
* :pep:`448`, additional unpacking generalizations
* :pep:`455`, key transforming dictionary


Copyright
=========

This document has been placed in the public domain.


id: pep-0401.rst
content:
PEP: 401
Title: BDFL Retirement
Author: Barry Warsaw, Brett Cannon
Status: April Fool!
Type: Process
Created: 01-Apr-2009
Post-History: 01-Apr-2009


Abstract
========

The BDFL, having shepherded Python development for 20 years,
officially announces his retirement, effective immediately.  Following
a unanimous vote, his replacement is named.


Rationale
=========

Guido wrote the original implementation of Python in 1989, and after
nearly 20 years of leading the community, has decided to step aside as
its Benevolent Dictator For Life.  His official title is now
Benevolent Dictator Emeritus Vacationing Indefinitely from the
Language (BDEVIL).  Guido leaves Python in the good hands of its new
leader and its vibrant community, in order to train for his lifelong
dream of climbing Mount Everest.

After unanimous vote of the Python Steering Union (not to be confused
with the Python Secret Underground, which emphatically does not exist)
at the 2009 Python Conference (PyCon_ 2009), Guido's successor has been
chosen: Barry Warsaw, or as he is affectionately known, Uncle Barry.
Uncle Barry's official title is Friendly Language Uncle For Life (FLUFL).

.. _PyCon: http://us.pycon.org/


Official Acts of the FLUFL
==========================

FLUFL Uncle Barry enacts the following decisions, in order to
demonstrate his intention to lead the community in the same
responsible and open manner as his predecessor, whose name escapes
him:

* Recognized that the selection of ``Hg`` as the DVCS of choice was
  clear proof of the onset of the BDEVIL's insanity, and reverting
  this decision to switch to ``Bzr`` instead, the only true choice.

* Recognized that the ``!=`` inequality operator in Python 3.0 was a
  horrible, finger pain inducing mistake, the FLUFL reinstates the
  ``<>`` diamond operator as the sole spelling.  This change is
  important enough to be implemented for, and released in Python
  3.1. To help transition to this feature, a new future statement,
  ``from __future__ import barry_as_FLUFL`` has been added.

* Recognized that the ``print`` function in Python 3.0 was a horrible,
  pain-inducing mistake, the FLUFL reinstates the ``print``
  statement.  This change is important enough to be implemented for,
  and released in Python 3.0.2.

* Recognized that the disappointing adoption curve of Python 3.0
  signals its abject failure, all work on Python 3.1 and subsequent
  Python 3.x versions is hereby terminated.  All features in Python
  3.0 shall be back ported to Python 2.7 which will be the official
  and sole next release.  The Python 3.0 string and bytes types will
  be back ported to Python 2.6.2 for the convenience of developers.

* Recognized that C is a 20th-century language with almost universal
  rejection by programmers under the age of 30, the CPython
  implementation will terminate with the release of Python 2.6.2 and
  3.0.2.  Thereafter, the reference implementation of Python will
  target the Parrot [1]_ virtual machine.  Alternative implementations
  of Python (e.g. Jython [2]_, IronPython [3]_, and PyPy [4]_) are
  officially discouraged but tolerated.

* Recognized that the Python Software Foundation [5]_ having fulfilled
  its mission admirably, is hereby disbanded.  The Python Steering
  Union [6]_ (not to be confused with the Python Secret Underground,
  which emphatically does not exist), is now the sole steward for all
  of Python's intellectual property.  All PSF funds are hereby
  transferred to the PSU (not that PSU, the other PSU).


References
==========

.. [1] http://www.parrot.org

.. [2] http://www.jython.org

.. [3] http://www.ironpython.com

.. [4] http://www.codespeak.net/pypy

.. [5] http://www.python.org/psf

.. [6] http://www.pythonlabs.com


Copyright
=========

This document is the property of the Python Steering Union (not to be
confused with the Python Secret Underground, which emphatically does
not exist).  We suppose it's okay for you to read this, but don't even
think about quoting, copying, modifying, or distributing it.


id: pep-0367.rst
content:
PEP: 367
Title: New Super
Author: Calvin Spealman <ironfroggy@gmail.com>,
        Tim Delaney <timothy.c.delaney@gmail.com>
Status: Superseded
Type: Standards Track
Created: 28-Apr-2007
Python-Version: 2.6
Post-History: `28-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072807.html>`__,
              `29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072835.html>`__,
              `29-Apr-2007 <https://mail.python.org/pipermail/python-dev/2007-April/072858.html>`__,
              `14-May-2007 <https://mail.python.org/pipermail/python-dev/2007-May/073127.html>`__

Numbering Note
==============

This PEP has been renumbered to :pep:`3135`.  The text below is the last
version submitted under the old number.

Abstract
========

This PEP proposes syntactic sugar for use of the ``super`` type to automatically
construct instances of the super type binding to the class that a method was
defined in, and the instance (or class object for classmethods) that the method
is currently acting upon.

The premise of the new super usage suggested is as follows::

    super.foo(1, 2)

to replace the old::

    super(Foo, self).foo(1, 2)

and the current ``__builtin__.super`` be aliased to ``__builtin__.__super__``
(with ``__builtin__.super`` to be removed in Python 3.0).

It is further proposed that assignment to ``super`` become a ``SyntaxError``,
similar to the behaviour of ``None``.


Rationale
=========

The current usage of super requires an explicit passing of both the class and
instance it must operate from, requiring a breaking of the DRY (Don't Repeat
Yourself) rule. This hinders any change in class name, and is often considered
a wart by many.


Specification
=============

Within the specification section, some special terminology will be used to
distinguish similar and closely related concepts. "super type" will refer to
the actual builtin type named "super". A "super instance" is simply an instance
of the super type, which is associated with a class and possibly with an
instance of that class.

Because the new ``super`` semantics are not backwards compatible with Python
2.5, the new semantics will require a ``__future__`` import::

    from __future__ import new_super

The current ``__builtin__.super`` will be aliased to ``__builtin__.__super__``.
This will occur regardless of whether the new ``super`` semantics are active.
It is not possible to simply rename ``__builtin__.super``, as that would affect
modules that do not use the new ``super`` semantics. In Python 3.0 it is
proposed that the name ``__builtin__.super`` will be removed.

Replacing the old usage of super, calls to the next class in the MRO (method
resolution order) can be made without explicitly creating a ``super``
instance (although doing so will still be supported via ``__super__``). Every
function will have an implicit local named ``super``. This name behaves
identically to a normal local, including use by inner functions via a cell,
with the following exceptions:

1. Assigning to the name ``super`` will raise a ``SyntaxError`` at compile time;

2. Calling a static method or normal function that accesses the name ``super``
   will raise a ``TypeError`` at runtime.

Every function that uses the name ``super``, or has an inner function that
uses the name ``super``, will include a preamble that performs the equivalent
of::

    super = __builtin__.__super__(<class>, <instance>)

where ``<class>`` is the class that the method was defined in, and
``<instance>`` is the first parameter of the method (normally ``self`` for
instance methods, and ``cls`` for class methods). For static methods and normal
functions, ``<class>`` will be ``None``, resulting in a ``TypeError`` being
raised during the preamble.

Note: The relationship between ``super`` and ``__super__`` is similar to that
between ``import`` and ``__import__``.

Much of this was discussed in the thread of the python-dev list, "Fixing super
anyone?" [1]_.


Open Issues
-----------


Determining the class object to use
'''''''''''''''''''''''''''''''''''

The exact mechanism for associating the method with the defining class is not
specified in this PEP, and should be chosen for maximum performance. For
CPython, it is suggested that the class instance be held in a C-level variable
on the function object which is bound to one of ``NULL`` (not part of a class),
``Py_None`` (static method) or a class object (instance or class method).


Should ``super`` actually become a keyword?
'''''''''''''''''''''''''''''''''''''''''''

With this proposal, ``super`` would become a keyword to the same extent that
``None`` is a keyword. It is possible that further restricting the ``super``
name may simplify implementation, however some are against the actual
keyword-ization of super. The simplest solution is often the correct solution
and the simplest solution may well not be adding additional keywords to the
language when they are not needed. Still, it may solve other open issues.


Closed Issues
-------------

super used with __call__ attributes
'''''''''''''''''''''''''''''''''''

It was considered that it might be a problem that instantiating super instances
the classic way, because calling it would lookup the __call__ attribute and
thus try to perform an automatic super lookup to the next class in the MRO.
However, this was found to be false, because calling an object only looks up
the __call__ method directly on the object's type. The following example shows
this in action.

::

    class A(object):
        def __call__(self):
            return '__call__'
        def __getattribute__(self, attr):
            if attr == '__call__':
                return lambda: '__getattribute__'
    a = A()
    assert a() == '__call__'
    assert a.__call__() == '__getattribute__'

In any case, with the renaming of ``__builtin__.super`` to
``__builtin__.__super__`` this issue goes away entirely.


Reference Implementation
========================

It is impossible to implement the above specification entirely in Python. This
reference implementation has the following differences to the specification:

1. New ``super`` semantics are implemented using bytecode hacking.

2. Assignment to ``super`` is not a ``SyntaxError``. Also see point #4.

3. Classes must either use the metaclass ``autosuper_meta`` or inherit from
   the base class ``autosuper`` to acquire the new ``super`` semantics.

4. ``super`` is not an implicit local variable. In particular, for inner
   functions to be able to use the super instance, there must be an assignment
   of the form ``super = super`` in the method.

The reference implementation assumes that it is being run on Python 2.5+.

::

    #!/usr/bin/env python
    #
    # autosuper.py

    from array import array
    import dis
    import new
    import types
    import __builtin__
    __builtin__.__super__ = __builtin__.super
    del __builtin__.super

    # We need these for modifying bytecode
    from opcode import opmap, HAVE_ARGUMENT, EXTENDED_ARG

    LOAD_GLOBAL = opmap['LOAD_GLOBAL']
    LOAD_NAME = opmap['LOAD_NAME']
    LOAD_CONST = opmap['LOAD_CONST']
    LOAD_FAST = opmap['LOAD_FAST']
    LOAD_ATTR = opmap['LOAD_ATTR']
    STORE_FAST = opmap['STORE_FAST']
    LOAD_DEREF = opmap['LOAD_DEREF']
    STORE_DEREF = opmap['STORE_DEREF']
    CALL_FUNCTION = opmap['CALL_FUNCTION']
    STORE_GLOBAL = opmap['STORE_GLOBAL']
    DUP_TOP = opmap['DUP_TOP']
    POP_TOP = opmap['POP_TOP']
    NOP = opmap['NOP']
    JUMP_FORWARD = opmap['JUMP_FORWARD']
    ABSOLUTE_TARGET = dis.hasjabs

    def _oparg(code, opcode_pos):
        return code[opcode_pos+1] + (code[opcode_pos+2] << 8)

    def _bind_autosuper(func, cls):
        co = func.func_code
        name = func.func_name
        newcode = array('B', co.co_code)
        codelen = len(newcode)
        newconsts = list(co.co_consts)
        newvarnames = list(co.co_varnames)

        # Check if the global 'super' keyword is already present
        try:
     


id: pep-0373.rst
content:
PEP: 373
Title: Python 2.7 Release Schedule
Author: Benjamin Peterson <benjamin@python.org>
Status: Final
Type: Informational
Topic: Release
Created: 03-Nov-2008
Python-Version: 2.7


Abstract
========

This document describes the development and release schedule for
Python 2.7.

Python 2.7 is the end of the Python 2.x series, and is succeeded by
Python 3. See `the "Sunsetting Python 2" FAQ on python.org
<https://www.python.org/doc/sunset-python-2/>`_ for a general
overview.


Update (April 2014)
===================

The End Of Life date (EOL, sunset date) for Python 2.7 has been moved
five years into the future, to 2020.  This decision was made to
clarify the status of Python 2.7 and relieve worries for those users
who cannot yet migrate to Python 3.  See also :pep:`466`.

This declaration does not guarantee that bugfix releases will be made
on a regular basis, but it should enable volunteers who want to
contribute bugfixes for Python 2.7 and it should satisfy vendors who
still have to support Python 2 for years to come.

There will be no Python 2.8 (see :pep:`404`).


Release Manager and Crew
========================

============================ ==================
Position                     Name
============================ ==================
2.7 Release Manager          Benjamin Peterson
Windows installers           Steve Dower
Mac installers               Ned Deily
============================ ==================


Maintenance releases
====================

Being the last of the 2.x series, 2.7 received bugfix support until
2020. Support officially stopped January 1 2020, and 2.7.18 code
freeze occurred on January 1 2020, but the final release occurred
after that date.

Dates of previous maintenance releases:

- 2.7.1 2010-11-27
- 2.7.2 2011-07-21
- 2.7.3rc1 2012-02-23
- 2.7.3rc2 2012-03-15
- 2.7.3 2012-03-09
- 2.7.4rc1 2013-03-23
- 2.7.4 2013-04-06
- 2.7.5 2013-05-12
- 2.7.6rc1 2013-10-26
- 2.7.6 2013-11-10
- 2.7.7rc1 2014-05-17
- 2.7.7 2014-05-31
- 2.7.8 2014-06-30
- 2.7.9rc1 2014-11-26
- 2.7.9 2014-12-10
- 2.7.10rc1 2015-05-09
- 2.7.10 2015-05-23
- 2.7.11rc1 2015-11-21
- 2.7.11 2015-12-05
- 2.7.12 2016-06-25
- 2.7.13rc1 2016-12-03
- 2.7.13 2016-12-17
- 2.7.14rc1 2017-08-26
- 2.7.14 2017-09-16
- 2.7.15rc1 2018-04-14
- 2.7.15 2018-05-01
- 2.7.16rc 2019-02-16
- 2.7.16 2019-03-02
- 2.7.17rc1 2019-10-05
- 2.7.17 2019-10-19
- 2.7.18rc1 2020-04-04
- 2.7.18 2020-04-20

2.7.0 Release Schedule
======================

The release schedule for 2.7.0 was:

- 2.7 alpha 1 2009-12-05
- 2.7 alpha 2 2010-01-09
- 2.7 alpha 3 2010-02-06
- 2.7 alpha 4 2010-03-06
- 2.7 beta 1 2010-04-03
- 2.7 beta 2 2010-05-08
- 2.7 rc1 2010-06-05
- 2.7 rc2 2010-06-19
- 2.7 final 2010-07-03

Possible features for 2.7
=========================

Nothing here. [Note that a moratorium on core language changes is in effect.]


References
==========

- `"The Python 2 death march" on python-dev
  <https://mail.python.org/archives/list/python-dev@python.org/thread/APWHFYQDKNVYQAK3HZMBGQIZHAVRHCV2/>`_
- `Petition: abandon plans to ship a 2.7.18 in April <https://discuss.python.org/t/petition-abandon-plans-to-ship-a-2-7-18-in-april/2946/>`_
- `[RELEASE] Python 2.7.18, the end of an era <https://mail.python.org/archives/list/python-announce-list@python.org/thread/OFCIETIXLX34X7FVK5B5WPZH22HXV342/>`_


Copyright
=========

This document has been placed in the public domain.


id: pep-0415.rst
content:
PEP: 415
Title: Implement context suppression with exception attributes
Author: Benjamin Peterson <benjamin@python.org>
BDFL-Delegate: Alyssa Coghlan
Status: Final
Type: Standards Track
Created: 26-Feb-2012
Python-Version: 3.3
Post-History: 26-Feb-2012
Replaces: 409
Resolution: https://mail.python.org/pipermail/python-dev/2012-May/119467.html


Abstract
========

:pep:`409` introduced support for the ``raise exc from None`` construct to
allow the display of the exception context to be explicitly suppressed.
This PEP retains the language level changes already implemented in :pep:`409`,
but replaces the underlying implementation mechanism with a simpler approach
based on a new ``__suppress_context__`` attribute on all ``BaseException``
instances.


PEP Acceptance
==============

This PEP was accepted by Alyssa Coghlan on the 14th of May, 2012.


Rationale
=========

:pep:`409` changes ``__cause__`` to be ``Ellipsis`` by default. Then if
``__cause__`` is set to ``None`` by ``raise exc from None``, no context or cause
will be printed should the exception be uncaught.

The main problem with this scheme is it complicates the role of
``__cause__``. ``__cause__`` should indicate the cause of the exception not
whether ``__context__`` should be printed or not. This use of ``__cause__`` is
also not easily extended in the future. For example, we may someday want to
allow the programmer to select which of ``__context__`` and ``__cause__`` will
be printed. The :pep:`409` implementation is not amenable to this.

The use of ``Ellipsis`` is a hack. Before :pep:`409`, ``Ellipsis`` was used
exclusively in extended slicing. Extended slicing has nothing to do with
exceptions, so it's not clear to someone inspecting an exception object why
``__cause__`` should be set to ``Ellipsis``. Using ``Ellipsis`` by default for
``__cause__`` makes it asymmetrical with ``__context__``.


Proposal
========

A new attribute on ``BaseException``, ``__suppress_context__``, will
be introduced. Whenever ``__cause__`` is set, ``__suppress_context__``
will be set to ``True``. In particular, ``raise exc from cause``
syntax will set ``exc.__suppress_context__`` to ``True``. Exception
printing code will check for that attribute to determine whether
context and cause will be printed. ``__cause__`` will return to its
original purpose and values.

There is precedence for ``__suppress_context__`` with the
``print_line_and_file`` exception attribute.

To summarize, ``raise exc from cause`` will be equivalent to::

    exc.__cause__ = cause
    raise exc

where ``exc.__cause__ = cause`` implicitly sets
``exc.__suppress_context__``.


Patches
=======

There is a patch on `Issue 14133`_.


References
==========

.. _issue 14133:
   http://bugs.python.org/issue14133


Copyright
=========

This document has been placed in the public domain.


id: pep-0603.rst
content:
PEP: 603
Title: Adding a frozenmap type to collections
Author: Yury Selivanov <yury@edgedb.com>
Discussions-To: https://discuss.python.org/t/pep-603-adding-a-frozenmap-type-to-collections/2318/
Status: Draft
Type: Standards Track
Created: 12-Sep-2019
Post-History: 12-Sep-2019


Abstract
========

A *persistent data structure* is defined as a data structure that
preserves the previous version of the data when the data is modified.
Such data structures are effectively *immutable*, as operations on
them do not update the structure in-place, but instead always yield
a new updated structure (see [0]_ for more details.)

This PEP proposes to add a new fully persistent and immutable mapping
type called ``frozenmap`` to the ``collections`` module.

The bulk of ``frozenmap``'s reference implementation is already
used in CPython to implement the ``contextvars`` module.


Rationale
=========

Python has two immutable collection types: ``tuple`` and
``frozenset``.  These types can be used to represent immutable lists
and sets. However, a way to represent immutable *mappings* does not yet
exist, and this PEP proposes a ``frozenmap`` to implement an
immutable *mapping*.

The proposed ``frozenmap`` type:

* implements the ``collections.abc.Mapping`` protocol,
* supports pickling, and
* provides an API for efficient creation of "modified" versions.

The following use cases illustrate why an immutable mapping is
desirable:

* Immutable mappings are hashable which allows their use
  as dictionary keys or set elements.

  This hashable property permits functions decorated with
  ``@functools.lru_cache()`` to accept immutable mappings as
  arguments. Unlike an immutable mapping, passing a plain ``dict``
  to such a function results in error.

* Immutable mappings can hold complex state. Since immutable mappings
  can be copied by reference, transactional mutation of state can be
  efficiently implemented.

* Immutable mappings can be used to safely share dictionaries across
  thread and asynchronous task boundaries. The immutability makes it
  easier to reason about threads and asynchronous tasks.

Lastly, CPython [1]_ already contains the main portion of the C code
required for the ``frozenmap`` implementation.  The C code already
exists to implement the ``contextvars`` module (see :pep:`567` for
more details.) Exposing this C code via a public collection type
drastically increases the number of users of the code.  This leads to
increased code quality by discovering bugs and improving performance
which without a ``frozenmap`` collection would be very challenging
because most programs use the ``contextvars`` module indirectly.


Specification
=============

A new public immutable type ``frozenmap`` is added to the
``collections`` module.

Construction
------------

``frozenmap`` implements a ``dict``-like construction API:

* ``frozenmap()`` creates a new empty immutable mapping;

* ``frozenmap(**kwargs)`` creates a mapping from ``**kwargs``, e.g.
  ``frozenmap(x=10, y=0, z=-1)``

* ``frozenmap(collection)`` creates a mapping from the passed
  ``collection`` object. The passed ``collection`` object can be:

  - a ``dict``,
  - another ``frozenmap``,
  - an object with an ``items()`` method that is expected to return
    a series of key/value tuples, or
  - an iterable of key/value tuples.

Data Access
-----------

``frozenmap`` implements the ``collection.abc.Mapping`` protocol.
Therefore, getters, membership checks, and iteration work the same
way that they would for a ``dict``::

  m = frozenmap(foo='bar')

  assert m['foo'] == 'bar'
  assert m.get('foo') == 'bar'
  assert 'foo' in m

  assert 'baz' not in m
  assert m.get('baz', 'missing') == 'missing'

  assert m == m
  assert m != frozenmap()  # m is not equal to an empty frozenmap

  assert len(m) == 1

  # etc.

Mutation
--------

``frozenmap`` instances are immutable. That said, it is possible
to efficiently produce mutated *copies* of the immutable instance.

The complexity of mutation operations is O(log N) and the resulting
``frozenmap`` copies often consume very little additional memory due
to the use of structural sharing (read [6]_ for more details.)

frozenmap.including(key, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The method creates a new ``frozenmap`` copy with a new *key* / *value*
pair::

  m = frozenmap(foo=1)
  m2 = m.including('bar', 100)

  print(m)   # will print frozenmap({'foo': 1})
  print(m2)  # will print frozenmap({'foo': 1, 'bar': 100})

frozenmap.excluding(key)
^^^^^^^^^^^^^^^^^^^^^^^^

The method produces a copy of the ``frozenmap`` which does not
include a deleted *key*::

  m = frozenmap(foo=1, bar=100)

  m2 = m.excluding('foo')

  print(m)   # will print frozenmap({'foo': 1, 'bar': 100})
  print(m2)  # will print frozenmap({'bar': 1})

  m3 = m.excluding('spam')  # will throw a KeyError('spam')

frozenmap.union(mapping=None, \*\*kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The method produces a copy of the ``frozenmap`` and adds or modifies
multiple key/values for the created copy.  The signature of
the method matches the signature of the ``frozenmap`` constructor::

  m = frozenmap(foo=1)

  m2 = m.union({'spam': 'ham'})
  print(m2)  # will print frozenmap({'foo': 1, 'spam': 'ham'})

  m3 = m.union(foo=100, y=2)
  print(m3)  # will print frozenmap({'foo': 100, 'y': 2})

  print(m)   # will print frozenmap({'foo': 1})

Calling the ``union()`` method to add/replace N keys is more efficient
than calling the ``including()`` method N times.

frozenmap.mutating()
^^^^^^^^^^^^^^^^^^^^

The method allows efficient copying of a ``frozenmap`` instance with
multiple modifications applied.  This method is especially useful
when the frozenmap in question contains thousands of key/value pairs
and there's a need to update many of them in a performance-critical
section of the code.

The ``frozenmap.mutating()`` method returns a mutable dict-like
copy of the ``frozenmap`` object: an instance of
``collections.FrozenMapCopy``.

The ``FrozenMapCopy`` objects:

* are copy-on-write views of the data of ``frozenmap`` instances
  they were created from;

* are mutable, although any mutations on them do not affect the
  ``frozenmap`` instances they were created from;

* can be passed to the ``frozenmap`` constructor; creating a
  frozenmap from a ``FrozenMapCopy`` object is an O(1)
  operation;

* have O(log N) complexity for get/set operations; creating
  them is an O(1) operation;

* have a ``FrozenMapCopy.close()`` method that prevents any
  further access/mutation of the data;

* can be used as a context manager.

The below example illustrates how ``mutating()`` can be used with
a context manager::

  numbers = frozenmap((i, i ** 2) for i in range(1_000_000))

  with numbers.mutating() as copy:
      for i in numbers:
          if not (numbers[i] % 997):
              del copy[i]

      numbers_without_997_multiples = frozenmap(copy)

      # at this point, *numbers* still has 1_000_000 key/values, and
      # *numbers_without_997_multiples* is a copy of *numbers* without
      # values that are multiples of 997.

      for i in numbers:
          if not (numbers[i] % 593):
              del copy[i]

      numbers_without_593_multiples = frozenmap(copy)

      print(copy[10])  # will print 100.

  print(copy[10])  # This will throw a ValueError as *copy*
                   # has been closed when the "with" block
                   # was executed.

Iteration
---------

As ``frozenmap`` implements the standard ``collections.abc.Mapping``
protocol, so all expected methods of iteration are supported::

  assert list(m) == ['foo']
  assert list(m.items()) == [('foo', 'bar')]
  assert list(m.keys()) == ['foo']
  assert list(m.values()) == ['bar']

Iteration in ``frozenmap``, unlike in ``dict``, does not preserve the
insertion order.

Hashing
-------

``frozenmap`` instances can be hashable just like ``tuple`` objects::

  hash(frozenmap(foo='bar'))  # works
  hash(frozenmap(foo=[]))     # will throw an error


id: pep-0617.rst
content:
PEP: 617
Title: New PEG parser for CPython
Author: Guido van Rossum <guido@python.org>,
        Pablo Galindo <pablogsal@python.org>,
        Lysandros Nikolaou <lisandrosnik@gmail.com>
Discussions-To: python-dev@python.org
Status: Final
Type: Standards Track
Created: 24-Mar-2020
Python-Version: 3.9
Post-History: 02-Apr-2020

.. canonical-doc:: :ref:`python:full-grammar-specification`

.. highlight:: PEG

========
Overview
========

This PEP proposes replacing the current LL(1)-based parser of CPython
with a new PEG-based parser. This new parser would allow the elimination of multiple
"hacks" that exist in the current grammar to circumvent the LL(1)-limitation.
It would substantially reduce the maintenance costs in some areas related to the
compiling pipeline such as the grammar, the parser and the AST generation. The new PEG
parser will also lift the LL(1) restriction on the current Python grammar.

===========================
Background on LL(1) parsers
===========================

The current Python grammar is an LL(1)-based grammar. A grammar can be said to be
LL(1) if it can be parsed by an LL(1) parser, which in turn is defined as a
top-down parser that parses the input from left to right, performing leftmost
derivation of the sentence, with just one token of lookahead.
The traditional approach to constructing or generating an LL(1) parser is to
produce a *parse table* which encodes the possible transitions between all possible
states of the parser. These tables are normally constructed from the *first sets*
and the *follow sets* of the grammar:

* Given a rule, the *first set* is the collection of all terminals that can occur
  first in a full derivation of that rule. Intuitively, this helps the parser decide
  among the alternatives in a rule. For
  instance, given the rule::

      rule: A | B

  if only ``A`` can start with the terminal *a* and only ``B`` can start with the
  terminal *b* and the parser sees the token *b* when parsing this rule, it knows
  that it needs to follow the non-terminal ``B``.

* An extension to this simple idea is needed when a rule may expand to the empty string.
  Given a rule, the *follow set* is the collection of terminals that can appear
  immediately to the right of that rule in a partial derivation. Intuitively, this
  solves the problem of the empty alternative. For instance,
  given this rule::

    rule: A 'b'

  if the parser has the token *b* and the non-terminal ``A`` can only start
  with the token *a*, then the parser can tell that this is an invalid program.
  But if ``A`` could expand to the empty string (called an Îµ-production),
  then the parser would recognise a valid empty ``A``,
  since the next token *b* is in the *follow set*  of ``A``.


The current Python grammar does not contain Îµ-productions, so the *follow sets* are not
needed when creating the parse tables. Currently, in CPython, a parser generator
program reads the grammar and produces a parsing table representing a set of
deterministic finite automata (DFA) that can be included in a C program, the
parser. The parser is a pushdown automaton that uses this data to produce a Concrete
Syntax Tree (CST) sometimes known directly as a "parse tree". In this process, the
*first sets* are used indirectly when generating the DFAs.

LL(1) parsers and grammars are usually efficient and simple to implement
and generate. However, it is not possible, under the LL(1) restriction,
to express certain common constructs in a way natural to the language
designer and the reader. This includes some in the Python language.

As LL(1) parsers can only look one token ahead to distinguish
possibilities, some rules in the grammar may be ambiguous. For instance the rule::

    rule: A | B

is ambiguous if the *first sets* of both ``A`` and ``B`` have some elements in
common. When the parser sees a token in the input
program that both *A* and *B* can start with, it is impossible for it to deduce
which option to expand, as no further token of the program can be examined to
disambiguate.
The rule may be transformed to equivalent LL(1) rules, but then it may
be harder for a human reader to grasp its meaning.
Examples later in this document show that the current LL(1)-based
grammar suffers a lot from this scenario.

Another broad class of rules precluded by LL(1) is left-recursive rules.
A rule is left-recursive if it can derive to a
sentential form with itself as the leftmost symbol. For instance this rule::

    rule: rule 'a'

is left-recursive because the rule can be expanded to an expression that starts
with itself. As will be described later, left-recursion is the natural way to
express certain desired language properties directly in the grammar.

=========================
Background on PEG parsers
=========================

A PEG (Parsing Expression Grammar) grammar differs from a context-free grammar
(like the current one) in the fact that the way it is written more closely
reflects how the parser will operate when parsing it. The fundamental technical
difference is that the choice operator is ordered. This means that when writing::

  rule: A | B | C

a context-free-grammar parser (like an LL(1) parser) will generate constructions
that given an input string will *deduce* which alternative (``A``, ``B`` or ``C``)
must be expanded, while a PEG parser will check if the first alternative succeeds
and only if it fails, will it continue with the second or the third one in the
order in which they are written. This makes the choice operator not commutative.

Unlike LL(1) parsers, PEG-based parsers cannot be ambiguous: if a string parses,
it has exactly one valid parse tree. This means that a PEG-based parser cannot
suffer from the ambiguity problems described in the previous section.

PEG parsers are usually constructed as a recursive descent parser in which every
rule in the grammar corresponds to a function in the program implementing the
parser and the parsing expression (the "expansion" or "definition" of the rule)
represents the "code" in said function. Each parsing function conceptually takes
an input string as its argument, and yields one of the following results:

* A "success" result. This result indicates that the expression can be parsed by
  that rule and the function may optionally move forward or consume one or more
  characters of the input string supplied to it.
* A "failure" result, in which case no input is consumed.

Notice that "failure" results do not imply that the program is incorrect or a
parsing failure because as the choice operator is ordered, a "failure" result
merely indicates "try the following option". A direct implementation of a PEG
parser as a recursive descent parser will present exponential time performance in
the worst case as compared with LL(1) parsers, because PEG parsers have infinite lookahead
(this means that they can consider an arbitrary number of tokens before deciding
for a rule). Usually, PEG parsers avoid this exponential time complexity with a
technique called "packrat parsing" [1]_ which not only loads the entire
program in memory before parsing it but also allows the parser to backtrack
arbitrarily. This is made efficient by memoizing the rules already matched for
each position. The cost of the memoization cache is that the parser will naturally
use more memory than a simple LL(1) parser, which normally are table-based. We
will explain later in this document why we consider this cost acceptable.

=========
Rationale
=========

In this section, we describe a list of problems that are present in the current parser
machinery in CPython that motivates the need for a new parser.

---------------------------------
Some rules are not actually LL(1)
---------------------------------

Although the Python grammar is technically an LL(1) grammar (because it is parsed by
an LL(1) parser) several rules are not LL(1) and several workarounds are
implemented in the grammar and in other parts of CPython to deal with this. For
e


id: pep-0788.rst
content:
PEP: 788
Title: Reimagining native threads
Author: Peter Bierma <zintensitydev@gmail.com>
Sponsor: Victor Stinner <vstinner@python.org>
Discussions-To: https://discuss.python.org/t/89863
Status: Draft
Type: Standards Track
Created: 23-Apr-2025
Python-Version: 3.15
Post-History: `10-Mar-2025 <https://discuss.python.org/t/83959>`__,
              `27-Apr-2025 <https://discuss.python.org/t/89863>`__,


Abstract
========

:c:func:`PyGILState_Ensure`, :c:func:`PyGILState_Release`, and other related
functions in the ``PyGILState`` family are the most common way to create
native threads that interact with Python. They have been the standard for over
twenty years (:pep:`311`). But, over time, these functions have
become problematic:

- They aren't safe for finalization, either causing the calling thread to hang or
  crashing it with a segmentation fault, preventing further execution.
- When they're called before finalization, they force the thread to be
  "daemon", meaning that an interpreter won't wait for it to reach any point
  of execution. This is mostly frustrating for developers, but can lead to
  deadlocks!
- Subinterpreters don't play nicely with them, because they all assume that
  the main interpreter is the only one that exists. A fresh thread (that is,
  has never had a thread state) that calls :c:func:`PyGILState_Ensure` will
  always be for the main interpreter.
- The term "GIL" in the name is quite confusing for users of free-threaded
  Python. There isn't a GIL, why do they still have to call it?

This PEP intends to fix all of these issues by providing two new functions,
:c:func:`PyThreadState_Ensure` and :c:func:`PyThreadState_Release`, as a more
correct and safer replacement for :c:func:`PyGILState_Ensure` and
:c:func:`PyGILState_Release`. For example:

.. code-block:: c

    if (PyThreadState_Ensure(interp) < 0) {
        fputs("Python is shutting down", stderr);
        return;
    }

    /* Interact with Python, without worrying about finalization. */
    // ...

    PyThreadState_Release();

This is achieved by introducing two concepts into the C API:

-  "Daemon" and "non-daemon" threads, similar to how it works in the
   :mod:`threading` module.
-  Interpreter reference counts which prevent an interpreter from finalizing.

In :c:func:`PyThreadState_Ensure`, both of these ideas are applied. The
calling thread is to store a reference to an interpreter via
:c:func:`PyInterpreterState_Hold`. :c:func:`PyInterpreterState_Hold`
increases the reference count of an interpreter, requiring the thread
to finish (by eventually calling :c:func:`PyThreadState_Release`) before
beginning finalization.

For example, creating a native thread with this API would look something
like this:

.. code-block:: c

    static PyObject *
    my_method(PyObject *self, PyObject *unused)
    {
        PyThread_handle_t handle;
        PyThead_indent_t indent;

        PyInterpreterState *interp = PyInterpreterState_Hold();
        if (PyThread_start_joinable_thread(thread_func, interp, &ident, &handle) < 0) {
            PyInterpreterState_Release(interp);
            return NULL;
        }
        /* The thread will always attach and finish, because we increased
           the reference count of the interpreter. */
        Py_RETURN_NONE;
    }

Motivation
==========

Native threads will always hang during finalization
---------------------------------------------------

Many codebases might need to call Python code in highly-asynchronous
situations where the interpreter is already finalizing, or might finalize, and
want to continue running code after the Python call. This desire has been
`brought up by users <https://discuss.python.org/t/78850/>`_.
For example, a callback that wants to call Python code might be invoked when:

- A kernel has finished running on a GPU.
- A network packet was received.
- A thread has quit, and a native library is executing static finalizers of
  thread local storage.

In the current C API, any non-Python thread (one not created via the
:mod:`threading` module) is considered to be "daemon", meaning that the interpreter
won't wait on that thread to finalize. Instead, the interpreter will hang the
thread when it goes to :term:`attach <attached thread state>` a :term:`thread state`,
making it unusable past that point. Attaching a thread state can happen at
any point when invoking Python, such as releasing the GIL in-between bytecode
instructions, or when a C function exits a :c:macro:`Py_BEGIN_ALLOW_THREADS`
block. (Note that hanging the thread is relatively new behavior; in prior
versions, the thread would terminate, but the issue is the same.)

This means that any non-Python thread may be terminated at any point, which
is severely limiting for users who want to do more than just execute Python
code in their stream of calls (for example, C++ executing finalizers in
*addition* to calling Python).

Using ``Py_IsFinalizing`` is insufficient
*****************************************

The :ref:`docs <python:gilstate>`
currently recommend :c:func:`Py_IsFinalizing` to guard against termination of
the thread:

    Calling this function from a thread when the runtime is finalizing will
    terminate the thread, even if the thread was not created by Python. You
    can use ``Py_IsFinalizing()`` or ``sys.is_finalizing()`` to check if the
    interpreter is in process of being finalized before calling this function
    to avoid unwanted termination.

Unfortunately, this isn't correct, because of time-of-call to time-of-use
issues; the interpreter might not be finalizing during the call to
:c:func:`Py_IsFinalizing`, but it might start finalizing immediately afterwards, which
would cause the attachment of a thread state (typically via
:c:func:`PyGILState_Ensure`) to hang the thread.

Daemon threads can cause finalization deadlocks
***********************************************

When acquiring locks, it's extremely important to detach the thread state to
prevent deadlocks. This is true on both the with-GIL and free-threaded builds.
When the GIL is enabled, a deadlock can occur pretty easily when acquiring a
lock if the GIL wasn't released, and lock-ordering deadlocks can still occur
free-threaded builds if the thread state wasn't detached.

So, all code that needs to work with locks need to detach the thread state.
In C, this is almost always done via :c:macro:`Py_BEGIN_ALLOW_THREADS` and
:c:macro:`Py_END_ALLOW_THREADS`, in a code block that looks something like this:

.. code-block:: c

    Py_BEGIN_ALLOW_THREADS
    acquire_lock();
    Py_END_ALLOW_THREADS

Again, in a daemon thread, :c:macro:`Py_END_ALLOW_THREADS` will hang the thread
if the interpreter is finalizing. But, :c:macro:`Py_BEGIN_ALLOW_THREADS` will
*not* hang the thread; the lock will be acquired, and *then* the thread will
be hung! Once that happens, nothing can try to acquire that lock without
deadlocking. The main thread will continue to run finalizers past that point,
though. If any of those finalizers try to acquire the lock, deadlock ensues.

This affects CPython itself, and there's not much that can be done
to fix it. For example, `python/cpython#129536 <https://github.com/python/cpython/issues/129536>`_
remarks that the :mod:`ssl` module can emit a fatal error when used at
finalization, because a daemon thread got hung while holding the lock. There
are workarounds for this for pure-Python code, but native threads don't have
such an option.

.. _pep-788-hanging-compat:

We can't change finalization behavior for ``PyGILState_Ensure``
***************************************************************

There will always have to be a point in a Python program where
:c:func:`PyGILState_Ensure` can no longer acquire the GIL (or more correctly,
attach a thread state). If the interpreter is long dead, then Python
obviously can't give a thread a way to invoke it.
:c:func:`PyGILState_Ensure` doesn't have any meaningful way to return a
failure, so it has no choice but to terminate the th


id: pep-0011.rst
content:
PEP: 11
Title: CPython platform support
Author: Martin von LÃ¶wis <martin@v.loewis.de>,
        Brett Cannon <brett@python.org>
Status: Active
Type: Process
Created: 07-Jul-2002
Post-History: `18-Aug-2007 <https://mail.python.org/archives/list/python-dev@python.org/thread/DSSGXU5LBCMKYMZBRVB6RF3YAB6ST5AV/>`__,
              `14-May-2014 <https://mail.python.org/archives/list/python-dev@python.org/thread/T7WTUJ6TD3IGYGWV3M4PHJWNLM2WPZAW/>`__,
              `20-Feb-2015 <https://mail.python.org/archives/list/python-dev@python.org/thread/OEQHRR2COYZDL6LZ42RBZOMIUB32WI34/>`__,
              `10-Mar-2022 <https://mail.python.org/archives/list/python-committers@python.org/thread/K757345KX6W5ZLTWYBUXOXQTJJTL7GW5/>`__,


Abstract
========

This PEP documents how an operating system (platform) becomes
supported in CPython, what platforms are currently supported, and
documents past support.


Rationale
=========

Over time, the CPython source code has collected various pieces of
platform-specific code, which, at some point in time, was
considered necessary to use CPython on a specific platform.
Without access to this platform, it is not possible to determine
whether this code is still needed.  As a result, this code may
either break during CPython's evolution, or it may become
unnecessary as the platforms evolve as well.

Allowing these fragments to grow poses the risk of
unmaintainability: without having experts for a large number of
platforms, it is not possible to determine whether a certain
change to the CPython source code will work on all supported
platforms.

To reduce this risk, this PEP specifies what is required for a
platform to be considered supported by CPython as well as providing a
procedure to remove code for platforms with few or no CPython
users.

This PEP also lists what platforms *are* supported by the CPython
interpreter. This lets people know what platforms are directly
supported by the CPython development team.


Support tiers
=============

Platform support is broken down into *tiers*. Each tier comes with
different requirements which lead to different promises being made
about support.

To be promoted to a tier, steering council support is required and is
expected to be driven by team consensus. Demotion to a lower tier
occurs when the requirements of the current tier are no longer met for
a platform for an extended period of time based on the judgment of
the release manager or steering council. For platforms which no longer
meet the requirements of any tier by b1 of a new feature release, an
announcement will be made to warn the community of the pending removal
of support for the platform (e.g. in the b1 announcement). If the
platform is not brought into line for at least one of the tiers by the
first release candidate, it will be listed as unsupported in this PEP.

Tier 1
------


- `STATUS <https://github.com/python/cpython/actions/workflows/build.yml?query=branch%3Amain+is%3Acompleted>`__
- CI failures block releases.
- Changes which would break the ``main`` branch are not allowed to be merged;
  any breakage should be fixed or reverted immediately.
- All core developers are responsible to keep ``main``, and thus these
  platforms, working.
- Failures on these platforms **block a release**.

========================= =====
Target Triple             Notes
========================= =====
aarch64-apple-darwin      clang
aarch64-unknown-linux-gnu glibc, gcc
i686-pc-windows-msvc
x86_64-pc-windows-msvc
x86_64-apple-darwin       BSD libc, clang
x86_64-unknown-linux-gnu  glibc, gcc
========================= =====


Tier 2
------

- `STATUS <https://buildbot.python.org/all/#/builders?tags=%2B3.x&tags=%2Btier-2>`__
- Must have a reliable buildbot.
- At least **two** core developers are signed up to support the platform.
- Changes which break any of these platforms are to be **fixed or
  reverted within 24 hours**.
- Failures on these platforms **block a release**.

============================= ========================== ========
Target Triple                 Notes                      Contacts
============================= ========================== ========
aarch64-unknown-linux-gnu     glibc, clang               Victor Stinner, Gregory P. Smith
wasm32-unknown-wasip1         WASI SDK, Wasmtime         Brett Cannon, Eric Snow
x86_64-unknown-linux-gnu      glibc, clang               Victor Stinner, Gregory P. Smith
============================= ========================== ========


Tier 3
------

- `STATUS <https://buildbot.python.org/all/#/builders?tags=%2B3.x&tags=%2Btier-3>`__
- Must have a reliable buildbot.
- At least **one** core developer is signed up to support the platform.
- No response SLA to failures.
- Failures on these platforms do **not** block a release.

================================ =========================== ========
Target Triple                    Notes                       Contacts
================================ =========================== ========
aarch64-linux-android                                        Russell Keith-Magee, Petr Viktorin
aarch64-pc-windows-msvc                                      Steve Dower
arm64-apple-ios                  iOS on device               Russell Keith-Magee, Ned Deily
arm64-apple-ios-simulator        iOS on M1 macOS simulator   Russell Keith-Magee, Ned Deily
armv7l-unknown-linux-gnueabihf   Raspberry Pi OS, glibc, gcc Gregory P. Smith
powerpc64le-unknown-linux-gnu    glibc, clang                Victor Stinner

                                 glibc, gcc                  Victor Stinner
s390x-unknown-linux-gnu          glibc, gcc                  Victor Stinner
x86_64-linux-android                                         Russell Keith-Magee, Petr Viktorin
x86_64-unknown-freebsd           BSD libc, clang             Victor Stinner
================================ =========================== ========


All other platforms
-------------------

Support for a platform may be partial within the code base, such as
from active development around platform support or accidentally.
Code changes to platforms not listed in the above tiers may be rejected
or removed from the code base without a deprecation process if they
cause a maintenance burden or obstruct general improvements.

Platforms not listed here may be supported by the wider Python
community in some way. If your desired platform is not listed above,
please perform a search online to see if someone is already providing
support in some form.


Notes
-----

Microsoft Windows
'''''''''''''''''

Windows versions prior to Windows 10 follow Microsoft's `Fixed Lifecycle Policy
<https://learn.microsoft.com/en-us/lifecycle/policies/fixed>`__,
with a mainstream support phase for 5 years after release,
where the product is generally commercially available,
and an additional 5 year extended support phase,
where paid support is still available and certain bug fixes are released.
`Extended Security Updates (ESU)
<https://learn.microsoft.com/en-us/lifecycle/faq/extended-security-updates>`_
is a paid program available to high-volume enterprise customers
as a "last resort" option to receive certain security updates after extended support ends.
ESU is considered a distinct phase that follows the expiration of extended support.

Windows 10 and later follow Microsoft's `Modern Lifecycle Policy
<https://learn.microsoft.com/en-us/lifecycle/policies/modern>`__,
which varies per-product, per-version, per-edition and per-channel.
Generally, feature updates (1709, 22H2) occur every 6-12 months
and are supported for 18-36 months;
Server and IoT editions, and LTSC channel releases are supported for 5-10 years,
and the latest feature release of a major version (Windows 10, Windows 11)
generally receives new updates for at least 10 years following release.
Microsoft's `Windows Lifecycle FAQ
<https://learn.microsoft.com/en-us/lifecycle/faq/windows>`_
has more specific and up-to-date guidance.

CPython's Windows support currently follows Microsoft's lifecycles


id: pep-0777.rst
content:
PEP: 777
Title: How to Re-invent the Wheel
Author: Emma Harper Smith <emma@python.org>
Sponsor: Barry Warsaw <barry@python.org>
PEP-Delegate: Paul Moore <p.f.moore@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-777-how-to-re-invent-the-wheel/67484
Status: Draft
Type: Standards Track
Topic: Packaging
Created: 09-Oct-2024
Post-History: `10-Oct-2024 <https://discuss.python.org/t/pep-777-how-to-re-invent-the-wheel/67484>`__

Abstract
========

The current :pep:`wheel 1.0 specification <427>` was written over a decade ago,
and has been extremely robust to changes in the Python packaging ecosystem.
Previous efforts to improve the wheel specification
:pep:`were deferred <491#pep-deferral>` to focus on other packaging
specifications. Meanwhile, the use of wheels has changed dramatically in the
last decade. There have been many requests for new wheel features over the
years; however, a fundamental obstacle to evolving the wheel specification has
been that there is no defined process for how to handle adding
backwards-incompatible features to wheels. Therefore, to enable other PEPs to
describe new enhancements to the wheel specification, **this PEP prescribes**
**compatibility requirements on future wheel revisions**. This PEP does *not*
specify a new wheel revision. The specification of a new wheel format
(âWheel 2.0â) is left to a future PEP.

Rationale
=========

Currently, wheel specification changes that require new installer behavior are backwards incompatible and require a major version increase in
the wheel metadata format. An increase of the wheel major version has yet to
happen, partially because such a change has the potential to be
catastrophically disruptive. Per
`the wheel specification <https://packaging.python.org/en/latest/specifications/binary-distribution-format/#installing-a-wheel-distribution-1-0-py32-none-any-whl>`_,
any installer that does not support the new major version must abort at install
time. This means that if the major version were to be incremented without
further planning, many users would see installation failures as older installers reject new wheels
uploaded to public package indices like the Python Package Index (PyPI). It is
critically important to carefully plan the interactions between build tools,
package indices, and package installers to avoid incompatibility issues,
especially considering the long tail of users who are slow to update their
installers.

The backward compatibility concerns have prevented valuable improvements
to the wheel file format, such as
`better compression <https://discuss.python.org/t/improving-wheel-compression-by-nesting-data-as-a-second-zip/1747>`_,
`wheel data format improvements <https://discuss.python.org/t/should-there-be-a-new-standard-for-installing-arbitrary-data-files/7853/7>`_,
`better information about what is included in a wheel <https://discuss.python.org/t/record-the-top-level-names-of-a-wheel-in-metadata/29494>`_,
and `JSON formatted metadata in the ".dist-info" folder <https://discuss.python.org/t/is-was-there-a-goal-with-pep-566s-json-encoding-section/12324/3>`_.

This PEP describes constraints and behavior for new wheel revisions to preserve
stability for existing tools that do not support a new major version of the wheel format.
This ensures that backwards incompatible changes to the wheel specification
will only affect users and tools that are properly set up to use the newer
wheels. With a clear path for evolving the wheel specification, future PEPs
will be able to improve the wheel format without needing to re-define a
completely new compatibility story.

Specification
=============

Add Wheel-Version Metadata Field to Core Metadata
-------------------------------------------------

Currently, the :pep:`wheel 1.0 PEP <427>`, PEP 427, specifies that wheel files
must contain a ``WHEEL`` metadata file that contains the version of the wheel
specification that the file conforms to. PEP 427 stipulates that installers
MUST warn on installation of a wheel with a minor version greater than supported,
and MUST abort on installation of wheels with a major version that is greater than
what the installer supports. This ensures that users do not get invalid
installations from wheels that installers cannot properly install.

However, resolvers do not currently exclude wheels with an incompatible wheel
version. There is also currently no way for a resolver to check a wheel's
version without downloading the wheel directly. To make wheel version filtering
easy for resolvers, the wheel version **MUST** be included in the relevant
metadata file (currently ``METADATA``). This will allow resolvers to
efficiently check the wheel version using the :pep:`658` metadata API without
needing to download and inspect the ``.dist-info/WHEEL`` file.

To accomplish this, a new field, ``Wheel-Version``, will be added to the
`Core Metadata Specification <https://packaging.python.org/specifications/core-metadata/>`_.
This field is single use, and must contain the exact same version specified as
the ``Wheel-Version`` entry in the ``WHEEL`` file, or any future replacement
file defining metadata about the wheel file. If ``Wheel-Version`` is absent
from the metadata file, then tools **MUST** infer the wheel file major
version as 1.

``Wheel-Version`` **MUST NOT** be included in source distribution metadata
(``PKG-INFO``) files. If a tool encounters ``Wheel-Version`` inside of a source
distribution metadata file, it **SHOULD** raise an error.

``Wheel-Version`` **MAY** be included in the metadata file for wheels of
version 1, but for wheels of version 2 or higher, the metadata file **MUST**
include ``Wheel-Version``. This enforces that future revisions of the wheel
specification can rely on resolvers skipping incompatible wheels by checking
the ``Wheel-Version`` field. Build backends are encouraged to include
``Wheel-Version`` in all wheels that they generate, regardless of version.

Installers **SHOULD** copy the metadata file in a wheel unmodified during
installation. This prevents the need to update the ``RECORD`` file, which is
an error prone process. Tools reading installed core metadata **SHOULD NOT**
assume that the field is present, as other installation formats may omit it.

When installing a wheel, installers **MUST** take the following steps:

1. Check that the values of ``Wheel-Version`` in both the core metadata file
   and wheel metadata file match. If they do not match, installers **MUST**
   abort installation. Neither value takes precedence.
2. Check that the installer is compatible with ``Wheel-Version``. If
   ``Wheel-Version`` is absent, assume the version is 1.0. Warn if minor
   version is greater, abort if major version is greater. This procedure is
   identital to that in :pep:`427`.
3. Proceed with installation as specified in the
   `Binary Distribution Format <https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format>`_
   specification.

Resolver Behavior Regarding ``Wheel-Version``
---------------------------------------------

Resolvers, in the process of selecting a wheel to install, **MUST** check a
candidate wheel's ``Wheel-Version``, and ignore incompatible wheel files.
Without ignoring these files, older installers might select a wheel that uses
an unsupported wheel version for that installer, and force the installer to
abort per :pep:`427`. By skipping incompatible wheel files, users will not see
installation errors when a project adopts a new wheel major version. As already
specified in PEP 427, installers **MUST** abort if a user tries to directly
install a wheel that is incompatible. If, in the process of resolving packages
found in multiple indices, a resolver comes across two wheels of the same
distribution and version, resolvers should prioritize the wheel of the highest
compatible version.

While the above protects users from unexpected breakages, users may miss a new
release of a distribution if the


id: pep-0763.rst
content:
PEP: 763
Title: Limiting deletions on PyPI
Author: William Woodruff <william@yossarian.net>,
        Alexis Challande <alexis.challande@trailofbits.com>
Sponsor: Donald Stufft <donald@stufft.io>
PEP-Delegate: Donald Stufft <donald@stufft.io>
Discussions-To: https://discuss.python.org/t/69487
Status: Draft
Type: Standards Track
Topic: Packaging
Created: 24-Oct-2024
Post-History: `09-Jul-2022 <https://discuss.python.org/t/17227>`__,
              `01-Oct-2024 <https://discuss.python.org/t/66351>`__,
              `28-Oct-2024 <https://discuss.python.org/t/69487>`__

Abstract
========

We propose limiting when users can delete files, releases, and projects
from PyPI. A project, release, or file may only be deleted within 72 hours
of when it is uploaded to the index. From this point, users may only use
the "yank" mechanism specified by :pep:`592`.

An exception to this restriction is made for releases and files that are
marked with :ref:`pre-release specifiers <packaging:version-specifiers>`,
which will remain deletable at any time.
The PyPI administrators will retain the ability to delete files, releases,
and projects at any time, for example for moderation or security purposes.

Rationale and Motivation
========================

As observed in :pep:`592`, user-level deletion of projects on PyPI
enables a catch-22 situation of dependency breakage:

    Whenever a project detects that a particular release on PyPI might be
    broken, they oftentimes will want to prevent further users from
    inadvertently using that version. However, the obvious solution of
    deleting the existing file from a repository will break users who have
    pinned to a specific version of the project.

    This leaves projects in a catch-22 situation where new projects may be pulling
    down this known broken version, but if they do anything to prevent that theyâll
    break projects that are already using it.

On a technical level, the problem of deletion is mitigated by
"yanking," also specified in :pep:`592`. However, deletions continue to be
allowed on PyPI, and have caused multiple notable disruptions to the Python
ecosystem over the interceding years:

* July 2022: `atomicwrites <https://pypi.org/project/atomicwrites/>`_
  was `deleted by its maintainer <https://github.com/untitaker/python-atomicwrites/issues/61>`_
  in an attempt to remove the project's "critical" designation, without the
  maintainer realizing that project deletion would also delete all previously
  uploaded releases.

  The project was subsequently restored with the maintainer's consent,
  but at the cost of manual administrator action and extensive downstream
  breakage to projects like `pytest <https://github.com/pytest-dev/pytest/issues/10114>`_.
  As of October 2024, atomicwrites is archived but still has
  around `4.5 million monthly downloads from PyPI <https://pypistats.org/packages/atomicwrites>`_.

* April 2023: `codecov <https://pypi.org/project/codecov/>`_ was deleted by
  its maintainers after a long deprecation period. This caused extensive
  breakage for many of Codecov's CI/CD users, who were unaware of the
  deprecation period due to limited observability of deprecation warnings
  within CI/CD logs.

  The project was
  `subsequently re-created <https://about.codecov.io/blog/message-regarding-the-pypi-package/>`_
  by its maintainers, with a new release published to compensate for the deleted releases
  (which were not restored), meaning that any pinned installations remained
  broken. As of October 2024, this single release remains the only release on
  PyPI and has around
  `1.5 million monthly downloads <https://pypistats.org/packages/codecov>`_.

* June 2023: `python-sonarqube-api <https://pypi.org/project/python-sonarqube-api/>`_
  deleted all released releases prior to 2.0.2.

  The project's maintainer subsequently
  `deleted conversations <https://discuss.python.org/t/stop-allowing-deleting-things-from-pypi/17227/114>`_
  and force-pushed over the tag history for ``python-sonarqube-api``'s source
  repository, impeding efforts by users to compare changes between
  releases.

* June 2024: `PySimpleGUI <https://pypi.org/project/PySimpleGUI/>`_ changed
  licenses and deleted
  `nearly all previous releases <https://discuss.python.org/t/48790/27>`_.
  This resulted in widespread disruption for users, who (prior to the
  relicensing) were downloading PySimpleGUI approximately 25,000 times a day.

In addition to their disruptive effect on downstreams, deletions
also have detrimental effects on PyPI's sustainability as well as the overall
security of the ecosystem:

* Deletions increase support workload for PyPI's administrators and
  moderators, as users mistakenly file support requests believing that PyPI
  is broken, or that the administrators themselves have removed the
  project.

* Deletions impair external (meaning end-user) incident response and analysis,
  making it difficult to distinguish "good faith" maintainer behavior from
  a malicious actor attempting the cover their tracks.

The Python ecosystem is continuing to grow,
meaning that future deletions of projects can be reasonably assumed to
be *just, as if not more,* disruptive than the deletions sampled above.

Given all of the above, this PEP concludes that deletions now present a greater
risk and detriment to the Python ecosystem than a benefit.

In addition to these technical arguments, there is also precedent
from other packaging ecosystems for limiting the ability of users to delete
projects and their constituent releases. This precedent is documented in
:ref:`Appendix A <pep763-appendix-a>`.

Specification
=============

There are three different types of deletable objects:

1. **Files**, which are individual project distributions (such as source
   distributions or wheels).

   Example: ``requests-2.32.3-py3-none-any.whl``.

2. **Releases**, which contain one or more files that share the same version
   number.

   Example: `requests v2.32.3 <https://pypi.org/project/requests/2.32.3/>`_.

3. **Projects**, which contain one or more releases.

   Example: `requests <https://pypi.org/project/requests>`_.

Deletion eligibility rules
--------------------------

This PEP proposes the following *deletion eligibility rules*:

* A **file** is deletable if and only if it was uploaded to
  PyPI less than 72 hours from the current time, **or** if it
  has a :ref:`pre-release specifier <packaging:version-specifiers>`.
* A **release** is deletable if and only if all of its
  contained files are deletable.
* A **project** is deletable if and only if all of its releases are deletable.

These rules allow new projects to be
deleted entirely, and allow old projects to delete new files or releases,
but do not allow old projects to delete old files or releases.

Implementation
==============

This PEP's implementation primarily concerns aspects of PyPI that are not
standardized or subject to standardization, such as the web interface and
signed-in user operations. As a result, this section describes its
implementation in behavioral terms.

Changes
-------

* Per the eligibility rules above, PyPI will reject web interface requests
  (using an appropriate HTTP response code of its choosing) for
  file, release, or project deletion if the respective object is not
  eligible for deletion.
* PyPI will amend its web interface to indicate a file/release/project's
  deletion ineligibility, e.g. by styling the relevant UI elements as "inactive"
  and making relevant bottoms/forms unclickable.

Security Implications
=====================

This PEP does not identify negative security implications associated with the
proposed approach.

This PEP identifies one minor positive security implication: by restricting
user-controlled deletions, this PEP makes it more difficult for a malicious
actor to cover their tracks by deleting malware from the index. This is
particularly useful for external (i.e. non-PyPI administrator) triage and
incident r


id: pep-0005.rst
content:
PEP: 5
Title: Guidelines for Language Evolution
Author: Paul Prescod <paul@prescod.net>
Status: Superseded
Type: Process
Created: 26-Oct-2000
Post-History:
Superseded-By: 387


Abstract
========

In the natural evolution of programming languages it is sometimes
necessary to make changes that modify the behavior of older programs.
This PEP proposes a policy for implementing these changes in a manner
respectful of the installed base of Python users.


Implementation Details
======================

Implementation of this PEP requires the addition of a formal warning
and deprecation facility that will be described in another proposal.


Scope
=====

These guidelines apply to future versions of Python that introduce
backward-incompatible behavior.  Backward incompatible behavior is a
major deviation in Python interpretation from an earlier behavior
described in the standard Python documentation.  Removal of a feature
also constitutes a change of behavior.

This PEP does not replace or preclude other compatibility strategies
such as dynamic loading of backwards-compatible parsers.  On the other
hand, if execution of "old code" requires a special switch or pragma
then that is indeed a change of behavior from the point of view of the
user and that change should be implemented according to these
guidelines.

In general, common sense must prevail in the implementation of these
guidelines.  For instance changing "sys.copyright" does not constitute
a backwards-incompatible change of behavior!


Steps For Introducing Backwards-Incompatible Features
=====================================================

1. Propose backwards-incompatible behavior in a PEP.  The PEP must
   include a section on backwards compatibility that describes in
   detail a plan to complete the remainder of these steps.

2. Once the PEP is accepted as a productive direction, implement an
   alternate way to accomplish the task previously provided by the
   feature that is being removed or changed.  For instance if the
   addition operator were scheduled for removal, a new version of
   Python could implement an "add()" built-in function.

3. Formally deprecate the obsolete construct in the Python
   documentation.

4. Add an optional warning mode to the parser that will inform users
   when the deprecated construct is used.  In other words, all
   programs that will behave differently in the future must trigger
   warnings in this mode.  Compile-time warnings are preferable to
   runtime warnings.  The warning messages should steer people from
   the deprecated construct to the alternative construct.

5. There must be at least a one-year transition period between the
   release of the transitional version of Python and the release of
   the backwards incompatible version.  Users will have at least a
   year to test their programs and migrate them from use of the
   deprecated construct to the alternative one.


id: pep-0549.rst
content:
PEP: 549
Title: Instance Descriptors
Author: Larry Hastings <larry@hastings.org>
Discussions-To: python-dev@python.org
Status: Rejected
Type: Standards Track
Created: 04-Sep-2017
Python-Version: 3.7
Post-History: 04-Sep-2017


Rejection Notice
================

https://mail.python.org/pipermail/python-dev/2017-November/150528.html

Abstract
========

Python's descriptor protocol requires that descriptors
be members of the *type* of an object.  This PEP proposes
an extension to the descriptor protocol allowing use of
the descriptor protocol for members of *instances.*  This
would permit using properties in modules.

Rationale
=========

Python's descriptor protocol guides programmers towards
elegant API design.  If your class supports a data-like
member, and you *might* someday need to run code when
changing the member's value, you're encouraged to
simply declare it as a simple data member of the class
for now.  If in the future you do need to run code, you
can change it to a "property", and happily the API doesn't
change.

But consider this second bit of best-practice Python API design:
if you're writing a singleton, don't write a class, just build
your code directly into a module.  Don't make your users
instantiate a singleton class, don't make your users have to
dereference through a singleton object stored in a module,
just have module-level functions and module-level data.

Unfortunately these two best practices are in opposition.
The problem is that properties aren't supported on modules.
Modules are instances of a single generic ``module`` type,
and it's not feasible to modify or subclass this type to add
a property to one's module.  This means that programmers
facing this API design decision, where the data-like member
is a singleton stored in a module, must preemptively add
ugly "getters" and "setters" for the data.

Adding support for module properties in pure Python has recently
become *possible;*
as of Python 3.5, Python permits assigning to the ``__class__``
attribute of module objects, specifically for this purpose.  Here's
an example of using this functionality to add a property to a module::

    import sys, types
    class _MyModuleType(types.ModuleType):
        @property
        def prop(self, instance, owner):
            ...

    sys.modules[__name__].__class__ = _MyModuleType

This works, and is supported behavior, but it's clumsy and obscure.

This PEP proposes a per-type opt-in extension to the descriptor
protocol specifically designed to enable properties in modules.
The mechanism is a way to honor the descriptor protocol for
members of *instances* of a class without the member being declared
as a class variable.

Although this is being proposed as a general mechanism, the author
currently only foresees this as being useful for module objects.

Implementation
==============

The basic idea is simple: modify the ``tp_descr_get`` and ``tp_descr_set``
functions exposed by ``PyModule_Type`` to inspect the attribute interacted
with, and if it supports the descriptor protocol, call the relevant
exposed function.

Our implementation faces two challenges:

1. Since this code will be run every time an attribute is looked up on a
   method, it needs to add very little overhead in the general case,
   where the object stored in the attribute is not a descriptor.

2. Since functions are descriptors, we must take care to *not* honor
   the descriptor protocol for all objects.  Otherwise, all module-level
   functions will suddenly become bound to the module instance as if
   they were method calls on the module object.  The module handle would
   be passed in as a "self" argument to all module-level functions.

Both challenges can be solved with the same approach: we define a new
"fast subclass" flag that means "This object is a descriptor, and it
should be honored directly when this object is looked up as an
attribute of an instance".  So far this flag is only set on two
types: ``property`` and ``collections.abc.InstanceDescriptor``.
The latter is an abstract base class, whose only purpose is
to allow user classes to inherit this "fast subclass" flag.

Prototype
=========

A prototype of this functionality is under development
at GitHub [github]_.

Acknowledgements
================

Armin Rigo essentially proposed this mechanism when presented
with the idea of "module properties", and educated the author
both on the complexities of the problem and the proper solution.
Nathaniel J. Smith pointed out the 3.5 extension about assigning
to ``__class__`` on module objects, and provided the example.

References
==========

.. [github]
   The branch is here:
       https://github.com/larryhastings/cpython/tree/module-properties
   A pull request against the main CPython repo is here:
       https://github.com/python/cpython/pull/3534

Copyright
---------

This document has been placed in the public domain.


id: pep-8101.rst
content:
PEP: 8101
Title: 2020 Term Steering Council election
Author: Ewa Jodlowska <ewa@python.org>, Ee Durbin <ee@python.org>
Sponsor: Brett Cannon <brett@python.org>
Status: Final
Type: Informational
Topic: Governance
Created: 16-Nov-2019


Abstract
========

This document describes the schedule and other details of the December
2019 election for the Python steering council, as specified in
:pep:`13`. This is steering council election for the 2020 term.


Election Administration
=======================

The steering council appointed the
`Python Software Foundation <https://www.python.org/psf-landing/>`__
Director of Infrastructure, Ee Durbin, to implement the election,
and `Python Software Foundation <https://www.python.org/psf-landing/>`__
Executive Director, Ewa Jodlowska, to communicate announcements
regarding the election.


Schedule
========

There will be a two-week nomination period, followed by a two-week
vote.

The nomination period was: November 1, 2019 through November 16, 2019 12:00 UTC
(The end of November 15, 2019 `Anywhere on Earth
<http://www.ieee802.org/16/aoe.html>`_). As announced on `python-committers
<https://mail.python.org/archives/list/python-committers@python.org/thread/4N6WEWVSE6JQ64KDFGQRWFG33MSHRMCK/>`_,
and took place on `discuss.python.org <https://discuss.python.org/t/about-the-steering-council-nominations-category/2459>`_.

The voting period is: December 1, 2019 12:00 UTC through December 16, 2019
12:00 UTC (The end of December 15, 2019 `Anywhere on Earth
<http://www.ieee802.org/16/aoe.html>`_).


Candidates
==========

Candidates must be nominated by a core team member. If the candidate
is a core team member, they may nominate themselves.

Nominees (in alphabetical order):

- `Brett Cannon <https://discuss.python.org/t/steering-council-nomination-brett-cannon-2020-term/2566>`_
- `Kushal Das <https://discuss.python.org/t/steering-council-nomination-kushal-das-2020-term/2662>`_
- `Christian Heimes <https://discuss.python.org/t/steering-council-nomination-christian-heimes-2020-term/2661>`_
- `Pablo Galindo Salgado <https://discuss.python.org/t/steering-council-nomination-pablo-galindo-salgado-2020-term/2667>`_
- `Gregory P. Smith <https://discuss.python.org/t/steering-council-nomination-gregory-p-smith-2020-term/2668>`_
- `Victor Stinner <https://discuss.python.org/t/steering-council-nomination-victor-stinner-2020-term/2658>`_
- `Barry Warsaw <https://discuss.python.org/t/steering-council-nomination-barry-warsaw-2020-term/2601>`_
- `Carol Willing <https://discuss.python.org/t/steering-council-nomination-carol-willing-2020-term/2593>`_
- `Thomas Wouters <https://discuss.python.org/t/steering-council-nomination-thomas-wouters-2020-term/2615>`_

Withdrawn nominations:

- `Guido van Rossum <https://discuss.python.org/t/steering-council-nomination-guido-van-rossum-2020-term/2657>`_. (`Withdrawal <https://discuss.python.org/t/steering-council-nomination-guido-van-rossum-2020-term/2657/11>`_)

Voter Roll
==========

All active Python core team members are eligible to vote.

Ballots will be distributed based on the `The Python Voter Roll for this
election
<https://github.com/python/voters/blob/master/voter-files/2019-12-01-2020-python-steering-council-election.csv>`_
[1]_.

While this file is not public as it contains private email addresses, the
`Complete Voter Roll`_ by name is available.

Election Implementation
=======================

The election will be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2020-python-steering-council``

Name: ``2020 Python Steering Council Election``

Description: ``Election for the Python steering council, as specified in PEP 13. This is steering council election for the 2020 term.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``ee@python.org``

Voting starts at: ``December 1, 2019 12:00 UTC``

Voting ends at: ``December 16, 2019 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``- (approval)`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``



Results
=======

Of 82 eligible voters, 63 cast ballots.

The top five vote-getters are:

* Barry Warsaw
* Brett Cannon
* Carol Willing
* Thomas Wouters
* Victor Stinner

No conflict of interest as defined in :pep:`13` were observed.

The full vote counts are as follows:

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Brett Cannon          | 54             |
+-----------------------+----------------+
| Carol Willing         | 54             |
+-----------------------+----------------+
| Barry Warsaw          | 50             |
+-----------------------+----------------+
| Thomas Wouters        | 40             |
+-----------------------+----------------+
| Victor Stinner        | 38             |
+-----------------------+----------------+
| Christian Heimes      | 37             |
+-----------------------+----------------+
| Pablo Galindo Salgado | 37             |
+-----------------------+----------------+
| Gregory P. Smith      | 32             |
+-----------------------+----------------+
| Kushal Das            | 30             |
+-----------------------+----------------+


Copyright
=========

This document has been placed in the public domain.


Complete Voter Roll
===================

Active Python core developers
-----------------------------

::

    Abhilash Raj
    Alex Gaynor
    Alex Martelli
    Alexander Belopolsky
    Andrew Kuchling
    Andrew Svetlov
    Antoine Pitrou
    Barry Warsaw
    Benjamin Peterson
    Berker PeksaÄ
    Brett Cannon
    Brian Curtin
    Brian Quinlan
    Carol Willing
    Cheryl Sabella
    Chris Withers
    Christian Heimes
    Christian Tismer
    Davin Potts
    Dino Viehland
    Donald Stufft
    Emily Morehouse
    Ãric Araujo
    Eric Snow
    Eric V. Smith
    Ethan Furman
    Ezio Melotti
    Facundo Batista
    Fred Drake
    Giampaolo RodolÃ 
    Gregory P. Smith
    Guido van Rossum
    Inada Naoki
    Ivan Levkivskyi
    Jason R. Coombs
    Jeremy Kloth
    JesÃºs Cea
    Joannah Nanjekye
    Julien Palard
    Kurt B. Kaiser
    Kushal Das
    Larry Hastings
    Lisa Roach
    Åukasz Langa
    Marc-AndrÃ© Lemburg
    Mariatta
    Mark Dickinson
    Mark Shannon
    Matthias Klose
    Michael Foord
    Nathaniel J. Smith
    Ned Deily
    Neil Schemenauer
    Alyssa Coghlan
    Pablo Galindo
    Paul Ganssle
    Paul Moore
    Petr Viktorin
    R. David Murray
    Raymond Hettinger
    Robert Collins
    Ronald Oussoren
    Senthil Kumaran
    Serhiy Storchaka
    Skip Montanaro
    Stefan Behnel
    Stefan Krah
    Steve Dower
    Steven D'Aprano
    StÃ©phane Wirtel
    Tal Einat
    Terry Jan Reedy
    Thomas Wouters
    Tim Golden
    Tim Peters
    Victor Stinner
    Vinay Sajip
    Walter DÃ¶rwald
    Xavier de Gaye
    Xiang Zhang
    Yury Selivanov
    Zachary Ware


.. [1] This repository is private and accessible only to Python Core
   Developers, administrators, and Python Software Foundation Staff as it
   contains personal email addresses.


id: pep-0575.rst
content:
PEP: 575
Title: Unifying function/method classes
Author: Jeroen Demeyer <J.Demeyer@UGent.be>
Status: Withdrawn
Type: Standards Track
Created: 27-Mar-2018
Python-Version: 3.8
Post-History: 31-Mar-2018, 12-Apr-2018, 27-Apr-2018, 05-May-2018


Withdrawal notice
=================

See :pep:`580` for a better solution to allowing fast calling of custom classes.

See :pep:`579` for a broader discussion of some of the other issues from this PEP.


Abstract
========

Reorganize the class hierarchy for functions and methods
with the goal of reducing the difference between
built-in functions (implemented in C) and Python functions.
Mainly, make built-in functions behave more like Python functions
without sacrificing performance.

A new base class ``base_function`` is introduced and the various function
classes, as well as ``method`` (renamed to ``bound_method``), inherit from it.

We also allow subclassing the Python ``function`` class.


Motivation
==========

Currently, CPython has two different function classes:
the first is Python functions, which is what you get
when defining a function with ``def`` or ``lambda``.
The second is built-in functions such as ``len``, ``isinstance`` or ``numpy.dot``.
These are implemented in C.

These two classes are implemented completely independently and have different functionality.
In particular, it is currently not possible to implement a function efficiently in C
(only built-in functions can do that)
while still allowing introspection like ``inspect.signature`` or ``inspect.getsourcefile``
(only Python functions can do that).
This is a problem for projects like Cython [#cython]_ that want to do exactly that.

In Cython, this was worked around by inventing a new function class called ``cyfunction``.
Unfortunately, a new function class creates problems:
the ``inspect`` module does not recognize such functions as being functions [#bpo30071]_
and the performance is worse
(CPython has specific optimizations for calling built-in functions).

A second motivation is more generally making built-in functions and methods
behave more like Python functions and methods.
For example, Python unbound methods are just functions but
unbound methods of extension types (e.g. ``dict.get``) are a distinct class.
Bound methods of Python classes have a ``__func__`` attribute,
bound methods of extension types do not.

Third, this PEP allows great customization of functions.
The ``function`` class becomes subclassable and custom function
subclasses are also allowed for functions implemented in C.
In the latter case, this can be done with the same performance
as true built-in functions.
All functions can access the function object
(the ``self`` in ``__call__``), paving the way for :pep:`573`.


New classes
===========

This is the new class hierarchy for functions and methods::

                       object
                          |
                          |
                   base_function
                  /       |     \
                 /        |      \
                /         |   defined_function
               /          |        \
         cfunction (*)    |         \
                          |       function
                          |
                    bound_method (*)

The two classes marked with (*) do *not* allow subclassing;
the others do.

There is no difference between functions and unbound methods,
while bound methods are instances of ``bound_method``.

base_function
-------------

The class ``base_function`` becomes a new base class for all function types.
It is based on the existing ``builtin_function_or_method`` class,
but with the following differences and new features:

#. It acts as a descriptor implementing ``__get__`` to turn a function into a method
   if ``m_self`` is ``NULL``.
   If ``m_self`` is not ``NULL``,
   then this is a no-op: the existing function is returned instead.

#. A new read-only attribute ``__parent__``, represented in the C structure as ``m_parent``.
   If this attribute exists, it represents the defining object.
   For methods of extension types, this is the defining class (``__class__`` in plain Python)
   and for functions of a module, this is the defining module.
   In general, it can be any Python object.
   If ``__parent__`` is a class, it carries special semantics:
   in that case, the function must be called with ``self`` being an instance of that class.
   Finally, ``__qualname__`` and ``__reduce__`` will use ``__parent__``
   as namespace (instead of ``__self__`` before).

#. A new attribute ``__objclass__`` which equals ``__parent__`` if ``__parent__``
   is a class. Otherwise, accessing ``__objclass__`` raises ``AttributeError``.
   This is meant to be backwards compatible with ``method_descriptor``.

#. The field ``ml_doc`` and the attributes ``__doc__`` and
   ``__text_signature__`` (see :pep:`Argument Clinic <436>`)
   are not supported.

#. A new flag ``METH_PASS_FUNCTION`` for ``ml_flags``.
   If this flag is set, the C function stored in ``ml_meth`` is called with
   an additional first argument equal to the function object.

#. A new flag ``METH_BINDING`` for ``ml_flags`` which only applies to
   functions of a module (not methods of a class).
   If this flag is set, then ``m_self`` is set to ``NULL`` instead
   of the module.
   This allows the function to behave more like a Python function
   as it enables ``__get__``.

#. A new flag ``METH_CALL_UNBOUND`` to disable `self slicing`_.

#. A new flag ``METH_PYTHON`` for ``ml_flags``.
   This flag indicates that this function should be treated as Python function.
   Ideally, use of this flag should be avoided because it goes
   against the duck typing philosophy.
   It is still needed in a few places though, for example `profiling`_.

The goal of ``base_function`` is that it supports all different ways
of calling functions and methods in just one structure.
For example, the new flag ``METH_PASS_FUNCTION``
will be used by the implementation of methods.

It is not possible to directly create instances of ``base_function``
(``tp_new`` is ``NULL``).
However, it is legal for C code to manually create instances.

These are the relevant C structures::

    PyTypeObject PyBaseFunction_Type;

    typedef struct {
        PyObject_HEAD
        PyCFunctionDef *m_ml;     /* Description of the C function to call */
        PyObject *m_self;         /* __self__: anything, can be NULL; readonly */
        PyObject *m_module;       /* __module__: anything (typically str) */
        PyObject *m_parent;       /* __parent__: anything, can be NULL; readonly */
        PyObject *m_weakreflist;  /* List of weak references */
    } PyBaseFunctionObject;

    typedef struct {
        const char *ml_name;   /* The name of the built-in function/method */
        PyCFunction ml_meth;   /* The C function that implements it */
        int ml_flags;          /* Combination of METH_xxx flags, which mostly
                                  describe the args expected by the C func */
    } PyCFunctionDef;

Subclasses may extend ``PyCFunctionDef`` with extra fields.

The Python attribute ``__self__`` returns ``m_self``,
except if ``METH_STATIC`` is set.
In that case or if ``m_self`` is ``NULL``,
then there is no ``__self__`` attribute at all.
For that reason, we write either ``m_self`` or ``__self__`` in this PEP
with slightly different meanings.

cfunction
---------

This is the new version of the old ``builtin_function_or_method`` class.
The name ``cfunction`` was chosen to avoid confusion with "built-in"
in the sense of "something in the ``builtins`` module".
It also fits better with the C API which use the ``PyCFunction`` prefix.

The class ``cfunction`` is a copy of ``base_function``, with the following differences:

#. ``m_ml`` points to a ``PyMethodDef`` structure,
   extending ``PyCFunctionDef`` with an additional ``ml_doc``
   field to implement ``__doc__`` and ``__text_signature__``
   as read-only attributes::

    typedef struct {
        const


id: pep-0213.rst
content:
PEP: 213
Title: Attribute Access Handlers
Author: Paul Prescod <paul@prescod.net>
Status: Deferred
Type: Standards Track
Created: 21-Jul-2000
Python-Version: 2.1
Post-History:


Introduction
============

It is possible (and even relatively common) in Python code and
in extension modules to "trap" when an instance's client code
attempts to set an attribute and execute code instead. In other
words, it is possible to allow users to use attribute assignment/
retrieval/deletion syntax even though the underlying implementation
is doing some computation rather than directly modifying a
binding.

This PEP describes a feature that makes it easier, more efficient
and safer to implement these handlers for Python instances.


Justification
=============

Scenario 1
----------

You have a deployed class that works on an attribute named
"stdout". After a while, you think it would be better to
check that stdout is really an object with a "write" method
at the moment of assignment. Rather than change to a
setstdout method (which would be incompatible with deployed
code) you would rather trap the assignment and check the
object's type.

Scenario 2
----------

You want to be as compatible as possible with an object
model that has a concept of attribute assignment. It could
be the W3C Document Object Model or a particular COM
interface (e.g. the PowerPoint interface). In that case
you may well want attributes in the model to show up as
attributes in the Python interface, even though the
underlying implementation may not use attributes at all.

Scenario 3
----------

A user wants to make an attribute read-only.

In short, this feature allows programmers to separate the
interface of their module from the underlying implementation
for whatever purpose. Again, this is not a new feature but
merely a new syntax for an existing convention.


Current Solution
================

To make some attributes read-only::

   class foo:
       def __setattr__( self, name, val ):
           if name=="readonlyattr":
               raise TypeError
           elif name=="readonlyattr2":
               raise TypeError
       ...
       else:
           self.__dict__["name"]=val

This has the following problems:

1. The creator of the method must be intimately aware of whether
   somewhere else in the class hierarchy ``__setattr__`` has also been
   trapped for any particular purpose. If so, she must specifically
   call that method rather than assigning to the dictionary. There
   are many different reasons to overload ``__setattr__`` so there is a
   decent potential for clashes. For instance object database
   implementations often overload setattr for an entirely unrelated
   purpose.

2. The string-based switch statement forces all attribute handlers
   to be specified in one place in the code. They may then dispatch
   to task-specific methods (for modularity) but this could cause
   performance problems.

3. Logic for the setting, getting and deleting must live in
   ``__getattr__``, ``__setattr__`` and ``__delattr__``. Once again, this can
   be mitigated through an extra level of method call but this is
   inefficient.


Proposed Syntax
===============

Special methods should declare themselves with declarations of the
following form::

   class x:
       def __attr_XXX__(self, op, val ):
           if op=="get":
               return someComputedValue(self.internal)
           elif op=="set":
               self.internal=someComputedValue(val)
           elif op=="del":
               del self.internal


Client code looks like this::

   fooval=x.foo
   x.foo=fooval+5
   del x.foo


Semantics
=========

Attribute references of all three kinds should call the method.
The op parameter can be "get"/"set"/"del". Of course this string
will be interned so the actual checks for the string will be
very fast.

It is disallowed to actually have an attribute named XXX in the
same instance as a method named __attr_XXX__.

An implementation of __attr_XXX__ takes precedence over an
implementation of ``__getattr__`` based on the principle that
``__getattr__`` is supposed to be invoked only after finding an
appropriate attribute has failed.

An implementation of __attr_XXX__ takes precedence over an
implementation of ``__setattr__`` in order to be consistent. The
opposite choice seems fairly feasible also, however. The same
goes for __del_y__.


Proposed Implementation
=======================

There is a new object type called an attribute access handler.
Objects of this type have the following attributes::

    name (e.g. XXX, not __attr__XXX__)
    method (pointer to a method object)

In PyClass_New, methods of the appropriate form will be detected and
converted into objects (just like unbound method objects). These are
stored in the class ``__dict__`` under the name XXX. The original method
is stored as an unbound method under its original name.

If there are any attribute access handlers in an instance at all,
a flag is set. Let's call it "I_have_computed_attributes" for
now. Derived classes inherit the flag from base classes. Instances
inherit the flag from classes.

A get proceeds as usual until just before the object is returned.
In addition to the current check whether the returned object is a
method it would also check whether a returned object is an access
handler. If so, it would invoke the getter method and return
the value. To remove an attribute access handler you could directly
fiddle with the dictionary.

A set proceeds by checking the "I_have_computed_attributes" flag. If
it is not set, everything proceeds as it does today. If it is set
then we must do a dictionary get on the requested object name. If it
returns an attribute access handler then we call the setter function
with the value. If it returns any other object then we discard the
result and continue as we do today. Note that having an attribute
access handler will mildly affect attribute "setting" performance for
all sets on a particular instance, but no more so than today, using
``__setattr__``. Gets are more efficient than they are today with
``__getattr__``.

The I_have_computed_attributes flag is intended to eliminate the
performance degradation of an extra "get" per "set" for objects not
using this feature. Checking this flag should have minuscule
performance implications for all objects.

The implementation of delete is analogous to the implementation
of set.


Caveats
=======

1. You might note that I have not proposed any logic to keep
   the I_have_computed_attributes flag up to date as attributes
   are added and removed from the instance's dictionary. This is
   consistent with current Python. If you add a ``__setattr__`` method
   to an object after it is in use, that method will not behave as
   it would if it were available at "compile" time. The dynamism is
   arguably not worth the extra implementation effort. This snippet
   demonstrates the current behavior::

       >>> def prn(*args):print args
       >>> class a:

       ...    __setattr__=prn
       >>> a().foo=5
       (<__main__.a instance at 882890>, 'foo', 5)

       >>> class b: pass
       >>> bi=b()
       >>> bi.__setattr__=prn
       >>> b.foo=5


2. Assignment to __dict__["XXX"] can overwrite the attribute
   access handler for __attr_XXX__. Typically the access handlers will
   store information away in private __XXX variables


3. An attribute access handler that attempts to call setattr or getattr
   on the object itself can cause an infinite loop (as with ``__getattr__``)
   Once again, the solution is to use a special (typically private)
   variable such as __XXX.


Note
====

The descriptor mechanism described in :pep:`252` is powerful enough
to support this more directly.  A 'getset' constructor may be
added to the language making this possible::

   class C:
       def get_x(self):
           return self.__x
       def set_x(self, v):
           self.__x = v
       x = getset(get_x, set_x)

Additional syntactic sugar might be added


id: pep-0207.rst
content:
PEP: 207
Title: Rich Comparisons
Author: Guido van Rossum <guido@python.org>, David Ascher <DavidA@ActiveState.com>
Status: Final
Type: Standards Track
Created: 25-Jul-2000
Python-Version: 2.1
Post-History:


Abstract
========

This PEP proposes several new features for comparisons:

- Allow separately overloading of <, >, <=, >=, ==, !=, both in
  classes and in C extensions.

- Allow any of those overloaded operators to return something else
  besides a Boolean result.


Motivation
==========

The main motivation comes from NumPy, whose users agree that A<B
should return an array of elementwise comparison outcomes; they
currently have to spell this as less(A,B) because A<B can only
return a Boolean result or raise an exception.

An additional motivation is that frequently, types don't have a
natural ordering, but still need to be compared for equality.
Currently such a type **must** implement comparison and thus define
an arbitrary ordering, just so that equality can be tested.

Also, for some object types an equality test can be implemented
much more efficiently than an ordering test; for example, lists
and dictionaries that differ in length are unequal, but the
ordering requires inspecting some (potentially all) items.


Previous Work
=============

Rich Comparisons have been proposed before; in particular by David
Ascher, after experience with Numerical Python:

  http://starship.python.net/crew/da/proposals/richcmp.html

It is also included below as an Appendix.  Most of the material in
this PEP is derived from David's proposal.


Concerns
========

1. Backwards compatibility, both at the Python level (classes using
   ``__cmp__`` need not be changed) and at the C level (extensions
   defining ``tp_comparea`` need not be changed, code using
   ``PyObject_Compare()`` must work even if the compared objects use
   the new rich comparison scheme).

2. When A<B returns a matrix of elementwise comparisons, an easy
   mistake to make is to use this expression in a Boolean context.
   Without special precautions, it would always be true.  This use
   should raise an exception instead.

3. If a class overrides x==y but nothing else, should x!=y be
   computed as not(x==y), or fail?  What about the similar
   relationship between < and >=, or between > and <=?

4. Similarly, should we allow x<y to be calculated from y>x?  And
   x<=y from not(x>y)?  And x==y from y==x, or x!=y from y!=x?

5. When comparison operators return elementwise comparisons, what
   to do about shortcut operators like A<B<C, ``A<B and C<D``,
   ``A<B or C<D``?

6. What to do about ``min()`` and ``max()``, the 'in' and 'not in'
   operators, ``list.sort()``, dictionary key comparison, and other
   uses of comparisons by built-in operations?


Proposed Resolutions
====================

1. Full backwards compatibility can be achieved as follows.  When
   an object defines ``tp_compare()`` but not ``tp_richcompare()``, and a
   rich comparison is requested, the outcome of ``tp_compare()`` is
   used in the obvious way.  E.g. if "<" is requested, an exception if
   ``tp_compare()`` raises an exception, the outcome is 1 if
   ``tp_compare()`` is negative, and 0 if it is zero or positive.  Etc.

   Full forward compatibility can be achieved as follows.  When a
   classic comparison is requested on an object that implements
   ``tp_richcompare()``, up to three comparisons are used: first == is
   tried, and if it returns true, 0 is returned; next, < is tried
   and if it returns true, -1 is returned; next, > is tried and if
   it returns true, +1 is returned.  If any operator tried returns
   a non-Boolean value (see below), the exception raised by
   conversion to Boolean is passed through.  If none of the
   operators tried returns true, the classic comparison fallbacks
   are tried next.

   (I thought long and hard about the order in which the three
   comparisons should be tried.  At one point I had a convincing
   argument for doing it in this order, based on the behavior of
   comparisons for cyclical data structures.  But since that code
   has changed again, I'm not so sure that it makes a difference
   any more.)

2. Any type that returns a collection of Booleans instead of a
   single boolean should define ``nb_nonzero()`` to raise an exception.
   Such a type is considered a non-Boolean.

3. The == and != operators are not assumed to be each other's
   complement (e.g. IEEE 754 floating point numbers do not satisfy
   this).  It is up to the type to implement this if desired.
   Similar for < and >=, or > and <=; there are lots of examples
   where these assumptions aren't true (e.g. tabnanny).

4. The reflexivity rules **are** assumed by Python.  Thus, the
   interpreter may swap y>x with x<y, y>=x with x<=y, and may swap
   the arguments of x==y and x!=y.  (Note: Python currently assumes
   that x==x is always true and x!=x is never true; this should not
   be assumed.)

5. In the current proposal, when A<B returns an array of
   elementwise comparisons, this outcome is considered non-Boolean,
   and its interpretation as Boolean by the shortcut operators
   raises an exception.  David Ascher's proposal tries to deal
   with this; I don't think this is worth the additional complexity
   in the code generator.  Instead of A<B<C, you can write
   (A<B)&(B<C).

6. The ``min()`` and ``list.sort()`` operations will only use the
   < operator; max() will only use the > operator.  The 'in' and
   'not in' operators and dictionary lookup will only use the ==
   operator.


Implementation Proposal
=======================

This closely follows David Ascher's proposal.

C API
-----

- New functions::

      PyObject *PyObject_RichCompare(PyObject *, PyObject *, int)

  This performs the requested rich comparison, returning a Python
  object or raising an exception.  The 3rd argument must be one of
  Py_LT, Py_LE, Py_EQ, Py_NE, Py_GT or Py_GE.

  ::

      int PyObject_RichCompareBool(PyObject *, PyObject *, int)

  This performs the requested rich comparison, returning a
  Boolean: -1 for exception, 0 for false, 1 for true.  The 3rd
  argument must be one of Py_LT, Py_LE, Py_EQ, Py_NE, Py_GT or
  Py_GE.  Note that when ``PyObject_RichCompare()`` returns a
  non-Boolean object, ``PyObject_RichCompareBool()`` will raise an
  exception.

- New typedef::

      typedef PyObject *(*richcmpfunc) (PyObject *, PyObject *, int);

- New slot in type object, replacing spare tp_xxx7::

      richcmpfunc tp_richcompare;

  This should be a function with the same signature as
  ``PyObject_RichCompare()``, and performing the same comparison.
  At least one of the arguments is of the type whose
  tp_richcompare slot is being used, but the other may have a
  different type.  If the function cannot compare the particular
  combination of objects, it should return a new reference to
  ``Py_NotImplemented``.

- ``PyObject_Compare()`` is changed to try rich comparisons if they
  are defined (but only if classic comparisons aren't defined).

Changes to the interpreter
--------------------------

- Whenever ``PyObject_Compare()`` is called with the intent of getting
  the outcome of a particular comparison (e.g. in ``list.sort()``, and
  of course for the comparison operators in ceval.c), the code is
  changed to call ``PyObject_RichCompare()`` or
  ``PyObject_RichCompareBool()`` instead; if the C code needs to know
  the outcome of the comparison, ``PyObject_IsTrue()`` is called on
  the result (which may raise an exception).

- Most built-in types that currently define a comparison will be
  modified to define a rich comparison instead.  (This is
  optional; I've converted lists, tuples, complex numbers, and
  arrays so far, and am not sure whether I will convert others.)

Classes
-------

- Classes can define new special methods ``__lt__``, ``__le__``, ``__eq__``,
  ``__ne__``, ``__gt__``, ``__ge__`` to override the corresponding operators.
  (I.e., <, <=, ==, !=, >, >=. You gotta love the Fortran
  her


id: pep-0561.rst
content:
PEP: 561
Title: Distributing and Packaging Type Information
Author: Emma Harper Smith <emma@python.org>
Status: Final
Type: Standards Track
Topic: Packaging, Typing
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017, 12-Apr-2018

.. canonical-typing-spec:: :ref:`typing:packaging-typed-libraries`

Abstract
========

:pep:`484` introduced type hinting to Python, with goals of making typing
gradual and easy to adopt. Currently, typing information must be distributed
manually. This PEP provides a standardized means to leverage existing tooling
to package and distribute type information with minimal work and an ordering
for type checkers to resolve modules and collect this information for type
checking.


Rationale
=========

Currently, package authors wish to distribute code that has inline type
information. Additionally, maintainers would like to distribute stub files
to keep Python 2 compatibility while using newer annotation syntax. However,
there is no standard method to distribute packages with type information.
Also, if one wished to ship stub files privately the only method available
would be via setting ``MYPYPATH`` or the equivalent to manually point to
stubs. If the package can be released publicly, it can be added to
typeshed [1]_. However, this does not scale and becomes a burden on the
maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
of the tool using typeshed.

:pep:`484` has a brief section on distributing typing information. In this
:pep:`section <484#storing-and-distributing-stub-files>`
the PEP recommends using ``shared/typehints/pythonX.Y/`` for
shipping stub files. However, manually adding a path to stub files for each
third party library does not scale. The simplest approach people have taken
is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
and Django).


Definition of Terms
===================

The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
to be interpreted as described in :rfc:`2119`.

"inline" - the types are part of the runtime code using :pep:`526` and
:pep:`3107` syntax (the filename ends in ``.py``).

"stubs" - files containing only type information, empty of runtime code
(the filename ends in ``.pyi``).

"Distributions" are the packaged files which are used to publish and distribute
a release. (:pep:`426`)

"Module" a file containing Python runtime code or stubbed type information.

"Package" a directory or directories that namespace Python modules.
(Note the distinction between packages and distributions.  While most
distributions are named after the one package they install, some
distributions install multiple packages.)


Specification
=============

There are several motivations and methods of supporting typing in a package.
This PEP recognizes three types of packages that users of typing wish to
create:

1. The package maintainer would like to add type information inline.

2. The package maintainer would like to add type information via stubs.

3. A third party or package maintainer would like to share stub files for
   a package, but the maintainer does not want to include them in the source
   of the package.

This PEP aims to support all three scenarios and make them simple to add to
packaging and deployment.

The two major parts of this specification are the packaging specifications
and the resolution order for resolving module type information. The type
checking spec is meant to replace the ``shared/typehints/pythonX.Y/``
:pep:`spec of PEP 484 <484#storing-and-distributing-stub-files>`.

New third party stub libraries SHOULD distribute stubs via the third party
packaging methods proposed in this PEP in place of being added to typeshed.
Typeshed will remain in use, but if maintainers are found, third party stubs
in typeshed MAY be split into their own package.


Packaging Type Information
--------------------------

In order to make packaging and distributing type information as simple and
easy as possible, packaging and distribution is done through existing
frameworks.

Package maintainers who wish to support type checking of their code MUST add
a marker file named ``py.typed`` to their package supporting typing. This marker applies
recursively: if a top-level package includes it, all its sub-packages MUST support
type checking as well. To have this file installed with the package,
maintainers can use existing packaging options such as ``package_data`` in
distutils, shown below.

Distutils option example::

    setup(
        ...,
        package_data = {
            'foopkg': ['py.typed'],
        },
        ...,
        )

For namespace packages (see :pep:`420`), the ``py.typed`` file should be in the
submodules of the namespace, to avoid conflicts and for clarity.

This PEP does not support distributing typing information as part of
module-only distributions or single-file modules within namespace packages.

The single-file module should be refactored into a package
and indicate that the package supports typing as described
above.

Stub-only Packages
''''''''''''''''''

For package maintainers wishing to ship stub files containing all of their
type information, it is preferred that the ``*.pyi`` stubs are alongside the
corresponding ``*.py`` files. However, the stubs can also be put in a separate
package and distributed separately. Third parties can also find this method
useful if they wish to distribute stub files. The name of the stub package
MUST follow the scheme ``foopkg-stubs`` for type stubs for the package named
``foopkg``. Note that for stub-only packages adding a ``py.typed`` marker is not
needed since the name ``*-stubs`` is enough to indicate it is a source of typing
information.

Third parties seeking to distribute stub files are encouraged to contact the
maintainer of the package about distribution alongside the package. If the
maintainer does not wish to maintain or package stub files or type information
inline, then a third party stub-only package can be created.

In addition, stub-only distributions SHOULD indicate which version(s)
of the runtime package are supported by indicating the runtime distribution's
version(s) through normal dependency data. For example, the
stub package ``flyingcircus-stubs`` can indicate the versions of the
runtime ``flyingcircus`` distribution it supports through ``install_requires``
in distutils-based tools, or the equivalent in other packaging tools. Note that
in pip 9.0, if you update ``flyingcircus-stubs``, it will update
``flyingcircus``. In pip 9.0, you can use the
``--upgrade-strategy=only-if-needed`` flag. In pip 10.0 this is the default
behavior.

For namespace packages (see :pep:`420`), stub-only packages should
use the ``-stubs`` suffix on only the root namespace package.
All stub-only namespace packages should omit ``__init__.pyi`` files. ``py.typed``
marker files are not necessary for stub-only packages, but similarly
to packages with inline types, if used, they should be in submodules of the namespace to
avoid conflicts and for clarity.

For example, if the ``pentagon`` and ``hexagon`` are separate distributions
installing within the namespace package ``shapes.polygons``
The corresponding types-only distributions should produce packages
laid out as follows::

    shapes-stubs
    âââ polygons
        âââ pentagon
            âââ __init__.pyi

    shapes-stubs
    âââ polygons
        âââ hexagon
         Â Â  âââ __init__.pyi

.. _mro:

Type Checker Module Resolution Order
------------------------------------

The following is the order in which type checkers supporting this PEP SHOULD
resolve modules containing type information:


1. Stubs or Python source manually put in the beginning of the path. Type
   checkers SHOULD provide this to allow the user complete control of which
   stubs to use, and to patch broken stubs/i


id: pep-0559.rst
content:
PEP: 559
Title: Built-in noop()
Author: Barry Warsaw <barry@python.org>
Status: Rejected
Type: Standards Track
Created: 08-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-September/149438.html


Abstract
========

This PEP proposes adding a new built-in function called ``noop()`` which does
nothing but return ``None``.


Rationale
=========

It is trivial to implement a no-op function in Python.  It's so easy in fact
that many people do it many times over and over again.  It would be useful in
many cases to have a common built-in function that does nothing.

One use case would be for :pep:`553`, where you could set the breakpoint
environment variable to the following in order to effectively disable it::

    $ setenv PYTHONBREAKPOINT=noop


Implementation
==============

The Python equivalent of the ``noop()`` function is exactly::

    def noop(*args, **kws):
        return None

The C built-in implementation is available as a pull request [1]_.


Rejected alternatives
=====================

``noop()`` returns something
----------------------------

YAGNI.

This is rejected because it complicates the semantics.  For example, if you
always return both ``*args`` and ``**kws``, what do you return when none of
those are given?  Returning a tuple of ``((), {})`` is kind of ugly, but
provides consistency.  But you might also want to just return ``None`` since
that's also conceptually what the function was passed.

Or, what if you pass in exactly one positional argument, e.g. ``noop(7)``.  Do
you return ``7`` or ``((7,), {})``?  And so on.

The author claims that you won't ever need the return value of ``noop()`` so
it will always return ``None``.

Coghlan's Dialogs (edited for formatting):

    My counterargument to this would be ``map(noop, iterable)``,
    ``sorted(iterable, key=noop)``, etc. (``filter``, ``max``, and
    ``min`` all accept callables that accept a single argument, as do
    many of the itertools operations).

    Making ``noop()`` a useful default function in those cases just
    needs the definition to be::

       def noop(*args, **kwds):
           return args[0] if args else None

    The counterargument to the counterargument is that using ``None``
    as the default in all these cases is going to be faster, since it
    lets the algorithm skip the callback entirely, rather than calling
    it and having it do nothing useful.


References
==========

.. [1] https://github.com/python/cpython/pull/3480


Copyright
=========

This document has been placed in the public domain.


id: pep-8105.rst
content:
PEP: 8105
Title: 2024 Term Steering Council election
Author: Ee Durbin <ee@python.org>
Sponsor: Thomas Wouters <thomas@python.org>
Status: Final
Type: Informational
Topic: Governance
Created: 23-Oct-2023


Abstract
========

This document describes the schedule and other details of the
2023 election for the Python steering council, as specified in
:pep:`13`. This is the steering council election for the 2024 term
(i.e. Python 3.13).


Election Administration
=======================

The steering council appointed the
`Python Software Foundation <https://www.python.org/psf-landing/>`__
Director of Infrastructure, Ee Durbin, to administer the election.


Schedule
========

There was a two-week nomination period, followed by a two-week
vote.

The nomination period was: November 9, 2023 through `November 22, 2023 AoE
<https://www.timeanddate.com/worldclock/fixedtime.html?msg=Python+Steering+Council+nominations+close&iso=20231123T00&p1=3399>`_ [#note-aoe]_.

The voting period was: November 27, 2023 through `December 11, 2023 AoE
<https://www.timeanddate.com/worldclock/fixedtime.html?msg=Python+Steering+Council+voting+closes&iso=20231212T00&p1=3399>`_ [#note-aoe]_.


Candidates
==========

Candidates must be nominated by a core team member. If the candidate
is a core team member, they may nominate themselves.

Nominees (in alphabetical order by first name):

- `Barry Warsaw <https://discuss.python.org/t/steering-council-nomination-barry-warsaw-2024-term/39380>`_
- `Donghee Na <https://discuss.python.org/t/steering-council-nomination-donghee-na-2024-term/39280>`_
- `Emily Morehouse <https://discuss.python.org/t/steering-council-nomination-emily-morehouse-2024-term/39288/1>`_
- `Gregory P. Smith <https://discuss.python.org/t/steering-council-nomination-gregory-p-smith-2024-term/39381>`_
- `Mariatta <https://discuss.python.org/t/steering-council-nomination-mariatta-2024-term/39396>`_
- `Pablo Galindo Salgado <https://discuss.python.org/t/steering-council-nomination-pablo-galindo-salgado-2024-term/39237>`_
- `Thomas Wouters <https://discuss.python.org/t/steering-council-nomination-thomas-wouters-2024-term/39215>`_
- `Victor Stinner <https://discuss.python.org/t/steering-council-nomination-victor-stinner-2024-term/39349>`_

Withdrawn nominations:

- None

Voter Roll
==========

All active Python core team members are eligible to vote. Active status
is determined as :pep:`described in PEP 13 <13#membership>`
and implemented via the software at `python/voters <https://github.com/python/voters>`_
[#note-voters]_.

Ballots were be distributed based on the the `Python Voter Roll
<https://github.com/python/voters/blob/master/voter-files/>`_ [#note-voters]_
for this election.

While this file is not public as it contains private email addresses,
the `Complete Voter Roll`_ is available with a list of all eligible voters by name.

Election Implementation
=======================

The election was be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2024-python-steering-council``

Name: ``2024 Python Steering Council Election``

Description: ``Election for the Python steering council, as specified in PEP 13. This is steering council election for the 2024 term.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``psf-election@python.org``

Voting starts at: ``November 27, 2023 12:00 UTC``

Voting ends at: ``December 11, 2023 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``- (approval)`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``


Extended Election Implementation
================================

Due to a mismatch between announced timeline and the initial election implementation
an extended election is held to provide the additional day for voters who were disenfranchised
by the initial election.

Results from the initial election were combined with the results from the extended election.
Only voters who **DID NOT** cast ballots in the initial election were able to vote.

The election was be conducted using the `Helios Voting Service
<https://heliosvoting.org>`__.


Configuration
-------------

Short name: ``2024-python-steering-council-extension``

Name: ``2024 Python Steering Council Election Extension``

Description: ``Election for the Python steering council, as specified in PEP 13. This is steering council election for the 2024 term.``

type: ``Election``

Use voter aliases: ``[X]``

Randomize answer order: ``[X]``

Private: ``[X]``

Help Email Address: ``psf-election@python.org``

Voting starts at: ``December 11, 2023 12:00 UTC``

Voting ends at: ``December 12, 2023 12:00 UTC``

This will create an election in which:

* Voting is not open to the public, only those on the `Voter Roll`_ may
  participate. Ballots will be emailed when voting starts.
* Candidates are presented in random order, to help avoid bias.
* Voter identities and ballots are protected against cryptographic advances.

Questions
---------

Question 1
~~~~~~~~~~

Select between ``0`` and ``- (approval)`` answers. Result Type: ``absolute``

Question: ``Select candidates for the Python Steering Council``

Answer #1 - #N: ``Candidates from Candidates_ Section``


Results
=======

Of 87 eligible voters, 68 cast ballots.

The top five vote-getters are:

* Pablo Galindo Salgado
* Gregory P. Smith
* Barry Warsaw
* Emily Morehouse
* Thomas Wouters

No conflict of interest as defined in :pep:`13` were observed.

The full vote counts are as follows:

**Initial Election**

56 of 87 eligible voters cast ballots.

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Barry Warsaw          | 35             |
+-----------------------+----------------+
| Donghee Na            | 27             |
+-----------------------+----------------+
| Emily Morehouse       | 38             |
+-----------------------+----------------+
| Gregory P. Smith      | 42             |
+-----------------------+----------------+
| Mariatta              | 25             |
+-----------------------+----------------+
| Pablo Galindo Salgado | 48             |
+-----------------------+----------------+
| Thomas Wouters        | 30             |
+-----------------------+----------------+
| Victor Stinner        | 27             |
+-----------------------+----------------+

**Election Extension**

12 of 31 eligible voters disenfranchised in the initial election cast ballots.

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Barry Warsaw          | 8              |
+-----------------------+----------------+
| Donghee Na            | 5              |
+-----------------------+----------------+
| Emily Morehouse       | 8              |
+-----------------------+----------------+
| Gregory P. Smith      | 10             |
+-----------------------+----------------+
| Mariatta              | 5              |
+-----------------------+----------------+
| Pablo Galindo Salgado | 12             |
+-----------------------+----------------+
| Thomas Wouters        | 10             |
+-----------------------+----------------+
| Victor Stinner        | 8              |
+-----------------------+----------------+

**Combined**

68 of 87 eligible voters cast ballots

+-----------------------+----------------+
| Candidate             | Votes Received |
+=======================+================+
| Barry Warsaw       


id: pep-0565.rst
content:
PEP: 565
Title: Show DeprecationWarning in __main__
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Final
Type: Standards Track
Created: 12-Nov-2017
Python-Version: 3.7
Post-History: 12-Nov-2017, 25-Nov-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151224.html


Abstract
========

In Python 2.7 and Python 3.2, the default warning filters were updated to hide
``DeprecationWarning`` by default, such that deprecation warnings in development
tools that were themselves written in Python (e.g. linters, static analysers,
test runners, code generators), as well as any other applications that merely
happened to be written in Python, wouldn't be visible to their users unless
those users explicitly opted in to seeing them.

However, this change has had the unfortunate side effect of making
``DeprecationWarning`` markedly less effective at its primary intended purpose:
providing advance notice of breaking changes in APIs (whether in CPython, the
standard library, or in third party libraries) to users of those APIs.

To improve this situation, this PEP proposes a single adjustment to the
default warnings filter: displaying deprecation warnings attributed to the main
module by default.

This change will mean that code entered at the interactive prompt and code in
single file scripts will revert to reporting these warnings by default, while
they will continue to be silenced by default for packaged code distributed as
part of an importable module.

The PEP also proposes a number of small adjustments to the reference
interpreter and standard library documentation to help make the warnings
subsystem more approachable for new Python developers.

As part of the documentation updates, it will be made clearer that the
``unittest`` test runner displays all warnings by default when executing
test cases, and that other test runners are advised to follow that example.


Specification
=============


New default warnings filter entry
---------------------------------

The current set of default warnings filters consists of::

    ignore::DeprecationWarning
    ignore::PendingDeprecationWarning
    ignore::ImportWarning
    ignore::BytesWarning
    ignore::ResourceWarning

The default ``unittest`` test runner then uses ``warnings.catch_warnings()``
``warnings.simplefilter('default')`` to override the default filters while
running test cases.

The change proposed in this PEP is to update the default warning filter list
to be::

    default::DeprecationWarning:__main__
    ignore::DeprecationWarning
    ignore::PendingDeprecationWarning
    ignore::ImportWarning
    ignore::BytesWarning
    ignore::ResourceWarning

This means that in cases where the nominal location of the warning (as
determined by the ``stacklevel`` parameter to ``warnings.warn``) is in the
``__main__`` module, the first occurrence of each ``DeprecationWarning`` will once
again be reported.

This change will lead to ``DeprecationWarning`` being displayed by default for:

* code executed directly at the interactive prompt
* code executed directly as part of a single-file script

While continuing to be hidden by default for:

* code imported from another module in a ``zipapp`` archive's ``__main__.py``
  file
* code imported from another module in an executable package's ``__main__``
  submodule
* code imported from an executable script wrapper generated at installation time
  based on a ``console_scripts`` or ``gui_scripts`` entry point definition

This means that tool developers that create an installable or executable
artifact (such as a ``zipapp`` archive) for distribution to their users
shouldn't see any change from the status quo, while users of more ad hoc
personal or locally distributed scripts are likely to start seeing relevant
deprecation warnings again (as they did in Python 2.6 and earlier).


Additional use case for ``FutureWarning``
-----------------------------------------

The standard library documentation will be updated to explicitly recommend the
use of ``FutureWarning`` (rather than ``DeprecationWarning``) for backwards
compatibility warnings that are intended to be seen by *users* of an
application. (This will be in addition to the existing use of ``FutureWarning``
to warn about constructs that will remain valid code in the future,
but will have different semantics).

This will give the following three distinct categories of backwards
compatibility warning, with three different intended audiences:

* ``PendingDeprecationWarning``: hidden by default for all code.
  The intended audience is Python developers that take an active interest in
  ensuring the future compatibility of their software (e.g. professional
  Python application developers with specific support obligations).
* ``DeprecationWarning``: reported by default for code that runs directly in
  the ``__main__`` module (as such code is considered relatively unlikely to
  have a dedicated test suite), but hidden by default for code in other modules.
  The intended audience is Python developers that are at risk of upgrades to
  their dependencies (including upgrades to Python itself) breaking their
  software (e.g. developers using Python to script environments where someone
  else is in control of the timing of dependency upgrades).
* ``FutureWarning``: reported by default for all code.
  The intended audience is users of applications written in Python, rather than
  other Python developers (e.g. warning about use of a deprecated setting in a
  configuration file format).

For library and framework authors that want to ensure their API compatibility
warnings are more reliably seen by their users, the recommendation is to use a
custom warning class that derives from ``DeprecationWarning`` in Python 3.7+,
and from ``FutureWarning`` in earlier versions.


Recommended filter settings for test runners
--------------------------------------------

Developers of test runners are advised to implement logic equivalent to the
following when determining their default warnings filters::

    if not sys.warnoptions:
        warnings.simplefilter("default")

This effectively enables all warnings by default, as if the ``-Wd`` command
line option had been passed.

Note that actually enabling ``BytesWarning`` in a test suite still requires
passing the ``-b`` option to the interpreter at the command line. For implicit
bytes conversion and bytes comparison warnings, the warnings filter machinery
is only used to determine whether they should be printed as warnings or raised
as exceptions - when the command line flag isn't set, the interpreter doesn't
even emit the warning in the first place.


Recommended filter settings for interactive shells
--------------------------------------------------

Developers of interactive shells are advised to add a filter that enables
``DeprecationWarning`` in the namespace where user code is entered and executed.

If that namespace is ``__main__`` (as it is for the default CPython REPL), then
no changes are needed beyond those in this PEP.

Interactive shell implementations which use a namespace other than
``__main__`` will need to add their own filter. For example, IPython uses the
following command ([6]_) to set up a suitable filter::

    warnings.filterwarnings("default", category=DeprecationWarning,
                                       module=self.user_ns.get("__name__"))


Other documentation updates
---------------------------

The current reference documentation for the warnings system is relatively short
on specific *examples* of possible settings for the ``-W`` command line option
or the ``PYTHONWARNINGS`` environment variably that achieve particular end
results.

The following improvements are proposed as part of the implementation of this
PEP:

* Explicitly list the following entries under the description of the
  ``PYTHONWARNINGS`` environment variable::

        PYTHONWARNINGS=error # Convert to exceptions
        PYTHONWARNINGS=always # Warn every time
        PYTHONWA


id: pep-0203.rst
content:
PEP: 203
Title: Augmented Assignments
Author: Thomas Wouters <thomas@python.org>
Status: Final
Type: Standards Track
Created: 13-Jul-2000
Python-Version: 2.0
Post-History: 14-Aug-2000


Introduction
============

This PEP describes the *augmented assignment* proposal for Python 2.0.  This
PEP tracks the status and ownership of this feature, slated for introduction
in Python 2.0.  It contains a description of the feature and outlines changes
necessary to support the feature.  This PEP summarizes discussions held in
mailing list forums [1]_, and provides URLs for further information where
appropriate.  The CVS revision history of this file contains the definitive
historical record.


Proposed Semantics
==================

The proposed patch that adds augmented assignment to Python introduces the
following new operators::

+= -= *= /= %= **= <<= >>= &= ^= |=

They implement the same operator as their normal binary form, except that the
operation is done *in-place* when the left-hand side object supports it, and
that the left-hand side is only evaluated once.

They truly behave as augmented assignment, in that they perform all of the
normal load and store operations, in addition to the binary operation they are
intended to do.  So, given the expression::

    x += y

The object ``x`` is loaded, then ``y`` is added to it, and the resulting
object is stored back in the original place.  The precise action performed on
the two arguments depends on the type of ``x``, and possibly of ``y``.

The idea behind augmented assignment in Python is that it isn't just an easier
way to write the common practice of storing the result of a binary operation
in its left-hand operand, but also a way for the left-hand operand in question
to know that it should operate *on itself*, rather than creating a modified
copy of itself.

To make this possible, a number of new *hooks* are added to Python classes and
C extension types, which are called when the object in question is used as the
left hand side of an augmented assignment operation.  If the class or type
does not implement the *in-place* hooks, the normal hooks for the particular
binary operation are used.

So, given an instance object ``x``, the expression::

    x += y

tries to call ``x.__iadd__(y)``, which is the *in-place* variant of
``__add__`` .  If ``__iadd__`` is not present, ``x.__add__(y)`` is attempted,
and finally ``y.__radd__(x)`` if ``__add__`` is missing too.  There is no
*right-hand-side* variant of ``__iadd__``, because that would require for
``y`` to know how to in-place modify ``x``, which is unsafe to say the least.
The ``__iadd__`` hook should behave similar to ``__add__``, returning the
result of the operation (which could be ``self``) which is to be assigned to
the variable ``x``.

For C extension types, the *hooks* are members of the ``PyNumberMethods`` and
``PySequenceMethods`` structures.  Some special semantics apply to make the
use of these methods, and the mixing of Python instance objects and C types,
as unsurprising as possible.

In the generic case of ``x <augop> y`` (or a similar case using the
``PyNumber_InPlace`` API functions) the principal object being operated on is
``x``.  This differs from normal binary operations, where ``x`` and ``y``
could be considered *co-operating*, because unlike in binary operations, the
operands in an in-place operation cannot be swapped.  However, in-place
operations do fall back to normal binary operations when in-place modification
is not supported, resulting in the following rules:

- If the left-hand object (``x``) is an instance object, and it has a
  ``__coerce__`` method, call that function with ``y`` as the argument.  If
  coercion succeeds, and the resulting left-hand object is a different object
  than ``x``, stop processing it as in-place and call the appropriate function
  for the normal binary operation, with the coerced ``x`` and ``y`` as
  arguments.  The result of the operation is whatever that function returns.

  If coercion does not yield a different object for ``x``, or ``x`` does not
  define a ``__coerce__`` method, and ``x`` has the appropriate ``__ihook__``
  for this operation, call that method with ``y`` as the argument, and the
  result of the operation is whatever that method returns.

- Otherwise, if the left-hand object is not an instance object, but its type
  does define the in-place function for this operation, call that function
  with ``x`` and ``y`` as the arguments, and the result of the operation is
  whatever that function returns.

  Note that no coercion on either ``x`` or ``y`` is done in this case, and
  it's perfectly valid for a C type to receive an instance object as the
  second argument; that is something that cannot happen with normal binary
  operations.

- Otherwise, process it exactly as a normal binary operation (not in-place),
  including argument coercion.  In short, if either argument is an instance
  object, resolve the operation through ``__coerce__``, ``__hook__`` and
  ``__rhook__``.  Otherwise, both objects are C types, and they are coerced
  and passed to the appropriate function.

- If no way to process the operation can be found, raise a ``TypeError`` with
  an error message specific to the operation.

- Some special casing exists to account for the case of ``+`` and ``*``,
  which have a special meaning for sequences: for ``+``, sequence
  concatenation, no coercion what so ever is done if a C type defines
  ``sq_concat`` or ``sq_inplace_concat``.  For ``*``, sequence repeating,
  ``y`` is converted to a C integer before calling either
  ``sq_inplace_repeat`` and ``sq_repeat``.  This is done even if ``y`` is an
  instance, though not if ``x`` is an instance.

The in-place function should always return a new reference, either to the
old ``x`` object if the operation was indeed performed in-place, or to a new
object.


Rationale
=========

There are two main reasons for adding this feature to Python: simplicity of
expression, and support for in-place operations. The end result is a tradeoff
between simplicity of syntax and simplicity of expression; like most new
features, augmented assignment doesn't add anything that was previously
impossible. It merely makes these things easier to do.

Adding augmented assignment will make Python's syntax more complex.  Instead
of a single assignment operation, there are now twelve assignment operations,
eleven of which also perform a binary operation.  However, these eleven new
forms of assignment are easy to understand as the coupling between assignment
and the binary operation, and they require no large conceptual leap to
understand.  Furthermore, languages that do have augmented assignment have
shown that they are a popular, much used feature. Expressions of the form::

    <x> = <x> <operator> <y>

are common enough in those languages to make the extra syntax worthwhile, and
Python does not have significantly fewer of those expressions.  Quite the
opposite, in fact, since in Python you can also concatenate lists with a
binary operator, something that is done quite frequently.  Writing the above
expression as::

    <x> <operator>= <y>

is both more readable and less error prone, because it is instantly obvious to
the reader that it is ``<x>`` that is being changed, and not ``<x>`` that is
being replaced by something almost, but not quite, entirely unlike ``<x>``.

The new in-place operations are especially useful to matrix calculation and
other applications that require large objects.  In order to efficiently deal
with the available program memory, such packages cannot blindly use the
current binary operations.  Because these operations always create a new
object, adding a single item to an existing (large) object would result in
copying the entire object (which may cause the application to run out of
memory), add the single item, and then possibly delete the original object,
depending on reference count.

To work around this problem, the packages curr


id: pep-0217.rst
content:
PEP: 217
Title: Display Hook for Interactive Use
Author: Moshe Zadka <moshez@zadka.site.co.il>
Status: Final
Type: Standards Track
Created: 31-Jul-2000
Python-Version: 2.1
Post-History:


Abstract
========

Python's interactive mode is one of the implementation's great
strengths -- being able to write expressions on the command line
and get back a meaningful output.  However, the output function
cannot be all things to all people, and the current output
function too often falls short of this goal.  This PEP describes a
way to provides alternatives to the built-in display function in
Python, so users will have control over the output from the
interactive interpreter.


Interface
=========

The current Python solution has worked for many users, and this
should not break it. Therefore, in the default configuration,
nothing will change in the REPL loop. To change the way the
interpreter prints interactively entered expressions, users
will have to rebind ``sys.displayhook`` to a callable object.
The result of calling this object with the result of the
interactively entered expression should be print-able,
and this is what will be printed on ``sys.stdout``.


Solution
========

The bytecode ``PRINT_EXPR`` will call ``sys.displayhook(POP())``.
A ``displayhook()`` will be added to the sys builtin module, which is
equivalent to::

    import __builtin__
    def displayhook(o):
        if o is None:
            return
        __builtin__._ = None
        print `o`
        __builtin__._ = o


Jython Issues
=============

The method ``Py.printResult`` will be similarly changed.


id: pep-0571.rst
content:
PEP: 571
Title: The manylinux2010 Platform Tag
Author: Mark  Williams <mrw@enotuniq.org>,
        Geoffrey Thomas <geofft@ldpreload.com>,
        Thomas Kluyver <thomas@kluyver.me.uk>
BDFL-Delegate: Alyssa Coghlan <ncoghlan@gmail.com>
Discussions-To: distutils-sig@python.org
Status: Superseded
Type: Informational
Topic: Packaging
Created: 05-Feb-2018
Post-History:
Superseded-By: 600
Resolution: https://mail.python.org/pipermail/distutils-sig/2018-April/032156.html


Abstract
========

This PEP proposes the creation of a ``manylinux2010`` platform tag to
succeed the ``manylinux1`` tag introduced by :pep:`513`.  It also
proposes that PyPI and ``pip`` both be updated to support uploading,
downloading, and installing ``manylinux2010`` distributions on compatible
platforms.

Rationale
=========

True to its name, the ``manylinux1`` platform tag has made the
installation of binary extension modules a reality on many Linux
systems.  Libraries like ``cryptography`` [2]_ and ``numpy`` [3]_ are
more accessible to Python developers now that their installation on
common architectures does not depend on fragile development
environments and build toolchains.

``manylinux1`` wheels achieve their portability by allowing the
extension modules they contain to link against only a small set of
system-level shared libraries that export versioned symbols old enough
to benefit from backwards-compatibility policies.  Extension modules
in a ``manylinux1`` wheel that rely on ``glibc``, for example, must be
built against version 2.5 or earlier; they may then be run systems
that provide more recent ``glibc`` version that still export the
required symbols at version 2.5.

:pep:`513` drew its whitelisted shared libraries and their symbol
versions from CentOS 5.11, which was the oldest supported CentOS
release at the time of its writing.  Unfortunately, CentOS 5.11
reached its end-of-life on March 31st, 2017 with a clear warning
against its continued use. [4]_ No further updates, such as security
patches, will be made available.  This means that its packages will
remain at obsolete versions that hamper the efforts of Python software
packagers who use the ``manylinux1`` Docker image.

CentOS 6 is now the oldest supported CentOS release, and will receive
maintenance updates through November 30th, 2020. [5]_ We propose that
a new :pep:`425`-style platform tag called ``manylinux2010`` be derived
from CentOS 6 and that the ``manylinux`` toolchain, PyPI, and ``pip``
be updated to support it.

This was originally proposed as ``manylinux2``, but the versioning has
been changed to use calendar years (also known as CalVer [22]_). This
makes it easier to define future *manylinux* tags out of order: for
example, a hypothetical ``manylinux2017`` standard may be defined via
a new PEP before ``manylinux2014``, or a ``manylinux2007`` standard
might be defined that targets systems older than this PEP but newer
than ``manylinux1``.

Calendar versioning also gives a rough idea of which Linux
distribution versions support which tag: ``manylinux2010`` will work
on most distribution versions released since 2010. This is only an
approximation, however: the actual compatibility rules are defined
below, and some newer distributions may not meet them.

The ``manylinux2010`` policy
============================

The following criteria determine a ``linux`` wheel's eligibility for
the ``manylinux2010`` tag:

1. The wheel may only contain binary executables and shared objects
   compiled for one of the two architectures supported by CentOS 6:
   x86_64 or i686. [5]_
2. The wheel's binary executables or shared objects may not link
   against externally-provided libraries except those in the following
   whitelist: ::

       libgcc_s.so.1
       libstdc++.so.6
       libm.so.6
       libdl.so.2
       librt.so.1
       libc.so.6
       libnsl.so.1
       libutil.so.1
       libpthread.so.0
       libresolv.so.2
       libX11.so.6
       libXext.so.6
       libXrender.so.1
       libICE.so.6
       libSM.so.6
       libGL.so.1
       libgobject-2.0.so.0
       libgthread-2.0.so.0
       libglib-2.0.so.0

   This list is identical to the externally-provided libraries
   whitelisted for ``manylinux1``, minus ``libncursesw.so.5`` and
   ``libpanelw.so.5``. [7]_ ``libpythonX.Y`` remains ineligible for
   inclusion for the same reasons outlined in :pep:`513`.

   ``libcrypt.so.1`` was retrospectively removed from the whitelist after
   Fedora 30 was released with ``libcrypt.so.2`` instead.

   On Debian-based systems, these libraries are provided by the packages:

   ============  =======================================================
   Package       Libraries
   ============  =======================================================
   libc6         libdl.so.2, libresolv.so.2, librt.so.1, libc.so.6,
                 libpthread.so.0, libm.so.6, libutil.so.1, libnsl.so.1
   libgcc1       libgcc_s.so.1
   libgl1        libGL.so.1
   libglib2.0-0  libgobject-2.0.so.0, libgthread-2.0.so.0, libglib-2.0.so.0
   libice6       libICE.so.6
   libsm6        libSM.so.6
   libstdc++6    libstdc++.so.6
   libx11-6      libX11.so.6
   libxext6      libXext.so.6
   libxrender1   libXrender.so.1
   ============  =======================================================

   On RPM-based systems, they are provided by these packages:

   ============  =======================================================
   Package       Libraries
   ============  =======================================================
   glib2         libglib-2.0.so.0, libgthread-2.0.so.0, libgobject-2.0.so.0
   glibc         libresolv.so.2, libutil.so.1, libnsl.so.1, librt.so.1,
                 libpthread.so.0, libdl.so.2, libm.so.6, libc.so.6
   libICE        libICE.so.6
   libX11        libX11.so.6
   libXext:      libXext.so.6
   libXrender    libXrender.so.1
   libgcc:       libgcc_s.so.1
   libstdc++     libstdc++.so.6
   mesa          libGL.so.1
   ============  =======================================================

3. If the wheel contains binary executables or shared objects linked
   against any whitelisted libraries that also export versioned
   symbols, they may only depend on the following maximum versions::

       GLIBC_2.12
       CXXABI_1.3.3
       GLIBCXX_3.4.13
       GCC_4.5.0

   As an example, ``manylinux2010`` wheels may include binary artifacts
   that require ``glibc`` symbols at version ``GLIBC_2.4``, because
   this an earlier version than the maximum of ``GLIBC_2.12``.
4. If a wheel is built for any version of CPython 2 or CPython
   versions 3.0 up to and including 3.2, it *must* include a CPython
   ABI tag indicating its Unicode ABI.  A ``manylinux2010`` wheel built
   against Python 2, then, must include either the ``cpy27mu`` tag
   indicating it was built against an interpreter with the UCS-4 ABI
   or the ``cpy27m`` tag indicating an interpreter with the UCS-2
   ABI. (:pep:`3149`, [9]_)
5. A wheel *must not* require the ``PyFPE_jbuf`` symbol.  This is
   achieved by building it against a Python compiled *without* the
   ``--with-fpectl`` ``configure`` flag.

Compilation of Compliant Wheels
===============================

Like ``manylinux1``, the ``auditwheel`` tool adds ``manylinux2010``
platform tags to ``linux`` wheels built by ``pip wheel`` or
``bdist_wheel`` in a ``manylinux2010`` Docker container.

Docker Image
------------

Two ``manylinux2010`` Docker images based on CentOS 6 are
provided for building binary ``linux`` wheels that can reliably be
converted to ``manylinux2010`` wheels.  [10]_ The x86_64 and i686 images comes with a
new compiler suite installed (``gcc``, ``g++``, and ``gfortran``
from ``devtoolset-8``) as well as the latest releases of Python and ``pip``.

Compatibility with kernels that lack ``vsyscall``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A Docker container assumes that its userland is compatible with its
host's kernel.  Unfortunately, an increasingly common kernel
configuration breaks th


id: pep-0001.rst
content:
PEP: 1
Title: PEP Purpose and Guidelines
Author: Barry Warsaw, Jeremy Hylton, David Goodger, Alyssa Coghlan
Status: Active
Type: Process
Created: 13-Jun-2000
Post-History: 21-Mar-2001, 29-Jul-2002, 03-May-2003, 05-May-2012,
              07-Apr-2013


What is a PEP?
==============

PEP stands for Python Enhancement Proposal.  A PEP is a design
document providing information to the Python community, or describing
a new feature for Python or its processes or environment.  The PEP
should provide a concise technical specification of the feature and a
rationale for the feature.

We intend PEPs to be the primary mechanisms for proposing major new
features, for collecting community input on an issue, and for
documenting the design decisions that have gone into Python.  The PEP
author is responsible for building consensus within the community and
documenting dissenting opinions.

Because the PEPs are maintained as text files in a versioned
repository, their revision history is the historical record of the
feature proposal. This historical record is available by the normal git
commands for retrieving older revisions, and can also be browsed
`on GitHub <https://github.com/python/peps>`__.


PEP Audience
============

The typical primary audience for PEPs are the core developers of the CPython
reference interpreter and their elected Steering Council, as well as developers
of other implementations of the Python language specification.

However, other parts of the Python community may also choose to use the process
(particularly for Informational PEPs) to document expected API conventions and
to manage complex design coordination problems that require collaboration across
multiple projects.


PEP Types
=========

There are three kinds of PEP:

1. A **Standards Track** PEP describes a new feature or implementation
   for Python. It may also describe an interoperability standard that will
   be supported outside the standard library for current Python versions
   before a subsequent PEP adds standard library support in a future
   version.

2. An **Informational** PEP describes a Python design issue, or
   provides general guidelines or information to the Python community,
   but does not propose a new feature.  Informational PEPs do not
   necessarily represent a Python community consensus or
   recommendation, so users and implementers are free to ignore
   Informational PEPs or follow their advice.

3. A **Process** PEP describes a process surrounding Python, or
   proposes a change to (or an event in) a process.  Process PEPs are
   like Standards Track PEPs but apply to areas other than the Python
   language itself.  They may propose an implementation, but not to
   Python's codebase; they often require community consensus; unlike
   Informational PEPs, they are more than recommendations, and users
   are typically not free to ignore them.  Examples include
   procedures, guidelines, changes to the decision-making process, and
   changes to the tools or environment used in Python development.
   Any meta-PEP is also considered a Process PEP.


PEP Workflow
============

Python's Steering Council
-------------------------

There are several references in this PEP to the "Steering Council" or "Council".
This refers to the current members of the elected Steering Council described
in :pep:`13`, in their role as the final authorities on whether or not PEPs
will be accepted or rejected.


Python's Core Developers
------------------------

There are several references in this PEP to "core developers". This refers to
the currently active Python core team members described in :pep:`13`.


Python's BDFL
-------------

Previous versions of this PEP used the title "BDFL-Delegate" for PEP decision
makers. This was a historical reference to Python's previous governance model,
where all design authority ultimately derived from Guido van Rossum, the
original creator of the Python programming language. By contrast, the Steering
Council's design authority derives from their election by the currently active
core developers. Now, PEP-Delegate is used in place of BDFL-Delegate.


PEP Editors
-----------

The PEP editors are individuals responsible for managing the administrative
and editorial aspects of the PEP workflow (e.g. assigning PEP numbers and
changing their status).  See `PEP Editor Responsibilities & Workflow`_ for
details.

PEP editorship is by invitation of the current editors, and they can be
contacted by mentioning ``@python/pep-editors`` on GitHub.  All of the PEP
workflow can be conducted via the GitHub `PEP repository`_ issues and pull
requests.


Start with an idea for Python
-----------------------------

The PEP process begins with a new idea for Python.  It is highly
recommended that a single PEP contain a single key proposal or new
idea; the more focused the PEP, the more successful it tends to be.
Most enhancements and bug fixes don't need a PEP and
can be submitted directly to the `Python issue tracker`_.
The PEP editors reserve the
right to reject PEP proposals if they appear too unfocused or too
broad.  If in doubt, split your PEP into several well-focused ones.

Each PEP must have a champion -- someone who writes the PEP using the style
and format described below, shepherds the discussions in the appropriate
forums, and attempts to build community consensus around the idea.  The PEP
champion (a.k.a. Author) should first attempt to ascertain whether the idea is
PEP-able.  Posting to the `Ideas category`_ of the `Python Discourse`_ is usually
the best way to go about this, unless a more specialized venue is appropriate,
such as the `Typing category`_ (for static typing ideas)
or `Packaging category`_ (for packaging ideas) on the Python Discourse.

Vetting an idea publicly before going as far as writing a PEP is meant
to save the potential author time. Many ideas have been brought
forward for changing Python that have been rejected for various
reasons. Asking the Python community first if an idea is original
helps prevent too much time being spent on something that is
guaranteed to be rejected based on prior discussions (searching
the internet does not always do the trick). It also helps to make sure
the idea is applicable to the entire community and not just the author.
Just because an idea sounds good to the author does not
mean it will work for most people in most areas where Python is used.

Once the champion has asked the Python community as to whether an
idea has any chance of acceptance, a draft PEP should be presented to
the appropriate venue mentioned above.
This gives the author a chance to flesh out the draft
PEP to make properly formatted, of high quality, and to address
initial concerns about the proposal.


Submitting a PEP
----------------

Following the above initial discussion, the workflow varies based on whether
any of the PEP's co-authors are core developers. If one or more of the PEP's
co-authors are core developers, they are responsible for following the process
outlined below. Otherwise (i.e. none of the co-authors are core developers),
then the PEP author(s) will need to find a sponsor for the PEP.

Ideally, a core developer sponsor is identified, but non-core sponsors may also
be selected with the approval of the Steering Council.  Members of the GitHub
"PEP editors" team and members of the Typing Council (:pep:`729`) are
pre-approved to be sponsors.  The sponsor's job is to
provide guidance to the PEP author to help them through the logistics of the
PEP process (somewhat acting like a mentor).  Being a sponsor does **not**
disqualify that person from becoming a co-author or PEP-Delegate later on (but
not both).  The sponsor of a PEP is recorded in the "Sponsor:" field of the
header.

Once the sponsor or the core developer(s) co-authoring the PEP deem the PEP
ready for submission, the proposal should be submitted as a draft PEP via a
`GitHub pull request`_.  The draft must be written in PEP style as described
below, else it will 


id: pep-0767.rst
content:
PEP: 767
Title: Annotating Read-Only Attributes
Author: Eneg <eneg at discuss.python.org>
Sponsor: Carl Meyer <carl@oddbird.net>
Discussions-To: https://discuss.python.org/t/pep-767-annotating-read-only-attributes/73408
Status: Draft
Type: Standards Track
Topic: Typing
Created: 18-Nov-2024
Python-Version: 3.15
Post-History: `09-Oct-2024 <https://discuss.python.org/t/expanding-readonly-to-normal-classes-protocols/67359>`__


Abstract
========

:pep:`705` introduced the :external+py3.13:data:`typing.ReadOnly` type qualifier
to allow defining read-only :class:`typing.TypedDict` items.

This PEP proposes using ``ReadOnly`` in :term:`annotations <annotation>` of class and protocol
:term:`attributes <attribute>`, as a single concise way to mark them read-only.

Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correct
usage of read-only attributes is intended to be enforced only by static type checkers.


Motivation
==========

The Python type system lacks a single concise way to mark an attribute read-only.
This feature is present in other statically and gradually typed languages
(such as `C# <https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/readonly>`_
or `TypeScript <https://www.typescriptlang.org/docs/handbook/2/objects.html#readonly-properties>`_),
and is useful for removing the ability to reassign or ``del``\ ete an attribute
at a type checker level, as well as defining a broad interface for structural subtyping.

.. _classes:

Classes
-------

Today, there are three major ways of achieving read-only attributes, honored by type checkers:

* annotating the attribute with :data:`typing.Final`::

    class Foo:
        number: Final[int]

        def __init__(self, number: int) -> None:
            self.number = number


    class Bar:
        def __init__(self, number: int) -> None:
            self.number: Final = number

  - Supported by :mod:`dataclasses` (and type checkers since `typing#1669 <https://github.com/python/typing/pull/1669>`_).
  - Overriding ``number`` is not possible - the specification of ``Final``
    imposes that the name cannot be overridden in subclasses.

* read-only proxy via ``@property``::

    class Foo:
        _number: int

        def __init__(self, number: int) -> None:
            self._number = number

        @property
        def number(self) -> int:
            return self._number

  - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_
  - Read-only at runtime. [#runtime]_
  - Requires extra boilerplate.
  - Supported by :mod:`dataclasses`, but does not compose well - the synthesized
    ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name.

* using a "freezing" mechanism, such as :func:`dataclasses.dataclass` or :class:`typing.NamedTuple`::

    @dataclass(frozen=True)
    class Foo:
        number: int  # implicitly read-only


    class Bar(NamedTuple):
        number: int  # implicitly read-only

  - Overriding ``number`` is possible in the ``@dataclass`` case.
  - Read-only at runtime. [#runtime]_
  - No per-attribute control - these mechanisms apply to the whole class.
  - Frozen dataclasses incur some runtime overhead.
  - ``NamedTuple`` is still a ``tuple``. Most classes do not need to inherit
    indexing, iteration, or concatenation.

.. _protocols:

Protocols
---------

Suppose a :class:`~typing.Protocol` member ``name: T`` defining two requirements:

1. ``hasattr(obj, "name")``
2. ``isinstance(obj.name, T)``

Those requirements are satisfiable at runtime by all of the following:

* an object with an attribute ``name: T``,
* a class with a class variable ``name: ClassVar[T]``,
* an instance of the class above,
* an object with a ``@property`` ``def name(self) -> T``,
* an object with a custom descriptor, such as :func:`functools.cached_property`.

The current `typing spec <https://typing.python.org/en/latest/spec/protocol.html#protocol-members>`_
allows creation of such protocol members using (abstract) properties::

    class HasName(Protocol):
        @property
        def name(self) -> T: ...

This syntax has several drawbacks:

* It is somewhat verbose.
* It is not obvious that the quality conveyed here is the read-only character of a property.
* It is not composable with :external+typing:term:`type qualifiers <type qualifier>`.
* Not all type checkers agree [#property_in_protocol]_ that all of the above five
  objects are assignable to this structural type.

Rationale
=========

These problems can be resolved by an attribute-level type qualifier.
``ReadOnly`` has been chosen for this role, as its name conveys the intent well,
and the newly proposed changes complement its semantics defined in :pep:`705`.

A class with a read-only instance attribute can now be defined as::

    from typing import ReadOnly


    class Member:
        def __init__(self, id: int) -> None:
            self.id: ReadOnly[int] = id

...and the protocol described in :ref:`protocols` is now just::

    from typing import Protocol, ReadOnly


    class HasName(Protocol):
        name: ReadOnly[str]


    def greet(obj: HasName, /) -> str:
        return f"Hello, {obj.name}!"

* A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a
  :term:`descriptor`. It can also :external+typing:term:`narrow` the type.
* The ``HasName`` protocol has a more succinct definition, and is agnostic
  to the writability of the attribute.
* The ``greet`` function can now accept a wide variety of compatible objects,
  while being explicit about no modifications being done to the input.


Specification
=============

The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier`
becomes a valid annotation for :term:`attributes <attribute>` of classes and protocols.
It can be used at class-level or within ``__init__`` to mark individual attributes read-only::

    class Book:
        id: ReadOnly[int]

        def __init__(self, id: int, name: str) -> None:
            self.id = id
            self.name: ReadOnly[str] = name

Type checkers should error on any attempt to reassign or ``del``\ ete an attribute
annotated with ``ReadOnly``.
Type checkers should also error on any attempt to delete an attribute annotated as ``Final``.
(This is not currently specified.)

Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning
(such as local/global variables or function parameters) is considered out of scope
for this PEP.

Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how
type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs <abstract base class>`
and :mod:`containers <collections.abc>` may be used in combination with ``ReadOnly``
to forbid mutation of such values at a type checker level:

.. code-block:: python

    from collections import abc
    from dataclasses import dataclass
    from typing import Protocol, ReadOnly


    @dataclass
    class Game:
        name: str


    class HasGames[T: abc.Collection[Game]](Protocol):
        games: ReadOnly[T]


    def add_games(shelf: HasGames[list[Game]]) -> None:
        shelf.games.append(Game("Half-Life"))  # ok: list is mutable
        shelf.games[-1].name = "Black Mesa"    # ok: "name" is not read-only
        shelf.games = []                       # error: "games" is read-only
        del shelf.games                        # error: "games" is read-only and cannot be deleted


    def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None:
        shelf.games.append(...)             # error: "Sequence" has no attribute "append"
        shelf.games[0].name = "Blue Shift"  # ok: "name" is not read-only
        shelf.games = []                    # error: "games" is read-only


All instance attributes of frozen dataclasses and ``NamedTuple`` should be
implied to be read-only. Type checkers may inform that annotating such attributes
with ``ReadOnly`` 


id: pep-0773.rst
content:
PEP: 773
Title: A Python Installation Manager for Windows
Author: Steve Dower
Discussions-To: https://discuss.python.org/t/77900/
Status: Accepted
Type: Standards Track
Topic: Release
Created: 21-Jan-2025
Post-History:
   `18-Dec-2024 <https://discuss.python.org/t/74556/>`__,
   `21-Jan-2025 <https://discuss.python.org/t/77900/>`__,
Replaces: 397, 486
Resolution: `25-Apr-2025 <https://discuss.python.org/t/77900/180>`__


Abstract
========

Installation of the python.org Python distribution on Windows is complex.
There are three main approaches with roughly equivalent levels of user
experience, and yet all of these suffer from different limitations, including
failing to satisfy modern usage scenarios. This PEP proposes a design for
a single Windows install workflow tool that satisfies all the needs of the
existing installers for the platform, while avoiding most of their limitations,
and provides the core team with the ability to manage releases for many years
to come.

With the new installer, the recommended command to launch the default Python on
Windows will be ``python``, while the command for managing multiple versions
(including launching a specific one) will be ``py``. Users can also opt-in to
having global ``python3.x`` commands by adding an extra entry to their ``PATH``
environment variable. We recommend starting with the "How to teach this" section
for the most concise high-level overview of the features.

The existing ``.exe`` installer and ``py`` launcher will be deprecated and no
longer released from two years after this PEP is accepted. The existing
embeddable distro will no longer be listed as a download, but will be available
through the install manager. The Windows Store app will be immediately replaced
with the install manager.

Reader's Note
-------------

This document is a detailed *proposal*, intended to assist in an "approve" or
"reject" decision on the replacement of the existing install formats for Python
on Windows with a new format. It is not intended to be a binding specification,
nor is it user documentation or an interoperability specification. The actual
implementation may vary over time as needs change, and that is not to be
considered a "violation" of this PEP. Any interfaces and protocols meant for
public use will be independently documented and maintained in accordance with
standard deprecation timelines.

Documentation on installing Python on Windows can be found at
`docs.python.org/using/windows.html <https://docs.python.org/using/windows.html>`__.

Background
==========

There are a large range of needs users may have that lead to them wanting
to install a Python runtime. Many, likely most, are interested in running
(perhaps writing) short scripts, such as those that perform a simple task,
or help teach someone a concept. Some users are looking for a specific version
to integrate with existing code or another application. Some are after a full
set of different interpreter versions to perform testing.

In this section, we discuss the expectations that users have of "installing
Python", provide an overview of the existing installers for Windows, and
identify some of the gaps and challenges inherent in these offerings.

Expectations
------------

Based on significant anecdotal experience and analysis of quantitative data
available (though not necessarily public), we make the following assertions
about the majority of Python users on Windows:

* most users just want the latest stable version
* most users want a "one-click" (or fewer) install
* most users do not want to use administrator privileges
* most users will benefit from installing maintenance updates
* most users expect the ``python`` command to work after installation

The primary support for these assertions is that the most popular installers
actively chosen by users are the latest stable release on python.org, and
the latest stable release on the Windows Store, both of which meet these
requirements.

We make the following assumptions about other significant sets of users.
These may have some overlap between groups, and at least some users expect
all of them.

* some users want to install Python programmatically
* some users want to install a particular version
* some users want to install many versions
* some users want to install for all users of their machine
* some users do not want Start Menu shortcuts
* some users want to install as part of their project's build process
* some users want to install as part of their project's install process
* some users intend to never update their install
* some users expect the ``py`` command to work after installation

These assumptions have all been demonstrated over time to exist, though the
relative importance has not been quantified. The :ref:`NuGet packages
<windows-nuget>` and the :ref:`embeddable distro
<windows-embeddable>` can meet most of these needs.

Traditional Installer
---------------------

The traditional installer is an executable downloadable directly from
python.org that installs the entire development kit for Python. This includes
the CPython interpreter, the standard library, Python headers and import
libraries, builds of Tcl and Tk, the documentation as HTML files, the runtime
and standard library test suite, Start Menu shortcuts for Python and IDLE,
debugging symbols and debug builds of the binaries, the ``py.exe`` launcher
and its file associations, and functionality to modify the user's ``PATH``
environment variable, enable long-path support on their system, pre-generate
``.pyc`` files for the standard library, and install pip.
As of 3.13, it also includes a set of experimental free-threaded binaries.
Many of these components are optional.

After downloading the executable, users are presented with a "quick install"
option, which installs into their user directory with most options enabled.
We believe that most users select this option.

A second option alongside the quick install takes the user to two pages worth
of options, listing the components that they need not install, as well as other
options such as the install directory and whether to install for all users.

All of these options may be specified on the command line, and there is also an
option to proceed with the install without displaying any UI.
Based on feedback and bug reports, all of these options are used by at least
some users. However, as we do not track install telemetry, we have no way to
know which options are more important than others.

Behind the scenes, the traditional installer is a Burn bundle, generated using
the Wix Toolset installer framework, containing one or more MSI files for each
feature. This framework is used extensively by Microsoft themselves, and
provides the most direct method of using Windows Installer. The bundle is a
custom C++ application, based on their template, which allows us to customise
the overall behaviour of the installer to determine precisely which MSI files
should actually be installed. The process of copying files, updating the
registry, and generating shortcuts is handled entirely by Windows Installer.

As well as the intended uses, it is understood that many users will (attempt to)
use the traditional installer for other scenarios, such as unregistered installs
and automated CI system installs. While better alternatives are available, they
are not as obvious, and the hope is that a future design would make these
scenarios easier.

Windows Store
-------------

The Windows Store packages for CPython are produced as part of our normal
release process using almost all identical binaries to the other packages.
Due to being in an app store package, the primary ``python.exe`` is enhanced
to be able to determine its location properly, and alternative ``pip.exe`` and
other shortcuts are included to make up for the lack of ``PATH`` environment
variable settings. These are implemented in :file:`PC\\python_uwp.cpp` in our repo.

These packages are installed by searching for Python in the Mi


id: pep-0613.rst
content:
PEP: 613
Title: Explicit Type Aliases
Author: Shannon Zhu <szhu@fb.com>
Sponsor: Guido van Rossum <guido@python.org>
Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/MWRJOBEEEMFVXE7CAKO7B4P46IPM4AN3/
Status: Final
Type: Standards Track
Topic: Typing
Created: 21-Jan-2020
Python-Version: 3.10
Post-History: 21-Jan-2020

.. canonical-typing-spec:: :ref:`typing:type-aliases` and
                           :py:data:`typing.TypeAlias`

Abstract
========

Type aliases are user-specified types which may be as complex as any type hint,
and are specified with a simple variable assignment on a module top level.

This PEP formalizes a way to explicitly declare an assignment as a type alias.

Motivation
==========

Type aliases are declared as top level variable assignments.
In :pep:`PEP 484 <484#type-aliases>`,
the distinction between a valid type alias and a global variable was implicitly
determined: if a top level assignment is unannotated, and the assigned value is
a valid type, then the name being assigned to is a valid type alias. Otherwise,
that name is simply a global value that cannot be used as a type hint.

These implicit type alias declaration rules create confusion when type aliases
involve forward references, invalid types, or violate other restrictions
enforced on type alias declaration. Because the distinction between an
unannotated value and a type alias is implicit, ambiguous or incorrect type
alias declarations implicitly default to a valid value assignment. This creates
expressions that are impossible to express as type aliases and punts error
diagnosis of malformed type aliases downstream.

The following examples each include an illustration of some of the suboptimal
or confusing behaviors resulting from existing implicit alias declarations.
We also introduce explicit aliases of the format ``TypeName: TypeAlias = Expression``
here for the sake of comparison, but the syntax is discussed in further detail
in later sections.

Forward References:
*******************

::

  MyType = "ClassName"
  def foo() -> MyType: ...

This code snippet should not error so long as ``ClassName`` is defined
later on. However, a type checker is forced to assume that MyType is a value
assignment rather than a type alias, and therefore may throw spurious errors
that (1) ``MyType`` is an unannotated global string, and (2) ``MyType``
cannot be used as a return annotation because it is not a valid type.

::

  MyType: TypeAlias = "ClassName"
  def foo() -> MyType: ...

Explicit aliases remove ambiguity so neither of the above errors will be
thrown. Additionally, if something is wrong with ``ClassName``
(i.e., itâs not actually defined later), the type checker can throw an error.


Error Messaging:
****************

::

  MyType1 = InvalidType
  MyType2 = MyGeneric(int)  # i.e., intention was MyGeneric[int]

A type checker should warn on this code snippet that ``InvalidType`` is not
a valid type, and therefore cannot be used to annotate an expression or to
construct a type alias. Instead, type checkers are forced to throw spurious
errors that (1) ``MyType`` is a global expression missing an annotation,
and (2) ``MyType`` is not a valid type in all usages of ``MyType``
across the codebase.

::

  MyType1: TypeAlias = InvalidType
  MyType2: TypeAlias = MyGeneric(int)

With explicit aliases, the type checker has enough information to error on the
actual definition of the bad type alias, and explain why: that ``MyGeneric(int)``
and ``InvalidType`` are not valid types. When the value expression is no longer
evaluated as a global value, unactionable type errors on all usages of ``MyType``
across the codebase can be suppressed.

Scope Restrictions:
*******************

::

  class Foo:
    x = ClassName
    y: TypeAlias = ClassName
    z: Type[ClassName] = ClassName

Type aliases are valid within class scope, both implicitly (``x``) and
explicitly (``y``). If the line should be interpreted as a class
variable, it must be explicitly annotated (``z``).

::

  x = ClassName
  def foo() -> None:
    x = ClassName

The outer ``x`` is a valid type alias, but type checkers must error if the
inner ``x`` is ever used as a type because type aliases cannot be defined
inside of a function.
This is confusing because the alias declaration rule is not explicit, and because
a type error will not be thrown on the location of the inner type alias declaration
but rather on every one of its subsequent use cases.

::

  x: TypeAlias = ClassName
  def foo() -> None:
    x = ClassName
  def bar() -> None:
    x: TypeAlias = ClassName

With explicit aliases, the outer assignment is still a valid type variable.
Inside ``foo``, the inner assignment should be interpreted as ``x: Type[ClassName]``.
Inside ``bar``, the type checker should raise a clear error, communicating
to the author that type aliases cannot be defined inside a function.


Specification
=============

The explicit alias declaration syntax clearly differentiates between the three
possible kinds of assignments: typed global expressions, untyped global
expressions, and type aliases. This avoids the existence of assignments that
break type checking when an annotation is added, and avoids classifying the
nature of the assignment based on the type of the value.

Implicit syntax (pre-existing):

::

  x = 1  # untyped global expression
  x: int = 1  # typed global expression

  x = int  # type alias
  x: Type[int] = int  # typed global expression


Explicit syntax:

::

  x = 1  # untyped global expression
  x: int = 1  # typed global expression

  x = int  # untyped global expression (see note below)
  x: Type[int] = int  # typed global expression

  x: TypeAlias = int  # type alias
  x: TypeAlias = "MyClass"  # type alias


Note: The examples above illustrate implicit and explicit alias declarations in
isolation. For the sake of backwards compatibility, type checkers should support
both simultaneously, meaning an untyped global expression ``x = int`` will
still be considered a valid type alias.


Backwards Compatibility
=======================

Explicit aliases provide an alternative way to declare type aliases, but all
pre-existing code and old alias declarations will work as before.


Reference Implementation
========================

The `Pyre <https://pyre-check.org/>`_ type checker supports explicit type
alias declarations.


Rejected Ideas
==============

Some alternative syntaxes were considered for explicit aliases:

::

  MyType: TypeAlias[int]

This looks a lot like an uninitialized variable.

::

  MyType = TypeAlias[int]

Along with the option above, this format potentially adds confusion around
what the runtime value of ``MyType`` is.


In comparison, the chosen syntax option ``MyType: TypeAlias = int`` is
appealing because it still sticks with the ``MyType = int`` assignment
syntax, and adds some information for the type checker purely as an annotation.


Version History
===============

* 2021-11-16

  * Allow ``TypeAlias`` inside class scope


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


id: pep-0607.rst
content:
PEP: 607
Title: Reducing CPython's Feature Delivery Latency
Author: Åukasz Langa <lukasz@python.org>,
        Steve Dower <steve.dower@python.org>,
        Alyssa Coghlan <ncoghlan@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-607-shared-background-for-the-release-cadence-peps/2528
Status: Final
Type: Informational
Created: 11-Oct-2019
Python-Version: 3.9
Post-History: 20-Oct-2019

Abstract
========

:pep:`602` and :pep:`605` describe two alternative approaches to delivering smaller
collections of features to Python's users more frequently (as compared to the
current approach of offering new feature releases every 18-24 months, with
the first binary alpha release taking place 6-8 months before the final release).

Both PEPs also propose moving to a release cadence that results in full releases
occurring at a consistent time of year (every year for :pep:`602`, every other
year for :pep:`605`).

This PEP (from the authors of both competing proposals) provides common
background on *why* a change in the release cadence is considered desirable,
as well as the perceived risks that both PEPs attempt to mitigate.


Rationale for change
====================

Reducing the size of feature delivery batches
---------------------------------------------

When multiple large changes are delivered together, a complex investigation
may be required to determine the root cause of any new issues that arise.
Large batch sizes also make it more likely that problems *will* be encountered,
given that they include larger pieces of relatively untested code.

The easiest way to simplify those investigations and reduce the likelihood of
users encountering problems is to reduce the size of the batches being shipped.

:pep:`602` proposes to address this problem via the straightforward approach of
reducing CPython's typical batch size by 50%, shipping 12 months of changes
each time, rather than accumulating 18+ months of changes.

:pep:`605` proposes to address it by regularly delivering 2 months worth of changes
to a subset of Python's user base that opts in to running a rolling stream of
beta releases (similar to running Windows Insider builds instead of the Windows
retail release, or running Debian testing instead of Debian stable).


Reducing the latency of feature delivery
----------------------------------------

When only stable releases are seeing significant user adoption, and there's a
long period of time between stable releases, it creates an incredibly strong
temptation for developers to push changes into stable releases before they're
really ready for general use.

:pep:`602` proposes to address this problem by reducing the period of time
between stable releases to 12 months rather than 18 months.

:pep:`605` proposes to address it by actively creating a community of
Python users that regularly install and use CPython beta releases, providing an
incentive for core developers to start shipping changes earlier in the
pre-release cycle, in order to obtain feedback before the feature gets locked
down in a stable release.


Aligning the release cadence with the calendar year
---------------------------------------------------

While the current release cadence is nominally 18-24 months, in practice it has
consistently been towards the 18 month end of that range. This means that the
target dates for pre-releases and final releases move around from release to
release, and the only way to remember them is to either look at the release PEP,
or else to add those dates to your calendar. This is annoying for both
individual volunteers and for corporate contributors, and also complicates
alignment with events like PyCon US (typically April/May) and the now-annual
core development sprints (typically in September).

:pep:`602` proposes to address this problem by publishing a new release in October
every year, and basing the pre-release calendar for each year off that.

:pep:`605` proposes to address this problem by alternating between release years
(where a new stable release is published in August), and non-release years
(where only maintenance releases and new rolling beta releases are published).


Improving the pre-release design feedback cycle
-----------------------------------------------

One of the challenges of designing changes to the core interpreter and standard
library APIs is that the user base in a position to provide feedback on
nightly builds and the current pre-releases is relatively limited. This means
that much user feedback isn't received until after an API design has already
shipped in a full X.Y.0 release.

If the API is a regular API, then deprecation cycles mean that it may take
literally years to correct any design mistakes identified at that point.
Marking APIs as provisional nominally offers a way to avoid that constraint,
but actually taking advantage of that freedom causes other problems.

:pep:`602` proposes to address this problem by starting the alpha period
immediately after the previous stable release.

:pep:`605` proposes to address this problem by actively promoting adoption of
CPython pre-releases for running production workloads (not just for library and
application compatibility testing), and adjusting the pre-release management
process as necessary to make that a reasonable thing to do.

(Note: some standard library APIs are amenable to initially being shipped as
part of separately versioned packages via PyPI, and only later incorporated
into the standard library. This section is more about the lower level APIs
and non-library features where that approach to obtaining early design
feedback doesn't apply)


Risks to be mitigated
=====================

While the status quo could stand to be improved in some respects, Python's
popularity indicates that a lot of users and other participants in the wider
Python ecosystem are happy enough with the current release management process.

Python's user base is too large and
`too varied <https://www.curiousefficiency.org/posts/2017/10/considering-pythons-target-audience.html>`__
to cover all the potential downsides of changing our release cadence here, so
instead this section just covers some of the points that have been specifically
taken into account in the design of the PEPs.


Impact on users and redistributors that already skip some releases
------------------------------------------------------------------

It is already the case that not all users and redistributors update to every
published CPython release series (for example, Debian stable and Ubuntu LTS
sometimes skip releases due to the mismatch between their 24-month release
cycles and CPython's typically 18-month cycle).

The faster 12-month full release cadence in :pep:`602` means that users in this
category may end up skipping two releases where they would previously have only
skipped one. However, the extended notice period for deprecations means that
skipping a single release should no longer result in missed deprecation warnings.

The slower 24-month full release cadence in :pep:`605` may move some of the users
that have historically been in this category into the "update to every stable
release" category.


Impact on users and redistributors that update to every release
---------------------------------------------------------------

Many of Python's users never install a pre-release, but do update to every
stable release series at some point after it is published.

:pep:`602` aims to mitigate the potential negative impact on members of this group
by keeping the minimum gap between releases to 12 months, and retaining the
18 month full support period for each release.

Keeping the 18-month full support period for each release branch means that the
branches will spend roughly the same amount of time in full support and
security-fix-only mode as they do now (~18 months and ~42 months, respectively).

:pep:`605` aims to mitigate the potential negative impact on members of this group
by increasing use during the pre-relea


id: pep-0439.rst
content:
PEP: 439
Title: Inclusion of implicit pip bootstrap in Python installation
Author: Richard Jones <richard@python.org>
BDFL-Delegate:  Alyssa Coghlan <ncoghlan@gmail.com>
Discussions-To: distutils-sig@python.org
Status: Rejected
Type: Standards Track
Topic: Packaging
Created: 18-Mar-2013
Python-Version: 3.4
Post-History: 19-Mar-2013
Resolution: https://mail.python.org/pipermail/distutils-sig/2013-August/022527.html


Abstract
========

This PEP proposes the inclusion of a pip bootstrap executable in the
Python installation to simplify the use of 3rd-party modules by Python
users.

This PEP does not propose to include the pip implementation in the
Python standard library.  Nor does it propose to implement any package
management or installation mechanisms beyond those provided by PEP
427 ("The Wheel Binary Package Format 1.0") and TODO distlib PEP.


PEP Rejection
=============

This PEP has been rejected in favour of a more explicit mechanism that
should achieve the same end result in a more reliable fashion. The more
explicit bootstrapping mechanism is described in :pep:`453`.

Rationale
=========

Currently the user story for installing 3rd-party Python modules is
not as simple as it could be.  It requires that all 3rd-party modules
inform the user of how to install the installer, typically via a link
to the installer.  That link may be out of date or the steps required
to perform the install of the installer may be enough of a roadblock
to prevent the user from further progress.

Large Python projects which emphasise a low barrier to entry have
shied away from depending on third party packages because of the
introduction of this potential stumbling block for new users.

With the inclusion of the package installer command in the standard
Python installation the barrier to installing additional software is
considerably reduced.  It is hoped that this will therefore increase
the likelihood that Python projects will reuse third party software.

The Python community also has an issue of complexity around the current
bootstrap procedure for pip and setuptools. They all have
their own bootstrap download file with slightly different usages and
even refer to each other in some cases. Having a single bootstrap which
is common amongst them all, with a simple usage, would be far preferable.

It is also hoped that this is reduces the number of proposals to
include more and more software in the Python standard library, and
therefore that more popular Python software is more easily upgradeable
beyond requiring Python installation upgrades.


Proposal
========

The bootstrap will install the pip implementation, setuptools by downloading
their installation files from PyPI.

This proposal affects two components of packaging: `the pip bootstrap`_ and,
thanks to easier package installation, `modifications to publishing
packages`_.

The core of this proposal is that the user experience of using pip should not
require the user to install pip.


The pip bootstrap
-----------------

The Python installation includes an executable called "pip3" (see :pep:`394` for
naming rationale etc.) that attempts to import pip machinery.  If it can then
the pip command proceeds as normal.  If it cannot it will bootstrap pip by
downloading the pip implementation and setuptools wheel files. Hereafter the
installation of the "pip implementation" will imply installation of setuptools
and virtualenv.  Once installed, the pip command proceeds as normal. Once the
bootstrap process is complete the "pip3" command is no longer the bootstrap
but rather the full pip command.

A bootstrap is used in the place of a the full pip code so that we don't have
to bundle pip and also pip is upgradeable outside of the regular Python
upgrade timeframe and processes.

To avoid issues with sudo we will have the bootstrap default to
installing the pip implementation to the per-user site-packages
directory defined in :pep:`370` and implemented in Python 2.6/3.0.  Since
we avoid installing to the system Python we also avoid conflicting
with any other packaging system (on Linux systems, for example.) If
the user is inside a :pep:`405` virtual environment then the pip
implementation will be installed into that virtual environment.

The bootstrap process will proceed as follows:

1. The user system has Python (3.4+) installed.  In the "scripts"
   directory of the Python installation there is the bootstrap script
   called "pip3".
2. The user will invoke a pip command, typically "pip3 install
   <package>", for example "pip3 install Django".
3. The bootstrap script will attempt to import the pip implementation.
   If this succeeds, the pip command is processed normally. Stop.
4. On failing to import the pip implementation the bootstrap notifies
   the user that it needs to "install pip". It will ask the user whether it
   should install pip as a system-wide site-packages or as a user-only
   package. This choice will also be present as a command-line option to pip
   so non-interactive use is possible.
5. The bootstrap will and contact PyPI to obtain the latest download wheel
   file (see :pep:`427`.)
6. Upon downloading the file it is installed using "python setup.py install".
7. The pip tool may now import the pip implementation and continues to
   process the requested user command normally.

Users may be running in an environment which cannot access the public
Internet and are relying solely on a local package repository.  They
would use the "-i" (Base URL of Python Package Index) argument to the
"pip3 install" command. This simply overrides the default index URL pointing
to PyPI.

Some users may have no Internet access suitable for fetching the pip
implementation file. These users can manually download and install the
setuptools and pip tar files. Adding specific support for this use-case is
unnecessary.

The download of the pip implementation install file will be performed
securely.  The transport from pypi.python.org will be done over HTTPS with the
CA certificate check performed. This facility will be present in Python 3.4+
using Operating System certificates (see PEP XXXX).

Beyond those arguments controlling index location and download
options, the "pip3" bootstrap command may support further standard pip
options for verbosity, quietness and logging.

The "pip3" command will support two new command-line options that are used
in the bootstrapping, and otherwise ignored. They control where the pip
implementation is installed:

``--bootstrap``
  Install to the user's packages directory. The name of this option is chosen
  to promote it as the preferred installation option.

``--bootstrap-to-system``
  Install to the system site-packages directory.

These command-line options will also need to be implemented, but otherwise
ignored, in the pip implementation.

Consideration should be given to defaulting pip to install packages to the
user's packages directory if pip is installed in that location.

The "--no-install" option to the "pip3" command will not affect the
bootstrapping process.


Modifications to publishing packages
------------------------------------

An additional new Python package is proposed, "pypublish", which
will be a tool for publishing packages to PyPI.  It would replace the
current "python setup.py register" and "python setup.py upload"
distutils commands.  Again because of the measured Python release
cycle and extensive existing Python installations these commands are
difficult to bugfix and extend.  Additionally it is desired that the
"register" and "upload" commands be able to be performed over HTTPS
with certificate validation.  Since shipping CA certificate keychains
with Python is not really feasible (updating the keychain is quite
difficult to manage) it is desirable that those commands, and the
accompanying keychain, be made installable and upgradeable outside of
Python itself.

The existing distutils mechanisms for package registration and upload would
remain, though with a depreca


id: pep-0411.rst
content:
PEP: 411
Title: Provisional packages in the Python standard library
Author: Alyssa Coghlan <ncoghlan@gmail.com>,
        Eli Bendersky <eliben@gmail.com>
Status: Superseded
Type: Informational
Created: 10-Feb-2012
Python-Version: 3.3
Post-History: 10-Feb-2012, 24-Mar-2012

.. note::

   This PEP has been marked as *Superseded*. A decade after this PEP
   was written, experience has shown this is a rarely used feature in
   managing the standard library. It has also not helped prevent
   people from relying too heavily on provisional modules, such that
   changes can still cause significant breakage in the community.


Abstract
========

The process of including a new package into the Python standard library is
hindered by the API lock-in and promise of backward compatibility implied by
a package being formally part of Python.  This PEP describes a methodology
for marking a standard library package "provisional" for the period of a single
feature release.  A provisional package may have its API modified prior to
"graduating" into a "stable" state.  On one hand, this state provides the
package with the benefits of being formally part of the Python distribution.
On the other hand, the core development team explicitly states that no promises
are made with regards to the stability of the package's API, which may
change for the next release.  While it is considered an unlikely outcome,
such packages may even be removed from the standard library without a
deprecation period if the concerns regarding their API or maintenance prove
well-founded.


Proposal - a documented provisional state
=========================================

Whenever the Python core development team decides that a new package should be
included into the standard library, but isn't entirely sure about whether the
package's API is optimal, the package can be included and marked as
"provisional".

In the next feature release, the package may either be "graduated" into a normal
"stable" state in the standard library, remain in provisional state, or be
rejected and removed entirely from the Python source tree.  If the package ends
up graduating into the stable state after being provisional, its API may
be changed according to accumulated feedback.  The core development team
explicitly makes no guarantees about API stability and backward compatibility
of provisional packages.


Marking a package provisional
-----------------------------

A package will be marked provisional by a notice in its documentation page and
its docstring. The following paragraph will be added as a note at the top of
the documentation page:

    The <X> package has been included in the standard library on a
    provisional basis.  Backwards incompatible changes (up to and including
    removal of the package) may occur if deemed necessary by the core
    developers.

The phrase "provisional basis" will then be a link to the glossary term
"provisional package", defined as:

    A provisional package is one which has been deliberately excluded from the
    standard library's backwards compatibility guarantees.  While major
    changes to such packages are not expected, as long as they are marked
    provisional, backwards incompatible changes (up to and including removal of
    the package) may occur if deemed necessary by core developers.  Such changes
    will not be made gratuitously -- they will occur only if serious flaws are
    uncovered that were missed prior to the inclusion of the package.

    This process allows the standard library to continue to evolve over time,
    without locking in problematic design errors for extended periods of time.
    See :pep:`411` for more details.

The following will be added to the start of the package's docstring:

    The API of this package is currently provisional.  Refer to the
    documentation for details.

Moving a package from the provisional to the stable state simply implies
removing these notes from its documentation page and docstring.


Which packages should go through the provisional state
------------------------------------------------------

We expect most packages proposed for addition into the Python standard library
to go through a feature release in the provisional state. There may, however,
be some exceptions, such as packages that use a pre-defined API (for example
``lzma``, which generally follows the API of the existing ``bz2`` package),
or packages with an API that has wide acceptance in the Python development
community.

In any case, packages that are proposed to be added to the standard library,
whether via the provisional state or directly, must fulfill the acceptance
conditions set by :pep:`2`.

Criteria for "graduation"
-------------------------

In principle, most provisional packages should eventually graduate to the
stable standard library.  Some reasons for not graduating are:

* The package may prove to be unstable or fragile, without sufficient developer
  support to maintain it.
* A much better alternative package may be found during the preview release.

Essentially, the decision will be made by the core developers on a per-case
basis.  The point to emphasize here is that a package's inclusion in the
standard library as "provisional" in some release does not guarantee it will
continue being part of Python in the next release.  At the same time, the bar
for making changes in a provisional package is quite high.  We expect that
most of the API of most provisional packages will be unchanged at graduation.
Withdrawals are expected to be rare.


Rationale
=========

Benefits for the core development team
--------------------------------------

Currently, the core developers are really reluctant to add new interfaces to
the standard library.  This is because as soon as they're published in a
release, API design mistakes get locked in due to backward compatibility
concerns.

By gating all major API additions through some kind of a provisional mechanism
for a full release, we get one full release cycle of community feedback
before we lock in the APIs with our standard backward compatibility guarantee.

We can also start integrating provisional packages with the rest of the standard
library early, so long as we make it clear to packagers that the provisional
packages should not be considered optional.  The only difference between
provisional APIs and the rest of the standard library is that provisional APIs
are explicitly exempted from the usual backward compatibility guarantees.

Benefits for end users
----------------------

For future end users, the broadest benefit lies in a better "out-of-the-box"
experience - rather than being told "oh, the standard library tools for task X
are horrible, download this 3rd party library instead", those superior tools
are more likely to be just be an import away.

For environments where developers are required to conduct due diligence on
their upstream dependencies (severely harming the cost-effectiveness of, or
even ruling out entirely, much of the material on PyPI), the key benefit lies
in ensuring that all packages in the provisional state are clearly under
python-dev's aegis from at least the following perspectives:

* Licensing:  Redistributed by the PSF under a Contributor Licensing Agreement.
* Documentation: The documentation of the package is published and organized via
  the standard Python documentation tools (i.e. ReST source, output generated
  with Sphinx and published on http://docs.python.org).
* Testing: The package test suites are run on the python.org buildbot fleet
  and results published via http://www.python.org/dev/buildbot.
* Issue management: Bugs and feature requests are handled on
  http://bugs.python.org
* Source control: The master repository for the software is published
  on http://hg.python.org.


Candidates for provisional inclusion into the standard library
==============================================================

For Python 3.3, there are a number of clear current ca


id: pep-0377.rst
content:
PEP: 377
Title: Allow __enter__() methods to skip the statement body
Author: Alyssa Coghlan <ncoghlan@gmail.com>
Status: Rejected
Type: Standards Track
Created: 08-Mar-2009
Python-Version: 2.7, 3.1
Post-History: 08-Mar-2009


Abstract
========

This PEP proposes a backwards compatible mechanism that allows ``__enter__()``
methods to skip the body of the associated ``with`` statement. The lack of
this ability currently means the ``contextlib.contextmanager`` decorator
is unable to fulfil its specification of being able to turn arbitrary
code into a context manager by moving it into a generator function
with a yield in the appropriate location. One symptom of this is that
``contextlib.nested`` will currently raise ``RuntimeError`` in
situations where writing out the corresponding nested ``with``
statements would not [1]_.

The proposed change is to introduce a new flow control exception
``SkipStatement``, and skip the execution of the ``with``
statement body if ``__enter__()`` raises this exception.


PEP Rejection
=============

This PEP was rejected by Guido [4]_ as it imposes too great an increase
in complexity without a proportional increase in expressiveness and
correctness. In the absence of compelling use cases that need the more
complex semantics proposed by this PEP the existing behaviour is
considered acceptable.


Proposed Change
===============

The semantics of the ``with`` statement will be changed to include a
new ``try``/``except``/``else`` block around the call to ``__enter__()``.
If ``SkipStatement`` is raised by the ``__enter__()`` method, then
the main section of the ``with`` statement (now located in the ``else``
clause) will not be executed. To avoid leaving the names in any ``as``
clause unbound in this case, a new ``StatementSkipped`` singleton
(similar to the existing ``NotImplemented`` singleton) will be
assigned to all names that appear in the ``as`` clause.

The components of the ``with`` statement remain as described in :pep:`343`::

    with EXPR as VAR:
        BLOCK

After the modification, the ``with`` statement semantics would
be as follows::

    mgr = (EXPR)
    exit = mgr.__exit__  # Not calling it yet
    try:
        value = mgr.__enter__()
    except SkipStatement:
        VAR = StatementSkipped
        # Only if "as VAR" is present and
        # VAR is a single name
        # If VAR is a tuple of names, then StatementSkipped
        # will be assigned to each name in the tuple
    else:
        exc = True
        try:
            try:
                VAR = value  # Only if "as VAR" is present
                BLOCK
            except:
                # The exceptional case is handled here
                exc = False
                if not exit(*sys.exc_info()):
                    raise
                # The exception is swallowed if exit() returns true
        finally:
            # The normal and non-local-goto cases are handled here
            if exc:
                exit(None, None, None)

With the above change in place for the ``with`` statement semantics,
``contextlib.contextmanager()`` will then be modified to raise
``SkipStatement`` instead of ``RuntimeError`` when the underlying
generator doesn't yield.


Rationale for Change
====================

Currently, some apparently innocuous context managers may raise
``RuntimeError`` when executed. This occurs when the context
manager's ``__enter__()`` method encounters a situation where
the written out version of the code corresponding to the
context manager would skip the code that is now the body
of the ``with`` statement. Since the ``__enter__()`` method
has no mechanism available to signal this to the interpreter,
it is instead forced to raise an exception that not only
skips the body of the ``with`` statement, but also jumps over
all code until the nearest exception handler. This goes against
one of the design goals of the ``with`` statement, which was to
be able to factor out arbitrary common exception handling code
into a single context manager by putting into a generator
function and replacing the variant part of the code with a
``yield`` statement.

Specifically, the following examples behave differently if
``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
then handles and suppresses::

  with cmA():
    with cmB():
      do_stuff()
  # This will resume here without executing "do_stuff()"

  @contextlib.contextmanager
  def combined():
    with cmA():
      with cmB():
        yield

  with combined():
    do_stuff()
  # This will raise a RuntimeError complaining that the context
  # manager's underlying generator didn't yield

  with contextlib.nested(cmA(), cmB()):
    do_stuff()
  # This will raise the same RuntimeError as the contextmanager()
  # example (unsurprising, given that the nested() implementation
  # uses contextmanager())

  # The following class based version shows that the issue isn't
  # specific to contextlib.contextmanager() (it also shows how
  # much simpler it is to write context managers as generators
  # instead of as classes!)
  class CM(object):
    def __init__(self):
      self.cmA = None
      self.cmB = None

    def __enter__(self):
      if self.cmA is not None:
        raise RuntimeError("Can't re-use this CM")
      self.cmA = cmA()
      self.cmA.__enter__()
      try:
        self.cmB = cmB()
        self.cmB.__enter__()
      except:
        self.cmA.__exit__(*sys.exc_info())
        # Can't suppress in __enter__(), so must raise
        raise

    def __exit__(self, *args):
      suppress = False
      try:
        if self.cmB is not None:
          suppress = self.cmB.__exit__(*args)
      except:
        suppress = self.cmA.__exit__(*sys.exc_info()):
        if not suppress:
          # Exception has changed, so reraise explicitly
          raise
      else:
        if suppress:
          # cmB already suppressed the exception,
          # so don't pass it to cmA
          suppress = self.cmA.__exit__(None, None, None):
        else:
          suppress = self.cmA.__exit__(*args):
      return suppress

With the proposed semantic change in place, the contextlib based examples
above would then "just work", but the class based version would need
a small adjustment to take advantage of the new semantics::

  class CM(object):
    def __init__(self):
      self.cmA = None
      self.cmB = None

    def __enter__(self):
      if self.cmA is not None:
        raise RuntimeError("Can't re-use this CM")
      self.cmA = cmA()
      self.cmA.__enter__()
      try:
        self.cmB = cmB()
        self.cmB.__enter__()
      except:
        if self.cmA.__exit__(*sys.exc_info()):
          # Suppress the exception, but don't run
          # the body of the with statement either
          raise SkipStatement
        raise

    def __exit__(self, *args):
      suppress = False
      try:
        if self.cmB is not None:
          suppress = self.cmB.__exit__(*args)
      except:
        suppress = self.cmA.__exit__(*sys.exc_info()):
        if not suppress:
          # Exception has changed, so reraise explicitly
          raise
      else:
        if suppress:
          # cmB already suppressed the exception,
          # so don't pass it to cmA
          suppress = self.cmA.__exit__(None, None, None):
        else:
          suppress = self.cmA.__exit__(*args):
      return suppress

There is currently a tentative suggestion [3]_ to add import-style syntax to
the ``with`` statement to allow multiple context managers to be included in
a single ``with`` statement without needing to use ``contextlib.nested``. In
that case the compiler has the option of simply emitting multiple ``with``
statements at the AST level, thus allowing the semantics of actual nested
``with`` statements to be reproduced accurately. However, such a change
would highlight rather than alleviate the problem the current PEP aims to
address: it would not be possible to use ``contextlib.contextmanager`` to
reliably factor out such ``with


id: pep-0363.rst
content:
PEP: 363
Title: Syntax For Dynamic Attribute Access
Author: Ben North <ben@redfrontdoor.org>
Status: Rejected
Type: Standards Track
Created: 29-Jan-2007
Post-History: 12-Feb-2007

.. rejected::

   See :ref:`363-rejection` for more information.

Abstract
========

Dynamic attribute access is currently possible using the "getattr"
and "setattr" builtins.  The present PEP suggests a new syntax to
make such access easier, allowing the coder for example to write::

    x.('foo_%d' % n) += 1

    z = y.('foo_%d' % n).('bar_%s' % s)

instead of::

    attr_name = 'foo_%d' % n
    setattr(x, attr_name, getattr(x, attr_name) + 1)

    z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)


Rationale
=========

Dictionary access and indexing both have a friendly invocation
syntax: instead of ``x.__getitem__(12)`` the coder can write ``x[12]``.
This also allows the use of subscripted elements in an augmented
assignment, as in "x[12] += 1".  The present proposal brings this
ease-of-use to dynamic attribute access too.

Attribute access is currently possible in two ways:

* When the attribute name is known at code-writing time, the
  ".NAME" trailer can be used, as in::

      x.foo = 42
      y.bar += 100

* When the attribute name is computed dynamically at run-time, the
  "getattr" and "setattr" builtins must be used::

      x = getattr(y, 'foo_%d' % n)
      setattr(z, 'bar_%s' % s, 99)

  The "getattr" builtin also allows the coder to specify a default
  value to be returned in the event that the object does not have
  an attribute of the given name::

      x = getattr(y, 'foo_%d' % n, 0)

This PEP describes a new syntax for dynamic attribute access ---
"x.(expr)" --- with examples given in the Abstract above.

(The new syntax could also allow the provision of a default value in
the "get" case, as in::

    x = y.('foo_%d' % n, None)

This 2-argument form of dynamic attribute access would not be
permitted as the target of an (augmented or normal) assignment.  The
"Discussion" section below includes opinions specifically on the
2-argument extension.)

Finally, the new syntax can be used with the "del" statement, as in::

    del x.(attr_name)


Impact On Existing Code
=======================

The proposed new syntax is not currently valid, so no existing
well-formed programs have their meaning altered by this proposal.

Across all "\*.py" files in the 2.5 distribution, there are around
600 uses of "getattr", "setattr" or "delattr".  They break down as
follows (figures have some room for error because they were
arrived at by partially-manual inspection)::

    c.300 uses of plain "getattr(x, attr_name)", which could be
          replaced with the new syntax;

    c.150 uses of the 3-argument form, i.e., with the default
          value; these could be replaced with the 2-argument form
          of the new syntax (the cases break down into c.125 cases
          where the attribute name is a literal string, and c.25
          where it's only known at run-time);

    c.5   uses of the 2-argument form with a literal string
          attribute name, which I think could be replaced with the
          standard "x.attribute" syntax;

    c.120 uses of setattr, of which 15 use getattr to find the
          new value; all could be replaced with the new syntax,
          the 15 where getattr is also involved would show a
          particular increase in clarity;

    c.5   uses which would have to stay as "getattr" because they
          are calls of a variable named "getattr" whose default
          value is the builtin "getattr";

    c.5   uses of the 2-argument form, inside a try/except block
          which catches AttributeError and uses a default value
          instead; these could use 2-argument form of the new
          syntax;

    c.10  uses of "delattr", which could use the new syntax.

As examples, the line::

    setattr(self, attr, change_root(self.root, getattr(self, attr)))

from Lib/distutils/command/install.py could be rewritten::

    self.(attr) = change_root(self.root, self.(attr))

and the line::

    setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten::

    self.(method_name) = self.metadata.(method_name)


Performance Impact
==================

Initial pystone measurements are inconclusive, but suggest there may
be a performance penalty of around 1% in the pystones score with the
patched version.  One suggestion is that this is because the longer
main loop in ceval.c hurts the cache behaviour, but this has not
been confirmed.

On the other hand, measurements suggest a speed-up of around 40--45%
for dynamic attribute access.


Error Cases
===========

Only strings are permitted as attribute names, so for instance the
following error is produced::

    >>> x.(99) = 8
       Traceback (most recent call last):
         File "<stdin>", line 1, in <module>
       TypeError: attribute name must be string, not 'int'

This is handled by the existing ``PyObject_GetAttr`` function.


Draft Implementation
====================

A draft implementation adds a new alternative to the "trailer"
clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in
Python.asdl, with accompanying changes to symtable.c, ast.c, and
compile.c, and three new opcodes (load/store/del) with
accompanying changes to opcode.h and ceval.c.  The patch consists
of c.180 additional lines in the core code, and c.100 additional
lines of tests.  It is available as sourceforge patch #1657573 [1]_.


.. _363-rejection:

Mailing Lists Discussion
========================

Initial posting of this PEP in draft form was to python-ideas on
20070209 [2]_, and the response was generally positive.  The PEP was
then posted to python-dev on 20070212 [3]_, and an interesting
discussion ensued.  A brief summary:

Initially, there was reasonable (but not unanimous) support for the
idea, although the precise choice of syntax had a more mixed
reception.  Several people thought the "." would be too easily
overlooked, with the result that the syntax could be confused with a
method/function call.  A few alternative syntaxes were suggested::

    obj.(foo)
    obj.[foo]
    obj.{foo}
    obj{foo}
    obj.*foo
    obj->foo
    obj<-foo
    obj@[foo]
    obj.[[foo]]

with "obj.[foo]" emerging as the preferred one.  In this initial
discussion, the two-argument form was universally disliked, so it
was to be taken out of the PEP.

Discussion then took a step back to whether this particular feature
provided enough benefit to justify new syntax.  As well as requiring
coders to become familiar with the new syntax, there would also be
the problem of backward compatibility --- code using the new syntax
would not run on older pythons.

Instead of new syntax, a new "wrapper class" was proposed, with the
following specification / conceptual implementation suggested by
Martin von LÃ¶wis::

    class attrs:
       def __init__(self, obj):
         self.obj = obj
       def __getitem__(self, name):
         return getattr(self.obj, name)
       def __setitem__(self, name, value):
         return setattr(self.obj, name, value)
       def __delitem__(self, name):
         return delattr(self, name)
       def __contains__(self, name):
         return hasattr(self, name)

This was considered a cleaner and more elegant solution to the
original problem.  (Another suggestion was a mixin class providing
dictionary-style access to an object's attributes.)

The decision was made that the present PEP did not meet the burden
of proof for the introduction of new syntax, a view which had been
put forward by some from the beginning of the discussion.  The
wrapper class idea was left open as a possibility for a future PEP.


References
==========

.. [1] Sourceforge patch #1657573
       http://sourceforge.net/tracker/index.php?func=detail&aid=1657573&group_id=5470&atid=305470

.. [2] https://mail.python.org/pipermail/python-ideas/2007-February/000210.html
       and following 


id: pep-0405.rst
content:
PEP: 405
Title: Python Virtual Environments
Author: Carl Meyer <carl@oddbird.net>
BDFL-Delegate: Alyssa Coghlan
Status: Final
Type: Standards Track
Topic: Packaging
Created: 13-Jun-2011
Python-Version: 3.3
Post-History: 24-Oct-2011, 28-Oct-2011, 06-Mar-2012, 24-May-2012
Resolution: https://mail.python.org/pipermail/python-dev/2012-May/119668.html


Abstract
========

This PEP proposes to add to Python a mechanism for lightweight
"virtual environments" with their own site directories, optionally
isolated from system site directories.  Each virtual environment has
its own Python binary (allowing creation of environments with various
Python versions) and can have its own independent set of installed
Python packages in its site directories, but shares the standard
library with the base installed Python.


Motivation
==========

The utility of Python virtual environments has already been well
established by the popularity of existing third-party
virtual-environment tools, primarily Ian Bicking's `virtualenv`_.
Virtual environments are already widely used for dependency management
and isolation, ease of installing and using Python packages without
system-administrator access, and automated testing of Python software
across multiple Python versions, among other uses.

Existing virtual environment tools suffer from lack of support from
the behavior of Python itself.  Tools such as `rvirtualenv`_, which do
not copy the Python binary into the virtual environment, cannot
provide reliable isolation from system site directories.  Virtualenv,
which does copy the Python binary, is forced to duplicate much of
Python's ``site`` module and manually symlink/copy an ever-changing
set of standard-library modules into the virtual environment in order
to perform a delicate boot-strapping dance at every startup.
(Virtualenv must copy the binary in order to provide isolation, as
Python dereferences a symlinked executable before searching for
``sys.prefix``.)

The ``PYTHONHOME`` environment variable, Python's only existing
built-in solution for virtual environments, requires
copying/symlinking the entire standard library into every environment.
Copying the whole standard library is not a lightweight solution, and
cross-platform support for symlinks remains inconsistent (even on
Windows platforms that do support them, creating them often requires
administrator privileges).

A virtual environment mechanism integrated with Python and drawing on
years of experience with existing third-party tools can lower
maintenance, raise reliability, and be more easily available to all
Python users.

.. _virtualenv: http://www.virtualenv.org

.. _rvirtualenv: https://github.com/kvbik/rvirtualenv


Specification
=============

When the Python binary is executed, it attempts to determine its
prefix (which it stores in ``sys.prefix``), which is then used to find
the standard library and other key files, and by the ``site`` module
to determine the location of the site-package directories.  Currently
the prefix is found (assuming ``PYTHONHOME`` is not set) by first
walking up the filesystem tree looking for a marker file (``os.py``)
that signifies the presence of the standard library, and if none is
found, falling back to the build-time prefix hardcoded in the binary.

This PEP proposes to add a new first step to this search.  If a
``pyvenv.cfg`` file is found either adjacent to the Python executable or
one directory above it (if the executable is a symlink, it is not
dereferenced), this file is scanned for lines of the form ``key =
value``.  If a ``home`` key is found, this signifies that the Python
binary belongs to a virtual environment, and the value of the ``home``
key is the directory containing the Python executable used to create
this virtual environment.

In this case, prefix-finding continues as normal using the value of
the ``home`` key as the effective Python binary location, which finds
the prefix of the base installation.  ``sys.base_prefix`` is set to
this value, while ``sys.prefix`` is set to the directory containing
``pyvenv.cfg``.

(If ``pyvenv.cfg`` is not found or does not contain the ``home`` key,
prefix-finding continues normally, and ``sys.prefix`` will be equal to
``sys.base_prefix``.)

Also, ``sys.base_exec_prefix`` is added, and handled similarly with
regard to ``sys.exec_prefix``. (``sys.exec_prefix`` is the equivalent of
``sys.prefix``, but for platform-specific files; by default it has the
same value as ``sys.prefix``.)

The ``site`` and ``sysconfig`` standard-library modules are modified
such that the standard library and header files are found relative
to ``sys.base_prefix`` / ``sys.base_exec_prefix``, while site-package
directories ("purelib" and "platlib", in ``sysconfig`` terms) are still
found relative to ``sys.prefix`` / ``sys.exec_prefix``.

Thus, a Python virtual environment in its simplest form would consist
of nothing more than a copy or symlink of the Python binary
accompanied by a ``pyvenv.cfg`` file and a site-packages directory.


Isolation from system site-packages
-----------------------------------

By default, a virtual environment is entirely isolated from the
system-level site-packages directories.

If the ``pyvenv.cfg`` file also contains a key
``include-system-site-packages`` with a value of ``true`` (not case
sensitive), the ``site`` module will also add the system site
directories to ``sys.path`` after the virtual environment site
directories.  Thus system-installed packages will still be importable,
but a package of the same name installed in the virtual environment
will take precedence.

:pep:`370` user-level site-packages are considered part of the system
site-packages for venv purposes: they are not available from an
isolated venv, but are available from an
``include-system-site-packages = true`` venv.


Creating virtual environments
-----------------------------

This PEP also proposes adding a new ``venv`` module to the standard
library which implements the creation of virtual environments.  This
module can be executed using the ``-m`` flag::

    python3 -m venv /path/to/new/virtual/environment

A ``pyvenv`` installed script is also provided to make this more
convenient::

    pyvenv /path/to/new/virtual/environment

Running this command creates the target directory (creating any parent
directories that don't exist already) and places a ``pyvenv.cfg`` file
in it with a ``home`` key pointing to the Python installation the
command was run from.  It also creates a ``bin/`` (or ``Scripts`` on
Windows) subdirectory containing a copy (or symlink) of the ``python3``
executable, and the ``pysetup3`` script from the ``packaging`` standard
library module (to facilitate easy installation of packages from PyPI
into the new venv).  And it creates an (initially empty)
``lib/pythonX.Y/site-packages`` (or ``Lib\site-packages`` on Windows)
subdirectory.

If the target directory already exists an error will be raised, unless
the ``--clear`` option was provided, in which case the target
directory will be deleted and virtual environment creation will
proceed as usual.

The created ``pyvenv.cfg`` file also includes the
``include-system-site-packages`` key, set to ``true`` if ``pyvenv`` is
run with the ``--system-site-packages`` option, ``false`` by default.

Multiple paths can be given to ``pyvenv``, in which case an identical
venv will be created, according to the given options, at each
provided path.

The ``venv`` module also places "shell activation scripts" for POSIX and
Windows systems in the ``bin`` or ``Scripts`` directory of the
venv. These scripts simply add the virtual environment's ``bin`` (or
``Scripts``) directory to the front of the user's shell PATH.  This is
not strictly necessary for use of a virtual environment (as an explicit
path to the venv's python binary or scripts can just as well be used),
but it is convenient.

In order to allow ``pysetup`` and other Python package managers to
install packages into the virtual environment the same


id: pep-0389.rst
content:
PEP: 389
Title: argparse - New Command Line Parsing Module
Author: Steven Bethard <steven.bethard@gmail.com>
Status: Final
Type: Standards Track
Created: 25-Sep-2009
Python-Version: 2.7, 3.2
Post-History: 27-Sep-2009, 24-Oct-2009


Acceptance
==========
This PEP was approved by Guido on python-dev on February 21, 2010 [17]_.


Abstract
========
This PEP proposes inclusion of the argparse [1]_ module in the Python
standard library in Python 2.7 and 3.2.


Motivation
==========
The argparse module is a command line parsing library which provides
more functionality than the existing command line parsing modules in
the standard library, getopt [2]_ and optparse [3]_. It includes
support for positional arguments (not just options), subcommands,
required options, options syntaxes like "/f" and "+rgb", zero-or-more
and one-or-more style arguments, and many other features the other
two lack.

The argparse module is also already a popular third-party replacement
for these modules. It is used in projects like IPython (the Scipy
Python shell) [4]_, is included in Debian testing and unstable [5]_,
and since 2007 has had various requests for its inclusion in the
standard library [6]_ [7]_ [8]_. This popularity suggests it may be
a valuable addition to the Python libraries.


Why aren't getopt and optparse enough?
======================================
One argument against adding argparse is that there are "already two
different option parsing modules in the standard library" [9]_. The
following is a list of features provided by argparse but not present
in getopt or optparse:

* While it is true there are two *option* parsing libraries, there
  are no full command line parsing libraries -- both getopt and
  optparse support only options and have no support for positional
  arguments. The argparse module handles both, and as a result, is
  able to generate better help messages, avoiding redundancies like
  the ``usage=`` string usually required by optparse.

* The argparse module values practicality over purity. Thus, argparse
  allows required options and customization of which characters are
  used to identify options, while optparse explicitly states "the
  phrase 'required option' is self-contradictory" and that the option
  syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file``
  "are not supported by optparse, and they never will be".

* The argparse module allows options to accept a variable number of
  arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The
  optparse module provides an untested recipe for some part of this
  functionality [10]_ but admits that "things get hairy when you want
  an option to take a variable number of arguments."

* The argparse module supports subcommands, where a main command
  line parser dispatches to other command line parsers depending on
  the command line arguments. This is a common pattern in command
  line interfaces, e.g. ``svn co`` and ``svn up``.


Why isn't the functionality just being added to optparse?
=========================================================
Clearly all the above features offer improvements over what is
available through optparse. A reasonable question then is why these
features are not simply provided as patches to optparse, instead of
introducing an entirely new module. In fact, the original development
of argparse intended to do just that, but because of various fairly
constraining design decisions of optparse, this wasn't really
possible. Some of the problems included:

* The optparse module exposes the internals of its parsing algorithm.
  In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed
  to be available to callbacks [11]_. This makes it extremely
  difficult to improve the parsing algorithm as was necessary in
  argparse for proper handling of positional arguments and variable
  length arguments. For example, ``nargs='+'`` in argparse is matched
  using regular expressions and thus has no notion of things like
  ``parser.largs``.

* The optparse extension APIs are extremely complex. For example,
  just to use a simple custom string-to-object conversion function,
  you have to subclass ``Option``, hack class attributes, and then
  specify your custom option type to the parser, like this::

    class MyOption(Option):
        TYPES = Option.TYPES + ("mytype",)
        TYPE_CHECKER = copy(Option.TYPE_CHECKER)
        TYPE_CHECKER["mytype"] = check_mytype
    parser = optparse.OptionParser(option_class=MyOption)
    parser.add_option("-m", type="mytype")

  For comparison, argparse simply allows conversion functions to be
  used as ``type=`` arguments directly, e.g.::

    parser = argparse.ArgumentParser()
    parser.add_option("-m", type=check_mytype)

  But given the baroque customization APIs of optparse, it is unclear
  how such a feature should interact with those APIs, and it is
  quite possible that introducing the simple argparse API would break
  existing custom Option code.

* Both optparse and argparse parse command line arguments and assign
  them as attributes to an object returned by ``parse_args``.
  However, the optparse module guarantees that the ``take_action``
  method of custom actions will always be passed a ``values`` object
  which provides an ``ensure_value`` method [12]_, while the argparse
  module allows attributes to be assigned to any object, e.g.::

    foo_object = ...
    parser.parse_args(namespace=foo_object)
    foo_object.some_attribute_parsed_from_command_line

  Modifying optparse to allow any object to be passed in would be
  difficult because simply passing the ``foo_object`` around instead
  of a ``Values`` instance will break existing custom actions that
  depend on the ``ensure_value`` method.

Because of issues like these, which made it unreasonably difficult
for argparse to stay compatible with the optparse APIs, argparse was
developed as an independent module. Given these issues, merging all
the argparse features into optparse with no backwards
incompatibilities seems unlikely.


Deprecation of optparse
=======================
Because all of optparse's features are available in argparse, the
optparse module will be deprecated. However, because of the
widespread use of optparse, the deprecation strategy contains only
documentation changes and warnings that will not be visible by
default:

* Python 2.7+ and 3.2+ -- The following note will be added to the
  optparse documentation:

    The optparse module is deprecated and will not be developed
    further; development will continue with the argparse module.

* Python 2.7+ -- If the Python 3 compatibility flag, ``-3``, is
  provided at the command line, then importing optparse will issue a
  DeprecationWarning. Otherwise no warnings will be issued.

* Python 3.2+ -- Importing optparse will issue a
  PendingDeprecationWarning, which is not displayed by default.

Note that no removal date is proposed for optparse.


Updates to getopt documentation
===============================
The getopt module will not be deprecated. However, its documentation
will be updated to point to argparse in a couple of places. At the
top of the module, the following note will be added:

  The getopt module is a parser for command line options whose API
  is designed to be familiar to users of the C getopt function.
  Users who are unfamiliar with the C getopt function or who would
  like to write less code and get better help and error messages
  should consider using the argparse module instead.

Additionally, after the final getopt example, the following note will
be added:

  Note that an equivalent command line interface could be produced
  with less code by using the argparse module::

    import argparse

    if __name__ == '__main__':
        parser = argparse.ArgumentParser()
        parser.add_argument('-o', '--output')
        parser.add_argument('-v', dest='verbose', action='store_true')
        args = parser.parse_args()
        # ... do something with args.output


id: pep-0362.rst
content:
PEP: 362
Title: Function Signature Object
Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>,
        Yury Selivanov <yury@edgedb.com>, Larry Hastings <larry@hastings.org>
Status: Final
Type: Standards Track
Created: 21-Aug-2006
Python-Version: 3.3
Post-History: 04-Jun-2012
Resolution: https://mail.python.org/pipermail/python-dev/2012-June/120682.html


Abstract
========

Python has always supported powerful introspection capabilities,
including introspecting functions and methods (for the rest of
this PEP, "function" refers to both functions and methods).  By
examining a function object you can fully reconstruct the function's
signature.  Unfortunately this information is stored in an inconvenient
manner, and is spread across a half-dozen deeply nested attributes.

This PEP proposes a new representation for function signatures.
The new representation contains all necessary information about a function
and its parameters, and makes introspection easy and straightforward.

However, this object does not replace the existing function
metadata, which is used by Python itself to execute those
functions.  The new metadata object is intended solely to make
function introspection easier for Python programmers.


Signature Object
================

A Signature object represents the call signature of a function and
its return annotation.  For each parameter accepted by the function
it stores a `Parameter object`_ in its ``parameters`` collection.

A Signature object has the following public attributes and methods:

* return_annotation \: object
    The "return" annotation for the function. If the function
    has no "return" annotation, this attribute is set to
    ``Signature.empty``.

* parameters \: OrderedDict
    An ordered mapping of parameters' names to the corresponding
    Parameter objects.

* bind(\*args, \*\*kwargs) -> BoundArguments
    Creates a mapping from positional and keyword arguments to
    parameters.  Raises a ``TypeError`` if the passed arguments do
    not match the signature.

* bind_partial(\*args, \*\*kwargs) -> BoundArguments
    Works the same way as ``bind()``, but allows the omission
    of some required arguments (mimics ``functools.partial``
    behavior.)  Raises a ``TypeError`` if the passed arguments do
    not match the signature.

* replace(parameters=<optional>, \*, return_annotation=<optional>) -> Signature
    Creates a new Signature instance based on the instance
    ``replace`` was invoked on.  It is possible to pass different
    ``parameters`` and/or ``return_annotation`` to override the
    corresponding properties of the base signature.  To remove
    ``return_annotation`` from the copied ``Signature``, pass in
    ``Signature.empty``.

    Note that the '=<optional>' notation, means that the argument is
    optional.  This notation applies to the rest of this PEP.

Signature objects are immutable.  Use ``Signature.replace()`` to
make a modified copy:
::

    >>> def foo() -> None:
    ...     pass
    >>> sig = signature(foo)

    >>> new_sig = sig.replace(return_annotation="new return annotation")
    >>> new_sig is not sig
    True
    >>> new_sig.return_annotation != sig.return_annotation
    True
    >>> new_sig.parameters == sig.parameters
    True

    >>> new_sig = new_sig.replace(return_annotation=new_sig.empty)
    >>> new_sig.return_annotation is Signature.empty
    True

There are two ways to instantiate a Signature class:

* Signature(parameters=<optional>, \*, return_annotation=Signature.empty)
    Default Signature constructor.  Accepts an optional sequence
    of ``Parameter`` objects, and an optional ``return_annotation``.
    Parameters sequence is validated to check that there are no
    parameters with duplicate names, and that the parameters
    are in the right order, i.e. positional-only first, then
    positional-or-keyword, etc.
* Signature.from_function(function)
    Returns a Signature object reflecting the signature of the
    function passed in.

It's possible to test Signatures for equality.  Two signatures are
equal when their parameters are equal, their positional and
positional-only parameters appear in the same order, and they
have equal return annotations.

Changes to the Signature object, or to any of its data members,
do not affect the function itself.

Signature also implements ``__str__``:
::

    >>> str(Signature.from_function((lambda *args: None)))
    '(*args)'

    >>> str(Signature())
    '()'


Parameter Object
================

Python's expressive syntax means functions can accept many different
kinds of parameters with many subtle semantic differences.  We
propose a rich Parameter object designed to represent any possible
function parameter.

A Parameter object has the following public attributes and methods:

* name \: str
    The name of the parameter as a string.  Must be a valid
    python identifier name (with the exception of ``POSITIONAL_ONLY``
    parameters, which can have it set to ``None``.)

* default \: object
    The default value for the parameter.  If the parameter has no
    default value, this attribute is set to ``Parameter.empty``.

* annotation \: object
    The annotation for the parameter.  If the parameter has no
    annotation, this attribute is set to ``Parameter.empty``.

* kind
    Describes how argument values are bound to the parameter.
    Possible values:

    * ``Parameter.POSITIONAL_ONLY`` - value must be supplied
      as a positional argument.

      Python has no explicit syntax for defining positional-only
      parameters, but many built-in and extension module functions
      (especially those that accept only one or two parameters)
      accept them.

    * ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
      supplied as either a keyword or positional argument
      (this is the standard binding behaviour for functions
      implemented in Python.)

    * ``Parameter.KEYWORD_ONLY`` - value must be supplied
      as a keyword argument.  Keyword only parameters are those
      which appear after a "*" or "\*args" entry in a Python
      function definition.

    * ``Parameter.VAR_POSITIONAL`` - a tuple of positional
      arguments that aren't bound to any other parameter.
      This corresponds to a "\*args" parameter in a Python
      function definition.

    * ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
      that aren't bound to any other parameter. This corresponds
      to a "\*\*kwargs" parameter in a Python function definition.

    Always use ``Parameter.*`` constants for setting and checking
    value of the ``kind`` attribute.

* replace(\*, name=<optional>, kind=<optional>, default=<optional>, annotation=<optional>) -> Parameter
    Creates a new Parameter instance based on the instance
    ``replaced`` was invoked on.  To override a Parameter
    attribute, pass the corresponding argument.  To remove
    an attribute from a ``Parameter``, pass ``Parameter.empty``.


Parameter constructor:

* Parameter(name, kind, \*, annotation=Parameter.empty, default=Parameter.empty)
   Instantiates a Parameter object. ``name`` and ``kind`` are required,
   while ``annotation`` and ``default`` are optional.

Two parameters are equal when they have equal names, kinds, defaults,
and annotations.

Parameter objects are immutable.  Instead of modifying a Parameter object,
you can use ``Parameter.replace()`` to create a modified copy like so:
::

    >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42)
    >>> str(param)
    'foo=42'

    >>> str(param.replace())
    'foo=42'

    >>> str(param.replace(default=Parameter.empty, annotation='spam'))
    "foo:'spam'"


BoundArguments Object
=====================

Result of a ``Signature.bind`` call.  Holds the mapping of arguments
to the function's parameters.

Has the following public attributes:

* arguments \: OrderedDict
    An ordered, mutable mapping of parameters' names to arguments' values.
    Contains only explicitly bound argum


id: pep-0404.rst
content:
PEP: 404
Title: Python 2.8 Un-release Schedule
Author: Barry Warsaw <barry@python.org>
Status: Final
Type: Informational
Topic: Release
Created: 09-Nov-2011
Python-Version: 2.8


Abstract
========

This document describes the un-development and un-release schedule for Python
2.8.


Un-release Manager and Crew
===========================

============================ ==================
Position                     Name
============================ ==================
2.8 Un-release Manager       Cardinal Biggles
============================ ==================


Un-release Schedule
===================

The current un-schedule is:

- 2.8 final Never


Official pronouncement
======================

Rule number six: there is *no* official Python 2.8 release.  There never will
be an official Python 2.8 release.  It is an ex-release.  Python 2.7
is the end of the Python 2 line of development.


Upgrade path
============

The official upgrade path from Python 2.7 is to Python 3.


And Now For Something Completely Different
==========================================

In all seriousness, there are important reasons why there won't be an
official Python 2.8 release, and why you should plan to migrate
instead to Python 3.

Python is (as of this writing) more than 20 years old, and Guido and the
community have learned a lot in those intervening years.  Guido's
original concept for Python 3 was to make changes to the language
primarily to remove the warts that had grown in the preceding
versions.  Python 3 was not to be a complete redesign, but instead an
evolution of the language, and while maintaining full backward
compatibility with Python 2 was explicitly off-the-table, neither were
gratuitous changes in syntax or semantics acceptable.  In most cases,
Python 2 code can be translated fairly easily to Python 3, sometimes
entirely mechanically by such tools as `2to3`_ (there's also a non-trivial
subset of the language that will run without modification on both 2.7 and
3.x).

Because maintaining multiple versions of Python is a significant drag
on the resources of the Python developers, and because the
improvements to the language and libraries embodied in Python 3 are so
important, it was decided to end the Python 2 lineage with Python
2.7.  Thus, all new development occurs in the Python 3 line of
development, and there will never be an official Python 2.8 release.
Python 2.7 will however be maintained for longer than the usual period
of time.

Here are some highlights of the significant improvements in Python 3.
You can read in more detail on the differences_ between Python 2 and
Python 3.  There are also many good guides on porting_ from Python 2
to Python 3.


Strings and bytes
-----------------

Python 2's basic original strings are called 8-bit strings, and
they play a dual role in Python 2 as both ASCII text and as byte
sequences.  While Python 2 also has a unicode string type, the
fundamental ambiguity of the core string type, coupled with Python 2's
default behavior of supporting automatic coercion from 8-bit strings
to unicode objects when the two are combined, often leads to
``UnicodeError``\ s. Python 3's standard string type is Unicode based, and
Python 3 adds a dedicated bytes type, but critically, no automatic coercion
between bytes and unicode strings is provided. The closest the language gets
to implicit coercion are a few text-based APIs that assume a default
encoding (usually UTF-8) if no encoding is explicitly stated. Thus, the core
interpreter, its I/O libraries, module names, etc. are clear in their
distinction between unicode strings and bytes.  Python 3's unicode
support even extends to the filesystem, so that non-ASCII file names are
natively supported.

This string/bytes clarity is often a source of difficulty in
transitioning existing code to Python 3, because many third party
libraries and applications are themselves ambiguous in this
distinction.  Once migrated though, most ``UnicodeError``\ s can be
eliminated.


Numbers
-------

Python 2 has two basic integer types, a native machine-sized ``int``
type, and an arbitrary length ``long`` type.  These have been merged in
Python 3 into a single ``int`` type analogous to Python 2's ``long``
type.

In addition, integer division now produces floating point numbers for
non-integer results.


Classes
-------

Python 2 has two core class hierarchies, often called *classic
classes* and *new-style classes*.  The latter allow for such things as
inheriting from the builtin basic types, support descriptor based tools
like the ``property`` builtin and provide a generally more sane and coherent
system for dealing with multiple inheritance.  Python 3 provided the
opportunity to completely drop support for classic classes, so all classes
in Python 3 automatically use the new-style semantics (although that's a
misnomer now). There is no need to explicitly inherit from ``object`` or set
the default metatype to enable them (in fact, setting a default metatype at
the module level is no longer supported - the default metatype is always
``object``).

The mechanism for explicitly specifying a metaclass has also changed to use
a ``metaclass`` keyword argument in the class header line rather than a
``__metaclass__`` magic attribute in the class body.


Multiple spellings
------------------

There are many cases in Python 2 where multiple spellings of some
constructs exist, such as ``repr()`` and *backticks*, or the two
inequality operators ``!=`` and ``<>``.  In all cases, Python 3 has chosen
exactly one spelling and removed the other (e.g. ``repr()`` and ``!=``
were kept).


Imports
-------

In Python 3, implicit relative imports within packages are no longer
available - only absolute imports and explicit relative imports are
supported. In addition, star imports (e.g. ``from x import *``) are only
permitted in module level code.

Also, some areas of the standard library have been reorganized to make
the naming scheme more intuitive.  Some rarely used builtins have been
relocated to standard library modules.


Iterators and views
-------------------

Many APIs, which in Python 2 returned concrete lists, in Python 3 now
return iterators or lightweight *views*.


Copyright
=========

This document has been placed in the public domain.


.. _`2to3`: http://docs.python.org/library/2to3.html
.. _differences: http://docs.python.org/release/3.0.1/whatsnew/3.0.html
.. _porting: http://python3porting.com/


id: pep-0410.rst
content:
PEP: 410
Title: Use decimal.Decimal type for timestamps
Author: Victor Stinner <vstinner@python.org>
Status: Rejected
Type: Standards Track
Created: 01-Feb-2012
Python-Version: 3.3
Resolution: https://mail.python.org/pipermail/python-dev/2012-February/116837.html


Rejection Notice
================

This PEP is rejected.
See https://mail.python.org/pipermail/python-dev/2012-February/116837.html.


Abstract
========

Decimal becomes the official type for high-resolution timestamps to make Python
support new functions using a nanosecond resolution without loss of precision.


Rationale
=========

Python 2.3 introduced float timestamps to support sub-second resolutions.
os.stat() uses float timestamps by default since Python 2.5. Python 3.3
introduced functions supporting nanosecond resolutions:

* os module: futimens(), utimensat()
* time module: clock_gettime(), clock_getres(), monotonic(), wallclock()

os.stat() reads nanosecond timestamps but returns timestamps as float.

The Python float type uses binary64 format of the IEEE 754 standard. With a
resolution of one nanosecond (10\ :sup:`-9`), float timestamps lose precision
for values bigger than 2\ :sup:`24` seconds (194 days: 1970-07-14 for an Epoch
timestamp).

Nanosecond resolution is required to set the exact modification time on
filesystems supporting nanosecond timestamps (e.g. ext4, btrfs, NTFS, ...). It
helps also to compare the modification time to check if a file is newer than
another file. Use cases: copy the modification time of a file using
shutil.copystat(), create a TAR archive with the tarfile module, manage a
mailbox with the mailbox module, etc.

An arbitrary resolution is preferred over a fixed resolution (like nanosecond)
to not have to change the API when a better resolution is required. For
example, the NTP protocol uses fractions of 2\ :sup:`32` seconds
(approximately 2.3 Ã 10\ :sup:`-10` second), whereas the NTP protocol version
4 uses fractions of 2\ :sup:`64` seconds (5.4 Ã 10\ :sup:`-20` second).

.. note::
   With a resolution of 1 microsecond (10\ :sup:`-6`), float timestamps lose
   precision for values bigger than 2\ :sup:`33` seconds (272 years: 2242-03-16
   for an Epoch timestamp). With a resolution of 100 nanoseconds
   (10\ :sup:`-7`, resolution used on Windows), float timestamps lose precision
   for values bigger than 2\ :sup:`29` seconds (17 years: 1987-01-05 for an
   Epoch timestamp).


Specification
=============

Add decimal.Decimal as a new type for timestamps. Decimal supports any
timestamp resolution, support arithmetic operations and is comparable. It is
possible to coerce a Decimal to float, even if the conversion may lose
precision. The clock resolution can also be stored in a Decimal object.

Add an optional *timestamp* argument to:

* os module: fstat(), fstatat(), lstat(), stat() (st_atime,
  st_ctime and st_mtime fields of the stat structure),
  sched_rr_get_interval(), times(), wait3() and wait4()
* resource module: ru_utime and ru_stime fields of getrusage()
* signal module: getitimer(), setitimer()
* time module: clock(), clock_gettime(), clock_getres(),
  monotonic(), time() and wallclock()

The *timestamp* argument value can be float or Decimal, float is still the
default for backward compatibility. The following functions support Decimal as
input:

* datetime module: date.fromtimestamp(), datetime.fromtimestamp() and
  datetime.utcfromtimestamp()
* os module: futimes(), futimesat(), lutimes(), utime()
* select module: epoll.poll(), kqueue.control(), select()
* signal module: setitimer(), sigtimedwait()
* time module: ctime(), gmtime(), localtime(), sleep()

The os.stat_float_times() function is deprecated: use an explicit cast using
int() instead.

.. note::
   The decimal module is implemented in Python and is slower than float, but
   there is a new C implementation which is almost ready for inclusion in
   CPython.


Backwards Compatibility
=======================

The default timestamp type (float) is unchanged, so there is no impact on
backward compatibility nor on performances. The new timestamp type,
decimal.Decimal, is only returned when requested explicitly.


Objection: clocks accuracy
==========================

Computer clocks and operating systems are inaccurate and fail to provide
nanosecond accuracy in practice. A nanosecond is what it takes to execute a
couple of CPU instructions.  Even on a real-time operating system, a
nanosecond-precise measurement is already obsolete when it starts being
processed by the higher-level application. A single cache miss in the CPU will
make the precision worthless.

.. note::
   Linux *actually* is able to measure time in nanosecond precision, even
   though it is not able to keep its clock synchronized to UTC with a
   nanosecond accuracy.


Alternatives: Timestamp types
=============================

To support timestamps with an arbitrary or nanosecond resolution, the following
types have been considered:

* decimal.Decimal
* number of nanoseconds
* 128-bits float
* datetime.datetime
* datetime.timedelta
* tuple of integers
* timespec structure

Criteria:

* Doing arithmetic on timestamps must be possible
* Timestamps must be comparable
* An arbitrary resolution, or at least a resolution of one nanosecond without
  losing precision
* It should be possible to coerce the new timestamp to float for backward
  compatibility


A resolution of one nanosecond is enough to support all current C functions.

The best resolution used by operating systems is one nanosecond. In practice,
most clock accuracy is closer to microseconds than nanoseconds. So it sounds
reasonable to use a fixed resolution of one nanosecond.


Number of nanoseconds (int)
---------------------------

A nanosecond resolution is enough for all current C functions and so a
timestamp can simply be a number of nanoseconds, an integer, not a float.

The number of nanoseconds format has been rejected because it would require to
add new specialized functions for this format because it not possible to
differentiate a number of nanoseconds and a number of seconds just by checking
the object type.


128-bits float
--------------

Add a new IEEE 754-2008 quad-precision binary float type. The IEEE 754-2008
quad precision float has 1 sign bit, 15 bits of exponent and 112 bits of
mantissa.  128-bits float is supported by GCC (4.3), Clang and ICC compilers.

Python must be portable and so cannot rely on a type only available on some
platforms. For example, Visual C++ 2008 doesn't support 128-bits float, whereas
it is used to build the official Windows executables. Another example: GCC 4.3
does not support __float128 in 32-bit mode on x86 (but GCC 4.4 does).

There is also a license issue: GCC uses the MPFR library for 128-bits float,
library distributed under the GNU LGPL license. This license is not compatible
with the Python license.

.. note::
  The x87 floating point unit of Intel CPU supports 80-bit floats. This format
  is not supported by the SSE instruction set, which is now preferred over
  float, especially on x86_64. Other CPU vendors don't support 80-bit float.



datetime.datetime
-----------------

The datetime.datetime type is the natural choice for a timestamp because it is
clear that this type contains a timestamp, whereas int, float and Decimal are
raw numbers. It is an absolute timestamp and so is well defined. It gives
direct access to the year, month, day, hours, minutes and seconds. It has
methods related to time like methods to format the timestamp as string (e.g.
datetime.datetime.strftime).

The major issue is that except os.stat(), time.time() and
time.clock_gettime(time.CLOCK_GETTIME), all time functions have an unspecified
starting point and no timezone information, and so cannot be converted to
datetime.datetime.

datetime.datetime has also issues with timezone. For example, a datetime object
without timezone (unaware) and a datetime with a timezone (aware) cannot be
compared. There is also an ordering issues 


id: pep-0376.rst
content:
PEP: 376
Title: Database of Installed Python Distributions
Author: Tarek ZiadÃ© <tarek@ziade.org>
Status: Final
Type: Standards Track
Topic: Packaging
Created: 22-Feb-2009
Python-Version: 2.7, 3.2
Post-History: `22-Jun-2009 <https://mail.python.org/archives/list/python-dev@python.org/thread/ILLTIOZAULMDY5CAS6GOITEYJ4HNFATQ/>`__


.. canonical-pypa-spec:: :ref:`packaging:core-metadata`


Abstract
========

The goal of this PEP is to provide a standard infrastructure to manage
project distributions installed on a system, so all tools that are
installing or removing projects are interoperable.

To achieve this goal, the PEP proposes a new format to describe installed
distributions on a system. It also describes a reference implementation
for the standard library.

In the past an attempt was made to create an installation database
(see :pep:`262`).

Combined with :pep:`345`, the current proposal supersedes :pep:`262`.

Note: the implementation plan didn't go as expected, so it should be
considered informative only for this PEP.


Rationale
=========

There are two problems right now in the way distributions are installed in
Python:

- There are too many ways to do it and this makes interoperation difficult.
- There is no API to get information on installed distributions.

How distributions are installed
-------------------------------

Right now, when a distribution is installed in Python, every element can
be installed in a different directory.

For instance, ``Distutils`` installs the pure Python code in the ``purelib``
directory, which is ``lib/python2.6/site-packages`` for unix-like systems and
Mac OS X, or ``Lib\site-packages`` under Python's installation directory for
Windows.

Additionally, the ``install_egg_info`` subcommand of the Distutils ``install``
command adds an ``.egg-info`` file for the project into the ``purelib``
directory.

For example, for the ``docutils`` distribution, which contains one package an
extra module and executable scripts, three elements are installed in
``site-packages``:

- ``docutils``: The ``docutils`` package.
- ``roman.py``: An extra module used by ``docutils``.
- ``docutils-0.5-py2.6.egg-info``: A file containing the distribution metadata
  as described in :pep:`314`. This file corresponds to the file
  called ``PKG-INFO``, built by the ``sdist`` command.

Some executable scripts, such as ``rst2html.py``, are also added in the
``bin`` directory of the Python installation.

Another project called ``setuptools`` [#setuptools]_ has two other formats
to install distributions, called ``EggFormats`` [#eggformats]_:

- a self-contained ``.egg`` directory, that contains all the distribution files
  and the distribution metadata in a file called ``PKG-INFO`` in a subdirectory
  called ``EGG-INFO``. ``setuptools`` creates other files in that directory that can
  be considered as complementary metadata.

- an ``.egg-info`` directory installed in ``site-packages``, that contains the same
  files ``EGG-INFO`` has in the ``.egg`` format.

The first format is automatically used when you install a distribution that
uses the ``setuptools.setup`` function in its setup.py file, instead of
the ``distutils.core.setup`` one.

``setuptools`` also add a reference to the distribution into an
``easy-install.pth`` file.

Last, the ``setuptools`` project provides an executable script called
``easy_install`` [#easyinstall]_ that installs all distributions, including
distutils-based ones in self-contained ``.egg`` directories.

If you want to have standalone ``.egg-info`` directories for your distributions,
e.g. the second ``setuptools`` format, you have to force it when you work
with a setuptools-based distribution or with the ``easy_install`` script.
You can force it by using the ``--single-version-externally-managed`` option
**or** the ``--root`` option. This will make the ``setuptools`` project install
the project like distutils does.

This option is used by :

- the ``pip`` [#pip]_ installer
- the Fedora packagers [#fedora]_.
- the Debian packagers [#debian]_.

Uninstall information
---------------------

Distutils doesn't provide an ``uninstall`` command. If you want to uninstall
a distribution, you have to be a power user and remove the various elements
that were installed, and then look over the ``.pth`` file to clean them if
necessary.

And the process differs depending on the tools you have used to install the
distribution and if the distribution's ``setup.py`` uses Distutils or
Setuptools.

Under some circumstances, you might not be able to know for sure that you
have removed everything, or that you didn't break another distribution by
removing a file that is shared among several distributions.

But there's a common behavior: when you install a distribution, files are
copied in your system. And it's possible to keep track of these files for
later removal.

Moreover, the Pip project has gained an ``uninstall`` feature lately. It
records all installed files, using the ``record`` option of the ``install``
command.

What this PEP proposes
----------------------

To address those issues, this PEP proposes a few changes:

- A new ``.dist-info`` structure using a directory, inspired on one format of
  the ``EggFormats`` standard from ``setuptools``.
- New APIs in ``pkgutil`` to be able to query the information of installed
  distributions.
- An uninstall function and an uninstall script in Distutils.


One .dist-info directory per installed distribution
===================================================

This PEP proposes an installation format inspired by one of the options in the
``EggFormats`` standard, the one that uses a distinct directory located in the
site-packages directory.

This distinct directory is named as follows::

    name + '-' + version + '.dist-info'

This ``.dist-info`` directory can contain these files:

- ``METADATA``: contains metadata, as described in :pep:`345`, :pep:`314` and :pep:`241`.
- ``RECORD``: records the list of installed files
- ``INSTALLER``: records the name of the tool used to install the project
- ``REQUESTED``: the presence of this file indicates that the project
  installation was explicitly requested (i.e., not installed as a dependency).

The METADATA, RECORD and INSTALLER files are mandatory, while REQUESTED may
be missing.

This proposal will not impact Python itself because the metadata files are not
used anywhere yet in the standard library besides Distutils.

It will impact the ``setuptools`` and ``pip`` projects but, given the fact that
they already work with a directory that contains a ``PKG-INFO`` file, the change
will have no deep consequences.


RECORD
------

A ``RECORD`` file is added inside the ``.dist-info`` directory at installation
time when installing a source distribution using the ``install`` command.
Notice that when installing a binary distribution created with ``bdist`` command
or a ``bdist``-based command, the ``RECORD`` file will be installed as well since
these commands use the ``install`` command to create binary distributions.

The ``RECORD`` file holds the list of installed files. These correspond
to the files listed by the ``record`` option of the ``install`` command, and will
be generated by default. This allows the implementation of an uninstallation
feature, as explained later in this PEP. The ``install`` command also provides
an option to prevent the ``RECORD`` file from being written and this option
should be used when creating system packages.

Third-party installation tools also should not overwrite or delete files
that are not in a RECORD file without prompting or warning.

This RECORD file is inspired from :pep:`262` FILES.

The ``RECORD`` file is a CSV file, composed of records, one line per
installed file. The ``csv`` module is used to read the file, with
these options:

- field delimiter : ``,``
- quoting char :  ``"``.
- line terminator : ``os.linesep`` (so ``\r\n`` or ``\n``)

When a distribution is installed, files can be installed under:

- the **b


id: pep-0438.rst
content:
PEP: 438
Title: Transitioning to release-file hosting on PyPI
Author: Holger Krekel <holger@merlinux.eu>, Carl Meyer <carl@oddbird.net>
BDFL-Delegate: Richard Jones <richard@python.org>
Discussions-To: distutils-sig@python.org
Status: Superseded
Type: Process
Topic: Packaging
Created: 15-Mar-2013
Post-History: 19-May-2013
Superseded-By: 470
Resolution: https://mail.python.org/pipermail/distutils-sig/2013-May/020773.html


Abstract
========

This PEP proposes a backward-compatible two-phase transition process
to speed up, simplify and robustify installing from the
pypi.python.org (PyPI) package index.  To ease the transition and
minimize client-side friction, **no changes to distutils or existing
installation tools are required in order to benefit from the first
transition phase, which will result in faster, more reliable installs
for most existing packages**.

The first transition phase implements easy and explicit means for a
package maintainer to control which release file links are served to
present-day installation tools.  The first phase also includes the
implementation of analysis tools for present-day packages, to support
communication with package maintainers and the automated setting of
default modes for controlling release file links.  The first phase
also will default newly-registered projects on PyPI to only serve
links to release files which were uploaded to PyPI.

The second transition phase concerns end-user installation tools,
which shall default to only install release files that are hosted on
PyPI and tell the user if external release files exist, offering a
choice to automatically use those external files.  External release
files shall in the future be registered together with a checksum
hash so that installation tools can verify the integrity of the
eventual download (PyPI-hosted release files always carry such
a checksum).

Alternative PyPI server implementations should implement the new
simple index serving behaviour of transition phase 1 to avoid
installation tools treating their release links as external ones in
phase 2.


Rationale
=========

.. _history:

History and motivations for external hosting
--------------------------------------------

When PyPI went online, it offered release registration but had no
facility to host release files itself.  When hosting was added, no
automated downloading tool existed yet.  When Phillip Eby implemented
automated downloading (through setuptools), he made the choice to
allow people to use download hosts of their choice.  The finding of
externally-hosted packages was implemented as follows:

#. The PyPI ``simple/`` index for a package contains all links found
   by scraping them from that package's long_description metadata for
   any release. Links in the "Download-URL" and "Home-page" metadata
   fields are given ``rel=download`` and ``rel=homepage`` attributes,
   respectively.

#. Any of these links whose target is a file whose name appears to be
   in the form of an installable source or binary distribution, with
   name in the form "packagename-version.ARCHIVEEXT", is considered a
   potential installation candidate by installation tools.

#. Similarly, any links suffixed with an "#egg=packagename-version"
   fragment are considered an installation candidate.

#. Additionally, the ``rel=homepage`` and ``rel=download`` links are
   crawled by installation tools and, if HTML, are themselves scraped
   for release-file links in the above formats.

See the easy_install documentation for a complete description of this
behavior. [1]_

Today, most packages indexed on PyPI host their release files on
PyPI. Out of 29,117 total projects on PyPI, only 2,581 (less than 10%)
include any links to installable files that are available only
off-PyPI. [2]_

There are many reasons [3]_ why people have chosen external
hosting. To cite just a few:

- release processes and scripts have been developed already and upload
  to external sites

- it takes too long to upload large files from some places in the
  world

- export restrictions e.g. for crypto-related software

- company policies which require offering open source packages through
  own sites

- problems with integrating uploading to PyPI into one's release
  process (because of release policies)

- desiring download statistics different from those maintained by PyPI

- perceived bad reliability of PyPI

- not aware that PyPI offers file-hosting

Irrespective of the present-day validity of these reasons, there
clearly is a history why people choose to host files externally and it
even was for some time the only way you could do things.  This PEP
takes the position that there remain some valid reasons for
external hosting even today.

Problem
-------

**Today, python package installers (pip, easy_install, buildout, and
others) often need to query many non-PyPI URLs even if there are no
externally hosted files**.  Apart from querying pypi.python.org's
simple index pages, also all homepages and download pages ever
specified with any release of a package are crawled by an installer.
The need for installers to crawl external sites slows down
installation and makes for a brittle and unreliable installation
process.  Those sites and packages also don't take part in the
:pep:`381` mirroring infrastructure, further decreasing reliability
and speed of automated installation processes around the world.

Most packages are hosted directly on pypi.python.org [2]_.  Even for
these packages, installers still crawl their homepage and
download-url, if specified.  Many package uploaders are not aware that
specifying the "homepage" or "download-url" in their package metadata
will needlessly slow down the installation process for all users.

Relying on third party sites also opens up more attack vectors for
injecting malicious packages into sites using automated installs.  A
simple attack might just involve getting hold of an old now-unused
homepage domain and placing malicious packages there.  Moreover,
performing a Man-in-The-Middle (MITM) attack between an installation
site and any of the download sites can inject malicious packages on
the installation site.  As many homepages and download locations are
using HTTP and not HTTPS, such attacks are not hard to launch.  Such
MITM attacks can easily happen even for packages which never intended
to host files externally as their homepages are contacted by
installers anyway.

There is currently no way for package maintainers to avoid
external-link crawling, other than removing all homepage/download url
metadata for all historic releases.  While a script [4]_ has been
written to perform this action, it is not a good general solution
because it removes useful metadata from PyPI releases.

Even if the sites referenced by "Homepage" and "Download-URL" links
were not scraped for further links, there is no obvious way under the
current system for a package owner to link to an installable file from
a long_description metadata field (which is shown as package
documentation on ``/pypi/PKG``) without installation tools
automatically considering that file a candidate for installation.
Conversely, there is no way to explicitly register multiple external
release files without putting them in metadata fields.


Goals
-----

These are the goals to be achieved by implementation of this PEP:

* Package owners should be able to explicitly control which files are
  presented by PyPI to installer tools as installation
  candidates. Installation should not be slowed and made less reliable
  by extensive and unnecessary crawling of links that package owners
  did not explicitly nominate as installation files.

* It should remain possible for package owners to choose to host their
  release files on their own hosting, external to PyPI. It should be
  easy for a user to request the installation of such releases using
  automated installer tools, especially if the external release files
  were registered together with a checksum hash.

* Auto


id: pep-0606.rst
content:
PEP: 606
Title: Python Compatibility Version
Author: Victor Stinner <vstinner@python.org>
Status: Rejected
Type: Standards Track
Created: 18-Oct-2019
Python-Version: 3.9


Abstract
========

Add ``sys.set_python_compat_version(version)`` to enable partial
compatibility with requested Python version. Add
``sys.get_python_compat_version()``.

Modify a few functions in the standard library to implement partial
compatibility with Python 3.8.

Add ``sys.set_python_min_compat_version(version)`` to deny backward
compatibility with Python versions older than *version*.

Add ``-X compat_version=VERSION`` and ``-X min_compat_version=VERSION``
command line options. Add ``PYTHONCOMPATVERSION`` and
``PYTHONCOMPATMINVERSION`` environment variables.


Rationale
=========

The need to evolve frequently
-----------------------------

To remain relevant and useful, Python has to evolve frequently; some
enhancements require incompatible changes. Any incompatible change can
break an unknown number of Python projects.  Developers can decide to
not implement a feature because of that.

Users want to get the latest Python version to obtain new features and
better performance. A few incompatible changes can prevent them from using their
applications on the latest Python version.

This PEP proposes to add a partial compatibility with old Python
versions as a tradeoff to fit both use cases.

The main issue with the migration from Python 2 to Python 3 is not that
Python 3 is backward incompatible, but how incompatible changes were
introduced.


Partial compatibility to minimize the Python maintenance burden
---------------------------------------------------------------

While technically it would be possible to provide full compatibility
with old Python versions, this PEP proposes to minimize the number of
functions handling backward compatibility to reduce the maintenance
burden of the Python project (CPython).

Each change introducing backport compatibility to a function should be
properly discussed to estimate the maintenance cost in the long-term.

Backward compatibility code will be dropped on each Python release, on a
case-by-case basis. Each compatibility function can be supported for a
different number of Python releases depending on its maintenance cost
and the estimated risk (number of broken projects) if it's removed.

The maintenance cost does not only come from the code implementing the
backward compatibility, but also comes from the additional tests.


Cases excluded from backward compatibility
------------------------------------------

The performance overhead of any compatibility code must be low when
``sys.set_python_compat_version()`` is not called.

The C API is out of the scope of this PEP: ``Py_LIMITED_API`` macro and
the stable ABI are solving this problem differently, see the :pep:`PEP 384:
Defining a Stable ABI <384>`.

Security fixes which break backward compatibility on purpose will
not get a compatibility layer; security matters more than compatibility.
For example, ``http.client.HTTPSConnection`` was modified in Python
3.4.3 to performs all the necessary certificate and hostname checks by
default. It was a deliberate change motivated by :pep:`PEP 476: Enabling
certificate verification by default for stdlib http clients
<476>` (`bpo-22417
<https://bugs.python.org/issue22417>`_).

The Python language does not provide backward compatibility.

Changes which are not clearly incompatible are not covered by this PEP.
For example, Python 3.9 changed the default protocol in the ``pickle``
module to Protocol 4 which was first introduced in Python 3.4. This
change is backward compatible up to Python 3.4. There is no need to use
the Protocol 3 by default when compatibility with Python 3.8 is
requested.

The new ``DeprecationWarning`` and ``PendingDeprecatingWarning`` warnings
in Python 3.9 will not be disabled in Python 3.8 compatibility mode.
If a project runs its test suite using ``-Werror`` (treat any warning as
an error), these warnings must be fixed, or specific deprecation
warnings must be ignored on a case-by-case basis.


Upgrading a project to a newer Python
-------------------------------------

Without backward compatibility, all incompatible changes must be fixed
at once, which can be a blocker issue. It is even worse when a project
is upgraded to a newer Python which is separated by multiple releases
from the old Python.

Postponing an upgrade only makes things worse: each skipped release adds
more incompatible changes. The technical debt only steadily
increases over time.

With backward compatibility, it becomes possible to upgrade Python
incrementally in a project, without having to fix all of the issues at once.

The "all-or-nothing" is a showstopper to port large Python 2 code bases
to Python 3. The list of incompatible changes between Python 2 and
Python 3 is long, and it's getting longer with each Python 3.x release.


Cleaning up Python and DeprecationWarning
-----------------------------------------

One of the :pep:`Zen of Python (PEP 20)
<20>` motto is:

    There should be one-- and preferably only one --obvious way to do
    it.

When Python evolves, new ways inevitably emerge. ``DeprecationWarning``\ s
are emitted to suggest using the new way, but many developers ignore
these warnings, which are silent by default (except in the ``__main__``
module: see the :pep:`565`).
Some developers simply ignore all warnings when there are too many
warnings, thus only bother with exceptions when the deprecated code is
removed.

Sometimes, supporting both ways has a minor maintenance cost, but
developers prefer to drop the old way to clean up their code. These kinds of
changes are backward incompatible.

Some developers can take the end of the Python 2 support as an
opportunity to push even more incompatible changes than usual.

Adding an opt-in backward compatibility prevents the breaking of
applications and allows developers to continue doing these cleanups.


Redistribute the maintenance burden
-----------------------------------

The backward compatibility involves authors of incompatible
changes more in the upgrade path.


Examples of backward compatibility
==================================

collections ABC aliases
-----------------------

``collections.abc`` aliases to ABC classes have been removed from the
``collections`` module in Python 3.9, after being deprecated since
Python 3.3. For example, ``collections.Mapping`` no longer exists.

In Python 3.6, aliases were created in ``collections/__init__.py`` by
``from _collections_abc import *``.

In Python 3.7, a ``__getattr__()`` has been added to the ``collections``
module to emit a DeprecationWarning upon first access to an
attribute::

    def __getattr__(name):
        # For backwards compatibility, continue to make the collections ABCs
        # through Python 3.6 available through the collections module.
        # Note: no new collections ABCs were added in Python 3.7
        if name in _collections_abc.__all__:
            obj = getattr(_collections_abc, name)
            import warnings
            warnings.warn("Using or importing the ABCs from 'collections' instead "
                          "of from 'collections.abc' is deprecated since Python 3.3, "
                          "and in 3.9 it will be removed.",
                          DeprecationWarning, stacklevel=2)
            globals()[name] = obj
            return obj
        raise AttributeError(f'module {__name__!r} has no attribute {name!r}')

Compatibility with Python 3.8 can be restored in Python 3.9 by adding
back the ``__getattr__()`` function, but only when backward
compatibility is requested::

    def __getattr__(name):
        if (sys.get_python_compat_version() < (3, 9)
           and name in _collections_abc.__all__):
            ...
        raise AttributeError(f'module {__name__!r} has no attribute {name!r}')


Deprecated open() "U" mode
--------------------------

The ``"U"`` mode of ``open()`` is deprecate


id: pep-0160.rst
content:
PEP: 160
Title: Python 1.6 Release Schedule
Author: Fred L. Drake, Jr. <fred@fdrake.net>
Status: Final
Type: Informational
Topic: Release
Created: 25-Jul-2000
Python-Version: 1.6
Post-History:


Introduction
============

This PEP describes the Python 1.6 release schedule.  The CVS
revision history of this file contains the definitive historical
record.

This release will be produced by BeOpen PythonLabs staff for the
Corporation for National Research Initiatives (CNRI).


Schedule
========

* August 1: 1.6 beta 1 release (planned).
* August 3: 1.6 beta 1 release (actual).
* August 15: 1.6 final release (planned).
* September 5: 1.6 final release (actual).


Features
========

A number of features are required for Python 1.6 in order to
fulfill the various promises that have been made.  The following
are required to be fully operational, documented, and forward
compatible with the plans for Python 2.0:

* Unicode support: The Unicode object defined for Python 2.0 must be provided,
  including all methods and codec support.

* SRE: Fredrik Lundh's new regular expression engine will be used
  to provide support for both 8-bit strings and Unicode strings. It must pass
  the regression test used for the pcre-based version of the re module.

* The curses module was in the middle of a transformation to a package, so the
  final form was adopted.


Mechanism
=========

The release will be created as a branch from the development tree
rooted at CNRI's close of business on 16 May 2000.  Patches
required from more recent checkins will be merged in by moving the
branch tag on individual files whenever possible in order to
reduce mailing list clutter and avoid divergent and incompatible
implementations.

The branch tag is "cnri-16-start".

Patches and features will be merged to the extent required to pass
regression tests in effect on 16 May 2000.

The beta release is tagged "r16b1" in the CVS repository, and the
final Python 1.6 release is tagged "release16" in the repository.


Copyright
=========

This document has been placed in the public domain.


id: pep-0612.rst
content:
PEP: 612
Title: Parameter Specification Variables
Author: Mark Mendoza <mendoza.mark.a@gmail.com>
Sponsor: Guido van Rossum <guido@python.org>
BDFL-Delegate: Guido van Rossum <guido@python.org>
Discussions-To: typing-sig@python.org
Status: Final
Type: Standards Track
Topic: Typing
Created: 18-Dec-2019
Python-Version: 3.10
Post-History: 18-Dec-2019, 13-Jul-2020

.. canonical-typing-spec:: :ref:`typing:paramspec` and
                           :py:class:`typing.ParamSpec`

Abstract
--------

There currently are two ways to specify the type of a callable, the
``Callable[[int, str], bool]`` syntax defined in  :pep:`484`,
and callback protocols from :pep:`PEP
544 <544#callback-protocols>`. Neither of
these support forwarding the parameter types of one callable over to another
callable, making it difficult to annotate function decorators. This PEP proposes
``typing.ParamSpec`` and ``typing.Concatenate`` to
support expressing these kinds of relationships.

Motivation
----------

The existing standards for annotating higher order functions donât give us the
tools to annotate the following common decorator pattern satisfactorily:

.. code-block::

   from typing import Awaitable, Callable, TypeVar

   R = TypeVar("R")

   def add_logging(f: Callable[..., R]) -> Callable[..., Awaitable[R]]:
     async def inner(*args: object, **kwargs: object) -> R:
       await log_to_database()
       return f(*args, **kwargs)
     return inner

   @add_logging
   def takes_int_str(x: int, y: str) -> int:
     return x + 7

   await takes_int_str(1, "A")
   await takes_int_str("B", 2) # fails at runtime

``add_logging``\ , a decorator which logs before each entry into the decorated
function, is an instance of the Python idiom of one function passing all
arguments given to it over to another function.  This is done through the
combination of the ``*args`` and ``**kwargs`` features in both parameters and in
arguments. When one defines a function (like ``inner``\ ) that takes ``(*args,
**kwargs)`` and goes on to call another function with ``(*args, **kwargs)``,
the wrapping function can only be safely called in all of the ways that the
wrapped function could be safely called. To type this decorator, weâd like to be
able to place a dependency between the parameters of the callable ``f`` and the
parameters of the returned function. :pep:`484`
supports dependencies between
single types, as in ``def append(l: typing.List[T], e: T) -> typing.List[T]:
...``\ , but there is no existing way to do so with a complicated entity like
the parameters of a function.

Due to the limitations of the status quo, the ``add_logging`` example will type
check but will fail at runtime. ``inner`` will pass the string âBâ into
``takes_int_str``\, which will try to add 7 to it, triggering a type error.
This was not caught by the type checker because the decorated ``takes_int_str``
was given the type ``Callable[..., Awaitable[int]]`` (an ellipsis in place of
parameter types is specified to mean that we do no validation on arguments).

Without the ability to define dependencies between the parameters of different
callable types, there is no way, at present, to make ``add_logging`` compatible
with all functions, while still preserving the enforcement of the parameters of
the decorated function.

With the addition of the ``ParamSpec`` variables proposed by this
PEP, we can rewrite the previous example in a way that keeps the flexibility of
the decorator and the parameter enforcement of the decorated function.

.. code-block::

   from typing import Awaitable, Callable, ParamSpec, TypeVar

   P = ParamSpec("P")
   R = TypeVar("R")

   def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]:
     async def inner(*args: P.args, **kwargs: P.kwargs) -> R:
       await log_to_database()
       return f(*args, **kwargs)
     return inner

   @add_logging
   def takes_int_str(x: int, y: str) -> int:
     return x + 7

   await takes_int_str(1, "A") # Accepted
   await takes_int_str("B", 2) # Correctly rejected by the type checker

Another common decorator pattern that has previously been impossible to type is
the practice of adding or removing arguments from the decorated function.  For
example:

.. code-block::

   class Request:
     ...

   def with_request(f: Callable[..., R]) -> Callable[..., R]:
     def inner(*args: object, **kwargs: object) -> R:
       return f(Request(), *args, **kwargs)
     return inner

   @with_request
   def takes_int_str(request: Request, x: int, y: str) -> int:
     # use request
     return x + 7

   takes_int_str(1, "A")
   takes_int_str("B", 2) # fails at runtime


With the addition of the ``Concatenate`` operator from this PEP, we can even
type this more complex decorator.

.. code-block::

   from typing import Concatenate

   def with_request(f: Callable[Concatenate[Request, P], R]) -> Callable[P, R]:
     def inner(*args: P.args, **kwargs: P.kwargs) -> R:
       return f(Request(), *args, **kwargs)
     return inner

   @with_request
   def takes_int_str(request: Request, x: int, y: str) -> int:
     # use request
     return x + 7

   takes_int_str(1, "A") # Accepted
   takes_int_str("B", 2) # Correctly rejected by the type checker


Specification
-------------

``ParamSpec`` Variables
^^^^^^^^^^^^^^^^^^^^^^^

Declaration
````````````

A parameter specification variable is defined in a similar manner to how a
normal type variable is defined with ``typing.TypeVar``.

.. code-block::

   from typing import ParamSpec
   P = ParamSpec("P")         # Accepted
   P = ParamSpec("WrongName") # Rejected because P =/= WrongName

The runtime should accept ``bound``\ s and ``covariant`` and ``contravariant``
arguments in the declaration just as ``typing.TypeVar`` does, but for now we
will defer the standardization of the semantics of those options to a later PEP.

Valid use locations
```````````````````

Previously only a list of parameter arguments (``[A, B, C]``) or an ellipsis
(signifying "undefined parameters") were acceptable as the first "argument" to
``typing.Callable`` .  We now augment that with two new options: a parameter
specification variable (``Callable[P, int]``\ ) or a concatenation on a
parameter specification variable (``Callable[Concatenate[int, P], int]``\ ).

.. code-block::

   callable ::= Callable "[" parameters_expression, type_expression "]"

   parameters_expression ::=
     | "..."
     | "[" [ type_expression ("," type_expression)* ] "]"
     | parameter_specification_variable
     | concatenate "["
                      type_expression ("," type_expression)* ","
                      parameter_specification_variable
                   "]"

where ``parameter_specification_variable`` is a ``typing.ParamSpec`` variable,
declared in the manner as defined above, and ``concatenate`` is
``typing.Concatenate``.

As before, ``parameters_expression``\ s by themselves are not acceptable in
places where a type is expected

.. code-block::

   def foo(x: P) -> P: ...                           # Rejected
   def foo(x: Concatenate[int, P]) -> int: ...       # Rejected
   def foo(x: typing.List[P]) -> None: ...           # Rejected
   def foo(x: Callable[[int, str], P]) -> None: ...  # Rejected


User-Defined Generic Classes
````````````````````````````

Just as defining a class as inheriting from ``Generic[T]`` makes a class generic
for a single parameter (when ``T`` is a ``TypeVar``\ ), defining a class as
inheriting from ``Generic[P]`` makes a class generic on
``parameters_expression``\ s (when ``P`` is a ``ParamSpec``).

.. code-block::

   T = TypeVar("T")
   P_2 = ParamSpec("P_2")

   class X(Generic[T, P]):
     f: Callable[P, int]
     x: T

   def f(x: X[int, P_2]) -> str: ...                    # Accepted
   def f(x: X[int, Concatenate[int, P_2]]) -> str: ...  # Accepted
   def f(x: X[int, [int, bool]]) -> str: ...            # Accepted
   def f(x: X[int, ...]) -> str: ...                    # Accepted
   def f(x: X[int, int]) -> str: ...       


id: pep-0772.rst
content:
PEP: 772
Title: Packaging governance process
Author: Barry Warsaw <barry@python.org>,
        Deb Nicholson <deb@python.org>,
        Pradyun Gedam <pradyunsg@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-772-packaging-governance-process/79724
Status: Draft
Type: Process
Topic: Governance, Packaging
Created: 21-Jan-2025
Replaces: 609


========
Abstract
========

This PEP proposes a Python Packaging Council with broad authority over
packaging standards, tools, and implementations. Like the Python Steering
Council, the Packaging Council seeks to exercise this authority as rarely as
possible; instead, they use this power to establish standard processes.

Like :pep:`8016`, instead of trying to do everything in one PEP, this PEP
focuses on providing a minimal-but-solid foundation for further governance
decisions.

========================
Motivation and Rationale
========================

As Python packaging has matured, several interrelated problems with the current
way of managing the technical development, decision making and processes have
become apparent.

----
PyPA
----

The `Python Packaging Authority (PyPA)`_ was created to take over the
maintenance of `pip`_ and `virtualenv`_ from Ian Bicking, led by Brian Rosner,
Carl Meyer and Jannis Leidel. Over the years, additional projects got added to
the PyPA and the tooling maintained by the PyPA became the baseline tooling for
Python packaging, with additional package ecosystems using and building on top
of the tools and interoperability standards maintained by the PyPA.

:pep:`609` formalised PyPA's authority over the existing packaging tools, the
interoperability standards and their operation as a group of independent
projects under the same umbrella that aim to be interoperable. It also stated
that PyPA should be expected to provide opinions, insight and experience when
ecosystem-wide changes are being proposed. In particular, the work to support
existing projects and maintain a user guide has been excellent.

At the time of writing, PyPA does not have a community elected body and
individuals within the group meet ad hoc, rather than on a regular cadence. The
PyPA itself is defined by a group of projects rather than as a group of
individuals, with no direct oversight over its individuals.

------------
Packaging-WG
------------

The PSF's `Packaging Workgroup`_ was created to support the larger efforts of
improving and maintaining the packaging ecosystem in Python through fundraising
and disbursement of raised funds. It was expected to largely focus on efforts
such as Python Package Index (PyPI), pip, packaging.python.org, Setuptools, and
cross-project efforts.

The group made very good progress here in the past, but the scope and scale of
supporting Python's packaging ecosystem has grown considerably in the last few
years. There is both more work to be done and more stakeholders than ever
before. The Packaging Workgroup is also not a community elected body and does
not have a regular mechanism for changing, adding or removing members.

--------------------------
Interoperability Standards
--------------------------

The PyPA is responsible for creating and maintaining the interoperability
standards for Python packaging tools. The decision making for updating these
standards involves a standing delegation from the Python Steering Council to
specific individuals based on specific topics, and these individuals have the
ability to further delegate the decision making on the changes to other
individuals.

We know that this process is not sustainable and there has been a desire
expressed by many to have a designated body to make these decisions (including
the individuals with the standing delegations).

-----------------------
Python Steering Council
-----------------------

While there is overlap between the PyPA, Packaging-WG, and the Python core
team, the Steering Council is not well-placed to make decisions over Python
packaging matters directly. Packaging only tangentially intersects with Python
language evolution, packaging requires specialised expertise and deep domain
knowledge (much like typing or the C API), and there is a different cohort of
potential electors for a Packaging Council than the Steering Council.

------------
Expectations
------------

An elected Packaging Council would have authority over the interoperability
standards, would guide the packaging tooling provided with CPython, and have
the mandate to coordinate Python packaging efforts. This would provide a
better, more consistent experience to packagers, package consumers, tool
developers, PyPI, and other index maintainers. With greater transparency and
clearer goals via an elected Packaging Council, the PSF will also be able to
offer more tactical work, longer term strategic vision, and fundraising support
to all parts of this ecosystem.

Some expectations around the formation of a Packaging Council are that they
will facilitate:

* Work with PSF staff and the newly formed `User Success Workgroup`_ toward
  improving the user experience around packaging.
* A more stable packaging ecosystem that is more responsive to community input.
* Increased transparency and clearer shared goals for the packaging ecosystem.
* Increased tactical and fundraising support from the PSF, to increase capacity
  and funding available to packaging tools.

=============
Specification
=============

-----------------
Packaging Council
-----------------

The Packaging Council will be composed of five individuals.

Mandate
=======

The council shall work to:

* Maintain the quality and stability of the Python packaging standards.
* Formalise and maintain the relationship with the core team as well as the
  PSF.
* Establish appropriate decision-making processes.
* Improve Python packaging's user experience.
* Make contributing as accessible, inclusive, and sustainable as possible.
* Strive to seek consensus among contributors before acting in a formal
  capacity.

Responsibilities
================

The council shall:

* Have broad authority over the Python packaging standards and Python Packaging
  User Guide, that are maintained on https://packaging.python.org.
* Establish processes for making binding decisions regarding packaging
  standards, tools and implementations as well as for considering
  ecosystem-wide changes.
* Look for ways to use their direct authority as little as possible - seeking
  consensus and consent over voting.

To use its powers, the council votes. Every council member must either vote or
explicitly abstain. Members with conflicts of interest on a particular vote
must abstain. Passing requires support from a majority of non-abstaining
council members.

Whenever possible, the council is expected to share its decisions and processes
publicly in a timely fashion.

Delegations
-----------

The Packaging Council gains its authority over packaging matters via delegation from the Python
Steering Council.  Upon acceptance of this PEP, the Steering Council will formally issue a standing
delegation to the Packaging Council for PEPs related to Python packaging. The bodies would work
together on issues that intersect the packaging domain and language stewardship (including the
CPython implementation, standard library, and distribution).

The PSF Board is encouraged to formally deactivate the Packaging Workgroup and
the Packaging Council would take on the responsibilities of the PSF's Packaging
Workgroup.

The PyPA is expected to work with the Packaging Council to establish a decision
making process that governs the technical projects under the PyPA umbrella.

Processes
=========

Election of the council
-----------------------

A council election consists of two phases:

* Phase 1: A :ref:`voting member <voting-member>` can nominate any individual for the council
  elections, including themselves.  Nominations must include information about the nominee's
  relevant affiliations.  Nominees for the Packaging Council do not need


id: pep-0766.rst
content:
PEP: 766
Title: Explicit Priority Choices Among Multiple Indexes
Author: Michael Sarahan <msarahan@gmail.com>
Sponsor: Barry Warsaw <barry@python.org>
PEP-Delegate: Paul Moore <p.f.moore@gmail.com>
Discussions-To: https://discuss.python.org/t/pep-for-handling-multiple-indexes-index-priority/71589
Status: Draft
Type: Informational
Topic: Packaging
Created: 18-Nov-2024
Post-History: `18-Nov-2024 <https://discuss.python.org/t/pep-for-handling-multiple-indexes-index-priority/71589>`__,

Abstract
========

Package resolution is a key part of the Python user experience as the means of
extending Python's core functionality. The experience of package resolution is
mostly taken for granted until someone encounters a situation where the package
installer does something they don't expect.  The installer behavior with
multiple indexes has been `a common source of unexpected behavior
<https://github.com/pypa/pip/issues/8606>`__.  Through its ubiquity, pip has
long defined the standard expected behavior across other tools in the ecosystem,
but Python installers are diverging with respect to how they handle multiple
indexes. At the core of this divergence is whether index contents are combined
before resolving distributions, or each index is handled individually in order.
pip merges all indexes before matching distributions, while uv matches
distributions on one index before moving on to the next. Each approach has
advantages and disadvantages.  This PEP aims to describe each of these
behaviors, which are referred to as âversion priorityâ and âindex priorityâ
respectively, so that community discussions and troubleshooting can share a
common vocabulary, and so that tools can implement predictable behavior based on
these descriptions.

Motivation
==========

Python package users frequently find themselves in need of specifying an index
or package source other than PyPI. There are many reasons for external indexes
to exist:

- File size/quota limitations on PyPI
- Implementation variants, such as `different GPU library builds in PyTorch <https://pytorch.org/get-started/locally/>`__
- `Local builds of packages shared internally at an organization <https://github.com/pypa/pip/issues/8606>`__
- `Situations where a local package has remote dependencies
  <https://github.com/pypa/pip/issues/11624>`__, and the user wishes to prioritize
  local packages over remote dependencies, while still falling back to remote
  dependencies where needed

In most of these cases, it is not desirable to completely forego PyPI. Instead,
users generally want PyPI to still be a source of packages, but a lower priority
source. Unfortunately, `pip's current design precludes this concept of priority <https://github.com/pypa/pip/issues/8606>`__.
Some Python installer tools have developed alternative ways to handle multiple
indexes that incorporate mechanisms to express index priority, such as `uv
<https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes>`__
and `PDM
<https://pdm-project.org/latest/usage/config/#respect-the-order-of-the-sources>`__.

The innovation and the potential for customization is exciting, but it comes at
the risk of further fragmenting the python packaging ecosystem, which is already
perceived as one of Python's weak points. The motivation of this PEP is to encourage
installers to provide more insight into how they handle multiple indexes, and to
provide a vocabulary that can be common to the broader community.

Specification
=============

âVersion priorityâ
------------------

This behavior is characterized by the installer always getting the
"best" version of a package, regardless of the index that it comes
from. "Best" is defined by the installer's algorithm for optimizing
the various traits of a package, also factoring in user input (such as
preferring only binaries, or no binaries). While installers may differ
in their optimization criteria and user options, the general trait that
all version priority installers share is that the index
contents are collated prior to candidate selection.

Version priority is most useful when all configured indexes are equally trusted
and well-behaved regarding the distribution interchangeability assumption.
Mirrors are especially well-behaved in this regard. That interchangeability
assumption is what makes comparing distributions of a given package meaningful.
Without it, the installer is no longer comparing âapples to apples.â In
practice, it is common for different indexes to have files that have different
contents than other indexes, such as builds for special hardware, or differing
metadata for the same package. Version priority behavior can lead to
undesirable, unexpected outcomes in these cases, and this is where `users
generally look for some kind of index priority
<https://github.com/pypa/pip/issues/8606>`__. Additionally, when there is a
difference in trust among indexes, version priority does not provide a way to
prefer more trusted indexes over less trusted indexes. This has been exploited by
dependency confusion attacks, and :pep:`708` was proposed as a way of
hard-coding a notion of trusted external indexes into the index.

The "version priority" name is new, and introduction of new terms should always
be minimized. This PEP looks toward the uv project, which refers to `its implementation of the version priority
behavior <https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes>`__
as â``unsafe-best-match``.â Naming is really hard here. On one hand, it
isnât accurate to call pipâs default behavior intrinsically âunsafe.â
The addition of possibly malicious indexes is what
introduces concern with this behavior. :pep:`708` added a way to restrict
installers from drawing packages from unexpected, potentially insecure
indexes. On the other hand, the term âbest-matchâ is technically
correct, but also misleading. The âbest matchâ varies by user and by
application. âBestâ is technically correct in the sense that it is a
global optimum according to the match criteria specified above, but that
is not necessarily what is âbestâ in a userâs eyes. âVersion priorityâ
is a proposed term that avoids the concerns with the uv terminology,
while approximating the behavior in the most user-identifiable way that
packages are compared.

âIndex priorityâ
----------------

In index priority, the resolver finds candidates for each index, one at a time.
The resolver proceeds to subsequent indexes only if the current package request
has no viable candidates. Index priority does not combine indexes into one
global, flat namespace. Because indexes are searched in order, the package from
an earlier index will be preferred over a package from a later index,
regardless of whether the later index had a better match with the installer's
optimization criteria. For a given installer, the optimization criteria and
selection algorithm should be the same for both index priority and version
priority. It is only the treatment of multiple indexes that differs: all
together for version priority, and individually for index priority.

The order of specification of indexes determines their priority in the
finding process. As a result, the way that installers load the index
configuration must be predictable and reproducible. This PEP does not prescribe
any particular mechanism, other than to say that installers should provide
a way of ordering their collection of sources. Installers should also
ideally provide optional debugging output that provides insight into
which index is being considered.

Each packageâs finder should start at the beginning of the list of indexes, so each
package starts over with the index list. In other words, if one package has no
valid candidates on the first index, but finds a hit on the second index,
subsequent packages should still start their search on the first index, rather than
starting on the second.

One desirable behavior


id: pep-0216.rst
content:
PEP: 216
Title: Docstring Format
Author: Moshe Zadka <moshez@zadka.site.co.il>
Status: Withdrawn
Type: Informational
Created: 31-Jul-2000
Post-History:
Superseded-By: 287


.. withdrawn::

   It has been superseded by :pep:`287`.


Abstract
========

Named Python objects, such as modules, classes and functions, have a
string attribute called ``__doc__``. If the first expression inside
the definition is a literal string, that string is assigned
to the ``__doc__`` attribute.

The ``__doc__`` attribute is called a documentation string, or docstring.
It is often used to summarize the interface of the module, class or
function. However, since there is no common format for documentation
string, tools for extracting docstrings and transforming those into
documentation in a standard format (e.g., DocBook) have not sprang
up in abundance, and those that do exist are for the most part
unmaintained and unused.


Perl Documentation
==================

In Perl, most modules are documented in a format called POD -- Plain
Old Documentation. This is an easy-to-type, very low level format
which integrates well with the Perl parser. Many tools exist to turn
POD documentation into other formats: info, HTML and man pages, among
others. However, in Perl, the information is not available at run-time.


Java Documentation
==================

In Java, special comments before classes and functions function to
document the code. A program to extract these, and turn them into
HTML documentation is called javadoc, and is part of the standard
Java distribution. However, the only output format that is supported
is HTML, and JavaDoc has a very intimate relationship with HTML.


Python Docstring Goals
======================

Python documentation string are easy to spot during parsing, and are
also available to the runtime interpreter. This double purpose is
a bit problematic, sometimes: for example, some are reluctant to have
too long docstrings, because they do not want to take much space in
the runtime. In addition, because of the current lack of tools, people
read objects' docstrings by "print"ing them, so a tendency to make them
brief and free of markups has sprung up. This tendency hinders writing
better documentation-extraction tools, since it causes docstrings to
contain little information, which is hard to parse.


High Level Solutions
====================

To counter the objection that the strings take up place in the running
program, it is suggested that documentation extraction tools will
concatenate a maximum prefix of string literals which appear in the
beginning of a definition. The first of these will also be available
in the interactive interpreter, so it should contain a few summary
lines.


Docstring Format Goals
======================

These are the goals for the docstring format, as discussed ad nauseam
in the doc-sig.

1. It must be easy to type with any standard text editor.
2. It must be readable to the casual observer.
3. It must not contain information which can be deduced from parsing
   the module.
4. It must contain sufficient information so it can be converted
   to any reasonable markup format.
5. It must be possible to write a module's entire documentation in
   docstrings, without feeling hampered by the markup language.


Docstring Contents
==================

For requirement 5. above, it is needed to specify what must be
in docstrings.

At least the following must be available:

a. A tag that means "this is a Python *something*, guess what"

   Example: In the sentence "The POP3 class", we need to markup "POP3"
   so. The parser will be able to guess it is a class from the contents
   of the ``poplib`` module, but we need to make it guess.

b. Tags that mean "this is a Python class/module/class var/instance var..."

   Example: The usual Python idiom for singleton class ``A`` is to have ``_A``
   as the class, and ``A`` a function which returns ``_A`` objects. It's usual
   to document the class, nonetheless, as being ``A``. This requires the
   strength to say "The class ``A``" and have ``A`` hyperlinked and marked-up
   as a class.

c. An easy way to include Python source code/Python interactive sessions

d. Emphasis/bold

e. List/tables


Docstring Basic Structure
=========================

The documentation strings will be in StructuredTextNG
(http://www.zope.org/Members/jim/StructuredTextWiki/StructuredTextNG)
Since StructuredText is not yet strong enough to handle (a) and (b)
above, we will need to extend it. I suggest using
``[<optional description>:python identifier]``.
E.g.: ``[class:POP3]``, ``[:POP3.list]``, etc. If the description is missing,
a guess will be made from the text.


Unresolved Issues
=================

Is there a way to escape characters in ST? If so, how?
(example: * at the beginning of a line without being bullet symbol)

Is my suggestion above for Python symbols compatible with ST-NG?
How hard would it be to extend ST-NG to support it?

How do we describe input and output types of functions?

What additional constraint do we enforce on each docstring?
(module/class/function)?

What are the guesser rules?


Rejected Suggestions
====================

XML -- it's very hard to type, and too cluttered to read it comfortably.


