Skip to content
155 changes: 152 additions & 3 deletions peps/pep-0694.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PEP: 694
Title: Upload 2.0 API for Python Package Indexes
Author: Barry Warsaw <barry@python.org>, Donald Stufft <donald@stufft.io>, Ee Durbin <ee@python.org>
Author: Barry Warsaw <barry@python.org>, Donald Stufft <donald@stufft.io>, Ee Durbin <ee@python.org>, Cary Hawkins <hawkinscary23@gmail.com>
PEP-Delegate: Dustin Ingram <di@python.org>
Discussions-To: https://discuss.python.org/t/pep-694-pypi-upload-api-2-0-round-2/101483
Status: Draft
Expand Down Expand Up @@ -29,6 +29,9 @@ Along with standardization, the upload API provides additional useful features s
* "staging" a release, which can be used to test uploads before publicly publishing them,
without the need for `test.pypi.org <https://test.pypi.org/>`__;

* entering the publishing session workflow from the existing legacy upload API, so that staging is
available to publishers before their tooling fully adopts this API;

* artifacts which can be overwritten and replaced, until a session is published;

* detailed status on the state of artifact uploads;
Expand Down Expand Up @@ -285,6 +288,28 @@ The unguessable :ref:`stage preview URL <staged-preview>` is a separate capabili
governed by this authorization check; it grants read-only preview access to any client that holds the token,
so that (for example) a CI job can install-test a staged release without project upload credentials.

As one such stricter policy, an index **MAY** require *additional* authorization, beyond upload permission, for
Comment thread
cjames23 marked this conversation as resolved.
the actions that decide a session's terminal outcome, namely :ref:`publishing <publishing-session-completion>`
it and :ref:`canceling <publishing-session-cancellation>` it, while still allowing session creation and file
upload with upload permission alone. This separates the authority to assemble a release from the authority to
publish or discard it: an automated system can hold a credential that creates a session and uploads files to it
but can neither publish nor cancel it, with that authority held elsewhere, so that compromise of the automated
system alone can neither publish a release nor discard a pending one.

Cancellation is protected because it is destructive: it discards the staged files and frees the name-version
pair. An index that left it at the upload-permission level would let a compromised upload credential delete a
Comment thread
cjames23 marked this conversation as resolved.
release a maintainer has staged and is waiting to publish (a denial-of-release), or discard a session that
has been flagged and is awaiting review. For a first release of a new project, cancellation also releases the
temporary project-name reservation the index holds for the session, so an index that did not protect it would
let a compromised upload credential discard the session and allow the name to be claimed by someone else. An
index that instead wants an automated uploader to be able to abort its *own* in-progress uploads **MAY** leave
cancellation at the upload-permission level.

How any additional authorization is expressed, and whether it is offered at all, is determined by the index
operator. An index **MAY** keep this choice operator-wide, or expose it to individual project owners, for
example by letting a project owner require the separate authorization for their own project, or allow a single
principal to hold both.


.. _session-errors:

Expand Down Expand Up @@ -336,9 +361,14 @@ interpretation to aid in diagnosing underlying issue.
Some responses may return more specific HTTP status codes as described in the text below.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self. Once PEP 847 is approved, we should update this PEP to specifically reference it. @woodruffw @dstufft


.. _publishing-session:
.. _session-creation:

Session Creation
----------------

Publishing Session
------------------
A publishing session can be created in two ways: directly through this API, described below, or from a
:ref:`legacy upload <legacy-interop>`. However it is created, the session is then managed through the same
endpoints, described in :ref:`Managing a Publishing Session <managing-session>`.

.. _publishing-session-create:

Expand Down Expand Up @@ -585,6 +615,73 @@ sub-mapping with the following keys:
these notices are specific to the referenced file.


.. _legacy-interop:

Create a Publishing Session from a Legacy Upload
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Publishers cannot use the features of this API until their upload tooling adopts it, and the legacy API is
expected to remain available anyway, as this PEP does not propose a deprecation schedule. To make staging available to those publishers sooner,
an index **MAY** allow a legacy upload to create a publishing session, so that from creation onward the session is the
same as one created directly.

An index that supports this **MUST** document it, and **MUST** accept a ``staged`` field with the value
``true`` in the legacy ``multipart/form-data`` upload request. When that field is present, the index creates a
publishing session in the ``open`` state for the uploaded file's project and version, adds the file to it as a
:ref:`completed <file-upload-session-states>` file upload, and does not publish it. The index **MUST**
return the :ref:`publishing session creation response body <publishing-session-response>` from that upload,
including the ``links`` and ``session-token`` keys, so that the publisher can then use the endpoints in this
PEP to :ref:`preview <staged-preview>`, :ref:`publish <publishing-session-completion>`, or :ref:`cancel
<publishing-session-cancellation>` the session. Returning the body is required because a client that sets
``staged=true`` but receives no session handle would be left unable to publish or cancel the staged release.
Note that a client that sets ``staged=true`` without understanding the response may record it (for example in
public CI logs); the ``session-token`` it contains grants only read-only :ref:`preview <staged-preview>`
access, but indexes and publishers **SHOULD** be aware that it can be exposed this way. An index **MAY** also create a session for an upload based on
its own policy or the project's configuration, without the field being present; this allows a project to
require that its releases are staged in a way that an upload client cannot bypass. In this policy driven
case no client change is required at all; the ``staged`` field lets a publisher opt in per upload with only a
minimal change to existing tooling.

Because the legacy API uploads a single file per request, subsequent legacy uploads for the same project and
Comment thread
cjames23 marked this conversation as resolved.
version **SHOULD** be added to the same open session when that session was itself created through a legacy
upload, so that the release is still published as a unit. Once such a session exists, a subsequent legacy
upload for that project and version is added to it whether or not it carries the ``staged`` field: the field
is not required again, and the index **MUST NOT** reject the upload for omitting it. The field is still
permitted on these uploads as an explicit statement of intent.

A session's creation path is fixed when it is created, and the two paths **MUST NOT** be mixed.
If a non-terminal session already exists for a name-version pair (see :ref:`publishing-session-multiple`), a request to
contribute to it through the *other* path **MUST** be rejected with a ``409 Conflict``: any legacy upload for
a pair that already has an open session created through the Upload 2.0 API is rejected (whether or not it
carries the ``staged`` field), and an Upload 2.0 request that would add to a session created through a legacy
upload is likewise rejected.

Fixing the path at creation avoids ambiguity in how the session reaches a terminal state. An Upload 2.0
session is driven to completion by a client that holds its ``session-token`` and calls the control plane
endpoints, whereas a legacy created session is completed by continued legacy uploads and by the index acting
on the publisher's behalf. Allowing a legacy ``staged=true`` upload to join an existing Upload
2.0 session would mean issuing a ``session-token`` for, and granting completion authority over, a session the
legacy uploader did not create; allowing an Upload 2.0 request to add to a legacy created session would
subject a legacy publisher to a completion model its tooling does not implement. Rejecting cross path
contribution keeps the authority to publish or cancel a session with the path that created it, which also
keeps any :ref:`separate publishing authorization <authentication>` on that session unambiguous.

A legacy client that is unaware of this PEP cannot issue a :ref:`publish request
<publishing-session-completion>`. Where an index has created a session on such a client's behalf, and the
session is subject only to automated processing, the index **MAY** publish the session itself once that

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the scenario where an index uses sessions by policy of the index/project, it seems this leaves clients who do not support the new protocol in bit of a lurch, unable to control when their staged files enter the processing state which may be before they want, or long after.

I assume there is some desire for this to eventually be the default on PyPI to provide a mechanism for pre-publication malware scanning, so I think it is a bit unwieldy to not have a story around how we will handle users of legacy clients who may never adopt explicit session support.

If I'm honest, I would prefer that we move very quickly to rollout Upload 2.0, and then very quickly to deprecate the legacy endpoint, as active uploaders are the some of the most easily "reached" clients we have. When their publication flows break, they are much more likely to be actively ready to repair them.

I'm a bit concerned that having the interop period, with itself potentially being divided between "opt-in" and "by policy" staging, creating more than one distinct migration/adoption period.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to agree I would definitely like the rollout of Upload 2.0 and deprecation of the legacy endpoint to be quick in succession. I think otherwise places a lot of burden on maintenance and as you stated it can create more than one adoption period. And speaking from JOB1 experience the longer that a migration is allowed to take the longer that the long tail becomes.

processing resolves without an adverse result, or once the period allowed for it elapses (see
:ref:`publishing-session-completion`). An index **MUST NOT** publish a session this way if the publisher has
configured the project to require a separate publishing authorization (see :ref:`authentication`).


.. _managing-session:

Managing a Publishing Session
-----------------------------

The endpoints in this section apply to a publishing session regardless of how it was created (see
:ref:`session-creation`).

.. _publishing-session-states:

Publishing Session States
Expand Down Expand Up @@ -743,6 +840,42 @@ change: deferred processing resolves to either ``published`` on success or ``err
resolves to ``error``, the session remains editable and the reason is reported in the session's ``notices``,
as described in :ref:`publishing-session-states`.

Reviewing an upload can produce two kinds of result, surfaced at different points. Correctness problems that
can be decided about a single file in isolation, such as its size, checksum, metadata, or compatibility tags,
are surfaced when that :ref:`file upload session <file-upload-session-states>` completes, moving the file to
the ``error`` state, from which it cannot be repaired in place. Results that concern whether the release may
become public, such as malware detection, instead gate the publish step and are described below. An index
**MAY** run malware review per file, as each file is uploaded, as well as (or instead of) at publication; a
file remains uploadable to a stage for some time before the release is published, so reviewing each file as it
arrives can shorten the window in which an unreviewed file sits in a stage.

The ``processing`` state **MAY** be used to run asynchronous review of a session's files before it is

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the intention that an index may perform similar asynchronous review per file as well or do we only expect such review to occur once a user requests publication of the entire release?

The processing state for file uploads (initiated by the caller submitting to the completion endpoint) was kind of where I had this processing occurring in my head. With more thought towards this PR, I think that there's a good argument for per-file and per-release processing, although I think I ultimately like gating on the results being part of the publication step.

An index may have an adverse result based on reviews around malware for a given file that is only exposed once publication is attempted, but any processing as it relates to correctness (file size, checksum, metadata, compatibility, etc...) is probably best raised immediately in the file session and treated as non-recoverable as it is currently written.

This is mostly me talking it out, but the question I think I'm raising now is:

Do we want to have a similar callout for what kinds of results are raised in the file-upload-session completion versus the release publication step?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good call out, I should definitely add something around what kinds of result are raised in the different steps.

I think in terms of the malware types of scans those definitely have to be per file just given that iirc there is a 14 day window for file uploads to a given version as of now. Since the primary goal of this type of scanning is to eliminate as many supply chain attacks as possible.

published, such as malware scanning. If the review completes without an adverse result, the session resolves
to ``published`` as normal. If the review does not complete within a period chosen by the index, the server
**MAY** treat it as though it had completed without an adverse result and publish the session, so that a
backlogged or unavailable review system does not indefinitely prevent publication. If the review produces an
adverse result, the session resolves to ``error`` with the reason reported in the session's ``notices``; the
server **MAY** decline to publish such a session on any subsequent retry, in which case it is eventually
Comment thread
cjames23 marked this conversation as resolved.
:ref:`canceled <publishing-session-cancellation>` and its data discarded. Where a session has resolved to
``error`` because of an adverse result, an index **MAY** provide a way for the publisher to request that the
result be re-examined. Such a re-examination **SHOULD** be performed by a human, to confirm whether the
adverse result was a false positive; if it was, the index **SHOULD** allow the session to be published. The
review itself, including which checks run, how long they are allowed to take, and how such requests are
handled, is determined by the index operator and is out of scope for this specification.

Because a session remains editable in the ``open`` and ``error`` states, its files can change after a review
has run. A server **MUST NOT** publish a session on the strength of a review of content the session no longer
contains: if any file is added, replaced, or deleted after a review, the prior result is invalidated and the
session **MUST** be reviewed again before it can be published. How an in-progress review reacts to such a
change is left to the index: it might run to completion, rescan only the changed files, or restart.

Because editing a session can trigger re-review, an index **SHOULD** guard against abuse of the review system.
Comment thread
cjames23 marked this conversation as resolved.
For example, a client might repeatedly upload a file it knows will be flagged, delete it, and re-upload it to
force repeated scans. Mitigations such as rate-limiting reviews, capping the number of review attempts for a
session, or moving a persistently adverse session to a terminal state are left to the index. An index can
also avoid this class of abuse entirely by performing the review only at publication time, when the session's
files are fixed for the duration of the publish and cannot be swapped out to force another scan.

A publish attempt that fails *synchronously* (i.e. within the publish request itself) is returned to the
client as an :ref:`error response <session-errors>` and leaves the session in its current editable state; it
does **not** move the session to ``error``.
Expand Down Expand Up @@ -1847,6 +1980,22 @@ as experience is gained operating Upload 2.0.
Change History
==============

* `01-Oct-2026 <https://discuss.python.org/t/pre-pep-staged-releases-separated-from-pep-694/107804/58>`__

* Add :ref:`Legacy Upload API Interoperability <legacy-interop>`, allowing a legacy upload to create a
publishing session (via a ``staged`` field, or by index or project configuration that an upload client
cannot bypass) so that staging is usable before upload tooling adopts this API, and allowing an index to
publish such a session itself once automated processing resolves or its window elapses. Require that an
index supporting this accept the ``staged`` field and return the session creation response body, specify
that subsequent legacy uploads to an open stage need not repeat the field, and forbid mixing the two
session-creation paths.
* Note that the ``processing`` state **MAY** be used for asynchronous review such as malware scanning, with
an index-chosen window after which the review is treated as having produced no adverse result. Distinguish
per-file correctness results from release-level review, allow review per file or at publication, and note
that reviewing at publication time avoids re-scan abuse.
* Allow an index to require additional authorization, beyond upload permission, to publish or cancel a
session, so that duties can be separated, optionally exposing that choice to project owners.

* `29-Jul-2026 <https://discuss.python.org/t/pep-694-pypi-upload-api-2-0-round-4/108320>`__

* Add an **Atomic Publication and Conflicts** section. Specify that publication is atomic with respect to
Expand Down
Loading