Instructions for aiohttp admins

This page is intended to document certain processes for admins of the aiohttp repository. For regular contributors, return to Contributing.

Running reproducer code

Warning

When evaluating a bug report or vulnerability report, treat reproducer code as untrusted. If you don’t understand what it does or are unfamiliar with a library it imports do not run it (and ask the reporter to provide a simpler reproducer). We also recommend that any reproducers you do run are executed in a container.

Creating a new release

Note

The example commands assume that origin refers to the aio-libs repository.

To create a new release:

  1. Start on the branch for the release you are planning (e.g. 3.8 for v3.8.6): git checkout 3.8 && git pull

  2. Update the version number in __init__.py.

  3. Run towncrier.

  4. Check and cleanup the changes in CHANGES.rst.

  5. Checkout a new branch: e.g. git checkout -b release/v3.8.6

  6. Commit and create a PR. Verify the changelog and release notes look good on Read the Docs. Once PR is merged, continue.

  7. Go back to the release branch: e.g. git checkout 3.8 && git pull

  8. Add a tag: e.g. git tag -a v3.8.6 -m 'Release 3.8.6' -s

  9. Push the tag: e.g. git push origin v3.8.6

  10. Monitor CI to ensure release process completes without errors.

Once released, we need to complete some cleanup steps (no further steps are needed for non-stable releases though). If doing a patch release, we need to do the below steps twice, first merge into the newer release branch (e.g. 3.8 into 3.9) and then to master (e.g. 3.9 into master). If a new minor release, then just merge to master.

  1. Switch to target branch: e.g. git checkout 3.9 && git pull

  2. Start a merge: e.g. git merge 3.8 --no-commit --no-ff --gpg-sign

  3. Carefully review the changes and revert anything that should not be included (most things outside the changelog). Use git checkout 3.9 path/to/file to revert files without aborting the merge.

  4. To ensure change fragments are cleaned up properly, run: python tools/cleanup_changes.py

  5. Complete the merge: git merge –continue.

  6. Push the branch directly to Github (because a PR would get squashed). When pushing, you may get a rejected message. Follow these steps to resolve:

  1. Checkout to a new branch and push: e.g. git checkout -b do-not-merge && git push

  2. Open a draft PR with a title of ‘DO NOT MERGE’.

  3. Once the CI has completed on that branch, you should be able to switch back and push the target branch (as tests have passed on the merge commit now).

  4. This should automatically consider the PR merged and delete the temporary branch.

Back on the original release branch, bump the version number and append .dev0 in __init__.py.

Post the release announcement to social media:

If doing a minor release:

  1. Create a new release branch for future features to go to: e.g. git checkout -b 3.10 3.9 && git push

  2. Update both target-branch backports for Dependabot to reference the new branch name in .github/dependabot.yml.

  3. Delete the older backport label (e.g. backport-3.8): https://github.com/aio-libs/aiohttp/labels

  4. Add a new backport label (e.g. backport-3.10).

Incident response

This section covers responding to a reported security vulnerability and to three classes of compromise – of the supply chain, of a maintainer account, and of the project infrastructure. It picks up after a report or a problem is in hand.

Note

Vulnerability reports arrive through the organization’s SECURITY.md – GitHub private vulnerability reporting, or email to the security coordinators. Never triage a suspected vulnerability in a public issue or pull request.

The security coordinator who first triages a report is its incident lead, and may explicitly hand off to another coordinator. The lead owns the GitHub Security Advisory (GHSA) draft, fix coordination, the release, and notification.

Severity tiers

aiohttp is volunteer-maintained, so there is no response-time commitment. The tier guides prioritization, the release decision, and how widely the fix is announced. All security fixes – regardless of tier – land on master and are backported to the currently supported x.y branch (and the next x.y branch, if one is in development).

High

A concrete remote impact an attacker can actually achieve against a default deployment. Examples: a single request that stops the server from handling further requests (a server-wide denial of service), reading files on the server outside the project root, remote code execution, authentication bypass. Cut a dedicated security release once the fix lands.

Medium

Bounded impact, or impact that needs a non-default option, an unusual configuration, or a local position. Examples: request smuggling demonstrated to cause a real issue against a common proxy, or a DoS that use significant server resources with a low effort sustained attack. Cut a dedicated security release.

Low

Limited or hard-to-exploit impact, or impact confined to debug or non-default paths. Examples: minor version or error-message disclosure, parser leniency with no demonstrated security impact, or request smuggling that has not been demonstrated to cause a real issue against a common proxy. May ride the next routine release rather than cutting a dedicated security release.

Severity is rated by demonstrated attacker impact. A denial of service that disables further request handling for the whole server from a single request leans High; one that requires sustained low-effort traffic to consume significant server resources leans Medium. Request smuggling and parser bugs are rated by what an attacker can actually achieve against a common proxy or in a realistic deployment, not by the shape of the bug.

Responding to a reported vulnerability

  1. Acknowledge and confirm. Reply to the reporter in the GitHub private report or the email thread. Reproduce the issue (but don’t run untrusted code) against master and the maintained x.y branches. If it is not reproducible or is out of scope, record the decision and close the report.

  2. Open the GHSA draft. For an email report, if the reporter can’t use Github for some reason, then open a new advisory to track the issue.

  3. Develop the fix privately in the GHSA’s temporary private fork (“Start a temporary private fork” in the draft) – never on a public branch or in a public pull request. Include a regression test unless it would provide too much information for an attacker to replicate.

  4. Title the advisory with the vulnerability class and affected component.

  5. Write the description using the GHSA description template:

    ### Summary

<one or two sentences describing the vulnerability>

### Impact

<who is affected, prerequisites, and what an attacker achieves>

### Workaround

<if relevant, any workarounds for users unable to upgrade>

---

Patch: <link to commit fixing the issue on the stable x.y branch>

    An important detail that Github will use to decide the CVE severity is exploit maturity. Try to clarify the likelihood of a vulnerability being attacked (e.g. if exploit code is available, or active attacks in-the-wild already exist).

  6. Set the affected and fixed versions.

  7. Assign a severity tier (see Severity tiers).

  8. Credit the reporter (if report came by email) in the GHSA Credits field.

  9. Credit the developer using “Remediation developer” if different from Reporter.

  10. Request a CVE by clicking the button in the advisory.

  11. Get the fix reviewed in the private fork by at least one other maintainer.

  12. Consider early notify for high severity issues (see Notifying about a disclosed vulnerability).

  13. Coordinate a release.

    1. Agree the timing. For an embargoed high-severity incident, set the release date to align with the lift of any private-list embargo.

    2. Merge all the private forks and create and merge the backports for each.

    3. Create the release. Follow Creating a new release.

  14. Update patch link in GHSA description.

  15. Publish the GHSA usually around 1 day after release.

  16. Notify according to the severity tier (see Notifying about a disclosed vulnerability).

  17. Run the post-incident steps (see Post-incident).

Notifying about a disclosed vulnerability

Notification is cumulative – each tier adds to the ones below it. Mind the timing: the linux-distros pre-notification happens under embargo, before the release, while everything else happens at or after the release.

Baseline (all severities)

The published GHSA (which feeds the CVE and the GitHub Advisory Database).

Higher (Medium-High)

For most high severity issues, additionally post to the public oss-security mailing list once the advisory is public.

Highest (High-Critical)

For the most serious issues, additionally before the public release, send an embargoed pre-notification to the private distribution-security lists – primarily linux-distros – and attempt to notify affected downstreams. Use the vuln_search.py dependents-enumeration script to search aiohttp’s GitHub dependents; it needs a GitHub token and may need to be run over a few days, so start it early.

Keep the embargo as short as practical. Typically an embargo of 1 or 2 days is expected.

Template for the embargoed linux-distros pre-notification:

To: linux-distros@vs.openwall.org
Subject: aiohttp: <one-line summary of the vulnerability>

We are coordinating disclosure of a security vulnerability in aiohttp
(https://github.com/aio-libs/aiohttp), the asyncio HTTP client/server
library.

Summary
  <2-3 sentences: the flaw and its impact>

Affected versions
  <e.g. aiohttp < 3.12.15 (all 3.11.x and 3.12.x)>

CVE
  <CVE-XXXX-XXXXX, or "requested via GitHub Security Advisory, ID pending">

Fix
  <link to the patch, or attach the diff>

Proposed public disclosure date
  <YYYY-MM-DD> -- as short as practical. On that date we will publish the
  GitHub Security Advisory, ship a patched PyPI release, and post to
  oss-security.

This issue is not yet public; please observe the embargo until that date.

Contact: <incident lead name and security email>

Template for the public oss-security disclosure:

To: oss-security@lists.openwall.com
Subject: CVE-XXXX-XXXXX: aiohttp <one-line summary>

A security vulnerability has been fixed in aiohttp, the asyncio HTTP
client/server library (https://github.com/aio-libs/aiohttp).

CVE: CVE-XXXX-XXXXX
Advisory: <GHSA URL>

Affected versions: <...>
Fixed versions:    <...>

Description
  <what the flaw is>

Impact
  <what an attacker can achieve, and any prerequisites>

Mitigation
  <workaround, or "upgrade to <version>">

Post-incident

  1. Update THREAT_MODEL.md. Have an AI coding agent fetch the new advisories and update the document. Review the changes.

  2. File follow-up hardening as ordinary public issues or pull requests.

Supply-chain or release compromise

A malicious or tampered release on PyPI, or a compromised publish pipeline.

  1. Pull the bad artifact. On PyPI, yank the affected release – this hides it from new resolves while keeping existing pins working. Delete it only if it is actively malicious and removal is the safer choice. This needs a PyPI project owner or maintainer account.

  2. Verify the legitimate artifacts with Sigstore. Every genuine release is signed by the release pipeline, with the .sigstore bundles attached to the GitHub Release. Verify the sdist and wheels; a divergence from what is on PyPI localizes the tampering.

  3. Lock the publish path. Publishing uses a PyPI OIDC trusted publisher, so there is no long-lived token to rotate. Instead, on PyPI temporarily remove the trusted-publisher binding, and on GitHub restrict or pause the pypi deployment environment used by the release job.

  4. Audit the release inputs. Review recent changes to the release workflow, its tag trigger, and the third-party action versions it uses.

  5. Audit the committed Cython sources. The generated .c files ship in the sdist and are not checked against their .pyx sources in CI (see THREAT_MODEL.md section 5.19). Regenerate them with make cythonize and git diff against the released revision. Re-verify the vendor/llhttp pin and package-lock.json.

  6. Reissue a clean release via Creating a new release once the cause is fixed. Never re-publish over a yanked version number.

  7. Notify per severity – a malicious published release is High.

Maintainer account compromise

A GitHub or PyPI account takeover, or leaked publishing or signing material.

  1. Suspend the account. Another GitHub organization owner can suspend it, revoke its repository access, and start the audit-log review. If the compromised account is the sole organization owner, or no other owner is reachable, contact GitHub Support at https://support.github.com/contact for emergency account lock down and organization recovery.

  2. Kill sessions and credentials on the affected account. On GitHub: change the password, sign out all sessions, revoke every personal access token, OAuth app, and SSH or GPG key, then re-enroll two-factor authentication. On PyPI: change the password, re-enroll two-factor authentication, and revoke API tokens.

  3. Audit PyPI project ownership. OIDC publishing means there is usually no long-lived PyPI token to leak, but a compromised project owner can add a trusted publisher or upload directly. Review the project collaborators and trusted-publisher bindings, and remove anything unrecognized.

  4. Audit what the account could have done from the GitHub audit log: pushes, tag creation, branch-protection or settings changes, new secrets or deploy keys, new collaborators, and workflow edits.

  5. Revert and re-verify. Force-revert any unauthorized commits or tags, and re-verify recent releases against their Sigstore bundles. If a malicious release shipped, escalate to Supply-chain or release compromise.

  6. Handle signing material. Sigstore signing is key-less, so there is no static signing key to rotate. Any separate GPG key used for signed tags must be treated as compromised, then revoked and rotated.

CI or infrastructure compromise

A compromise of GitHub Actions, repository settings, or branch protections.

  1. Freeze the release path. Pause the pypi deployment environment and stop tag-triggered deploys until the compromise is scoped.

  2. Review branch protection and required reviews on master and the x.y branches. These are enforced on GitHub and are not visible in the repository; confirm nothing was relaxed and restore the known-good configuration.

  3. Audit Actions secrets and environments – the repository and organization secrets, the pypi environment’s protection rules and reviewers, and any new deploy keys. Confirm the workflow keeps empty top-level permissions and that id-token: write is scoped to the deploy job only.

  4. Diff recently changed workflows with git log -- .github/workflows/, especially the release workflow, the auto-merge workflow (which runs in the privileged pull_request_target context), and the CodeQL workflow. Look for added steps, changed action references, or new triggers.

  5. Check the committed Cython sources as in Supply-chain or release compromise – regenerate them and git diff.

  6. Re-pin and rebuild. If a third-party action was compromised, pin the affected actions by full commit hash, re-run CI from a clean known-good revision, and re-verify with Sigstore any release made during the suspected window.

  7. Escalate to Supply-chain or release compromise if a release shipped through the compromised pipeline.