Skip to content

Commit 7e1bc1e

Browse files
AlexWaygoodAlexWaygood
authored
Add agent skills for working on ty (#25422)
## Summary This PR adds a `.agents/skills` directory to the repo, and adds three skills to help agents work on ty more effectively: - A skill to help them write better ty diagnostics - A skill to help them minimize a single ecosystem change more effectively - A skill to help them analyze and summarise ecosystem reports more effectively The skills have been designed so that they will (hopefully) work well whether a contributor is using Codex or Claude as their agent locally. Codex should discover these skills automatically if you're working on the repo; if you use Claude locally, you _should_ hopefully be prompted by Claude to install the local skills due to them being listed as recommended skills in `.claude/settings.json`. ## Test Plan I ran `/review` locally with Codex, instructing it to be as nitpicky as possible, until it could find no more issues with these skills.
1 parent 574e107 commit 7e1bc1e

9 files changed

Lines changed: 184 additions & 14 deletions

File tree

‎.agents/.claude-plugin/marketplace.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "ruff-agent-skills",
3+
"owner": {
4+
"name": "Ruff contributors"
5+
},
6+
"description": "Local Claude Code skills for working on ty in the Ruff repository",
7+
"plugins": [
8+
{
9+
"name": "ty-skills",
10+
"source": "./",
11+
"description": "Local skills for ty diagnostics and ecosystem workflows"
12+
}
13+
]
14+
}

‎.agents/.claude-plugin/plugin.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "ty-skills",
3+
"description": "Local skills for working on ty in the Ruff repository"
4+
}

‎.agents/skills/adding-ty-diagnostics/SKILL.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
name: adding-ty-diagnostics
3+
description: Use when adding, updating, or reviewing ty checks, diagnostics, diagnostic messages, subdiagnostics, or concise output behavior.
4+
---
5+
6+
# Adding Ty Diagnostics
7+
8+
Use this skill when adding or changing a ty diagnostic, especially as part of a new ty check.
9+
10+
**Keep error messages concise.** Think about how the diagnostic will look on a narrow terminal screen.
11+
12+
Put extra detail in subdiagnostics or secondary annotations when that helps, but make sure the primary diagnostic is understandable on its own.
13+
14+
Always check that the diagnostic still makes sense when the user passes `--output-format=concise`.
15+
16+
If the error code is entirely new or if you have changed the documentation for the error code,
17+
you will need to run `cargo dev generate-all` after making your changes, to update the generated schema for ty
18+
and the generated `.md` documentation files.
19+
20+
Diagnostics should usually be tested using mdtests. If you are changing behaviour for an existing diagnostic,
21+
you should usually add your tests to a pre-existing `.md` file; otherwise, it may be appropriate to add a new
22+
`.md` file for your tests. Snapshot tests are only usually necessary for diagnostics that use secondary annotations
23+
or subdiagnostics. If you want to add a snapshot, inline `# snapshot` comments are preferred over the legacy
24+
`<!-- snapshot-annotations -->` directive.
25+
26+
When using the `declare_lint!` macro, the `status` field should be set to `LintStatus::stable(<next version of ty>)`.
27+
You should determine what the next version of ty will be by inspecting https://pypi.org/pypi/ty/json, finding what
28+
the latest release of ty is, and incrementing the patch version by one. For example, if the latest release of ty is `0.5.3`, the status should be `LintStatus::stable("0.5.4")`.

‎.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
name: minimizing-ty-ecosystem-changes
3+
description: Use when reproducing, investigating, or minimizing behavior changes in ty ecosystem or primer projects.
4+
---
5+
6+
# Minimizing Ty Ecosystem Changes
7+
8+
Use this skill when asked to reproduce a ty ecosystem change, investigate a behavior difference in a primer project, or minimize a reproducer from a ty ecosystem project.
9+
10+
## Reproduce First
11+
12+
From the repository root, clone the project and install its dependencies into `.venv`:
13+
14+
```sh
15+
uv run scripts/setup_primer_project.py <project-name> <some-temp-dir>
16+
```
17+
18+
Confirm the behavior difference reproduces before minimizing or explaining it.
19+
20+
## Minimize
21+
22+
When asked to minimize an ecosystem change, start from the reproduced project. Reduce the Python code until only the smallest code needed to demonstrate the behavior difference remains. Even if the cause looks obvious, **do not skip ahead** to an explanation or a hand-written reproducer. Use a rigorous, systematic process and verify after each reduction that the behavior difference still reproduces.
23+
24+
An ideal minimized reproducer:
25+
26+
- Is a single file that is as small as possible.
27+
- Has as few third-party imports as possible, ideally none.
28+
- Uses smaller third-party packages with fewer dependencies when third-party imports are unavoidable. For example, prefer an import from `numpy` over one from `pandas`.
29+
- Has as few first-party and standard-library imports as possible.
30+
- Keeps imports from modules with highly special-cased symbols only when they are necessary (such modules may include `typing`, `abc`, `enum`, `types`, or `typing_extensions`).
31+
- Uses an absolute minimum of "advanced"/complex typing or language features. For example, if the original code
32+
uses a walrus operator, but the behavior difference still reproduces without it, remove the walrus operator. If the original code uses a `Protocol` from `typing_extensions`, but the behavior difference still reproduces without it, remove the `Protocol`.
33+
34+
Use a systematic loop. You do not need to apply every tool in every iteration, but keep looping until none of the available minimization tools can reduce the reproducer further while preserving the behavior difference.
35+
36+
Available minimization tools include:
37+
38+
1. Remove files that are not needed for the difference.
39+
2. Strip imports, definitions, and statements from the remaining files.
40+
3. Cut and paste definitions from one file into another file to reduce first-party or third-party imports.
41+
4. Inline the relevant parts of third-party dependencies into first-party code to reduce third-party imports.
42+
5. Inline the relevant parts of stdlib definitions from typeshed into first-party code to reduce stdlib imports.
43+
44+
After applying a minimization tool, re-run the comparison between the feature branch and `main`.
45+
Keep a reduction only when the behavior difference still reproduces.
46+
47+
Stop looping only when no further reductions could be applied without making the behavior difference disappear.

‎.agents/skills/summarise-ecosystem-results/SKILL.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
name: summarise-ecosystem-results
3+
description: Use when asked to summarise or summarize ty ecosystem results for a Ruff PR, including from a PR number, PR URL, GitHub ecosystem-results comment, or full detailed HTML report.
4+
---
5+
6+
# Summarise Ecosystem Results
7+
8+
Use this skill when asked to summarise ecosystem results for a Ruff PR with the `ty` label.
9+
10+
## Find the Report
11+
12+
Accept any of these inputs:
13+
14+
- A PR number.
15+
- A GitHub PR URL.
16+
- A GitHub comment URL on a PR, such as `https://github.com/astral-sh/ruff/pull/25342#issuecomment-4525002693`.
17+
- A full detailed HTML ecosystem report URL.
18+
19+
Determine the PR number first. If the user gave only a PR number, open `https://github.com/astral-sh/ruff/pull/<number>`.
20+
21+
Find the ty ecosystem-results comment on the PR. Search PR comments for terms such as "ecosystem", "full report", "HTML report", and "detailed report". From that comment, open the linked full detailed HTML report.
22+
23+
Use the PR comment as the change list and the full detailed HTML report as the source of detailed evidence. When the report includes exact project revisions, use those revisions rather than current upstream checkouts.
24+
25+
## Minimize in Parallel
26+
27+
Before minimizing, load and apply the `minimizing-ty-ecosystem-changes` skill to each ecosystem change.
28+
29+
If possible, use subagents to parallelize this work. Decide how to batch changes so the overall task finishes as quickly as possible while still allowing each subagent to work methodically. Reasonable batching strategies include grouping related changes by project, diagnostic code, suspected cause, or report section, while keeping large groups split enough to avoid one slow subagent blocking the whole task.
30+
31+
If subagents are not available, batch the minimization work manually and minimize the batches sequentially. Keep batches small enough that each pass can still be checked carefully.
32+
33+
Give each subagent a self-contained assignment:
34+
35+
- The PR number, PR URL, ecosystem comment URL, and detailed HTML report URL.
36+
- The exact ecosystem changes assigned to that subagent.
37+
- The requirement to use the `minimizing-ty-ecosystem-changes` process rigorously.
38+
- The expected Markdown output format for each minimized change.
39+
40+
Each subagent should proceed methodically through all assigned changes. If a subagent moves on to a new change and that change appears very similar to one it has already minimized, it may skip the new change, but it must record why the skipped change appears to demonstrate the same behavior.
41+
42+
## Collect Results
43+
44+
After all subagents finish, collect their minimizations into one Markdown file at the repository root:
45+
46+
```text
47+
PR_<number>_ECOSYSTEM_SUMMARY.md
48+
```
49+
50+
Remove minimizations that appear to demonstrate the same behavior change. Prefer the smallest and clearest minimized reproducer, especially one that is single-file and has fewer imports.
51+
52+
At the top of the file, add prose summarising the distinct behavior changes demonstrated by the retained minimizations. Then include the retained minimizations with enough detail for a reader to understand and reproduce them.
53+
54+
For each retained minimization, include:
55+
56+
- The original project and report entry.
57+
- The diagnostic or behavior change on `main` versus the PR.
58+
- The minimized code.
59+
- The commands or comparison method used to verify the minimization.
60+
- Any source links needed to trace the example back to the PR comment or full report.
61+
62+
Present `PR_<number>_ECOSYSTEM_SUMMARY.md` as the finished product.

‎.claude/settings.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,15 @@
11
{
2+
"enabledPlugins": {
3+
"ty-skills@ruff-agent-skills": true
4+
},
5+
"extraKnownMarketplaces": {
6+
"ruff-agent-skills": {
7+
"source": {
8+
"source": "directory",
9+
"path": ".agents"
10+
}
11+
}
12+
},
213
"hooks": {
314
"SessionStart": [
415
{

‎.pre-commit-config.yaml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,8 @@ repos:
8484
- mdformat-footnote==0.1.3
8585
exclude: |
8686
(?x)^(
87-
docs/formatter/black\.md
87+
\.agents/skills/.*
88+
| docs/formatter/black\.md
8889
| docs/\w+\.md
8990
)$
9091
priority: 0
@@ -109,7 +110,8 @@ repos:
109110
- id: markdownlint-fix
110111
exclude: |
111112
(?x)^(
112-
docs/formatter/black\.md
113+
\.agents/skills/.*
114+
| docs/formatter/black\.md
113115
| docs/\w+\.md
114116
)$
115117
priority: 1

‎AGENTS.md‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,17 +54,6 @@ Run ty:
5454
cargo run --bin ty -- check path/to/file.py
5555
```
5656

57-
## Reproducing and minimizing ty ecosystem changes
58-
59-
If asked to reproduce changes in the ty ecosystem, use this script to clone the project to some
60-
directory and install its dependencies into `.venv`:
61-
62-
```sh
63-
uv run scripts/setup_primer_project.py <project-name> <some-temp-dir>
64-
```
65-
66-
If asked to *minimize* a change in the ty ecosystem, you should start off with the above command to ensure that the change reproduces. You should then attempt to minimize the Python code required to demonstrate a behaviour difference between ty on your feature branch and ty on the main branch. Your minimization process should consist of systematically removing files from the cloned ecosystem project, and stripping content from existing files, until the behaviour difference between your branch and `main` no longer reproduces.
67-
6857
## Pull Requests
6958

7059
When working on ty, PR titles should start with `[ty]` and be tagged with the `ty` GitHub label.
@@ -83,7 +72,6 @@ When working on ty, PR titles should start with `[ty]` and be tagged with the `t
8372
- Prefer let chains (`if let` combined with `&&`) over nested `if let` statements to reduce indentation and improve readability. At the end of a task, always check your work to see if you missed opportunities to use `let` chains.
8473
- If you *have* to suppress a Clippy lint, prefer to use `#[expect()]` over `[allow()]`, where possible. But if a lint is complaining about unused/dead code, it's usually best to just delete the unused code.
8574
- Use comments purposefully. Don't use comments to narrate code, but do use them to explain invariants and why something unusual was done a particular way.
86-
- When adding new ty checks, it's important to make error messages concise. Think about how an error message would look on a narrow terminal screen. Sometimes more detail can be provided in subdiagnostics or secondary annotations, but it's also important to make sure that the diagnostic is understandable if the user has passed `--output-format=concise`.
8775
- **Salsa incrementality (ty):** Any method that accesses `.node()` must be `#[salsa::tracked]`, or it will break incrementality. Prefer higher-level semantic APIs over raw AST access.
8876
- Run `cargo dev generate-all` after changing configuration options, CLI arguments, lint rules, or environment variable definitions, as these changes require regeneration of schemas, docs, and CLI references.
8977
- Don't prefix tests with `test_`.

‎CONTRIBUTING.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,20 @@ We **require all use of AI in contributions to follow our
4444

4545
If your contribution does not follow the policy, it will be closed.
4646

47+
### Local skills for Codex or Claude
48+
49+
This repository includes local agent skills for ty development under `.agents/skills`.
50+
51+
Contributors using Codex for development should find that Codex auto-discovers the skills and uses
52+
them automatically when necessary. Claude Code users may be prompted to install the
53+
`ty-skills@ruff-agent-skills` local plugin. If the skills are not installed automatically, install
54+
them manually from the repository root by running these slash commands inside Claude Code:
55+
56+
```text
57+
/plugin marketplace add ./.agents
58+
/plugin install ty-skills@ruff-agent-skills
59+
```
60+
4761
## The Basics
4862

4963
### Prerequisites

0 commit comments

Comments
 (0)