fix: add per-instance spec overrides to handle astropy setuptools conflict#593
Open
Oxygen56 wants to merge 1 commit into
Open
fix: add per-instance spec overrides to handle astropy setuptools conflict#593Oxygen56 wants to merge 1 commit into
Oxygen56 wants to merge 1 commit into
Conversation
Introduce a MAP_INSTANCE_TO_SPECS dictionary that allows individual SWE-bench instances to override specific fields in their repo-version-level spec without affecting other instances of the same repo/version. The instance overrides are shallow-merged on top of the base spec in make_test_spec(), so only the keys that need changing need to be provided. Use this mechanism to fix astropy__astropy-8872: the instance uses a cached environment.yml that pins setuptools==68.0.0, whose vendored distutils emits a DeprecationWarning for LooseVersion usage. astropy 3.1's pytest config escalates warnings to errors, causing test collection to abort. Adding a pre_install step that runs `pip install 'setuptools<60'` downgrades setuptools before astropy is installed, bypassing the issue.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #584 — the gold patch for
astropy__astropy-8872cannot be scored becausesetuptools==68.0.0(from the repo-version spec) ships a vendoreddistutilswhoseLooseVersionemits aDeprecationWarning. Astropy 3.1's pytest config escalates warnings to errors, causing test collection to abort before any test runs.Changes
New
MAP_INSTANCE_TO_SPECSdictionary inswebench/harness/constants/__init__.py— a per-instance override layer that is shallow-merged on top of the repo-version-level spec inmake_test_spec(). Only the keys that need changing need to be provided.Override logic in
make_test_spec()— after fetchingspecsfromMAP_REPO_VERSION_TO_SPECS, checksMAP_INSTANCE_TO_SPECSfor the currentinstance_idand applies any overrides via{**specs, **instance_overrides}.Instance override for
astropy__astropy-8872— adds apre_installstep that runspython -m pip install 'setuptools<60'to downgrade setuptools before astropy is installed. This uses the stdlibdistutils(which does not emit theLooseVersionwarning on Python 3.9) instead of the vendored copy.Testing
tests/test_harness_utils.py) pass.make_test_specproduces a repo install script containing thesetuptools<60pin forastropy__astropy-8872.setuptools==68.0.0as before.