This is the main tracking issue for the move to Meson as our new build system, until we've made it the default in pyproject.toml and things are looking good. For context:

• distutils is removed from the stdlib in Python 3.12, and numpy.distutils is deprecated because of that. For details, see PEP 632, deprecation of distutils and numpy.distutils #18588
• What we need from a build system and why Meson: basically the same as for SciPy, see RFC: switch to Meson as a build system scipy/scipy#13615

Planning work and integration

This was initially written up in gh-22468:

• We use a meson branch in this repo: https://github.com/numpy/numpy/tree/meson
• CI:
  • we ensure regular CI does not run on meson, by removing regular CI jobs from the meson branch in order to not trigger a lot of unnecessary CI runs
  • we start with one new Linux CI job in the meson branch, and deal with other platforms and configs once things have started working on Linux
• Branch rules:
  • for simple changes where the author is fairly sure that the changes are correct, pushing directly to the meson branch is fine,
  • for more complex changes, let's work via PRs to ensure that we get at least cursory review early on
  • since it's a branch in the main repo, merging main into meson is preferred over rebases. @stefanv, @mattip and @rgommers will align in case the meson branch needs rebasing or other history rewriting, to ensure that we don't lose anyone's work.
  • we should avoid making code changes other than build/CI/docs in the meson branch; if we need fixes in NumPy itself, we should open separate PRs for those, get them merged quickly, and then merge main into meson
• Once we are ready to merge initial Meson support into main, we will make a careful PR with granular commits and co-authorship for folks who contributed to the code in that commit.
• For tracking the work, we will have:
  • This tracking issue
  • A project board with status per topic and assigned dev: Move build to Meson.
  • Documentation on how to test/use the build will live in the meson branch itself.
  • Writing a NEP may be useful at some point, but I'd like to postpone that.

Here is the project board: Move build to Meson (view). Self-assigning topic there is useful if you plan to work on something. Please feel free to add new tasks if you're clear on them being needed. Otherwise please feel free to discuss on this issue or on the #meson channel in Slack.

Timeline

This is probably 3-6 months of work. A key topic will be SIMD support, because pretty much everything else is known to work - it's already done in SciPy. Whatever is not done there is tracked in scipy/scipy#16293. The most important topic there is BLAS/LAPACK support; as of right now that works, but isn't as user-friendly as with numpy.distutils - more automatic detection is needed, as well as user-level ways to do things like express a preferred ordering of BLAS libraries.

The Meson 0.65.0 release will come out in ~3 months (quarterly release cycle, and 0.64.0 came out today). We should aim to get some of the things we need into that release.

We typically start testing a new CPython development version ~6 months before it is released. For Python 3.12 that would be Mar-Apr 2023.