Skip to content

bpo-45798: Move _decimal build setup into configure (GH-29541) #29541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 13, 2021
Merged

bpo-45798: Move _decimal build setup into configure (GH-29541) #29541

merged 2 commits into from
Nov 13, 2021

Conversation

tiran
Copy link
Member

@tiran tiran commented Nov 12, 2021 •
edited by bedevere-bot
Loading

Settings for :mod:decimal internal C extension are now detected by
:program:configure. The bundled libmpdec library is built in
Makefile.

Signed-off-by: Christian Heimes christian@python.org

https://bugs.python.org/issue45798

@tiran tiran force-pushed the libmpdec_configure branch from 81b53bc to ecac9ff Compare November 12, 2021 22:54
@tiran tiran added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 12, 2021
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @tiran for commit ecac9ff9a7b6933faa088261b347bed627ac3a08 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 12, 2021
@tiran tiran changed the title _decimal and libmpdec in configure bpo-45798: Move _decimal build setup into configure Nov 13, 2021
@tiran tiran force-pushed the libmpdec_configure branch from ecac9ff to 69d4720 Compare November 13, 2021 09:20
@tiran tiran marked this pull request as ready for review November 13, 2021 09:20
@tiran tiran force-pushed the libmpdec_configure branch from 69d4720 to 698947a Compare November 13, 2021 10:31
@tiran
bpo-45798: Move _decimal build setup into configure
78dabef 
Settings for :mod:`decimal` internal C extension are now detected by
:program:`configure`. The bundled `libmpdec` library is built in
``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
@tiran tiran force-pushed the libmpdec_configure branch from 698947a to 78dabef Compare November 13, 2021 11:37
mdickinson
mdickinson reviewed Nov 13, 2021
View reviewed changes
Makefile.pre.in
Modules/_decimal/libmpdec/fourstep.o \
Modules/_decimal/libmpdec/io.o \
Modules/_decimal/libmpdec/mpalloc.o \
Modules/_decimal/libmpdec/mpdecimal.o \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that mpsignal.o is missing here; presumably that's because we don't actually make any use of it? Is the omission worth a comment? (bench and bench_full are also missing, but the reason for those is more obvious)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't include it because setup.py's detect_decimal does not include mpsignal.o either. It doesn't hurt to include the object. The linker should drop any unused symbol from the final library anyway.

bench and bench_full are programs with main function. We cannot include them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a build rule and comment for mpsignal.o.

mdickinson
mdickinson approved these changes Nov 13, 2021
View reviewed changes
Copy link
Member

@mdickinson mdickinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me in principle; the code changes look good to my eyes, though I don't think I'm qualified to review the fine details here.

I'm currently testing the build on my home machine (macOS 10.14.6) both with bundled and external libmpdec (in the latter case, using mpdecimal 2.5.1 from MacPorts); I'll report back here if I see any issues.

@mdickinson
Copy link
Member

Some suspicious-looking output from the decimal test suite when using the bundled libmpdec:

test_maxcontext_exact_arith (__main__.CWhitebox) ... python.exe(34434,0x1075815c0) malloc: can't allocate region
*** mach_vm_map(size=842105263157895168) failed (error code=3)
python.exe(34434,0x1075815c0) malloc: *** set a breakpoint in malloc_error_break to debug
python.exe(34434,0x1075815c0) malloc: can't allocate region
*** mach_vm_map(size=842105263157895168) failed (error code=3)
python.exe(34434,0x1075815c0) malloc: *** set a breakpoint in malloc_error_break to debug
python.exe(34434,0x1075815c0) malloc: can't allocate region
*** mach_vm_map(size=421052631578947584) failed (error code=3)
python.exe(34434,0x1075815c0) malloc: *** set a breakpoint in malloc_error_break to debug
python.exe(34434,0x1075815c0) malloc: can't allocate region
*** mach_vm_map(size=421052631578947584) failed (error code=3)
python.exe(34434,0x1075815c0) malloc: *** set a breakpoint in malloc_error_break to debug
ok

I'm also seeing a number of tests skipped with:

skipped 'test requires build with -DEXTRA_FUNCTIONALITY'

@mdickinson
Copy link
Member

Some suspicious-looking output [...]

Scratch that. It also appears on the main branch, and there's an explicit notice at the top of the test suite output telling us to disregard. It shouldn't really be surprising that an attempt to allocate 0.4 exabytes of RAM fails.

The -DEXTRA_FUNCTIONALITY-related skips also appear on main.

@tiran
Copy link
Member Author

tiran commented Nov 13, 2021

Some suspicious-looking output from the decimal test suite when using the bundled libmpdec:

I cannot reproduce the issue locally. ./python -m test -uall test_decimal consumes a lot of memory. Can you reproduce the problem on main?

skipped 'test requires build with -DEXTRA_FUNCTIONALITY'

You have to enable the feature in setup.py. Extra functionality is disabled by default.

@tiran tiran changed the title bpo-45798: Move _decimal build setup into configure bpo-45798: Move _decimal build setup into configure (GH-29541) Nov 13, 2021
@tiran tiran merged commit 0486570 into python:main Nov 13, 2021
@tiran tiran deleted the libmpdec_configure branch November 13, 2021 12:56
@mdickinson
Copy link
Member

Okay, confession time: I don't actually know how to build Python against the MacPorts mpdecimal, either on main or on this branch. :-( That has little to do with this PR, but I would have liked to have checked that it still worked.

@tiran
Copy link
Member Author

tiran commented Nov 13, 2021

Okay, confession time: I don't actually know how to build Python against the MacPorts mpdecimal, either on main or on this branch. :-( That has little to do with this PR, but I would have liked to have checked that it still worked.

I verified --with-system-libmpdec on Fedora. You probably need to set some CFLAGS and LDFLAGS options to have Makefile and setup.py pick up the right files.

env CFLAGS=-I/path/to/port/include LDFLAGS=/path/to/port/lib ./configure --with-system-libmpdec
make

remykarem pushed a commit to remykarem/cpython that referenced this pull request Dec 7, 2021
@tiran @remykarem
bpo-45798: Move _decimal build setup into configure (pythonGH-29541)
e6445ab 
Settings for :mod:`decimal` internal C extension are now detected by
:program:`configure`. The bundled `libmpdec` library is built in
``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
@lithammer lithammer mentioned this pull request Dec 10, 2021
Closed
remykarem pushed a commit to remykarem/cpython that referenced this pull request Jan 30, 2022
@tiran @remykarem
bpo-45798: Move _decimal build setup into configure (pythonGH-29541)
880e59f 
Settings for :mod:`decimal` internal C extension are now detected by
:program:`configure`. The bundled `libmpdec` library is built in
``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
@erlend-aasland erlend-aasland mentioned this pull request Oct 24, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@mdickinson mdickinson mdickinson approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tiran @bedevere-bot @mdickinson @the-knights-who-say-ni