gh-82012: Deprecate bitwise inversion (~) of bool #103487

timhoffm · 2023-04-12T19:50:52Z

The bitwise inversion operator on bool returns the bitwise inversion of the underlying int value; i.e. ~True == -2 such that bool(~True) == True.

It's a common pitfall that users mistake ~ as negation operator and actually want not. Supporting ~ is an artifact of bool inheriting from int. Since there is no real use-case for the current behavior, let's deprecate ~ on bool and later raise an error. This removes a potential source errors for users.

Full reasoning: #82012 (comment)

Issue: bool(~True) == True #82012

Lib/test/test_bool.py

hugovk · 2023-04-28T02:37:23Z

Please document this in What's New at https://docs.python.org/3.12/whatsnew/3.12.html#deprecated

And is this operator documented in the main reference? Let's also mention the deprecation there.

Is the plan to turn this into an error in 3.14, or keep it open ended?

gvanrossum

LGTM. Let's just merge this.

iritkatriel · 2023-04-28T10:31:35Z

We should probably add a comment about this to https://github.com/python/cpython/blob/main/Doc/whatsnew/3.12.rst.

gvanrossum · 2023-04-28T14:28:53Z

We should probably add a comment about this to https://github.com/python/cpython/blob/main/Doc/whatsnew/3.12.rst.

Yes!

timhoffm · 2023-04-28T16:42:27Z

I've added the deprecation notice to whatsnew.

Is the plan to turn this into an error in 3.14, or keep it open ended?

I suggest to turn this into an error (and have indicated so in the whatsnew). There is a real danger that some users write if ~condition (which is always True). Warnings are often overlooked, so it's better to eventually error on this. In contrast, I can hardly imagine any intended use for ~some_bool, so this error should not annoy any rightful users.

JelleZijlstra · 2023-04-29T03:30:31Z

Lib/test/test_bool.py

-        self.assertEqual(~True, -2)
+        with self.assertWarns(DeprecationWarning):
+            # We need to put the bool in a variable, because the constant
+            # ~False is evaluated at compile time due to constant folding;


We should test this behavior separately (doing something like with assertWarns(DeprecationWarning): exec("~False")).

Do you mean instead of or in addition to the current test? We originally had eval("~True") but I figured it's clearer to have the warning context only around the operation and not around a comparably complex eval() or exec(). I'm happy to change if there's a benefit of these though.

We should have both. The existing tests check that the deprecation warning is emitted correctly at runtime. The new tests would ensure that if the operation occurs at compile time, we still emit the DeprecationWarning.

JelleZijlstra

The new behavior should also be mentioned in https://docs.python.org/3.12/library/stdtypes.html#boolean-values

It currently says "In numeric contexts (for example when used as the argument to an arithmetic operator), they behave like the integers 0 and 1, respectively.", which will no longer be true in all contexts.

Doc/whatsnew/3.12.rst

timhoffm · 2023-04-29T23:26:15Z

The new behavior should also be mentioned in https://docs.python.org/3.12/library/stdtypes.html#boolean-values

It currently says "In numeric contexts (for example when used as the argument to an arithmetic operator), they behave like the integers 0 and 1, respectively.", which will no longer be true in all contexts.

I suggest to not document this as it would become more confusing than helpful:

In my view, there should be no reason to use bitwise operators on bool; one should always use the boolean operators (or convert to int explicitly). I therefore consider all bitwise operators of bool an implementation detail. While this is public API, it's not something we should advertize.

In particular, if we document ~, we'd also need to leave some words on &, | and ^. And then it gets quite messy. If they operate on bool, they return bool. In contrast, ~ was returning int, will now warn and will error out in the future. Changing to returning bool for ~ meaning a logic negation was rejected in the original issue discussion as too much of an API break (still working but with changed behavior). If I had to document this correctly this would be:

The binary bitwise operators &, | and ^ on two bools return a bool and behave like their logical equivalents. When applying these binary bitwise operators to mixed bool and int arguments, they return an int and interpret the bool as the underlying int (i.e. 0 or 1). The bitwise negation ~ of a bool currently returns the bitwise complement of the underlying int. This is deprecated and will raise an error in 3.14. Generally, the use of bitwise operators on bools is discouraged. Use the logical operators and, or, not and xor instead, or convert to int explicitly.

But maybe you have better ideas what to document.

The bitwise inversion operator on bool returns the bitwise inversion of the underlying int value; i.e. `~True == -2` such that `bool(~True) == True`. It's a common pitfall that users mistake `~` as negation operator and actually want `not`. Supporting `~` is an artifact of bool inheriting from int. Since there is no real use-case for the current behavior, let's deprecate `~` on bool and later raise an error. This removes a potential source errors for users. Full reasoning: python#82012 (comment) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

JelleZijlstra · 2023-04-29T23:54:59Z

The new behavior should also be mentioned in https://docs.python.org/3.12/library/stdtypes.html#boolean-values
It currently says "In numeric contexts (for example when used as the argument to an arithmetic operator), they behave like the integers 0 and 1, respectively.", which will no longer be true in all contexts.

I suggest to not document this as it would become more confusing than helpful:

In my view, there should be no reason to use bitwise operators on bool; one should always use the boolean operators (or convert to int explicitly). I therefore consider all bitwise operators of bool an implementation detail. While this is public API, it's not something we should advertize.

Whether we like it or not, these operators have worked this way for a long time in Python, and backwards compatibility alone means we can't just make them go away. So it's better to document clearly how they work and what is changing.

In particular, if we document ~, we'd also need to leave some words on &, | and ^. And then it gets quite messy. If they operate on bool, they return bool. In contrast, ~ was returning int, will now warn and will error out in the future. Changing to returning bool for ~ meaning a logic negation was rejected in the original issue discussion as too much of an API break (still working but with changed behavior).

I think the whole reason that we're deprecating only the ~ operator is that it's unlike the others: the results of &, |, and ^ make sense whether you think of bools or ints, it's only ~ that turns something from a bool into a non-bool int.

If I had to document this correctly this would be:

The binary bitwise operators &, | and ^ on two bools return a bool and behave like their logical equivalents. When applying these binary bitwise operators to mixed bool and int arguments, they return an int and interpret the bool as the underlying int (i.e. 0 or 1). The bitwise negation ~ of a bool currently returns the bitwise complement of the underlying int. This is deprecated and will raise an error in 3.14. Generally, the use of bitwise operators on bools is discouraged. Use the logical operators and, or, not and xor instead, or convert to int explicitly.

But maybe you have better ideas what to document.

Your wording sounds good to me!

This also pulls the bool type to top-level of the type description page. Before it was only documented in the section "Other Built-in Types / Boolean Values".

timhoffm · 2023-04-30T21:46:29Z

@JelleZijlstra thanks for the feedback. I've rewritten and restructured the bool docs a bit so that it's hopefully more clear and still precise. Please check if this is ok. - It's in a seprate commit, so could be easily modified/reverted.

In particular, I've created a top-level section "Boolean Type - bool", which does the importance of the type more justice.
Before, the bool description was only in documented in the section "Other Built-in Types / Boolean Values". I've moved most of the description over from there, but left the section as a short stub; both because (1) I'm unclear whether we still need "Boolean Values" explicitly in addtion to "Boolean Type", and (2) because that keeps the link so that internal and external references are not broken..

JelleZijlstra

Looks good, but I'd like to have another core dev look at the docs before we merge.

hauntsaninja

I think I'd prefer to remove the Boolean Values section. We can preserve the .. _bltin-boolean-values: and index roles, which should hopefully preserve links.

Doc/library/stdtypes.rst

timhoffm · 2023-04-30T22:49:34Z

I think I'd prefer to remove the Boolean Values section. We can preserve the .. _bltin-boolean-values: and index roles, which should hopefully preserve links.

You mean moving them to the new "Boolean Type" section?

hauntsaninja · 2023-04-30T22:55:21Z

Yup!

timhoffm · 2023-05-01T00:52:42Z

I think I'd prefer to remove the Boolean Values section. We can preserve the .. _bltin-boolean-values: and index roles, which should hopefully preserve links.

HTML links to the section cannot be preseved the URL was https://docs.python.org/3.12/library/stdtypes.html#boolean-values and the anchor #boolean-values is constructed from the section title, not from the label.

I've rewritten internal references (because the label _booltype is more in line with the labeling conventation in the file than _bltin-boolean-values). And the moved index roll works.

Doc/library/stdtypes.rst

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

gvanrossum · 2023-05-03T06:54:33Z

Is this waiting for anyone?

hauntsaninja · 2023-05-03T07:01:00Z

Not anymore. Thanks all!

bedevere-bot mentioned this pull request Apr 12, 2023

bool(~True) == True #82012

Closed

bedevere-bot added the awaiting review label Apr 12, 2023

timhoffm commented Apr 12, 2023

View reviewed changes

Lib/test/test_bool.py Outdated Show resolved Hide resolved

timhoffm force-pushed the deprecate-bool-invert branch 2 times, most recently from b94d85b to 20dec73 Compare April 13, 2023 23:57

gvanrossum approved these changes Apr 28, 2023

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Apr 28, 2023

timhoffm force-pushed the deprecate-bool-invert branch from 8290ea7 to bf02fd4 Compare April 28, 2023 16:35

JelleZijlstra reviewed Apr 29, 2023

View reviewed changes

Doc/whatsnew/3.12.rst Outdated Show resolved Hide resolved

timhoffm force-pushed the deprecate-bool-invert branch from 3ac9b82 to a6e3366 Compare April 29, 2023 23:24

timhoffm force-pushed the deprecate-bool-invert branch 2 times, most recently from d599825 to edd07f2 Compare April 29, 2023 23:35

timhoffm force-pushed the deprecate-bool-invert branch from edd07f2 to 1f0351a Compare April 29, 2023 23:44

Document bool behavior for bitwise operations

3632ba5

This also pulls the bool type to top-level of the type description page. Before it was only documented in the section "Other Built-in Types / Boolean Values".

JelleZijlstra approved these changes Apr 30, 2023

View reviewed changes

hauntsaninja reviewed Apr 30, 2023

View reviewed changes

Doc/library/stdtypes.rst Outdated Show resolved Hide resolved

Doc/library/stdtypes.rst Outdated Show resolved Hide resolved

Doc/library/stdtypes.rst Outdated Show resolved Hide resolved

Move section "Boolean Values" into "Boolean Type"

2cbfe2d

hauntsaninja approved these changes May 1, 2023

View reviewed changes

Doc/library/stdtypes.rst Outdated Show resolved Hide resolved

Doc/library/stdtypes.rst Outdated Show resolved Hide resolved

timhoffm and others added 2 commits May 1, 2023 08:21

Update Doc/library/stdtypes.rst

e5ec711

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

Update Doc/library/stdtypes.rst

2db5864

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

hauntsaninja merged commit fdb3ef8 into python:main May 3, 2023
19 checks passed

bedevere-bot removed the awaiting merge label May 3, 2023

timhoffm deleted the deprecate-bool-invert branch May 3, 2023 07:22

gh-82012: Deprecate bitwise inversion (~) of bool #103487

gh-82012: Deprecate bitwise inversion (~) of bool #103487

timhoffm commented Apr 12, 2023 •

edited by bedevere-bot

hugovk commented Apr 28, 2023

gvanrossum left a comment

iritkatriel commented Apr 28, 2023

gvanrossum commented Apr 28, 2023

timhoffm commented Apr 28, 2023

JelleZijlstra Apr 29, 2023

timhoffm Apr 29, 2023 •

edited

JelleZijlstra Apr 29, 2023

JelleZijlstra left a comment

timhoffm commented Apr 29, 2023 •

edited

JelleZijlstra commented Apr 29, 2023

timhoffm commented Apr 30, 2023

JelleZijlstra left a comment

hauntsaninja left a comment

timhoffm commented Apr 30, 2023

hauntsaninja commented Apr 30, 2023

timhoffm commented May 1, 2023

gvanrossum commented May 3, 2023

hauntsaninja commented May 3, 2023

gh-82012: Deprecate bitwise inversion (~) of bool #103487

gh-82012: Deprecate bitwise inversion (~) of bool #103487

Conversation

timhoffm commented Apr 12, 2023 • edited by bedevere-bot

hugovk commented Apr 28, 2023

gvanrossum left a comment

Choose a reason for hiding this comment

iritkatriel commented Apr 28, 2023

gvanrossum commented Apr 28, 2023

timhoffm commented Apr 28, 2023

JelleZijlstra Apr 29, 2023

Choose a reason for hiding this comment

timhoffm Apr 29, 2023 • edited

Choose a reason for hiding this comment

JelleZijlstra Apr 29, 2023

Choose a reason for hiding this comment

JelleZijlstra left a comment

Choose a reason for hiding this comment

timhoffm commented Apr 29, 2023 • edited

JelleZijlstra commented Apr 29, 2023

timhoffm commented Apr 30, 2023

JelleZijlstra left a comment

Choose a reason for hiding this comment

hauntsaninja left a comment

Choose a reason for hiding this comment

timhoffm commented Apr 30, 2023

hauntsaninja commented Apr 30, 2023

timhoffm commented May 1, 2023

gvanrossum commented May 3, 2023

hauntsaninja commented May 3, 2023

timhoffm commented Apr 12, 2023 •

edited by bedevere-bot

timhoffm Apr 29, 2023 •

edited

timhoffm commented Apr 29, 2023 •

edited