Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation to note dangers of shared state in random module #93743

Closed
wants to merge 2 commits into from

Conversation

admackin
Copy link

@admackin admackin commented Jun 12, 2022

Context: I've been bitten recently by a call to a third-party library function indirectly calling random.seed and changing the behaviour of my own code. I think it's worth adding bit of extra warning about the dangers of shared state since this can lead to subtle, hard-to-detect and hard-to-explain bugs.

I thought this was small enough that it didn't need an issue (clarifying existing docs) but happy to be corrected.

@admackin admackin requested a review from rhettinger as a code owner Jun 12, 2022
@cpython-cla-bot
Copy link

@cpython-cla-bot cpython-cla-bot bot commented Jun 12, 2022

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Jun 12, 2022

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

@bedevere-bot bedevere-bot added docs awaiting review labels Jun 12, 2022
@arhadthedev
Copy link
Contributor

@arhadthedev arhadthedev commented Jun 12, 2022

You can ignore the bot; purely documentation changes need neither changelog entries nor issue numbers.

instance of the :class:`random.Random` class, and thus share state – in
particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
You should consider instantiating your own instance or instances of :class:`Random`
to get generators that don't share state.
Copy link
Member

@AlexWaygood AlexWaygood Jun 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing whitespace is making the CI check fail:

Suggested change
instance of the :class:`random.Random` class, and thus share state – in
particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
You should consider instantiating your own instance or instances of :class:`Random`
to get generators that don't share state.
instance of the :class:`random.Random` class, and thus share state – in
particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
You should consider instantiating your own instance or instances of :class:`Random`
to get generators that don't share state.

@rhettinger rhettinger self-assigned this Jun 12, 2022
instance of the :class:`random.Random` class, and thus share state – in
particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
Copy link
Member

@AA-Turner AA-Turner Jun 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
to `random.*` methods in the same process, which can lead to unexpected results.
to ``random.*`` methods in the same process, which can lead to unexpected results.

or

"other top-level functions in :mod:`random`"

You should consider instantiating your own instance or instances of :class:`Random`
to get generators that don't share state.
Copy link
Member

@AA-Turner AA-Turner Jun 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some applications, the sharing of state might not matter -- perhaps provide a heuristic for where I'd want to instantiate my own Random class?

particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
Copy link
Member

@AA-Turner AA-Turner Jun 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not reflowed; See my other suggestion on "random.*"

Suggested change
particular, calls to :meth:`random.set_seed` anywhere in application code
or any libraries will affect the sequence of random values produced by calls
to `random.*` methods in the same process, which can lead to unexpected results.
particular, a call to :meth:`random.set_seed` in any executed code
will affect the sequence of random values produced, as the random seed is shared.
This affects return values from calls to other `random.*` methods in the same process, potentially leading to unexpected results.

@@ -30,8 +30,12 @@ deterministic, it is not suitable for all purposes, and is completely unsuitable
for cryptographic purposes.

The functions supplied by this module are actually bound methods of a hidden
instance of the :class:`random.Random` class. You can instantiate your own
instances of :class:`Random` to get generators that don't share state.
instance of the :class:`random.Random` class, and thus share state – in
Copy link
Member

@AA-Turner AA-Turner Jun 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
instance of the :class:`random.Random` class, and thus share state – in
instance of the :class:`Random` class, and thus share state – in

@rhettinger
Copy link
Contributor

@rhettinger rhettinger commented Jun 12, 2022

Thank you for the suggestion, but we will decline.

In general, we don't modify the docs in response to someone having a bug in their code especially if the essential details have already been covered. Here, the docs already state, "you can instantiate your own instances of Random() to get generators that don’t share state." That gets to the heart of the matter.

Our style is to word the docs are affirmatively worded and not fill them with "warnings and dangers" unless a significant security risk is present.

It is just a general programming principle that changing a stateful object affects all users and viewers of the object. If a library function calls print() in the middle of a user's print routine, it will commingle the output. There are many examples. There isn't anything special about the random module in the regard.

Thanks again for the suggestion, but I think it doesn't improve the docs (and likely wouldn't have prevented your library author from writing the code the way they did). ISTM this intro section reads better without the programming practices side trip.

@rhettinger rhettinger closed this Jun 12, 2022
@AA-Turner
Copy link
Member

@AA-Turner AA-Turner commented Jun 12, 2022

@rhettinger -- I think it might be useful to at least note that one of the items of shared state is the seed, which a developer on a quick read through may not realise.

There isn't a 'how to' guide for random at the moment, so I believe the module documentation is the best place. (Such a guide would probably be useful, but a different matter).

A

@admackin
Copy link
Author

@admackin admackin commented Jun 13, 2022

Thank you for the suggestion, but we will decline.

In general, we don't modify the docs in response to someone having a bug in their code especially if the essential details have already been covered. Here, the docs already state, "you can instantiate your own instances of Random() to get generators that don’t share state." That gets to the heart of the matter.

Does it though? if two authors calling the library exactly as the docs suggest they should, using random.* methods, leads to code behaving unexpectedly in confusing ways, isn't that a warning sign?

Our style is to word the docs are affirmatively worded and not fill them with "warnings and dangers" unless a significant security risk is present.

This is fair, I would be more than happy to reword to be more advice than warning but it seems like there is not much point.

It is just a general programming principle that changing a stateful object affects all users and viewers of the object. If a library function calls print() in the middle of a user's print routine, it will commingle the output. There are many examples. There isn't anything special about the random module in the regard.

TBH I'm not sure that print() is good counterexample of this regard – a print() call from a library doesn't a) cause subtle bugs you may not notice b) modify the behaviour of every subsequent print call you make – the only global state it modifies is the text printed to stdout. Logging config would be a better example, and I would note that logging.basicConfig is carefully designed to try to avoid shared global logging config state getting overwritten by badly-behaved libraries where possible because shared state is a huge problem.

This function does nothing if the root logger already has handlers configured, unless the keyword argument force is set to True.

I'm struggling to think of other examples in the stdlib where global state (not just reading/writing from stdin/stdout) is the important (and also not transparent)

Thanks again for the suggestion, but I think it doesn't improve the docs (and likely wouldn't have prevented your library author from writing the code the way they did). ISTM this intro section reads better without the programming practices side trip.

I mean if the docs said "this way might be a good idea" I'd like to imagine at least some library authors would pay attention to it, otherwise why are the docs there at all? Sure not all would. and I'll try to avoid what you call "side trips" in future suggestions.

@admackin
Copy link
Author

@admackin admackin commented Jun 13, 2022

@rhettinger -- I think it might be useful to at least note that one of the items of shared state is the seed, which a developer on a quick read through may not realise.

There isn't a 'how to' guide for random at the moment, so I believe the module documentation is the best place. (Such a guide would probably be useful, but a different matter).

A

I think this is better than nothing – I've had a go over at #93755

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting review docs skip news
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants