Raise exceptions instead of using assertions for control flow #12789
Comments
|
That is an excellent point, and we welcome any PR that changes assert's to proper exception. We have been doing that for long error messages anyway, and I agree with you it's a better choice (as long as we raise the appropriate exception of course). |
|
I'm adding the good first issue label, this way if someone wants to take care of one file to remove all asserts and replace them with proper exceptions, they can make a PR with it! (Don't try to do all files of the library at one ;-) ) |
|
Hi @sgugger, I'd like to take up one of the files to start. Should I pick up modelling_gpt2.py and go ahead? |
|
Any of the files with assert statements is fair game :-) |
|
Hi @sgugger, |
|
Just adding my two cents but I wouldn’t raise bare `Exception`s but instead raise the appropriate type of exception given the error.
… On Aug 19, 2021, at 1:44 AM, Ambesh Shekhar ***@***.***> wrote:
Hi @sgugger,
I'd be happy to do this. So I need to change assert to 'Exception' type error for only one file?
(Not allowed to do this for every file?)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
I have created PR #13184 for this issue. Kindly go through it. |
|
@sgugger is this still open ? If so any files with assertion came under the issue right ? Can i join this issue if it's not completed |
|
@jeffin07 You can pick any file that has not already been treated. |
|
@sgugger I am also going to pick several files to kill my time. |
* Raise exceptions instead of using assertions for control flow #12789 * # coding=utf-8 * Raise exceptions instead of using assertions for control flow * Raise exceptions instead of using assertions for control flow * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Suraj Patil <surajp815@gmail.com> * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Raise exceptions instead of using assertions for control flow * test * Raise exceptions instead of using assertions for control flow Co-authored-by: MocktaiLEngineer <kavinarasu22@gmail.com> Co-authored-by: Suraj Patil <surajp815@gmail.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Raise exceptions instead of using assertions for control flow huggingface#12789 * # coding=utf-8 * Raise exceptions instead of using assertions for control flow * Raise exceptions instead of using assertions for control flow * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Suraj Patil <surajp815@gmail.com> * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Raise exceptions instead of using assertions for control flow * test * Raise exceptions instead of using assertions for control flow Co-authored-by: MocktaiLEngineer <kavinarasu22@gmail.com> Co-authored-by: Suraj Patil <surajp815@gmail.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
|
I've created PR #13894 . Please take a look. |
|
I've created PR #13909 towards this issue. Please take a look |
|
Hello, I'm new to open source. I wanted to know if this issue is still open and if so how do i check which files have not been edited already and which files require changes still? |
|
Hello @kushagrabhushan! You can do a search across the repository and look for the I recommend looking only for files in I would recommend cloning the repository and using a tool to search for these, as the GitHub search isn't ideal for a search like this. |
|
I would also like to help on this issue. If i understood it right, i should make a separate PR for every single file right? |
* Raise exceptions instead of using assertions for control flow huggingface#12789 * # coding=utf-8 * Raise exceptions instead of using assertions for control flow * Raise exceptions instead of using assertions for control flow * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Suraj Patil <surajp815@gmail.com> * Update src/transformers/tokenization_utils.py Raise exceptions instead of using assertions for control flow Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Raise exceptions instead of using assertions for control flow * test * Raise exceptions instead of using assertions for control flow Co-authored-by: MocktaiLEngineer <kavinarasu22@gmail.com> Co-authored-by: Suraj Patil <surajp815@gmail.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
|
In cases of assertions being made inside for loops, what would be the best practice? for sentence in sentences:
if len(sentence["tokens"]) != len(sentence["labels"]):
raise ValueError(f"Number of tokens {len(words)} and labels tokens {len(labels)} mismatch")or for sentence in sentences:
if len(sentence["tokens"]) != len(sentence["labels"]):
raise ValueError(f"Number of tokens {len(words)} and labels tokens {len(labels)} mismatch in sentence {sentence}")The second approach seems to be a friendly solution for the user since it will show exactly what was the problematic sentence, although, I have seen the first approach used in the PRs. This isn't a critique, I am trying to understand if the first option is better or not. |
|
The second approach is more informative indeed! |
Updated masked-language modeling examples in pytorch with convention defined by #12789
transformers/src/transformers/models/gpt2/modeling_gpt2.py
Line 698 in 546dc24
Assertions can't be relied upon for control flow because they can be disabled, as per the following:
From my understanding, this is why mypy has no qualms about using them to narrow types because you can turn them off at runtime and so they incur zero cost.
Would you be open to me changing these assertions to other appropriate exceptions as I encounter them?
The text was updated successfully, but these errors were encountered: