Skip to content

BUG: Fix new DTypes and new string promotion when signature is involved #26744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ngoldbaum merged 5 commits into from
Jul 6, 2024
Merged

BUG: Fix new DTypes and new string promotion when signature is involved #26744

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a4596d7
BUG,MAINT: Move `allow_legacy_promotion` check to later
seberg Jun 18, 2024
0b723bc
BUG: Fix string addition promoter to work with `dtype=`
seberg Jun 18, 2024
6347369
TST: Add regression test for bad promotion error/path
seberg Jun 18, 2024
817c9e4
STY: Apply style suggestion from review
seberg Jun 18, 2024
1d1c0c0
STY: Address review comments
seberg Jul 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
STY: Apply style suggestion from review 
Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
  • Loading branch information
@seberg @ngoldbaum
seberg and ngoldbaum committed Jul 6, 2024
commit 817c9e4c775033c8c80027f11819c25380ad7ebd
12 changes: 6 additions & 6 deletions numpy/_core/src/umath/stringdtype_ufuncs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1028,9 +1028,9 @@ all_strings_promoter(PyObject *NPY_UNUSED(ufunc),
PyArray_DTypeMeta *const signature[],
PyArray_DTypeMeta *new_op_dtypes[])
{
if (op_dtypes[0] != &PyArray_StringDType &&
op_dtypes[1] != &PyArray_StringDType &&
op_dtypes[2] != &PyArray_StringDType) {
if ((op_dtypes[0] != &PyArray_StringDType &&
op_dtypes[1] != &PyArray_StringDType &&
op_dtypes[2] != &PyArray_StringDType)) {
/*
* This promoter was triggered with only unicode arguments, so use

@mhvk mhvk Jun 19, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems confusing - should StringDType really decide the result for something that does not include itself? Shouldn't that be up to UnicodeDType? What happens if we return -1 here?

@seberg seberg Jul 4, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things would just fail the operation, I am very sure I added it for a reason. The problem is that you would have to decide that UU->U is clearly better than UU->T. But there is nothing to decide that by, so the machinery prefers the promoter, since the promoter has the ability to resolve which one to actual use (also go to UU->U).

The other thing you might not like is that it matches at all, but that would need one of two new features (which is fine):

  • Always call promoters right away even if there may be much better matches and give them the ability to say "don't know" (i.e. a -1 return might be that without an error set).
  • Allow some heuristic for it like, "must contain this DType" as an additional dtype.
  • Allow "matching" via a second function.

The other solution for the particular case is that if there wasn't legacy promotion involved, I would like a default promoter that ensures that ufunc(..., dtype=X) will search for ufunc(..., signature=(X, X, X)) as well if there is otherwise no match. That would do the right thing here.

@mhvk mhvk Jul 8, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was probably good to get this in, but it still feels weird to have a promotor for a given dtype return a result that does not involve that type at all - how can it decide for another type what is acceptable? Two of your solutions sound reasonable: returning the equivalent of NotImplemented (your -1 with no error set), or a default promotor. Maybe worth a new issue?

* unicode. This can happen due to `dtype=` support which sets the
Expand All @@ -1041,9 +1041,9 @@ all_strings_promoter(PyObject *NPY_UNUSED(ufunc),
new_op_dtypes[2] = NPY_DT_NewRef(&PyArray_UnicodeDType);
return 0;
}
if (signature[0] == &PyArray_UnicodeDType &&
signature[1] == &PyArray_UnicodeDType &&
signature[2] == &PyArray_UnicodeDType) {
if ((signature[0] == &PyArray_UnicodeDType &&
signature[1] == &PyArray_UnicodeDType &&
signature[2] == &PyArray_UnicodeDType)) {
/* Unicode forced, but didn't override a string input: invalid */
return -1;

@seberg seberg Jun 18, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part makes me wonder if I should just check it after the promoter is done and invalidate the result if this is violated. But it is OK here also.

@ngoldbaum ngoldbaum Jun 18, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it would be better to enforce that there, if only because IMO DType authors shouldn't have to worry about that case or add code to account for it to write a correct DType.

@mhvk mhvk Jun 19, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, it seems strange one would even get here if the signature is already clear that StringDType should not be involved.

}
Expand Down