Fix data cursor for images with additional transform #18737

alexschlueter · 2020-10-15T08:45:41Z

PR Summary

When an image artist gets an additional transform supplied via set_transform or the transform=... argument, the data cursor will give values at the wrong coordinates. Example:

import matplotlib.pyplot as plt
from matplotlib.transforms import Affine2D
import numpy as np

fig, ax = plt.subplots()
trans = Affine2D().scale(2).rotate(0.5)
im = ax.imshow(np.arange(100).reshape(10, 10),
                transform=trans + ax.transData)
ax.set_xlim(-12, 20)
ax.set_ylim(-5, 30)
plt.show()

This happens because the current AxesImage.get_cursor_data uses the xdata and ydata attributes in the MouseEvent, which only undo the axis transform, but not the additional artist transform.

This PR adds a test and fixes the issue by applying the inversion of the image artists transform directly to the MouseEvents x and y attributes.

PR Checklist

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (run flake8 on changed files to check).
New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs should build without error).
Conforms to Matplotlib style conventions (install flake8-docstrings and pydocstyle<4 and run flake8 --docstring-convention=all).
New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

jklymak · 2020-10-15T14:50:42Z

lib/matplotlib/image.py

-        array_extent = Bbox([[0, 0], arr.shape[:2]])
-        trans = BboxTransform(boxin=data_extent, boxout=array_extent)
-        point = trans.transform([event.ydata, event.xdata])
+        data_extent = Bbox([[xmin, ymin], [xmax, ymax]])


I'm confused - was the previous version just a huge bug?

Are you referring to the switched xdata and ydata coordinates in the original version? I think this was fine originally (this code was covered quite well by tests in test_cursor_data()).
The first index of the array runs along the y-axis, while the second index runs along the x-axis. This means that a flip has to happen somewhere.
Originally, the flip was in the line point = trans.transform([event.ydata, event.xdata]). However, since we now want to apply a different transform first, we have to input the points in the right order (first x, then y). The flip still has to happen though, so we instead flip arr.shape and change i, j to j, i when we get the result.

I got it - I didn't trace it all through so was confused...

jklymak · 2020-10-15T14:51:24Z

Can you provide a self-contained example in the description to make it easier for reviewers to test themselves?

alexschlueter · 2020-10-15T15:12:21Z

Example added 👍

QuLogic · 2020-10-17T00:48:44Z

lib/matplotlib/image.py

@@ -661,7 +661,8 @@ def contains(self, mouseevent):
        # collection on nonlinear transformed coordinates.
        # TODO: consider returning image coordinates (shouldn't
        # be too difficult given that the image is rectilinear
-        x, y = mouseevent.xdata, mouseevent.ydata


Since you aren't using xdata/ydata any more, does that mean the comment about about figimage is fixed?

I believe it should be fine to remove the self.axes is not mouseevent.inaxes check now and contains() should also work for FigureImage. However, FigureImage doesn't have a get_cursor_data() yet, so I think the only place where contains would be used for a FigureImage is for the pick events? I'm not sure about that though, maybe @anntzer can help since he wrote the original comment.

It's likely that this fixes some of the problems I had noticed when I put in the comment, but I haven't reviewed this carefully enough to see whether all problems are indeed fixed.

alexschlueter · 2020-10-17T13:46:37Z

Looking into this more, it seems like the current MouseEvent / contains code is built with the assumption that artists don't set their own transform. Otherwise, using the axes transData to set xdata and ydata in the event does not generally make sense, since artists would always have to invert their own specific transform to check for contains. Maybe this should be changed globally?

jklymak

Thanks, sorry this sat for so long!

alexschlueter added 2 commits October 15, 2020 10:20

Fix image data cursor when artist has additional transform

12fbe4a

Add test for data cursor on transformed image

7a91945

jklymak reviewed Oct 15, 2020

View changes

jklymak added this to the v3.4.0 milestone Oct 15, 2020

jklymak added the topic: widgets/UI label Oct 15, 2020

QuLogic approved these changes Oct 17, 2020

View changes

QuLogic added the status: needs comment/discussion label Jan 5, 2021

jklymak approved these changes Jan 5, 2021

View changes

jklymak merged commit 98023a2 into matplotlib:master Jan 5, 2021
9 checks passed

jklymak removed the status: needs comment/discussion label Jan 5, 2021

Fix data cursor for images with additional transform #18737

Fix data cursor for images with additional transform #18737

alexschlueter commented Oct 15, 2020 •

edited

jklymak Oct 15, 2020

alexschlueter Oct 15, 2020

jklymak Oct 15, 2020

jklymak commented Oct 15, 2020

alexschlueter commented Oct 15, 2020

QuLogic Oct 17, 2020

alexschlueter Oct 17, 2020

anntzer Oct 17, 2020

alexschlueter commented Oct 17, 2020

jklymak left a comment

Fix data cursor for images with additional transform #18737

Fix data cursor for images with additional transform #18737

Conversation

alexschlueter commented Oct 15, 2020 • edited

PR Summary

PR Checklist

jklymak Oct 15, 2020

Choose a reason for hiding this comment

alexschlueter Oct 15, 2020

Choose a reason for hiding this comment

jklymak Oct 15, 2020

Choose a reason for hiding this comment

jklymak commented Oct 15, 2020

alexschlueter commented Oct 15, 2020

QuLogic Oct 17, 2020

Choose a reason for hiding this comment

alexschlueter Oct 17, 2020

Choose a reason for hiding this comment

anntzer Oct 17, 2020

Choose a reason for hiding this comment

alexschlueter commented Oct 17, 2020

jklymak left a comment

Choose a reason for hiding this comment

alexschlueter commented Oct 15, 2020 •

edited