Improve jupyter backends by prabhuramachandran · Pull Request #1139 · enthought/mayavi

prabhuramachandran · 2022-03-10T05:57:49Z

Refactor notebook code and add itkwidgets support. The itkwidgets support allows us to use Mayavi in a headless manner unlike the other backends which require offscreen support.
Improve implementation of the different backends.
Fix issues with the test backend.
- mlab.clf() was not clearing the scene.
- obj.render() was not working correctly.
- The itkwidgets backend would not work correctly in some cases so we explicitly flush the pipeline before calling view.

The itkwidgets support allows us to use Mayavi in a headless manner unlike the other backends which require offscreen support.

- `mlab.clf()` was not clearing the scene. - `obj.render()` was not working correctly. - The itkwidgets backend would not work correctly in some cases so we explicitly flush the pipeline before calling view.

rahulporuri · 2022-03-10T06:53:11Z

-        if preference_manager.root.show_helper_nodes \
-                    and len(self.scenes) == 0:
+        if (preference_manager.root.show_helper_nodes and
+           len(self.scenes) == 0):


it looks like all of the changes in this file are cosmetic - two unused variables and a bunch of style changes.

Yes, I thought there was an issue there but there wasn't so sent in a separate commit with just PEP8 fixes for that file.

rahulporuri · 2022-03-10T06:55:27Z

            scene = figure
-        disable_render = scene.scene.disable_render
-        scene.scene.disable_render = True
+        if scene.scene is not None:


looks like this is the fix related to the scene not being cleared correctly?

Yes if you see each commit, the logs will make more sense. This fixes the issue with mlab.clf

rahulporuri · 2022-03-10T06:58:27Z

                    m = actor.mapper
                    if m is not None:
-                        m.update(0)
+                        m.update()


looks like the 0 we were passing before was the arg port - not sure why we don't need to pass it in anymore.

I am not sure, I think this was when there was issues with the old and new pipeline. The current test failures have nothing to do with this. Those have to do with the import of a UI toolkit when tvtk is imported -- this could be in some other library.

I need to see how this build is different from the headless test last Saturday.

rahulporuri · 2022-03-10T08:27:32Z

+
+
+def _ipython_display_(self):
+    '''Method attached to Mayavi objects.


It was not clear from the code that this special method is what IPython looks for. Ref https://ipython.readthedocs.io/en/stable/config/integrating.html#custom-methods. It'd be nice to document this in the function/method

I will add a comment.

Done in latest commit.

rahulporuri · 2022-03-10T08:32:03Z

+    def __init__(self, width=None, height=None, local=True):
+        super().__init__(width, height, local)
+        from mayavi import mlab
+        mlab.options.backend = 'test'


i'm not sure why we're setting the backend to 'test' here and if this is what we should be shipping.

The test backend does not instantiate a render window. If you do, you cannot run Mayavi in a headless fashion -- say via a remotely hosted jupyter notebook. The name "test" is not ideal but in the interest of backwards compatibility I am not sure we can or should change it.

Also use traitsui-7.2.1 in build to test if failures are related to this.

prabhuramachandran · 2022-03-10T18:45:08Z

@corranwebster -- the only major change I am able to see between the headless test here and the one that was run on schedule from 5 days ago (https://github.com/enthought/mayavi/actions/runs/1937040765) is that traitsui was upgraded to 7.3.0 from 7.2.1. Could it be that simply importing traitsui.api triggers either a wxPython or Qt import before a configure_traits is invoked? This is possibly breaking the tests. I am just pushing a new commit to test this hypothesis. Indeed, that was it, so somehow traitsui 7.3.0 is importing the UI toolkit libraries. Is there a chance this can be fixed in a future traitsui release?

corranwebster · 2022-03-14T09:43:54Z

Could it be that simply importing traitsui.api triggers either a wxPython or Qt import before a configure_traits is invoked?

Unfortunately the Menu code in API causes the Pyface toolkit to be selected, so you will get a Qt or Wx import when you import traitsui.api. But I am thinking that there may be a regression in the way that Color and Font traits are defined.

corranwebster · 2022-03-14T11:12:42Z

Quick question: did the version of Pyface also change?

prabhuramachandran · 2022-03-14T12:19:57Z

@corranwebster -- no the earlier code was also on pyface-7.4.1.

corranwebster · 2022-03-14T12:38:00Z

When I try to replicate the failing test in a Python 3.9 virtualenv (not an EDM env) with no backends installed, I see the following line in the verbose output from doing from tvtk.api import tvtk; p = tvtk.Property():

# /Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/__pycache__/__init__.cpython-39.pyc matches /Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/__init__.py
# code object from '/Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/__pycache__/__init__.cpython-39.pyc'
import 'pyface.ui.wx' # <_frozen_importlib_external.SourceFileLoader object at 0x132cd9bb0>
# /Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/__pycache__/init.cpython-39.pyc matches /Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/init.py
# code object from '/Users/cwebster/p3.9/lib/python3.9/site-packages/pyface/ui/wx/__pycache__/init.cpython-39.pyc'
# destroy pyface.ui.wx.init

and there are similar lines mentioning pyface.qt. However in this environment we can't be importing wx, because it isn't installed. But if I look at sys.modules['wx'] it isn't loaded and the selected toolkit is null as would be expected.

So I think that the failure is because some code is likely trying to load wx (and probably Qt) and failing, but things are showing up in the verbose logs that didn't before. The test probably needs to be refined to do something like inspecting sys.modules rather than just searching the logs.

prabhuramachandran · 2022-03-14T12:48:44Z

@corranwebster -- so sorry I sent you on a wild goose chase! I will look into writing a better and more robust test if that is possible. Thank you very much!

prabhuramachandran · 2022-03-26T14:09:50Z

@corranwebster -- I did some more investigation on this, while the test may not be ideal, it does do its job. It did detect what I think is a regression. If we just do from traitsui.view import View it imports a Qt or wx backend. To see where this actually happens I did some more digging:

I see that the place this first starts is in the import from traitsui.ui import UI
From here I looked at from traitsui.editor import Editor and I found that the culprit is the import from pyface.api import information in editor.py

I sent a simple PR to fix this (enthought/traitsui#1883) and all it requires is delaying the import of information in the error method of the editor. This fixes the problem but you are very likely not going to like my test case at all.

prabhuramachandran · 2022-03-26T14:13:00Z

@rahulporuri -- can you please review this so I can merge it sooner rather than later. I can fix any issues related to the traitsui issue in a later PR.

prabhuramachandran added 4 commits March 8, 2022 11:18

Refactor notebook code and add itkwidgets support.

12c193f

The itkwidgets support allows us to use Mayavi in a headless manner unlike the other backends which require offscreen support.

Improve implementation of the different backends.

07a6106

PEP8 issues.

9b5e41a

BUG: Fix issues with the test backend.

35187aa

- `mlab.clf()` was not clearing the scene. - `obj.render()` was not working correctly. - The itkwidgets backend would not work correctly in some cases so we explicitly flush the pipeline before calling view.

rahulporuri self-requested a review March 10, 2022 08:33

rahulporuri reviewed Mar 10, 2022

View reviewed changes

Add link to IPython documentation as per review.

1f32591

Also use traitsui-7.2.1 in build to test if failures are related to this.

prabhuramachandran mentioned this pull request Mar 12, 2022

Importing traitsui.api imports toolkits enthought/traitsui#1866

Open

prabhuramachandran mentioned this pull request Mar 26, 2022

Delay importing toolkit when importing View. enthought/traitsui#1883

Merged

Fix the weekly headless tests.

ef8980d

prabhuramachandran requested a review from rahulporuri March 26, 2022 14:58

rahulporuri approved these changes Mar 28, 2022

View reviewed changes

prabhuramachandran merged commit a0b9f1c into enthought:master Mar 28, 2022

prabhuramachandran deleted the improve-jupyter-backends branch March 28, 2022 07:12



		def _ipython_display_(self):
		'''Method attached to Mayavi objects.

Conversation

prabhuramachandran commented Mar 10, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prabhuramachandran commented Mar 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corranwebster commented Mar 14, 2022

Uh oh!

corranwebster commented Mar 14, 2022

Uh oh!

prabhuramachandran commented Mar 14, 2022

Uh oh!

corranwebster commented Mar 14, 2022

Uh oh!

prabhuramachandran commented Mar 14, 2022

Uh oh!

prabhuramachandran commented Mar 26, 2022

Uh oh!

prabhuramachandran commented Mar 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

prabhuramachandran commented Mar 10, 2022 •

edited

Loading