Issue replicating in tf-nightly. Thanks.

It looks like tf.nest is overly permissive with Mapping subclasses, because the contract of Mapping does not seem to specify an interface for constructors. Users may accept key-value pairs, **kwargs or just about anything else and they'd still have a valid Mapping. Moreover, detecting how to properly use that constructor is extremely difficult in Python.

The correct solution is either to only support dict and OrderedDict (treating everything else as opaque objects), or to explicitly require a certain signature for constructor (right now, that is: "it must accept key-value pairs").

At any rate, this limitation should be documented and a best-effort to detect errors could be made by catching anything exception that the constructor raises and adding a hint about this.

For future readers, the workaround is to provide a constructor that accepts key-value pairs, similar to dict's.

Hey! I would like to work on this. Could you guide me?

Sure, here are a few pointers -

First, the documentation of all @tf_export methods inside tf.nest needs to be brought to date and clarify what is a "structure". Some entries mention that you can have dicts, others do not. I think the module documentation should clarify that. The docstrings in there are used to generate the public docs.

Second, IsMappingHelper that verifies for supported types should check for MutableMapping rather than just Mapping. It should probably be renamed to IsNestCompatibleMapping on this occasion. The reason for that is because we need to know how to construct new mappings of the same type and the Mapping interface doesn't standardize that. See the Python reference for more details.

Third, the error messages need to be improved. the code of _sequence_like that attempts to reconstruct dict objects should catch exceptions and raise a more informative one (something like "could not reconstruct object of type X"). Also, the code of func_graph_from_py_func should again catch exceptions and add more information (something like "could not handlearguments of <function name>: <message of caught error>").

I hope this helps! As a general guidance, I recommend sending multiple small PRs instead of one big change, to make them easier to review.

Hey! Thanks for the input.

First, the documentation of all @tf_export methods inside tf.nest needs to be brought to date and clarify what is a "structure". Some entries mention that you can have dicts, others do not. I think the module documentation should clarify that. The docstrings in there are used to generate the public docs.

Regarding this if am not wrong a couple of the function's documentation need to updated for which I should create a separate PR is what you are suggesting and to do the same for the remaining two broken down problems?

I'd start with everything in one PR (docstring changes and small code changes), and if you end up changing lots of code, break it down into more pieces.

@mdanatg I've create a PR #36186 addressing this. Could you please review?

I would like to test tensorflow locally from the github repo, how do I set that up for python?

@namedtoaster see the guide: https://www.tensorflow.org/install/source; you may want to skip to the Docker section for a ready-made setup.

@mdanatg I downloaded tensorflow from the most current repo and built using the instructions you linked to. When I try to run the code from the notebook (or really any simple tensorflow command), I get the following error.

Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA

Is there a way to suppress those features from my CPU so I can run tf? Or do I have to rebuild a new binary that disables those features?

Edit: I think I found some possible answers. I'll come back when I've exhausted my attempts

@namedtoaster You can safely ignore that warning, but otherwise you want a binary that is built with those features enabled. Try to see if you can rebuild it with these bazel options: --copt=-mavx2 --copt=-msse4 --copt=-mfma

@mdanatg thanks for the info. I stand corrected on my last post, I am able to do a simple tensorflow command with the original binary installed. When I tried to re-install with the optimizations I got some compiler errors that I'll try to fix at a later time. I did what you said but I get this output:

clang: error: unsupported option '--config=opt'

My bigger confusion right now is that when I try to run the code from the notebook I get an error about using the wrong number of arguments:

2020-02-12 15:50:38.623392: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA Traceback (most recent call last): File "/Users/----/PycharmProjects/tensorflow_dev/test.py", line 27, in <module> test_tf_function(CustomMapping(theta=3.1415)) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/def_function.py", line 576, in __call__ result = self._call(*args, **kwds) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/def_function.py", line 623, in _call self._initialize(args, kwds, add_initializers_to=initializers) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/def_function.py", line 505, in _initialize *args, **kwds)) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/function.py", line 2440, in _get_concrete_function_internal_garbage_collected graph_function, _, _ = self._maybe_define_function(args, kwargs) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/function.py", line 2771, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/eager/function.py", line 2661, in _create_graph_function capture_by_value=self._capture_by_value), File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 898, in func_graph_from_py_func args, arg_names, flat_shapes=arg_shapes) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 1132, in _get_defun_inputs_from_args args, names, structure=args, flat_shapes=flat_shapes) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 1188, in _get_defun_inputs arg_value = nest.map_structure(_get_composite_tensor_spec, arg_value) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/util/nest.py", line 600, in map_structure expand_composites=expand_composites) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/util/nest.py", line 535, in pack_sequence_as return _pack_sequence_as(structure, flat_sequence, expand_composites) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/util/nest.py", line 499, in _pack_sequence_as return sequence_fn(structure, packed) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow_core/python/util/nest.py", line 145, in _sequence_like return instance_type((key, result[key]) for key in instance) TypeError: __init__() takes exactly 1 argument (2 given)

I'm looking into the error right now. I may have an incorrect understanding of kwargs

I think the error you see is expected - it's the bug described in the OP.

For the instruction set warning, try specifying -c opt as well. I came across this page where the author seemed to successfully use these options: https://gist.github.com/Brainiarc7/6d6c3f23ea057775b72c52817759b25c

@mdanatg that makes sense. I should've noticed that -- I will check out that link and give it a try. Thanks

Hello I thought this issue was solved but it's been open for quite some time.
I would like to contribute if contributions are still being accepted.

Hey @Joey155 ! PR #36186 has been opened regarding the same.