Thanks @RichardBronosky, I have an idea of what's going on here. @dputtick do you want to take a look?

This also happens with obj.put(. Luckily I can type obj.put.__call__(Body='bar') and not get a crash.

Thanks for looking into this!

@RichardBronosky I've fixed this in master, so you can

pip install git+git://github.com/bpython/bpython.git@master

to keep working :)

Issue not closed because we should:

• add to changelog
• add a test for this

@dputtick if you're interested in these two things let me know, I'll wait a couple days.

For later reference, I assume the problem was that docstring.split('\n'), in Python 2 where this file has the from __future__ import unicode_literals compiler flag set, causes an implicit conversion and the boto docstring had a non-ascii character in it.

You're close. Now I'm getting:

Traceback (most recent call last):
  File "/usr/local/bin/bpython", line 9, in <module>
    load_entry_point('bpython==0.17.dev6', 'console_scripts', 'bpython')()
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsies.py", line 190, in main
    exit_value = repl.mainloop()
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsies.py", line 125, in mainloop
    self.process_event_and_paint(e)
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsies.py", line 99, in process_event_and_paint
    array, cursor_pos = self.paint()
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsiesfrontend/repl.py", line 1447, in paint
    if self.matches_iter.completer else None)
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsiesfrontend/replpainter.py", line 179, in paint_infobox
    if docstring else [])
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsiesfrontend/replpainter.py", line 167, in formatted_docstring
    for line in docstring.splitlines()), [])
  File "/usr/local/lib/python2.7/site-packages/bpython/curtsiesfrontend/replpainter.py", line 167, in <genexpr>
    for line in docstring.splitlines()), [])
  File "/usr/local/lib/python2.7/site-packages/curtsies/formatstring.py", line 628, in fmtstr
    string = FmtStr.from_str(string)
  File "/usr/local/lib/python2.7/site-packages/curtsies/formatstring.py", line 180, in from_str
    return FmtStr(Chunk(s))
  File "/usr/local/lib/python2.7/site-packages/curtsies/formatstring.py", line 70, in __init__
    raise ValueError("unicode string required, got %r" % string)
ValueError: unicode string required, got 'Retrieves objects from Amazon S3.'

Ah I was using a different version of curtsies, whoops. Taking a look now.

What version of curtsies? I can try to upgrade.

That error case is really weird to me considering...

$ awk '624<NR && NR<629 {print}' /usr/local/lib/python2.7/site-packages/curtsies/formatstring.py
    if isinstance(string, FmtStr):
        pass
    elif isinstance(string, (bytes, unicode)):
        string = FmtStr.from_str(string)
$ awk '68<NR && NR<71 {print}' /usr/local/lib/python2.7/site-packages/curtsies/formatstring.py
        if not isinstance(string, unicode):
            raise ValueError("unicode string required, got %r" % string)

No I was accidentally using an older one that allows formatstrings to be bytes, the current one that we now both have requires they be unicode. Now the problem is that the .splitlines() works on bytes or unicode strings, so we're leaking a bytestring into curtsies, which it violently protests.

I should have checked this more carefully, .split('\n') was more right because it always returns unicode. I've backed out these changes.

Since https://github.com/thomasballinger/curtsies/branches/all shows that master has the most recent commits, I tried sudo pip install git+git://github.com/thomasballinger/curtsies.git@master but still got the most recent error.

Gotcha.

Learning my lesson this time, I'm going to be more rigorous about this, so don't expect a fix in the next 20 minutes or so :)

it looks like some boto3 docstrings are ActionDocstrings and we need to figure out how to deal with them.

(bpython)tomb@tom-mba (master) curtsies$ bpython
bpython version 0.17.dev5 on top of Python 2.7.10 /Users/tomb/.virtualenvs/bpython/bin/python2.7
>>> import boto3
>>> s3=boto3.resource('s3')
>>> obj=s3.Object('bucket','key') # this can literally be those 2 strings
>>> type(obj.get.__doc__)
<class 'boto3.docs.docstring.ActionDocstring'>

Ah great, I can reproduce in Python 2 with

>>> def foo():
...     u"åß∂ƒ"
...
>>> foo(

so we must have implicit conversion happening (caused by this split(u'\n')) when we should be checking the type.

Still working on this? Replicated it, would be interested in poking around tomorrow or this weekend. Also happy to write tests.

@dputtick Yeah I'm not going to finish this anytime soon. Here are some tests that we should make pass in python 2:

    def test_unicode_docstrings(self):
        "A bit of a special case in Python 2"
        # issue 653

        def foo():
            u"åß∂ƒ"

        actual = replpainter.formatted_docstring(
                     foo.__doc__, 40, config=setup_config())
        expected = fsarray([u'åß∂ƒ'])
        self.assertFSArraysEqualIgnoringFormatting(actual, expected)

    def test_nonsense_docstrings(self):
        for docstring in [123, {}, [], ]:
            try:
                replpainter.formatted_docstring(
                    docstring, 40, config=setup_config())
            except Exception:
                self.fail('bad docstring caused crash: {!r}'.format(docstring))

(added to class TestCurtsiesPaintingSimple(CurtsiesPaintingTest) in bpython/test/test_curtsies_painting.py)

The thing I don't understand yet is the LazyLoadedDocstring class in the library botocore, in botocore/docs/docstring.py.

Our goal should be to change formatted_docstring in bpython/curtsiesfrontend/replpainter.py so that it works with these funny boto docstring objects, but I think all we have to do is check the type of the docstring argument we're handed and do something like

    if isinstance(docstring, bytes):
        docstring = docstring.decode('utf8')
    elif isinstance(docstring, str if py3 else unicode):
        pass
    else:
        return []

I think this does it, but I'm not sure. I can finish this up too, just not going to get to it tonight.

@dputtick You know what, I'm deep in this, I should finish it. I ran into another bug though if you want to take a look at that: #654

Sorry I meant to leave this, but then I felt bad for @RichardBronosky :)

👍 fair enough. I'll check back tomorrow and see if you got it or look at the other bug.

@dputtick here's another: #655

Fixed by #656

@RichardBronosky You can try this branch

pip install git+git://github.com/bpython/bpython.git@fix-653

if you want. I sort of want to say this is actually a bug in boto.

That did it! Thank you so much for taking this so personal.

Now I can see the docstring. Though I don't know what was breaking the world here:

>>> x.get()
┌───────────────────────────────────────────────────────────────────┐
│ x.get: (self, *args, **kwargs)                                    │
│ get                          get_available_subresources           │
│ Retrieves objects from Amazon S3.                                 │
│                                                                   │
│ **Request Syntax**                                                │
│ ::                                                                │
│                                                                   │
│   response = object_summary.get(                                  │
│       IfMatch='string',                                           │
│       IfModifiedSince=datetime(2015, 1, 1),                       │
│       IfNoneMatch='string',                                       │
│       IfUnmodifiedSince=datetime(2015, 1, 1),                     │
│       Range='string',                                             │
│       ResponseCacheControl='string',                              │
│       ResponseContentDisposition='string',                        │
│       ResponseContentEncoding='string',                           │
│       ResponseContentLanguage='string',                           │
│       ResponseContentType='string',                               │
│       ResponseExpires=datetime(2015, 1, 1),                       │
│       VersionId='string',                                         │
│       SSECustomerAlgorithm='string',                              │
│       SSECustomerKey='string',                                    │
│       RequestPayer='requester',                                   │
│       PartNumber=123                                              │
│   )                                                               │
│ :type IfMatch: string                                             │
│ :param IfMatch: Return the object only if its entity tag (ETag) i │
│ s the same as the one specified, otherwise return a 412 (precondi │
│ tion failed).                                                     │
│                                                                   │
└───────────────────────────────────────────────────────────────────┘

And in case it mattered. I just confirmed that all is well using python 3.5.2 installed via homebrew.

BTW, is there a better way to have access the both 2.7 and 3.5 bpython other than python -m bpython for 2.7 and python3 -m bpython for 3.5? The cool thing is that python3 -m bpython -i hack.py actually works.

Two good options would to use aliases as described at http://docs.bpython-interpreter.org/tips.html, or to use something like pyenv or virtualenvs to manage your Python versions.

@RichardBronosky What you've described is what I do, plus the alias thing @dputtick points to. I figured we had a issue for making a bpython3 executable, but I don't see one. I think there was packaging discussion around this. I believe this has been decided already, but if it hasn't I'm in favor of a separate bpython3 executable to mirror that most of us are typing in python3 in many situations.

#656 has been merged.