Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ctypes.util.find_library raises FileNotFoundError for libsecp256k1 #93094

Open
relativisticelectron opened this issue May 23, 2022 · 2 comments
Open

ctypes.util.find_library raises FileNotFoundError for libsecp256k1 #93094

relativisticelectron opened this issue May 23, 2022 · 2 comments
Labels
expert-ctypes type-bug An unexpected behavior, bug, or error

Comments

@relativisticelectron
Copy link

relativisticelectron commented May 23, 2022 •
edited

Bug report

The following script

import ctypes
from ctypes import  util

print(f'ctypes version {ctypes.__version__}')


print('Test foofoofoo')
print(f"returns: {util.find_library('foofoofoo')}\n")

print('Test secp256k1')
print(f"returns: {util.find_library('secp256k1')}\n")

print('Test libsecp256k1')
print(f"returns: {util.find_library('libsecp256k1')}\n")

gives the output:

ctypes version 1.1.0
Test foofoofoo
returns: None

Test secp256k1
returns: libsecp256k1.so.0

Test libsecp256k1
Traceback (most recent call last):
  File "/home/<user>/programs/testctypes.py", line 14, in <module>
    print(f"returns: {util.find_library('libsecp256k1')}\n")
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 351, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 148, in _findLib_gcc
    if not _is_elf(file):
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
FileNotFoundError: [Errno 2] No such file or directory: b'liblibsecp256k1.a'

find_library should not fail, but return None if nothing was found. In Ubuntu 20.04 this script runs without any exception.

Your environment

  • ctypes version 1.1.0
  • Operating system and architecture: Kubuntu 22.04 LTS x86_64, linux 5.15.0-27-generic
  • Bug can be reproduced in python versions: python 3.10 (via anaconda and via ubuntu deb repositories), python 3.8 (via anaconda)
  • The installed libsecp256k1 packages are
    image

image contains the files

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libsecp256k1.so.0
/usr/lib/x86_64-linux-gnu/libsecp256k1.so.0.0.0
/usr/share
/usr/share/doc
/usr/share/doc/libsecp256k1-0
/usr/share/doc/libsecp256k1-0/changelog.Debian.gz
/usr/share/doc/libsecp256k1-0/copyright

And image contains the files

/.
/usr
/usr/include
/usr/include/secp256k1.h
/usr/include/secp256k1_ecdh.h
/usr/include/secp256k1_extrakeys.h
/usr/include/secp256k1_preallocated.h
/usr/include/secp256k1_recovery.h
/usr/include/secp256k1_schnorrsig.h
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libsecp256k1.a
/usr/lib/x86_64-linux-gnu/libsecp256k1.so
/usr/lib/x86_64-linux-gnu/pkgconfig
/usr/lib/x86_64-linux-gnu/pkgconfig/libsecp256k1.pc
/usr/share
/usr/share/doc
/usr/share/doc/libsecp256k1-dev
/usr/share/doc/libsecp256k1-dev/changelog.Debian.gz
/usr/share/doc/libsecp256k1-dev/copyright

And in /usr/lib/x86_64-linux-gnu/ are the following relevant files containing secp256k1:
image

This bug was noticed in cryptoadvance/specter-desktop#1710
@relativisticelectron relativisticelectron added the type-bug An unexpected behavior, bug, or error label May 23, 2022
Closed
@relativisticelectron
Copy link
Author

relativisticelectron commented Jun 1, 2022 •
edited

When I do a print(trace) just before

cpython/Lib/ctypes/util.py

Line 139 in 8241a69

res = re.findall(expr, trace)

I get

b'/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o\n/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o\n/usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o\n/usr/bin/ld: cannot find -llibsecp256k1: No such file or directory\n/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsecp256k1.a use -l:libsecp256k1.a or rename it to liblibsecp256k1.a\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so\n/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a\ncollect2: error: ld returned 1 exit status\n'

here with replaced \n


b'/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o
/usr/bin/ld: cannot find -llibsecp256k1: No such file or directory
/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsecp256k1.a use -l:libsecp256k1.a or rename it to liblibsecp256k1.a
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
collect2: error: ld returned 1 exit status
'

My interpretation of this is, that the re logic in

cpython/Lib/ctypes/util.py

Line 139 in 8241a69

res = re.findall(expr, trace)
is not sufficient to handle the ld message.

@relativisticelectron relativisticelectron mentioned this issue Jun 2, 2022
Closed
@cz6ace
Copy link

cz6ace commented Oct 12, 2022

I face the same issue, when the output is polluted with message from ld, in my case:
/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libusb-1.0.a use -l:libusb-1.0.a or rename it
reproduced with using pyInstaller package, but can be reproduced easily, see below.

Environment:
Ubuntu 22.04 LTS
uname: Linux ubu22-virtual 5.15.0-50-generic #56-Ubuntu SMP Tue Sep 20 13:23:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Python: 3.10.6

Steps to reproduce:

sudo apt install libusb-1.0-0-dev
/usr/bin/gcc -Wl,-t -o /tmp/tmpp455oz01 -llibusb-1.0

outputting

/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o
/usr/bin/ld: cannot find -llibusb-1.0: No such file or directory
/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libusb-1.0.a use -l:libusb-1.0.a or rename it to liblibusb-1.0.a
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
collect2: error: ld returned 1 exit status

in python3

from ctypes import util
util.find_library('libusb-1.0')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/ctypes/util.py", line 341, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
  File "/usr/lib/python3.10/ctypes/util.py", line 147, in _findLib_gcc
    if not _is_elf(file):
  File "/usr/lib/python3.10/ctypes/util.py", line 99, in _is_elf
    with open(filename, 'br') as thefile:
FileNotFoundError: [Errno 2] No such file or directory: b'liblibusb-1.0.a'

If the gcc produces only the list of files, then prepending ^ as looking for whole line could fix the issue.
expr = os.fsencode(r'[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name))
to
expr = os.fsencode(r'^[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
expert-ctypes type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@erlend-aasland @cz6ace @relativisticelectron