Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question on building a hybrid C/C++ extension. #21

Open
mreineck opened this issue Apr 12, 2017 · 11 comments
Open

Question on building a hybrid C/C++ extension. #21

mreineck opened this issue Apr 12, 2017 · 11 comments

Comments

@mreineck
Copy link

@mreineck mreineck commented Apr 12, 2017

While trying to use pybind11 for interfacing a hybrid C/C++ codebase, following your example project, I encountered the following problem:

If the source files are a mix of C and C++ files, some compilers (most notably clang) complain and stop if the "-std=c++14" flag is supplied when compiling a C file.

This is of course completely reasonable, but even after extensive searching on the web I did not find a way to specify separate C and C++ compilation flags to setuptools (in analogy to CFLAGS and CXXFLAGS use by autotools).

Do you happen to know a trick how to achieve this? If so, I think it would be great if this could be added to your test project, to illustrate how it is done!

@MathMagique
Copy link

@MathMagique MathMagique commented Sep 8, 2017

I do not know if it is possible for your use case, but you could try to separate the C code into a separate extension module. So you would have a C extension module and a C++ extension module that links to the C module. Then you can use separate flags for each module.

In my project, I use multiple extension modules for other reasons, but also require linking between them: https://github.com/blue-yonder/turbodbc/blob/master/setup.py#L116 (get_extension_modules() function).

@mreineck
Copy link
Author

@mreineck mreineck commented Sep 9, 2017

I'm not sure whether that works in my case. The problem is that the C library does not provide any Python interface, but is exclusively needed by the C++ code; so I would basically have to "misuse" the setup.py machinery to build/install a package with no Python interface at all. The problem will probably be the C header files, which the C++ library has to locate somehow.

I'll give it a try!

@MathMagique
Copy link

@MathMagique MathMagique commented Sep 9, 2017

My project has a libturbodbc "extension" that does not expose Python stuff at all. The header files are easily found since you have to put it in the package anyways. I think you will be fine!

@mreineck
Copy link
Author

@mreineck mreineck commented Sep 10, 2017

Thanks a lot! Looking at your setup.py has helped me write my own version, which works almost perfectly now.

The only remaining problem is that pybind11.get_include() reports incorrect paths ... I installed it via 'pip install --user', but it gives me a path of "/usr/local/include/python2.7", where there are no headers, of course. But that's a different issue...

@dean0x7d
Copy link
Member

@dean0x7d dean0x7d commented Sep 10, 2017

Use pybind11.get_include(True) to get the --user install path.

@SylvainCorlay
Copy link
Member

@SylvainCorlay SylvainCorlay commented Sep 10, 2017

@mreineck I should point out that user install are generally evil 👿, especially for python packages, jupyter extensions, and headers!

The reason is that any python installation would pick up these with a higher precedence than the ones of their prefix.

@dean0x7d
Copy link
Member

@dean0x7d dean0x7d commented Sep 10, 2017

I should point out that user install are generally evil

Less evil than sudo installs...

@SylvainCorlay
Copy link
Member

@SylvainCorlay SylvainCorlay commented Sep 10, 2017

Well, the right way to do things is a prefix install...

@mreineck
Copy link
Author

@mreineck mreineck commented Sep 11, 2017

I see that I'll need a lot more time before I grasp all the intricacies of Python packages ... but for the moment I think I have all the ingredients I need! Thank you all for the help!

@benmaier
Copy link

@benmaier benmaier commented Mar 29, 2018

short off-topic question concerning the --user install comment by @SylvainCorlay

I'm ususally compiling/installing my custom pybind11-packages on a sun grid engine compute cluster, where python is installed at root level using several specifically compiled libraries, e.g. LBLAS, etc. So I'm not going to install a new python environment. I found installing via --user extremely useful since I obviously don't have root permission, I only need to install it once and it works for every node (user directories are shared between compute nodes), and I can still use the packages installed at root level.
Is there an alternative, more canonic way to do what I need to do there? Otherwise I'd say --user is pretty useful in this case.

@SylvainCorlay
Copy link
Member

@SylvainCorlay SylvainCorlay commented Mar 29, 2018

User installs only start being an issue when you use other environments than the root one, so you should be fine.

As soon as you have more than one, it start being nearly impossible to keep a consistent state with user installs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.