[QST] Handling of missing CUDA driver (runtime loading vs linking)

[andrea-cimatoribus-pix4d]

Hi, and thanks for your work on this library. I am starting to experiment with it, and am getting seriously interested, as a potential replacement for much larger and difficult to handle dependencies.

I have however a question concerning the use of MatX on machines without a NVIDIA driver. It might sound silly, but it's actually a very common situation when integrating such a library in an application. Right now, my understanding is that such an application would fail to start, because the NVIDIA driver is to be linked directly, judging from your cmake and from the fact that I cannot find any occurence of dlopen or similar.

I was wondering how intentional this situation is, if you plan to change it or if on the other hand there are motivations against loading the driver only at runtime. I would consider proposing a patch for that, in case there's interest, even if right now I only started scratching the surface of the library, so that wouldn't be a short-term action.

Thank you.

[cliffburdick]

Hi @andrea-cimatoribus-pix4d, thanks for the interest. MatX requires CUDA to be installed right now to compile, but it does not require you to have a GPU present on the system (unless of course you're trying to execute on a GPU). The most common way to handle this when building is to link against the CUDA stub files that are included with the driver. This allows it to link properly, then at runtime it will dynamically link to the real CUDA driver files. This is what you'd do if you want to build on a system with no GPU, but deploy it on a system with a GPU. If all you want to do is run the host executor and execute on a CPU. you can link against the stubs and don't worry about running with a GPU. Does that help?

[andrea-cimatoribus-pix4d]

Hi @cliffburdick and thank you very much for your reply. I do link against the stubs, but in my tests I was observing that the driver was being loaded unconditionally, which would still lead to a failure if no driver is present. Your answer however makes me strongly suspect that my test is somehow flawed. I'll dig further, and report back in case I identify a more specific issue. Otherwise, I'll close the question as solved.

Thanks again!

[andrea-cimatoribus-pix4d]

Unless I still misunderstand something, I think that the situation is a bit more subtle than I thought.

MatX being header-only, one can selectively use pieces of it that do not rely on the NVIDIA driver without any issue. That's already great.

If however one tries to use CUDA-dependent parts of MatX, the library doesn't seem to have any check for a missing driver, libcuda is expected unconditionally (directly by the linker, not by any MatX code). However, this is not such a big deal in the end: I can add a thin wrapper around MatX that loads it at runtime if possible, and fails gracefully otherwise. This makes it possible to use MatX linked against the driver stubs in an application that might run on a machine with no actual NVIDIA GPU.

Having some sort of dynamic check for the driver presence in MatX would still be nice, I believe, but I am not sure if it would add significant complexity for only limited benefit. I'm also not sure if it would be possible at all in a header-only context, I don't have much experience on this.

In any case, I don't think this is worth a proper feature request at this stage. Maybe just something to keep on the (mental) radar for future developments.

Thanks for your help 👍

[cliffburdick]

Hi @andrea-cimatoribus-pix4d that's a reasonable request. Can you open an issue detailing your use case?

[andrea-cimatoribus-pix4d]

I plan to do that, but I want to provide also a tentative patch, thanks.