This repository was archived by the owner on Nov 17, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
RuntimeError: Cannot find the MXNet library. #20144
Description
Description
I am trying to build MxNet from source on my 32-bit Raspberry Pi 4. It is giving me the error mentioned below.
I think the bug is caused because it can't find libmxnet.so in mxnet/python/mxnet/. Here is the output of ls mxnet/python/mxnet/ from my home directory where I cloned the repo.
Hope someone has a solution :)
Error Message
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Obtaining file:///home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/setup.py'"'"'; __file__='"'"'/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-9ni2u7t1
cwd: /home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/
Complete output (12 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/setup.py", line 47, in <module>
LIB_PATH = libinfo['find_lib_path']()
File "/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/mxnet/libinfo.py", line 74, in find_lib_path
'List of candidates:\n' + str('\n'.join(dll_path)))
RuntimeError: Cannot find the MXNet library.
List of candidates:
/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/mxnet/libmxnet.so
/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/mxnet/../../lib/libmxnet.so
/home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python/mxnet/../../build/libmxnet.so
../../../libmxnet.so
----------------------------------------
WARNING: Discarding file:///home/pi/Downloads/apache-mxnet-src-1.4.0-incubating/python. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
To Reproduce
https://mxnet.apache.org/get_started/build_from_source#install-mxnet-for-python
Steps to reproduce
git clone --recursive https://github.com/apache/incubator-mxnet mxnetcd mxnetsudo apt-get updatesudo apt-get install -y build-essential git ninja-build ccache libopenblas-dev libopencv-dev cmakesudo apt install gfortranpython3 -m pip install --user -e ./python
What have you tried to solve it?
- Looking for
libmxnet.soin the directory. It was not there. - I tried downloading
libmxnet.sofrom https://github.com/awslabs/mxnet-lambda/blob/master/src/mxnet/libmxnet.so. Then I ran some commands in the terminal:mv Downloads/libmxnet.so mxnet/python/mxnet/cd mxnet/python/mxnet/lscd && cd mxnetpython3 -m pip install --user -e ./python
After running, it gives me the output that Successfully installed mxnet, but when I try to import mxnet in python3, it says: libf77blas.so.3: cannot open shared object file: No such file or directory
I have no clue what to do? 😕
Environment
We recommend using our script for collecting the diagnostic information with the following command
curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python3
Environment Information
----------Python Info----------
Version : 3.7.3
Compiler : GCC 8.3.0
Build : ('default', 'Jul 25 2020 13:03:44')
Arch : ('32bit', 'ELF')
------------Pip Info-----------
Version : 21.0.1
Directory : /home/pi/.local/lib/python3.7/site-packages/pip
----------MXNet Info-----------
An error occured trying to import mxnet.
This is very likely due to missing missing or incompatible library files.
Traceback (most recent call last):
File "<stdin>", line 96, in check_mxnet
AttributeError: module 'mxnet' has no attribute '__version__'
----------System Info----------
Platform : Linux-5.10.17-v7l+-armv7l-with-debian-10.8
system : Linux
node : raspberrypi
release : 5.10.17-v7l+
version : #1403 SMP Mon Feb 22 11:33:35 GMT 2021
----------Hardware Info----------
machine : armv7l
processor :
Architecture: armv7l
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Vendor ID: ARM
Model: 3
Model name: Cortex-A72
Stepping: r0p3
CPU max MHz: 1500.0000
CPU min MHz: 600.0000
BogoMIPS: 270.00
Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0104 sec, LOAD: 0.0684 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0031 sec, LOAD: 0.0587 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1056)>, DNS finished in 0.003855466842651367 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0018 sec, LOAD: 0.1406 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0050 sec, LOAD: 0.2012 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.006901264190673828 sec.
----------Environment----------
Activity
github-actions commentedon Apr 8, 2021
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.
rsinventor commentedon Apr 8, 2021
Update:
After a little more exploration, I found out that I forgot the compiling step, and that is why I can't find
libmxnet.so. Now I have a new bug to report.Here are the steps to reproduce:
git clone --recursive https://github.com/apache/incubator-mxnet mxnetcd mxnetsudo apt-get updatesudo apt-get install -y build-essential git ninja-build ccache libopenblas-dev libopencv-dev cmakesudo apt install gfortranmkdir buildcd build/cmake ..python3 -m pip install --user -e ./python@szha and @leezu Can you help me?
szha commentedon Apr 8, 2021
@Raunak-Singh-Inventor the error says nvcc not found, which means cmake is trying to look for CUDA support on your raspberry pi. I think the fix would be to simply turn the
USE_CUDAoff. In fact, you should be able to rely on the linux CPU build config for building on rpi. You can copy it to the root of mxnet and name it config.cmake to make it active.rsinventor commentedon Apr 8, 2021
Thanks for the reply @szha.
After adding
config.cmaketo the root of mxnet, I am getting a new error in the compilation:rsinventor commentedon Apr 8, 2021
I am also including the CMakeOutput.log:
CMakeOutput.log
rsinventor commentedon Apr 8, 2021
@szha I still have an error. Can you help me?
leezu commentedon Apr 8, 2021
You can update the config file to disable onednn
rsinventor commentedon Apr 9, 2021
@leezu Disabling
onednnfixed it. I and getting a new error now when runningcmake --build .rsinventor commentedon Apr 9, 2021
Also, here are some stats:
command:
free -hcommand:
df -hleezu commentedon Apr 9, 2021
You may not have enough memory to use parallel build. Try
cmake --build . -j 1Btw, it will be much faster for you to cross-compile libmxnet from your dev desktop, which likely has faster cpu and more memory then the RPi. (Assuming that you'd not be afraid to learn about cross-compile)
Cross-compile build env:
https://github.com/apache/incubator-mxnet/blob/8397422aeb4b09fa464b824bfe769f5fd8cb7969/ci/docker/Dockerfile.build.arm#L130-L150
Cross-compile build command
https://github.com/apache/incubator-mxnet/blob/8397422aeb4b09fa464b824bfe769f5fd8cb7969/ci/docker/runtime_functions.sh#L203-L211
rsinventor commentedon Apr 9, 2021
I am cross-compiling
libmxnetfrom my 64-bit windows desktop. I am setting up the build environment, and am using thedebian:busterbase image https://hub.docker.com/_/debian. When I build the environment Dockerfile, I get this error:Here is my Dockerfile with the base image of
debian:buster. I had to change the Dockerfile a bit to get it to run (installing pip, python3, cmake):@leezu Can you give me a complete build env dockerfile? It would be helpful.
leezu commentedon Apr 9, 2021
@Raunak-Singh-Inventor the complete dockerfile is linked above. It's tested on CI every day. Also note you're not required to use Docker here. I'm just providing it for your reference so you can see the dependencies. Generally you can use any Linux system to do the cross-compilation. I'd discourage the use of Windows.