Jul 24, 2023, 9:15 AM
@Jachyn 好
CM5如何安装网卡驱动？
-
我在装mellanox的时候遇到无法识别linux内核头文件的问题，这个该怎么解决呢？希望可以解答。 -
@Jake 这个需要把内核的头文件编译生成DEB包，然后系统上安装。
-
@george 好的，感谢，我去试试
-
This post is deleted! -
@george
目前的错误提示如上图所示，问一下怎么把内核的头文件编译生成deb包呢？
-
@Jachyn 我们提供的内核源码，你下载一份，先编译一次。
-
@大法师 大师 内核已经编译了 编译之后进行安装 又报如下的错
-
@Jachyn 你用的网卡驱动源码提供一个链接
-
@jack_admin https://network.nvidia.com/products/ethernet-drivers/linux/mlnx_en/
The NVIDIA ConnectX adapter family can run Ethernet traffic simultaneously on two ports Single software stack that operates across all available NVIDIA Ethernet devices and configurations such as mem-free, up to 400Gb/s, and PCI Express modes 3.0, 4.0 and 5.0 Support for HPC applications for scientific research and AI Support for data center applications such as Oracle 11g/10g RAC, IBM DB2, Financial services applications such as IBM WebSphere LLM, Red Hat MRG, NYSE Data Fabric -
@Jachyn said in CM5如何安装网卡驱动？:
@大法师 大师 内核已经编译了 编译之后进行安装 又报如下的错
更新内核源码编译目录链接（假定源码目录在/home/coolpi/kernel）
sudo ln -sf /home/coolpi/kernel /lib/modules/5.10.110/build sudo ln -sf /home/coolpi/kernel /lib/modules/5.10.110/sourceMLNX_EN_SRC-debian-23.04-1.1.3.0.tgz
MLNX_EN_SRC-debian-5.8-3.0.7.0.tgz
解压网卡驱动源码，编译coolpi@coolpi:~/MLNX_EN_SRC-23.04-1.1.3.0$ sudo ./install.pl Logs dir: /tmp/OFED.80654.logs General log file: /tmp/OFED.80654.logs/general.log Below is the list of OFED packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-ofed-kernel-utils mlnx-ofed-kernel-modules Checking SW Requirements... One or more required packages for installing OFED-internal are missing. Attempting to install the following missing packages: swig make python3 chrpath gcc quilt debhelper dh-python python3-distutils This program will install the OFED package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with OFED, do not reinstall them. Removing old packages get_requires: libibverbs1 is required by qemu-system-arm, but qemu-system-arm won't be removed. get_requires: librdmacm1 is required by qemu-system-arm, but qemu-system-arm won't be removed. Building packages Building DEB for ofed-scripts-23.04 (ofed-scripts)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ofed-scripts-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/MLNX_EN_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ofed-scripts_23.04-OFED.23.04.1.1.3_arm64.deb' Building DEB for mlnx-tools-23.04 (mlnx-tools)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mlnx-tools-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/MLNX_EN_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-tools_23.04-0.2304053_arm64.deb' Building DEB for mlnx-ofed-kernel-utils-23.04 (mlnx-ofed-kernel)... -W- --with-mlx5-ipsec is enabled Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules ... -
@大法师 更新源码目录链接显示如下错误
ln: /lib/modules/5.10.110/build/kernel: cannot overwrite directory ln: /lib/modules/5.10.110/source/kernel: cannot overwrite directory -
@Jachyn said in CM5如何安装网卡驱动？:
@大法师 更新源码目录链接显示如下错误
ln: /lib/modules/5.10.110/build/kernel: cannot overwrite directory ln: /lib/modules/5.10.110/source/kernel: cannot overwrite directorysudo unlink /lib/modules/5.10.110/build sudo unlink /lib/modules/5.10.110/source然后再操作。编译可能有错误，你先排查下
-
@大法师 编译内核使用命令
make Image -j4，然后编译模块make modules -j4，并没有报错，内核make modules_install也是成功的
当前日志显示coolpi@coolpi:~/Downloads/mlnx-en-23.04-0.5.3.3-ubuntu22.04-aarch64$ sudo ./install --force-dkms Logs dir: /tmp/mlnx-en.6425.logs General log file: /tmp/mlnx-en.6425.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-23.04... Installing mlnx-tools-23.04... Installing mlnx-en-utils-23.04... Installing mlnx-en-dkms-23.04... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 196397 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_23.04-0.5.3.3_arm64.deb ... Unpacking mlnx-fw-updater (23.04-0.5.3.3) ... Setting up mlnx-fw-updater (23.04-0.5.3.3) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.22.1002 14.32.1010 PXE 3.5.0403 3.6.0502 UEFI 14.15.0019 14.25.0017 Status: Update required --------- Found 1 device(s) requiring firmware update... Device #1: Updating FW ... Done Restart needed for updates to take effect. Log File: /tmp/C3m3Iz3_P3 Real log file: /tmp/mlnx-en.6425.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart -
@Jachyn 好
-
coolpi@coolpi:~/Desktop$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
@Jachyn 加 sudo 执行试下
-
coolpi@coolpi:~/Desktop$ sudo /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.9-0.5.6.0-ubuntu22.04-aarch64$ sudo ./install --add-kernel-support Note: This program will create mlnx-en TGZ for ubuntu22.04 under /tmp/mlnx-en-5.9-0.5.6.0-5.10.110 directory. See log file /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs/mlnx_ofed_iso.109096.log Checking if all needed packages are installed... Building mlnx-en DEBS . Please wait... ERROR: Failed executing "MLNX_EN_SRC-5.9-0.5.6.0/install.pl --tmpdir /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs --eth-only --kernel 5.10.110 --kernel-sources /lib/modules/5.10.110/build --builddir /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096 --without-dkms --without-debug-symbols --build-only --distro ubuntu22.04" ERROR: See /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs/mlnx_ofed_iso.109096.log Failed to build mlnx-en for 5.10.110是不是这系统就不支持这个驱动呢
-
@Jachyn said in CM5如何安装网卡驱动？:
coolpi@coolpi:~/Desktop$ sudo /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService执行这个之后，系统日志输出(dmesg)分析下错误原因。
-
@george 解决了这个驱动的问题，很感谢，但是我现在遇到了一个新的问题
我发现使用网卡后仍是pci进行操作的网卡，这个我该如何解决？ -
我是不是需要去内核修改一些相关的配置？
-
@Jake 有什么问题？网口节点出来了没
-
@jack_admin 用ifconfig/ip addr等命令查看并没有网卡，只有lspci | grep Eth可以看到有相关网卡pcie信息，但是查看pcie日志，并没有更换为网卡驱动
-
CMD: mlxup -u --log-on-update --ssl-certificate /tmp/SGKRmQP1We/mlxfwmanager_sriov_dis_aarch64_4115-dir/ca-bundle.crt --current-dir /opt/mellanox/mlnx-fw-updater/ -L /tmp/Nr5DC9bSSF -y -d 0001:11:00.0 Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: -- Description: PSID: PCI Device Name: 0001:11:00.0 Base MAC: N/A Versions: Current Available FW -- Status: Failed to open device --------- -E- Failed to query 0001:11:00.0 device, error : No such file or directory. MFE_ICMD_NOT_SUPPORTED EXIT_STATUS: 0安装网卡驱动后 显示如上的日志
-
coolpi@coolpi:~/Downloads/mlnx-en-5.8-1.0.1.1-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.738979.logs General log file: /tmp/mlnx-en.738979.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.8... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.8... Installing mlnx-en-dkms-5.8... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199866 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.8-1.0.1.1_arm64.deb ... Unpacking mlnx-fw-updater (5.8-1.0.1.1) ... Setting up mlnx-fw-updater (5.8-1.0.1.1) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/DR3i76jLzy Real log file: /tmp/mlnx-en.738979.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.8-1.0.1.1-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.3438.logs General log file: /tmp/mlnx-en.3438.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.8... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.8... Installing mlnx-en-dkms-5.8... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199866 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.8-2.0.3.0_arm64.deb ... Unpacking mlnx-fw-updater (5.8-2.0.3.0) ... Setting up mlnx-fw-updater (5.8-2.0.3.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/05iafkfb8h Real log file: /tmp/mlnx-en.3438.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ -
coolpi@coolpi:~/Downloads/mlnx-en-5.6-2.0.9.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.514532.logs General log file: /tmp/mlnx-en.514532.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.6... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.6... Installing mlnx-en-dkms-5.6... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199769 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.6-2.0.9.0_arm64.deb ... Unpacking mlnx-fw-updater (5.6-2.0.9.0) ... Setting up mlnx-fw-updater (5.6-2.0.9.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/07OcqgbJK0 Real log file: /tmp/mlnx-en.514532.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.6-2.0.9.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.7-1.0.2.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.759187.logs General log file: /tmp/mlnx-en.759187.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.7... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.7... Installing mlnx-en-dkms-5.7... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199851 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.7-1.0.2.0_arm64.deb ... Unpacking mlnx-fw-updater (5.7-1.0.2.0) ... Setting up mlnx-fw-updater (5.7-1.0.2.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/FFuM6_xJPq Real log file: /tmp/mlnx-en.759187.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.7-1.0.2.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
@Jachyn 需要把驱动加载过程，内核打印日志输出看看错误在哪，可以用dmesg命令
-
root@coolpi:/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0# ./install.pl Logs dir: /tmp/OFED.127412.logs General log file: /tmp/OFED.127412.logs/general.log Below is the list of OFED packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-ofed-kernel-utils mlnx-ofed-kernel-modules srp-modules mlnx-nvme-modules rdma-core libibverbs1 ibverbs-utils ibverbs-providers libibverbs-dev libibverbs1-dbg libibumad3 libibumad-dev ibacm librdmacm1 rdmacm-utils librdmacm-dev mstflint libibmad5 libibmad-dev libibnetdisc5 infiniband-diags kernel-mft-modules perftest ibsim ibsim-doc ucx openmpi mpitests knem-modules srptools mlnx-ethtool mlnx-iproute2 rshim ibarr Checking SW Requirements... One or more required packages for installing OFED-internal are missing. Attempting to install the following missing packages: build-essential autotools-dev pkg-config python3-docutils debhelper libstdc++6 pandoc bison dh-autoreconf devscripts libmnl-dev fakeroot libgfortran5 cython3 libnuma-dev libnl-route-3-dev libssl-dev python3-dev libnl-3-dev autoconf quilt libfuse-dev python3-distutils flex libsystemd-dev libselinux1-dev libudev-dev dh-python cmake libpci-dev sudo make libdb-dev libusb-1.0-0-dev libelf-dev libtool valgrind python3 gcc bzip2 This program will install the OFED package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with OFED, do not reinstall them. Removing old packages get_requires: libibverbs1 is required by qemu-system-arm, but qemu-system-arm won't be removed. get_requires: librdmacm1 is required by qemu-system-arm, but qemu-system-arm won't be removed. Error: One or more packages depends on MLNX_OFED. Those packages should be removed before uninstalling MLNX_OFED: licensecheck libarray-intspan-perl To force uninstallation use '--force' flag. root@coolpi:/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0# ./install.pl --force Logs dir: /tmp/OFED.129276.logs General log file: /tmp/OFED.129276.logs/general.log Below is the list of OFED packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-ofed-kernel-utils mlnx-ofed-kernel-modules srp-modules mlnx-nvme-modules rdma-core libibverbs1 ibverbs-utils ibverbs-providers libibverbs-dev libibverbs1-dbg libibumad3 libibumad-dev ibacm librdmacm1 rdmacm-utils librdmacm-dev mstflint libibmad5 libibmad-dev libibnetdisc5 infiniband-diags kernel-mft-modules perftest ibsim ibsim-doc ucx openmpi mpitests knem-modules srptools mlnx-ethtool mlnx-iproute2 rshim ibarr Checking SW Requirements... One or more required packages for installing OFED-internal are missing. Attempting to install the following missing packages: flex libelf-dev fakeroot autoconf autotools-dev dh-python libtool libfuse-dev sudo debhelper python3-docutils libnl-route-3-dev pandoc bison libselinux1-dev quilt python3 libgfortran5 build-essential valgrind libsystemd-dev libssl-dev cmake libnl-3-dev libudev-dev cython3 python3-distutils libdb-dev libpci-dev python3-dev devscripts bzip2 libmnl-dev gcc libstdc++6 pkg-config libnuma-dev libusb-1.0-0-dev make dh-autoreconf This program will install the OFED package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with OFED, do not reinstall them. Removing old packages get_requires: libibverbs1 is required by qemu-system-arm, but qemu-system-arm won't be removed. get_requires: librdmacm1 is required by qemu-system-arm, but qemu-system-arm won't be removed. Building packages Building DEB for ofed-scripts-23.04 (ofed-scripts)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ofed-scripts-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ofed-scripts_23.04-OFED.23.04.1.1.3_arm64.deb' Building DEB for mlnx-tools-23.04 (mlnx-tools)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mlnx-tools-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-tools_23.04-0.2304053_arm64.deb' Building DEB for mlnx-ofed-kernel-utils-23.04 (mlnx-ofed-kernel)... Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules Installing mlnx-ofed-kernel-utils-23.04... Running /usr/bin/dpkg -i --force-confnew --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-ofed-kernel-utils_23.04-OFED.23.04.1.1.3.1.kver.5.10.110_arm64.deb' Installing mlnx-ofed-kernel-modules-23.04... Running /usr/bin/dpkg -i --force-confnew --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-ofed-kernel-modules_23.04-OFED.23.04.1.1.3.1.kver.5.10.110_arm64.deb' Building DEB for srp-modules-23.04 (srp)... Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules Installing srp-modules-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/srp-modules_23.04-OFED.23.04.0.5.3.1.kver.5.10.110_arm64.deb' Building DEB for mlnx-nvme-modules-23.04 (mlnx-nvme)... Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules Installing mlnx-nvme-modules-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-nvme-modules_23.04-OFED.23.04.0.5.3.1.kver.5.10.110_arm64.deb' Building DEB for rdma-core-2304mlnx44 (rdma-core)... Running /usr/bin/dpkg-buildpackage -us -uc Installing rdma-core-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/rdma-core_2304mlnx44-1.2304113_arm64.deb' Installing libibverbs1-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibverbs1_2304mlnx44-1.2304113_arm64.deb' Installing ibverbs-utils-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibverbs-utils_2304mlnx44-1.2304113_arm64.deb' Installing ibverbs-providers-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibverbs-providers_2304mlnx44-1.2304113_arm64.deb' Installing libibverbs-dev-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibverbs-dev_2304mlnx44-1.2304113_arm64.deb' Installing libibverbs1-dbg-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibverbs1-dbg_2304mlnx44-1.2304113_arm64.deb' Installing libibumad3-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibumad3_2304mlnx44-1.2304113_arm64.deb' Installing libibumad-dev-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibumad-dev_2304mlnx44-1.2304113_arm64.deb' Installing ibacm-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibacm_2304mlnx44-1.2304113_arm64.deb' Installing librdmacm1-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/librdmacm1_2304mlnx44-1.2304113_arm64.deb' Installing rdmacm-utils-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/rdmacm-utils_2304mlnx44-1.2304113_arm64.deb' Installing librdmacm-dev-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/librdmacm-dev_2304mlnx44-1.2304113_arm64.deb' Building DEB for mstflint-4.16.1 (mstflint)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mstflint-4.16.1... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mstflint_4.16.1-2.2304053_arm64.deb' Installing libibmad5-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibmad5_2304mlnx44-1.2304113_arm64.deb' Installing libibmad-dev-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibmad-dev_2304mlnx44-1.2304113_arm64.deb' Installing libibnetdisc5-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/libibnetdisc5_2304mlnx44-1.2304113_arm64.deb' Installing infiniband-diags-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/infiniband-diags_2304mlnx44-1.2304113_arm64.deb' Building DEB for kernel-mft-modules-4.24.0 (kernel-mft)... Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules Installing kernel-mft-modules-4.24.0... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/kernel-mft-modules_4.24.0-72.kver.5.10.110_all.deb' Building DEB for perftest-23.04.0 (perftest)... Running /usr/bin/dpkg-buildpackage -us -uc Installing perftest-23.04.0... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/perftest_23.04.0-0.23.g63e250f.2304053_arm64.deb' Building DEB for ibsim-0.12 (ibsim)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ibsim-0.12... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibsim_0.12-1.2304053_arm64.deb' Installing ibsim-doc-0.12... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibsim-doc_0.12-1.2304053_all.deb' Building DEB for ucx-1.15.0 (ucx)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ucx-1.15.0... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ucx_1.15.0-1.2304053_arm64.deb' Building DEB for openmpi-4.1.5rc2 (openmpi)... Running mellanox_autodetect='yes' /usr/bin/dpkg-buildpackage -us -uc Installing openmpi-4.1.5rc2... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/openmpi_4.1.5rc2-1.2304053_all.deb' Building DEB for mpitests-3.2.20 (mpitests)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mpitests-3.2.20... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mpitests_3.2.20-de56b6b.2304053_arm64.deb' Building DEB for knem-modules-1.1.4.90mlnx2 (knem)... Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules Installing knem-modules-1.1.4.90mlnx2... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/knem-modules_1.1.4.90mlnx2-OFED.23.04.0.5.2.1.kver.5.10.110_arm64.deb' Installing srptools-2304mlnx44... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/srptools_2304mlnx44-1.2304113_arm64.deb' Building DEB for mlnx-ethtool-6.0 (mlnx-ethtool)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mlnx-ethtool-6.0... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-ethtool_6.0-1.2304053_arm64.deb' Building DEB for mlnx-iproute2-6.2.0 (mlnx-iproute2)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mlnx-iproute2-6.2.0... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-iproute2_6.2.0-1.2304053_arm64.deb' Building DEB for rshim-2.0.6 (rshim)... Running /usr/bin/dpkg-buildpackage -us -uc Installing rshim-2.0.6... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/rshim_2.0.6-19.g0873acd.2304113_arm64.deb' Building DEB for ibarr-0.1.3 (ibarr)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ibarr-0.1.3... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/Downloads/MLNX_OFED_LINUX-23.04-1.1.3.0-ubuntu22.04-aarch64/src/MLNX_OFED_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ibarr_0.1.3-1.2304053_arm64.deb' Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Installation passed successfully