さて今回は
OFED
https://www.openfabrics.org/linux-sources.html
こちらを使ってインストールしてみようかと。

OS・・・CentOS5.9 x64
Kernel・・・2.6.18-348.12.1.el5

結論から
OFED3.5はRDMAを使わなければ入りました。
OFED1.5系はエラーが出てしまいNG

OFED3.5

# ./install.pl --basic -vvv

Below is the list of OFED packages that you have chosen
(some may have been added by the installer due to package dependencies):
 
ofed-scripts
libibverbs
libibverbs-utils
libibverbs-devel
libmthca
libmlx4
libcxgb3
libcxgb4
libnes
librdmacm
librdmacm-utils
mstflint
ofed-docs
compat-rdma

Detected Linux Distribution: unsupported 

get_requires: libibverbs を必要とするパッケージは存在しません。 is required by libibverbs を必要とするパッケージは存在しません。
get_requires: libibverbs を必要とするパッケージは存在しません。 is required by libibverbs を必要とするパッケージは存在しません。
get_requires: libibverbs を必要とするパッケージは存在しません。 is required by libibverbs を必要とするパッケージは存在しません。
get_requires: libibverbs を必要とするパッケージは存在しません。 is required by libibverbs を必要とするパッケージは存在しません。
get_requires: libibverbs を必要とするパッケージは存在しません。 is required by libibverbs を必要とするパッケージは存在しません。

はっはー
まずエラー

問題を解決してあげる

# yum install libibverbs libibverbs-devel libibverbs-utils-1.1.6-1 

これだけだけど、依存で他のパッケージも入ります
(他にもあるかも・・・)

さて再度実行

# ./install.pl --basic -vvv

Below is the list of OFED packages that you have chosen
(some may have been added by the installer due to package dependencies):
 
ofed-scripts
libibverbs
libibverbs-utils
libibverbs-devel
libmthca
libmlx4
libcxgb3
libcxgb4
libnes
librdmacm
librdmacm-utils
mstflint
ofed-docs
compat-rdma

Detected Linux Distribution: unsupported 
Uninstalling the previous version of OFED 
　　　・
　　　・
　　　・
Build compat-rdma RPM
Running rpmbuild --rebuild  --define '_topdir /var/tmp//OFED_topdir' --nodeps --define '_dist .unsupported' --define 'configure_options   --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod --with-cxgb3-mod --with-cxgb4-mod --with-nes-mod --with-ipoib-mod' --define 'build_kernel_ib 1' --define 'build_kernel_ib_devel 1' --define 'KVERSION 2.6.18-348.12.1.el5' --define 'K_SRC /lib/modules/2.6.18-348.12.1.el5/build' --define '_release 2.6.18_348.12.1.el5' --define 'network_dir /etc/sysconfig/network-scripts' --define '_prefix /usr' --define '__arch_install_post %{nil}' /tmp/OFED-3.5/SRPMS/compat-rdma-3.5-OFED.3.5.src.rpm
Failed to build compat-rdma RPM 
See /tmp/OFED.6227.logs/compat-rdma.rpmbuild.log 

RDMA周りでエラーが出てしまいました。

エラー詳細を確認

# view /tmp/OFED.6227.logs/compat-rdma.rpmbuild.log


　　　・
　　　・
　　　・
include/linux/if.h:71:1: warning: this is the location of the previous definition
In file included from /var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-2.6.h:60,
                 from <command line>:1:
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.1.h: At top level:
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.1.h:113: warning: 'struct ida' declared inside parameter list
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.1.h:114: warning: 'struct ida' declared inside parameter list
In file included from /var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-2.6.h:61,
                 from <command line>:1:
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:99: error: redefinition of 'skb_frag_size'
include/linux/skbuff.h:140: error: previous definition of 'skb_frag_size' was here
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:104: error: redefinition of 'skb_frag_size_set'
include/linux/skbuff.h:145: error: previous definition of 'skb_frag_size_set' was here
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:109: error: redefinition of 'skb_frag_size_add'
include/linux/skbuff.h:150: error: previous definition of 'skb_frag_size_add' was here
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:114: error: redefinition of 'skb_frag_size_sub'
include/linux/skbuff.h:155: error: previous definition of 'skb_frag_size_sub' was here
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h: In function 'hex_byte_pack':
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:133: error: called object 'hex_asc' is not a function
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.2.h:134: error: called object 'hex_asc' is not a function
In file included from /var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-2.6.h:62,
                 from <command line>:1:
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.3.h: At top level:
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.3.h:76: warning: 'struct netdev_queue' declared inside parameter list
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.3.h:81: warning: 'struct netdev_queue' declared inside parameter list
/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/include/linux/compat-3.3.h:85: warning: 'struct netdev_queue' declared inside parameter list
make[3]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat/main.o] Error 1
make[2]: *** [/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5/compat] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/compat-rdma-3.5] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-348.12.1.el5-x86_64'
make: *** [kernel] Error 2
エラー: /var/tmp/rpm-tmp.60346 の不正な終了ステータス (%build)

すみません。
僕には解決出来そうにない。。。

解決法が分からないので
とりあえず、RDMAを無効にしてみます。

# vi ofed.conf

libibverbs=y
libibverbs-utils=y
libmthca=y
libmlx4=y
libcxgb3=y
libcxgb4=y
libnes=y
librdmacm=y
librdmacm-utils=y
mstflint=y
ofed-docs=y
ofed-scripts=y
#compat-rdma=y
core=y
mthca=y
mlx4=y
mlx4_en=y
cxgb3=y
cxgb4=y
nes=y
ipoib=y

無効化してあげるだけ。

さて再々度実行

# ./install.pl -c ofed.conf -vvv

Below is the list of OFED packages that you have chosen
(some may have been added by the installer due to package dependencies):
 
ofed-scripts
libibverbs
libibverbs-utils
libmthca
libmlx4
libcxgb3
libcxgb4
libnes
libibverbs-devel
librdmacm
librdmacm-utils
mstflint
ofed-docs

Detected Linux Distribution: unsupported 
Uninstalling the previous version of OFED 
Install ofed-scripts RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/ofed-scripts-3.5-OFED.3.5.x86_64.rpm
Install libibverbs RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libibverbs-1.1.6-1.x86_64.rpm
Install libibverbs-utils RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libibverbs-utils-1.1.6-1.x86_64.rpm
Install libmthca RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libmthca-1.0.5-0.1.gbe5eef3.x86_64.rpm
Install libmlx4 RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libmlx4-1.0.4-1.x86_64.rpm
Install libcxgb3 RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libcxgb3-1.3.1-1.x86_64.rpm
Install libcxgb4 RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libcxgb4-1.2.0-1.x86_64.rpm
Install libnes RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libnes-1.1.3-1.x86_64.rpm
Install libibverbs-devel RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/libibverbs-devel-1.1.6-1.x86_64.rpm
Install librdmacm RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/librdmacm-1.0.16-1.x86_64.rpm
Install librdmacm-utils RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/librdmacm-utils-1.0.16-1.x86_64.rpm
Install mstflint RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/mstflint-1.4-1.23.g5c2032a.x86_64.rpm
Install ofed-docs RPM:
Running rpm -iv  /tmp/OFED-3.5/RPMS/centos-release-5-9.el5.centos.1/x86_64/ofed-docs-3.5-OFED.3.5.x86_64.rpm
Device (15b3:673c):
	01:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
	Link Width: 8x
	PCI Link Speed: 2.5Gb/s


Installation finished successfully. 

入りましたね。
（僕はRDMAが使いたいので諦めてCentOS6を入れます・・・）

OFED1.5.4
pci_pcie_capでエラー

/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4/kernel_addons/backport/2.6.18-EL5.7/include/linux/pci.h:164: error: conflicting types for 'pci_pcie_cap'
include/linux/pci.h:1045: error: previous definition of 'pci_pcie_cap' was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4/drivers/infiniband/core/notice.o] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4/drivers/infiniband/core] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4/drivers/infiniband] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.4] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-348.12.1.el5-x86_64'
make: *** [kernel] Error 2
エラー: /var/tmp/rpm-tmp.55278 の不正な終了ステータス (%build)

OFED-1.5.3
VLANでエラー？

In file included from include/net/neighbour.h:55,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/net/neighbour.h:4,
                 from include/net/arp.h:6,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/drivers/infiniband/core/addr.c:39:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/linux/seq_file.h:8: error: static declaration of 'seq_list_start' follows non-static declaration
include/linux/seq_file.h:60: error: previous declaration of 'seq_list_start' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/linux/seq_file.h:19: error: static declaration of 'seq_list_start_head' follows non-static declaration
include/linux/seq_file.h:62: error: previous declaration of 'seq_list_start_head' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/linux/seq_file.h:27: error: static declaration of 'seq_list_next' follows non-static declaration
include/linux/seq_file.h:64: error: previous declaration of 'seq_list_next' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/linux/seq_file.h:37: error: conflicting types for '__seq_open_private'
include/linux/seq_file.h:49: error: previous declaration of '__seq_open_private' was here
In file included from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/include/rdma/ib_addr.h:45,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/drivers/infiniband/core/addr.c:45:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/kernel_addons/backport/2.6.18-EL5.6/include/linux/if_vlan.h:17: error: redefinition of 'is_vlan_dev'
include/linux/if_vlan.h:165: error: previous definition of 'is_vlan_dev' was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/drivers/infiniband/core/addr.o] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/drivers/infiniband/core] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3/drivers/infiniband] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.3] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-348.12.1.el5-x86_64'
make: *** [kernel] Error 2
エラー: /var/tmp/rpm-tmp.59045 の不正な終了ステータス (%build)

OFED-1.5.2
VLANでエラー？

include/linux/if_ether.h:125: error: previous declaration of 'print_mac' was here
In file included from include/net/neighbour.h:55,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/net/neighbour.h:4,
                 from include/net/arp.h:6,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/drivers/infiniband/core/addr.c:39:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:8: error: static declaration of 'seq_list_start' follows non-static declaration
include/linux/seq_file.h:60: error: previous declaration of 'seq_list_start' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:19: error: static declaration of 'seq_list_start_head' follows non-static declaration
include/linux/seq_file.h:62: error: previous declaration of 'seq_list_start_head' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:27: error: static declaration of 'seq_list_next' follows non-static declaration
include/linux/seq_file.h:64: error: previous declaration of 'seq_list_next' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:37: error: conflicting types for '__seq_open_private'
include/linux/seq_file.h:49: error: previous declaration of '__seq_open_private' was here
In file included from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/include/rdma/ib_addr.h:45,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/drivers/infiniband/core/addr.c:45:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:7: error: redefinition of 'vlan_group_get_device'
include/linux/if_vlan.h:152: error: previous definition of 'vlan_group_get_device' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:13: error: conflicting types for 'vlan_group_set_device'
include/linux/if_vlan.h:159: error: previous definition of 'vlan_group_set_device' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:28: error: redefinition of 'is_vlan_dev'
include/linux/if_vlan.h:165: error: previous definition of 'is_vlan_dev' was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/drivers/infiniband/core/addr.o] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/drivers/infiniband/core] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2/drivers/infiniband] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.2] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-348.12.1.el5-x86_64'
make: *** [kernel] Error 2
エラー: /var/tmp/rpm-tmp.28286 の不正な終了ステータス (%build)

OFED-1.5.2
VLANでエラー？

In file included from include/net/neighbour.h:55,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/net/neighbour.h:4,
                 from include/net/arp.h:6,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/drivers/infiniband/core/addr.c:39:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:8: error: static declaration of 'seq_list_start' follows non-static declaration
include/linux/seq_file.h:60: error: previous declaration of 'seq_list_start' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:19: error: static declaration of 'seq_list_start_head' follows non-static declaration
include/linux/seq_file.h:62: error: previous declaration of 'seq_list_start_head' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:27: error: static declaration of 'seq_list_next' follows non-static declaration
include/linux/seq_file.h:64: error: previous declaration of 'seq_list_next' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/seq_file.h:37: error: conflicting types for '__seq_open_private'
include/linux/seq_file.h:49: error: previous declaration of '__seq_open_private' was here
In file included from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/include/rdma/ib_addr.h:45,
                 from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/drivers/infiniband/core/addr.c:45:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:7: error: redefinition of 'vlan_group_get_device'
include/linux/if_vlan.h:152: error: previous definition of 'vlan_group_get_device' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:13: error: conflicting types for 'vlan_group_set_device'
include/linux/if_vlan.h:159: error: previous definition of 'vlan_group_set_device' was here
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/kernel_addons/backport/2.6.18-EL5.5/include/linux/if_vlan.h:28: error: redefinition of 'is_vlan_dev'
include/linux/if_vlan.h:165: error: previous definition of 'is_vlan_dev' was here
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/drivers/infiniband/core/addr.o] Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/drivers/infiniband/core] Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1/drivers/infiniband] Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.5.1] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-348.12.1.el5-x86_64'
make: *** [kernel] Error 2
エラー: /var/tmp/rpm-tmp.2554 の不正な終了ステータス (%build)

諦めたｗ

[tegaki]CentOS6を入れるｗ[/tegaki]