English dropdown-ico

Cisco GEM318P manual

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

Go to page of

Similar user manuals

A good user manual

The rules should oblige the seller to give the purchaser an operating instrucion of Cisco GEM318P, along with an item. The lack of an instruction or false information given to customer shall constitute grounds to apply for a complaint because of nonconformity of goods with the contract. In accordance with the law, a customer can receive an instruction in non-paper form; lately graphic and electronic forms of the manuals, as well as instructional videos have been majorly used. A necessary precondition for this is the unmistakable, legible character of an instruction.

What is an instruction?

The term originates from the Latin word „instructio”, which means organizing. Therefore, in an instruction of Cisco GEM318P one could find a process description. An instruction's purpose is to teach, to ease the start-up and an item's use or performance of certain activities. An instruction is a compilation of information about an item/a service, it is a clue.

Unfortunately, only a few customers devote their time to read an instruction of Cisco GEM318P. A good user manual introduces us to a number of additional functionalities of the purchased item, and also helps us to avoid the formation of most of the defects.

What should a perfect user manual contain?

First and foremost, an user manual of Cisco GEM318P should contain:
- informations concerning technical data of Cisco GEM318P
- name of the manufacturer and a year of construction of the Cisco GEM318P item
- rules of operation, control and maintenance of the Cisco GEM318P item
- safety signs and mark certificates which confirm compatibility with appropriate standards

Why don't we read the manuals?

Usually it results from the lack of time and certainty about functionalities of purchased items. Unfortunately, networking and start-up of Cisco GEM318P alone are not enough. An instruction contains a number of clues concerning respective functionalities, safety rules, maintenance methods (what means should be used), eventual defects of Cisco GEM318P, and methods of problem resolution. Eventually, when one still can't find the answer to his problems, he will be directed to the Cisco service. Lately animated manuals and instructional videos are quite popular among customers. These kinds of user manuals are effective; they assure that a customer will familiarize himself with the whole material, and won't skip complicated, technical information of Cisco GEM318P.

Why one should read the manuals?

It is mostly in the manuals where we will find the details concerning construction and possibility of the Cisco GEM318P item, and its use of respective accessory, as well as information concerning all the functions and facilities.

After a successful purchase of an item one should find a moment and get to know with every part of an instruction. Currently the manuals are carefully prearranged and translated, so they could be fully understood by its users. The manuals will serve as an informational aid.

Table of contents for the manual

  • Page 1

    Americas Headquarters Cisco Systems, In c. 170 West Tasman Drive San Jose, CA 951 34-1706 USA http://www.ci sco.com Tel: 408 526-4000 800 553-NETS (638 7) Fax: 408 527-0883 Cisco SFS Inf iniBand Host Driv er s User Guide f or Linux Re lease 3.2.0 June 20 07 Text Part Number: OL -12309-01[...]

  • Page 2

    THE SPECIFICATION S AND INFORMAT ION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT T O CHANGE WITH OUT NOTICE. A LL STATEMENTS , INFORMATION, AND RECOMMENDATI ONS IN THI S MANUAL ARE BE LIEVED TO BE A CCURATE BUT ARE PRESENTED WI THOUT WARRANTY OF ANY KIND, EX PRESS OR IMPLIED. USERS MUST TAKE FULL RESPO NSIBILITY FOR THEIR APPLICAT ION OF ANY [...]

  • Page 3

    iii Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 CONTENTS Preface vii Audience vii Organization vii Conventi ons viii Root and Non-root Con ventions in Examples ix Related Documentation ix Obtaining Documentation, Obtaining Support, and Security Guid elines ix CHAPTER 1 About Host Drivers 1-1 Introduction 1-1 Architecture 1-2[...]

  • Page 4

    Contents iv Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Subinterfaces 3-2 Creating a Subinterface Associated with a Specific IB Partition 3-3 Removing a Subinterface Associated with a Spe cific IB Partition 3-4 Verifying IPoIB Functionality 3-5 IPoIB Performance 3-6 Sample Startup Config uration File 3-8 IPoIB High Availabil[...]

  • Page 5

    Contents v Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 CHAPTER 7 MVAPICH MPI 7-1 Introduction 7-1 Initial Setup 7-2 Configuring SSH 7-2 Editing Environment Variables 7-5 Setting Environment Variables in System-Wide Startup File s 7-6 Editing Environment Variables in the Users Shell Startup Files 7-6 Editing Environment Varia[...]

  • Page 6

    Contents vi Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01[...]

  • Page 7

    vii Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 Preface This preface describes who shoul d read the Cisco SFS Infi niBand Host Drivers User Guide fo r Linux , ho w it is or ganized, and it s document con v entions. It in cludes the follo wing section s: • Audience, pag e vii • Org anization, page vii • Con v entions, p[...]

  • Page 8

    viii Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Preface Conventions Conventions This document uses the f ollo wing con ventions: Chapter 8 HCA Utilities and Diag nostics Describes the fundamen tal HCA utilities and diagnostics. Appendix A Acronyms an d Abbre viations Defines the acron yms and abbre viations that are used in[...]

  • Page 9

    ix Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 Preface Root and Non-root Conventions in E xamples Notes use the follo wing con vention: Note Means reader take note . Notes contain helpful suggestions or references to material not co vered in the manual. Cautions use the following con v ention: Caution Means reader be car efu[...]

  • Page 10

    x Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Preface Obtaining Documentation, Obtaining Support, and Security G uidelines[...]

  • Page 11

    CH A P T E R 1-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 1 About Host Drivers This chapter describes host driv ers and includes the following sections: • Introduction, page 1-1 • Architecture, page 1-2 • Supported Protoc ols, page 1-3 • Supported APIs, page 1-4 • HCA Utilities and D iagnostics, page 1-4 Note For[...]

  • Page 12

    1-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 1 Abo ut Host Drivers Architecture Note See the “Root and Non-root Con v entions in Examples” section on page ix for details about the significance of prompt s used in the e xamples in this chapter . Architecture Figure 1-1 displays the software architect ure of the[...]

  • Page 13

    1-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 1 About Hos t Drivers Supported Protocols Supported Protocols This section des cribes the suppo rted protocols an d includes the fol lowing topics: • IPoIB • SRP • SDP Protocol here refers to software in the networking layer in kernel space. IPoIB The IPoIB protoco[...]

  • Page 14

    1-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 1 Abo ut Host Drivers Supported APIs Supported APIs This section descri bes the supported APIs and includes t he follo wing topics: • MV APICH MPI • uD APL • Intel MPI • HP MPI API refers to software in the network ing layer in user space. MVAPICH MPI MPI is a s[...]

  • Page 15

    CH A P T E R 2-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 2 Installing Host Drivers The chapter includes th e following sections: • Introduction, page 2-1 • Contents of ISO Image, page 2-2 • Installing Host Driv ers from an ISO Image, page 2-2 • Uninstalling Host Drivers from an ISO Image, page 2-3 Note See the “[...]

  • Page 16

    2-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 2 Installing Host Drivers Contents of ISO Image Contents of ISO Image The ISO image contains the follo wing directori es and files: • docs/ This directory contains the related documents. • tsinstall This is the inst allation scri pt. • redhat/ This directory conta[...]

  • Page 17

    2-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 2 In stalling Host Drivers Uninstalling Host Drivers from an ISO Image topspin-ib-mpi-rhel4-3.2.0-136.x86_64 (MPI libraries, source code, docs, etc) topspin-ib-mod-rhel4-2.6.9-34.ELsmp-3.2.0-136.x86_64 (kernel modules) installing 100% ####################################[...]

  • Page 18

    2-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 2 Installing Host Drivers Uninstalling Host Drivers from an ISO Image[...]

  • Page 19

    CH A P T E R 3-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 3 IP over IB Protocol This chapter describe s IP over IB protocol and includes the following sections : • Introduction, page 3-1 • Manually Conf iguring IPoIB for Def ault IB Partit ion, page 3-2 • Subinterfaces, page 3- 2 • V erifying IPoIB Functi onality ,[...]

  • Page 20

    3-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 3 IP over IB Protocol Manually Configuring IPoI B for Default IB Part ition Manually Configuring IPoI B for Default IB Partition T o manually conf igure IPoIB for the def ault IB partition, perform th e follo wing steps: Step 1 Log in to your Linu x host. Step 2 T o con[...]

  • Page 21

    3-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 3 IP over IB Protocol Subinterfaces Creating a Subinterface Associated with a Specific IB Partition T o create a subinterface associat ed with a specific IB partition , perform the follo wing steps: Step 1 Create a partition on an IB SFS. Alternativ ely , you can choose [...]

  • Page 22

    3-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 3 IP over IB Protocol Subinterface s NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) V erify that you see the ib0.8002 output.[...]

  • Page 23

    3-5 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 3 IP over IB Protocol Verifying IPoIB Functionality TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1024 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP L[...]

  • Page 24

    3-6 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 3 IP over IB Protocol IPoIB Performa nce IPoIB Performance This section describes ho w to verify IPoIB performance by ru nning the Bandwidth test and the Latenc y test. These tests are described in de tail at the fo llowing URL: http://www .netperf.or g/netperf/t rainin[...]

  • Page 25

    3-7 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 3 IP over IB Protocol IPoIB Performance Step 5 Run the Netperf Latenc y test. Run the test once, and stop the server so that it does no t repeat the test. The follo wing e xample sho ws ho w to run the Latency test, and then stop the Netperf serv er: host2$ netperf -H 19[...]

  • Page 26

    3-8 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 3 IP over IB Protocol Sample Startup Configuration File Sample Startup Configuration File IP addresses that are con figur ed manually are not pe rsistent across reboots. Y ou must use a conf iguration file to config ure IPoIB when the host boots. T wo samp le config ura[...]

  • Page 27

    3-9 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 3 IP over IB Protocol IPoIB High Availability Step 3 T ake the i nterfaces of fline. Y ou cannot mer ge interfaces u ntil you bring them down. The follo wing e xample sho ws ho w to take the interf aces off line: host1# ifconfig ib0 down host1# ifconfig ib1 down Step 4 M[...]

  • Page 28

    3-10 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 3 IP over IB Protocol IPoIB High Availability Step 3 Display the a vai lable interfaces b y entering the ipoibcfg list command. The follo wing example sho ws how to display the a v ailable interfaces: host1# /usr/local/topspin/sbin/ipoibcfg list ib0 (P_Key 0xffff) (SL:[...]

  • Page 29

    CH A P T E R 4-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 4 SCSI RDMA Protocol This chapter describe s SCSI RDMA protocol and includes the following sections: • Introduction, page 4-1 • Config uring SRP , page 4-1 • V erifying SRP , page 4-7 Note See the “Root and Non-root Con v entions in Examples” section on pa[...]

  • Page 30

    4-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 4 SCSI RDMA Protocol Configuring SRP This section contains informati on on how to co nfigu re your IB fabric to connect an SRP host to a SAN and includes the follo wing topics: • Config uring ITLs when Using Fibr e Channel Gate way , page 4-2 • Config uring SRP Host[...]

  • Page 31

    4-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 4 SCSI RDMA Protocol Configuring SRP Step 3 Bring up the Fibre Channel gate ways on yo ur SFS, b y performing the follo wing steps: a. Launch Eleme nt Manager . b. Double-click the Fibre Chan nel gate way card that you w ant to bring up. The Fibre Channel Card windo w op[...]

  • Page 32

    4-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 4 SCSI RDMA Protocol Configuring SRP Configuring ITLs with Element Manager wh ile Global Policy Restrictions Apply This section descri bes how to conf igure ITLs with Element Manager while global po licy restrictions apply . These instructions apply to en vironmen ts wh[...]

  • Page 33

    4-5 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 4 SCSI RDMA Protocol Configuring SRP Step 9 Click the Next > b utton. The Def ine Ne w SRP Host windo w displays a recommend ed WWNN for the host and recommended WWPNs that represent the host on all exist ing and potential Fibre Channel gatew ay ports. Note Although y[...]

  • Page 34

    4-6 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 4 SCSI RDMA Protocol Configuring SRP Configuring SRP Host This section descri bes how to configu re the SRP host. The SRP host dri ver exposes a Fibre Chan nel target (identified by a WWPN) as a SCS I target to the Linux SCSI mid-layer . In turn, the mid-layer creates L[...]

  • Page 35

    4-7 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 4 SCSI RDMA Protocol Verifying SRP Verifying SRP This section describes h ow t o verify SRP f unctional ity and veri fy SRP host-to- storage connections with the Element Manager GUI and includes the following sections: • V erifying SRP Functional ity , page 4-7 • V e[...]

  • Page 36

    4-8 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 4 SCSI RDMA Protocol Verifying SRP 512000 inodes, 1023996 blocks 51199 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1048576000 32 block groups 32768 blocks per group, 32768 fragments per group 16000 inodes per group Superblock [...]

  • Page 37

    CH A P T E R 5-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 5 Sockets Direct Protocol This chapter describes the Sock ets Direct Protocol and in cludes the follo wing sections : • Introduction, page 5-1 • Config uring IPoIB Interf aces, page 5-1 . • Con v erting Sockets- Based Application, page 5-2 • SDP Performance,[...]

  • Page 38

    5-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 5 Soc kets Direct Protocol Converting Sockets-Based Application Converting Sockets-Based Application This section describes ho w to co n ver t sockets-based applications. Y ou can con vert your sockets-b ased applications to use SDP instead of TCP b y using one of two c[...]

  • Page 39

    5-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 5 Sockets Direct Protoco l Converting Sockets-Based Application Log Statement This section describes the lo g statement. The log di rect iv e allo ws the u ser to specif y which deb ug and error messages are sent and where they are se nt. The log stat ement format is as [...]

  • Page 40

    5-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 5 Soc kets Direct Protocol SDP Performance The ip_port matches against an IP address, prefix length, and port range. The format is as follo ws: ip_addr [/ prefix _length ][: start_po rt [- end_port ]] The pref ix length is option al and missing defaults t o /32 (length [...]

  • Page 41

    5-5 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 5 Sockets Direct Protoco l SDP Performance Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB 87380 16384 65536 10.00 6601.82 23.79 21.37 1.181 1.061 The follo [...]

  • Page 42

    5-6 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 5 Soc kets Direct Protocol Netperf Server with IPoIB and SDP The notable pe rformance v alues in the e xample above are as follo ws: Client CPU utilizat ion is 6.26 percent of client CPU. Server CPU ut ilization is 7.22 percent of serv er CPU. Latency is 18.01 microseco[...]

  • Page 43

    5-7 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 5 Sockets Direct Protoco l Netperf Server with IPo IB and SDP The follo wing list describes paramet ers for the netperf command: The notable pe rformance v alues in the e xample above are as follo ws: Throughput is 6.60 gigabits per second. Client CPU utilizat ion is 23.[...]

  • Page 44

    5-8 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 5 Soc kets Direct Protocol Netperf Server with IPoIB and SDP[...]

  • Page 45

    CH A P T E R 6-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 6 uDAPL This chapter describe s uD APL and includes the follo wing sections: • Introduction, page 6-1 • uD APL T est Performance, page 6-1 • Compiling uDAPL Programs, page 6-4 Note See the “Root and Non-root Con v entions in Examples” section on pag e ix f[...]

  • Page 46

    6-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 6 uDAPL uDAPL Throughput Test Performance The Throughput test measures RDMA WRITE th roughput using uD APL. T o perform a uD APL Throughput test perfor mance, perform the follo wing step s: Step 1 Start the Throu ghput test on the serv er host. The synt ax for the serv [...]

  • Page 47

    6-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 6 uDAPL Step 3 V iew the Throughput test results from the serv er . The follo wing e xample sho ws the Throughput test results: Created an EP with ep_handle = 0x2a95f8a300 queried max_recv_dtos = 256 queried max_request_dtos = 1024 Accept issued... Received an event on e[...]

  • Page 48

    6-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 6 uDAPL host2$ /usr/local/topspin/bin/lat_client.x ib0 192.168.0.1 200000 1 0 • ib0 is the name of the device. • 192.168.0.1 is the IPoIB address of the server host. • 200000 is the number of RDMAs to p erform for the test. • 1 is the size in bytes of t he RDMA [...]

  • Page 49

    CH A P T E R 7-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 7 MVAPICH MPI The chapter describes MV APICH MPI and i ncludes the follo wing sections: • Introduction, page 7-1 • Initial Setup, page 7- 2 • Config uring SSH, page 7-2 • Editing En viron ment V ariables, page 7-5 • MPI Bandwidth T est Performance, page 7-[...]

  • Page 50

    7-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Initial Setup Initial Setup This section describes the initial MPI setup. MPI can be used with either IPoIB or Ethernet IP addresses. The dri vers for MPI are aut omatically loaded at boot time if IPoIB or SDP is loaded. If neither IPo IB nor SRP are used,[...]

  • Page 51

    7-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI Configuring SSH T o conf igure SSH, perform t he follo wing steps: Step 1 Log in to the host th at you want to con figure as the local host, host1. The follo wing e xample sho ws ho w to log in to the host: login: username Password: password host1$ Note Y o[...]

  • Page 52

    7-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Configuring SSH Step 5 Change into the .ssh di rectory that you created. The follo wing e xample sho ws ho w to change into t he .ssh directory: host1$ cd .ssh Step 6 Copy the public key that w as just generated to the authorized keys file . The follo wing[...]

  • Page 53

    7-5 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI Editing Environment Variab les Step 9 Return to host1 and cop y the authorized k eys f ile from Step 6 to the directory that you created in Ste p 8 . The follo wing e xample sho ws ho w to return to host1 a nd cop y the authorized keys f ile to the d irecto[...]

  • Page 54

    7-6 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Editing Environment Variables Setting Environment Variables in System-Wide Startup Files This method is used to set a system-wide def ault for which MPI implementation is used. This method is the easiest for end users; users wh o log in automatically ha ve[...]

  • Page 55

    7-7 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI MPI Bandwidth Test Performa nce Editing Environment Variables Manually T ypically , you edit en viro nment va riables manually when it is necessary to run temporarily with a gi v en MPI implementation. For ex ample, when it is not de sirable to change the d[...]

  • Page 56

    7-8 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI MPI Latency Test Per formance When the test completes successfull y , you see output that is similar to the follo wing: # OSU MPI Bandwidth Test (Version 2.2) # Size Bandwidth (MB/s) 1 3.352541 2 6.701571 4 10.738255 8 20.703599 16 39.875389 32 75.128393 6[...]

  • Page 57

    7-9 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI Intel MPI Benchmarks (IM B) Test Performance • The name of the hostf ile • The latency ex ecutable name The follo wing e xample sho ws ho w to run the MV APICH MPI Latency test: host1$ mpirun_rsh -np 2 -hostfile /tmp/hostfile /usr/local/topspin/mpi/mp[...]

  • Page 58

    7-10 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Intel MPI Benchmar ks (IMB) Test Performance When your installation is no t working prop erly , the IMB test might lead to V API_RETR Y_EXEC errors. Y ou should check the ou tput of the PingPong, Ping Ping, and Sendrecv bandwidth measurements against know[...]

  • Page 59

    7-11 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI Intel MPI Benchmarks (IM B) Test Performance When the test completes su ccessfully , you see output similar to the follo wing: #--------------------------------------------------- # Intel (R) MPI Benchmark Suite V2.3, MPI-1 part #--------------------------[...]

  • Page 60

    7-12 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Compiling MPI Programs Compiling MPI Programs This section descri bes how to compile MPI progra ms. Compi ling MPI applications from source code requires adding se ve ral compiler and link er flags. MV APICH MPI pro vides wra pper compilers th at add all [...]

  • Page 61

    7-13 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 7 MVAPICH MPI Compiling MPI Programs Step 3 Select the language and compiler of your choice from the sele ction of com piler wrapper s av ailable in Ta b l e 7 - 2 . Step 4 Compile the e xamples as sho wn here: host1$ cd $HOME/mpi-examples host1$ mpicc.i -o hello_c hell[...]

  • Page 62

    7-14 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 7 MVAPICH MPI Compiling MPI Programs[...]

  • Page 63

    CH A P T E R 8-1 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 8 HCA Utilities and Diagnostics This chapte r describes the HC A utilities and diagnostics and includes t he follo wing sect ions: • Introduction, page 8-1 • hca_self_test Utility , page 8-1 • tvflash Utility , page 8-3 • Diagnostics, page 8-5 Introduction T[...]

  • Page 64

    8-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 8 HCA Utilities and Diagnostics hca_self_test Utility Host Driver Initialization ............. PASS Number of HCA Ports Active ............. 2 Port State of Port #0 on HCA #0 ........ UP 4X Port State of Port #1 on HCA #0 ........ UP 4X Error Counter Check on HCA #0 ...[...]

  • Page 65

    8-3 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 8 HCA Utilities and Diagnostics tvflash Utility tvflash Utility This section descri bes the tvflash utility and includes the fo llowi ng topics: • V iewing Card T ype and Firmware V ersion, page 8-3 • Upgrading Firmware, page 8-4 Note The firmw are upgrade is handled[...]

  • Page 66

    8-4 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 8 HCA Utilities and Diagnostics tvflash Utility Upgrading Firmware T o upgrade f irmware on your ho st, perform the follo wing steps: Note Upon installation of the host driv ers, the f irmware is auto matically updated, if required. Ho we ver , if you ha ve outdated fir[...]

  • Page 67

    8-5 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Chapter 8 HCA Utilities and Diagnostics Diagnostics Diagnostics This section includes diagnosti cs information. A fe w diagnost ic programs are included with the Linux IB host dri vers. The vstat utility prin ts IB information. The follo wing e xample sho ws a vstat utility d is[...]

  • Page 68

    8-6 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Chapter 8 HCA Utilities and Diagnostics Diagnostics[...]

  • Page 69

    A-1 Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 APPENDIX A Acronyms and Abbreviations Ta b l e A - 1 defines the acron yms and abbre viations th at are used in this guide. T able A -1 List of Acr on yms and Abbr eviations Acronym Expansion API Application Program Interf ace CLI command-line interf ace GUI graphical user inte[...]

  • Page 70

    A-2 Cisco SFS InfiniBand Host Drivers User Guide for Lin ux OL-12309-01 Appendix A Acronyms an d Abbreviations SSH Secure Shell Protocol TCP T ransmission Cont rol Protocol uD APL User Direct Access Programming Library ULP upper-l ev el protocol WWNN world-wide node name WWPN world-wide port name T able A -1 List of A cr on yms and Abbr eviations ([...]

  • Page 71

    IN-1 Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 INDEX A architecture, HCA supported 1-2 audience vii authenticity messages 7-5 B Bandwidth test default 3-6, 5-7 MPI 7-7 Netperf 5-6 using 3-6 C card type, view 8-3 CLI 4-2 command-line interface. See CLI. compile MPI programs 7-12 compiler GNU 7-1 Intel 7-1 configure IPoIB 3-[...]

  • Page 72

    Index IN-2 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 Gateway, Fibre Cha nnel 4-1 Globally Uni que Identifier. Se e GUID. global policy r estrictions 4-2, 4-4 GNU compiler 7-1 graphical u ser interface . See GUI. GUI 4-2 GUID 4-2 H HCA description 1-1 diagnostics 1-4, 8-1 firmware version 2-3 ports 2-3, 3-1 supported APIs 1-[...]

  • Page 73

    Index IN-3 Cisco SFS InfiniBand Host Dr ivers User Guide for Linux OL-12309-01 MPI Bandwidth test 7-7 compile programs 7-12 description 1-4, 7-1 Intel Benchmarks test 7-9 Latency test 7-8 MVAPICH 1-4, 7-1 tutorial 7-12 MPI implementation multiple 7-5 single 7-5 MVAPICH MPI 1-4, 7-1 N netmask 3-2 Netperf 3-6, 5-6 Netperf server 3-6, 5-6 O organizati[...]

  • Page 74

    Index IN-4 Cisco SFS InfiniBand Host Drivers User Guide for Linux OL-12309-01 log 5-3 match 5-3 storage area network. See SA N. stream sockets networking 5-1 subinterface create 3-3 description 3-2 remove 3-4 T test Bandwidth 3-6 Bandwidth, default 3-6, 5-7 Bandwidth, MPI 7-7 Bandwidth, with SDP 5-6 IMB 7-9 Intel MPI Benchmarks. See IMB. Latency 3-[...]