English dropdown-ico

Mellanox Technologies MNPH29D-XTR manual

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

Go to page of

Similar user manuals

A good user manual

The rules should oblige the seller to give the purchaser an operating instrucion of Mellanox Technologies MNPH29D-XTR, along with an item. The lack of an instruction or false information given to customer shall constitute grounds to apply for a complaint because of nonconformity of goods with the contract. In accordance with the law, a customer can receive an instruction in non-paper form; lately graphic and electronic forms of the manuals, as well as instructional videos have been majorly used. A necessary precondition for this is the unmistakable, legible character of an instruction.

What is an instruction?

The term originates from the Latin word „instructio”, which means organizing. Therefore, in an instruction of Mellanox Technologies MNPH29D-XTR one could find a process description. An instruction's purpose is to teach, to ease the start-up and an item's use or performance of certain activities. An instruction is a compilation of information about an item/a service, it is a clue.

Unfortunately, only a few customers devote their time to read an instruction of Mellanox Technologies MNPH29D-XTR. A good user manual introduces us to a number of additional functionalities of the purchased item, and also helps us to avoid the formation of most of the defects.

What should a perfect user manual contain?

First and foremost, an user manual of Mellanox Technologies MNPH29D-XTR should contain:
- informations concerning technical data of Mellanox Technologies MNPH29D-XTR
- name of the manufacturer and a year of construction of the Mellanox Technologies MNPH29D-XTR item
- rules of operation, control and maintenance of the Mellanox Technologies MNPH29D-XTR item
- safety signs and mark certificates which confirm compatibility with appropriate standards

Why don't we read the manuals?

Usually it results from the lack of time and certainty about functionalities of purchased items. Unfortunately, networking and start-up of Mellanox Technologies MNPH29D-XTR alone are not enough. An instruction contains a number of clues concerning respective functionalities, safety rules, maintenance methods (what means should be used), eventual defects of Mellanox Technologies MNPH29D-XTR, and methods of problem resolution. Eventually, when one still can't find the answer to his problems, he will be directed to the Mellanox Technologies service. Lately animated manuals and instructional videos are quite popular among customers. These kinds of user manuals are effective; they assure that a customer will familiarize himself with the whole material, and won't skip complicated, technical information of Mellanox Technologies MNPH29D-XTR.

Why one should read the manuals?

It is mostly in the manuals where we will find the details concerning construction and possibility of the Mellanox Technologies MNPH29D-XTR item, and its use of respective accessory, as well as information concerning all the functions and facilities.

After a successful purchase of an item one should find a moment and get to know with every part of an instruction. Currently the manuals are carefully prearranged and translated, so they could be fully understood by its users. The manuals will serve as an informational aid.

Table of contents for the manual

  • Page 1

    www .mellanox.com ConnectX ® -2 EN Dual Port SFP+ Ethernet Adapter Card User Manual P/N: MNPH29C-XSR, MNPH29C-XTR, MNPH29D-XSR, MNPH29D-XTR Rev 1.6[...]

  • Page 2

    Mellanox T echnologies 350 Oakmead Parkwa y Suite 100 Sunnyvale, CA 94085 U.S.A. www .mellanox.com T el: (408) 970-3400 Fax: (408) 970-34 03 Mellanox T echnologies, Ltd. PO Box 586 Hermon Building Y okneam 20692 Israel T el: +972-4-909-7200 Fax: +972-4-959-3245 © Copyright 201 1. Mellanox T echno logies. All rights reserved. Mellanox®, BridgeX®,[...]

  • Page 3

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 3 T able of Content s Table of Contents 3 List of Figures 5 List of Tables 6 Revision History 7 About this Manual 8 Intended Audience 8 Related Docume ntation 8 Online Resources 8 Document Conventions 8 Technical Support 8 Firmware and Soft ware Updates 9 Chapter 1 Overview 10 1.1 A[...]

  • Page 4

    Rev 1.6 Mellanox Technologies 4 Appendix E Sicherheitshinweise (Warnings in German) 46 Appendix F Advertencias de seguridad para la instalación (Warnings in Spanish) 47[...]

  • Page 5

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 5 List of Figures Figure 1: SFP+ NIC Component Side and Bracket Views 12 Figure 2: Card Product Label 14 Figure 3: Port Numbering 16 Figure 4: LED - Port Association 17 Figure 5: Physical and Log ical Link Indications 17 Figure 6: I2C Connecto r 18 Figure 7: Flash Jumper 19 Figure 8[...]

  • Page 6

    Rev 1.6 Mellanox Technologies 6 List of T ables Table 1: Revision History Table 7 Table 2: Documents List 8 Table 3: Network Interface Cards 11 Table 4: Mellanox Cards Part Numbering Key 13 Table 5: LEDs 17 Table 6: Jumper Configuration 19 Table 7: VPD Layout for MNPH29D-X[ST]R 19 Table 8: VPD Layout for MNPH29C-X[ST]R 20 Table 9: Hardware and Soft[...]

  • Page 7

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 7 Revision History This document was printed on 2/15 /1 1. T able 1 - Revision History T able Date Rev Comments/Changes Jan. 201 1 1.6 Changed Air flow requirement Added to the Set up section Dec. 19th, 2010 1.5 Minor formatting changes Nov . 22nd, 2010 1.4 Added Generation D ca rd [...]

  • Page 8

    Rev 1.6 Mellanox Technologies 8 About this Manual This User Manual describes Mellanox T echnologies ConnectX ® -2 Dual Port 10Gigabit Ethernet PCI Express x8 adapter cards. It provides details as to the interface s of the board, specifications, required software and firmware for opera ting the board , and relevant documentation. Intended Audience [...]

  • Page 9

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 9 • T el: +1.408.916.0055 Customers who purchased Mellanox M-1 Global Su pport Services, please see your contract for details regarding T echnical Support. Customers who purchased Mellanox prod ucts thro ugh a Mellanox approved reseller should first seek assistance through their r[...]

  • Page 10

    Overview Rev 1.6 Mellanox Technologies 10 1 Overview This document is a User Manu al for Mellanox T echnologies cards based on the ConnectX ® -2 ENintegrated circuit device. The cards described in this manu al have the following main features: • IEEE S td 802.3 compli ant • 10 Gb/s Ethernet • Dual SFP+ ports for conn ection Ethernet traf fic[...]

  • Page 11

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 11 1.1 Adapter Cards Covered in this Manual T able 3 lists the Ethernet adapte r cards described in this manual. T able 3 - Network Interface Cards Ordering Part Number (OPN) PCI Express SERDES Speed Data T ransmission Rate/ # of ports Short / T all Bracket RoHS Adapter IC Part Numb[...]

  • Page 12

    Overview Rev 1.6 Mellanox Technologies 12 Figure 1: SFP+ NIC Component Side and Bracke t V iews J1 U1 MNPH29C J1 U1 MNPH29D[...]

  • Page 13

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 13 1.2 Mellanox Part Numbering Legend T able 4 describes the Mellanox T e chnologies adapter cards part numbering legend. For example, the part number MNPH29C-X TR describes Mellanox T echnologies’ ConnectX EN adapter card with two SFP+ ports, a PCIe2.0 x8 5.0GT/s interface, no on[...]

  • Page 14

    Overview Rev 1.6 Mellanox Technologies 14 1.3 Finding the MAC and Serial Nu mber on the Adapter Cards All Mellanox adapter cards have a label on the printed side of th e adapter card that has the card serial number , and the card MAC . Figure 2: Card Product Label 1.4 Safety W arnings For safety warnings in French se e “ A vertissements de sécur[...]

  • Page 15

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 15 3. During Lightning - Electrical Hazard 4. Copper Cable Connecting/Disconnecting 5. Equipment Installation 6. Equipment Disposal 7. Local and National Electrical Codes During periods of lightning activi ty , do not work on the equipment or connect or dis- connect cables. Some cop[...]

  • Page 16

    Adapter Card Interfaces Rev 1.6 Mellanox Technologies 16 2 Adapter Card Interfaces 2.1 I/O Interfaces Each card includes the following interfaces: •S F P + p o r t s • I/O panel LEDs •I 2 C compatible connector (for debug) For dual port cards, p ort 1 connects to connecto r 1 of the device, while port 2 co nnects to connec- tor 2 of the devic[...]

  • Page 17

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 17 2.1.3 LED Assignm ent The board has I/O LEDs located on the I/O panel. The green LED, when lit, indicates that the driver is running and a valid ph ysical connection between nodes ex ists. If the green LED is blink- ing, it indicates a problem with the physical link. The yellow L[...]

  • Page 18

    Adapter Card Interfaces Rev 1.6 Mellanox Technologies 18 2.1.4 I 2 C Comp atible Interface A three-pin header on the card is provided as the I 2 C compatible interface. See Appendix A,“Specifications,” on page 34 for the locati on on the board. Figure 6: I 2 C Connector 2.2 Power All adapter c ards receive 12V and 3.3 V power from the PCI Expre[...]

  • Page 19

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 19 2.3.3 EEPROM Each board incorporates an EEPROM that is accessible through the I2C. The EEPROM is used for storing the V ital Product Data (V PD). The EEPROM capacity is 4Kb. 2.4 VPD Layout The PCI VPD (V ital Product Data) layout, for each of the d escribed Mellanox T echnologies[...]

  • Page 20

    Adapter Card Interfaces Rev 1.6 Mellanox Technologies 20 76 VPD Keywor d V0 STR Misc Informa tion 78 Length 0x10 79 Data PCIe Gen2 x8 STR_SPC 95 VPD Keyword R V STR 97 Length 0x1 98 Data 0,97 %CS0 99 Large Resource T ype VP D-W T ag (0x11) 0x91 100 Length [7:0] LSB 0x99 101 Length [15:8] MSB 0x00 102 VPD Keyword V1 STR EFI Driver version 104 Length[...]

  • Page 21

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 21 76 VPD Keyword V0 STR Misc Information 78 Length 0x10 79 Data PCIe Gen2 x8 STR_SPC 95 VPD Keyword R V STR 97 Length 0x1 98 Data 0,97 %CS0 99 Large Resou rce T ype VPD-W T ag (0x11) 0x91 100 Length [7:0] LSB 0x99 101 Length [15:8] MSB 0x00 102 VPD Keyword V1 STR EFI Driver version[...]

  • Page 22

    Driver Software and Firmware Rev 1.6 Mellanox Technologies 22 3 Driver Sof tware and Firmware 3.1 Driver Sof tware 3.1.1 Linux For Linux, download and install th e latest MLNX_ EN driver software package available via the Mellanox W eb site at: http://www .mellanox.com => Downloads => Ethernet SW/Drive rs. Follow the installation instruc- tio[...]

  • Page 23

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 23 3.3 Up d ating Card Firmware Each card is shipped with the latest version of qualified firmware at th e time of manufacturing. Firmware is updated occasionally , and the mo st recent firmware can be obtained from: http://www .mellanox.com =>Suppo rt > Downlo ad Center . Che[...]

  • Page 24

    Driver Software and Firmware Rev 1.6 Mellanox Technologies 24 Figure 8: Suppor t Download Assist ant 3.4 ConnectX EN PXE The Mellanox ConnectX EN PXE soluti on enables the booting of servers from an IP based LAN environment. T o enable or disable PXE use the following procedure: The user must use the MFT tools: T o disable one must delete the PXE R[...]

  • Page 25

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 25 4 Ethernet Adapter Card Inst allation 4.1 Hardware and Sof tware Requirement s Before installing the adapter card, please make su re that the system meets the hardware and soft- ware requirements listed in T able 9. Refer to Chapter 3,“Driver Software and Firmware” on page 22[...]

  • Page 26

    Ethernet Adapter Card Installation Rev 1.6 Mellanox Technologies 26 Any PCI slot with the proper configuration is acce pt able for connection. If the card is installed in a PCI slot with less lanes than the card requires then the adapter card will not provide the optimum data transfer . 4.3 Set Up The basic steps to embed Mellanox cards in your com[...]

  • Page 27

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 27 Figure 9: Hard wa re Devices 3. Select a PCI Device / InfiniBand Controller entry . 4. Right-click. 5. Select “ Pr operties to display the PCI Devic e Properties ” window . 6. Click the Detail s tab and select Device In stance Id (W indows 2003) or Hardware Ids (W indows 2008[...]

  • Page 28

    Ethernet Adapter Card Installation Rev 1.6 Mellanox Technologies 28 Figure 10: PCI De vice 7. In the V a lue display box, check the fields VE N and DEV (fields are sepa rated by ‘&’). In the display exam ple above, notice the sub-strin g “PCIVEN_15B3&DEV_6368”: VEN is equal to 0x15B3 – this is the V endor ID of Mellanox T ech nol[...]

  • Page 29

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 29 4.3.1.2 Linux Get the device location on the PCI bus by runnin g lspci and locating lines with the string “Mel- lanox T echnologies”: > lspci | grep Mellanox 2:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0 ) Make sur[...]

  • Page 30

    Ethernet Adapter Card Installation Rev 1.6 Mellanox Technologies 30 Rate: 10 Base lid: 6 LMC: 0 SM lid: 3 Capability mask: 0x0 251086a Port GUID: 0x0 002c903000c871 1 Link layer: IB Check the OFED version T o get the version of the running Mellanox OFED/BXOFED, run th e fo llowing co mmand: Host# ofed_info | head -1 BXOFED-1.5.1-1.3.7-rc19: T roubl[...]

  • Page 31

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 31 The “Usage: openibd {start|stop|re start|status}” co mmand to modify this f ile and thereby control the drivers. Ethernet Driver Usage and Configuration T o assign an IP address to the interface run: #> ifconfig eth<n> <ip> where 'n' is the OS assigne[...]

  • Page 32

    Ethernet Adapter Card Installation Rev 1.6 Mellanox Technologies 32 The adapter cards are shipped without optical modules. Me llanox 10GBASE-SR (MFM1T02A- SR) and 10GBASE-LR (MFM1T02A-LR) optical mo dules ar e recommen ded. The fig ure below shows the Mellanox SFP+ module. Inserting the Optica l T ransceiver Module T o insert the module into the ca[...]

  • Page 33

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 33 Care should be taken not to impede the air ex haust flow through the ventilation holes. Cable lengths should be used wh ich allow for routing horizontally around to the side of the chassis before bending upward or do wnward in the rack. T o remove a c able, disengage the locks an[...]

  • Page 34

    Rev 1.6 Mellanox Technologies 34 Appendix A: S pecifications A.1 MNPH29D-X[ST]R Specifications T able 10 - Specifications for MNPH29D-X[ ST]R Physical Power and Envir onmental Size: Air Flow: 10Gb/s Connector: 2.71in x5.60in (68.90mm x 142.25mm) 200LFM a SFP+ a. Air flow is measured ~1” from the heat sink between the heat sink and the cooling air[...]

  • Page 35

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 35 A.2 MNPH29C-X[ST]R Specifications A.3 Board Mechanical Drawing and Dimensions All of the cards covered in this User Manual have the same mechani cal drawing and share the same dimensions as depicted in Figure 1 3 or Figure 14. T able 1 1 - Specificati ons for MNPH29C-X[ST]R Physi[...]

  • Page 36

    Rev 1.6 Mellanox Technologies 36 Figure 13: Schematic of the Connect X-2 Small Form Factor MNPH29D Adapter Ca rd All dimensions are i n millimeters. All the mechanical tolerances are +/-0.1mm J4 J3 U1 J1 U3 Q1 L8 U4 U14 45.01 3.65 1.90 15.00 45.96 142.25 56. 15 12.75 64.40 Q2 J7 J8[...]

  • Page 37

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 37 Figure 14: Schematic of th e ConnectX-2 Card MNPH29C Adapter Card A.4 EMC Certification St atements T able 12 lists the approved certification stat us per in different reg ions of the world. T able 12 - Adapter Cards Certificat ion S tatus Adapter Card P/N FCC VCCI EN ICES CE CB [...]

  • Page 38

    Rev 1.6 Mellanox Technologies 38 A.4.1 FCC St atement s (USA) Class A S tatements: § 15.19(a)(4) This device complies with P art 15 of the FCC Rules. Operation is subject to th e following two conditions: 1. This device may not cause harmful interference, and 2. This device must accept any interference re ceived, including interference that may ca[...]

  • Page 39

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 39 A.4.4 VCCI St atement s (Japan) Class A S tatement: (T ranslation - "This is a Class A product based on the standard of the V oluntary Control Council for Interference by Information T echnology Equi pment (VCCI). If this equipment is used in a domestic environment, radio in[...]

  • Page 40

    Rev 1.6 Mellanox Technologies 40 Appendix B: Interface Connectors Pinout B.1 I 2 C-Comp atible Connector Pinout Figure 15: Compatible Connector Plug and Pinout B.2 PCI Express x8Connector Pinout These cards use a standard PCI Express x8 edge conne ctor and the PCI Express x8 standard pinout according to the PCI Express 2.0 s pecification. 5 1 2 3 4[...]

  • Page 41

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 41 B.3 PCI Express Connector Pinout B.4 SFP+ Connector Pinout Figure 16: Rear V iew of Module With Pin Placement 13.70 8.50 Top VeeT VeeT VeeT TX_Fault TX_Disable SDA SCL MOD_ABS RS0 RX_LO S RS1 VeeR VeeR VeeR VccT VccR TD+ TD- RD- RD+ 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 0 1 1 1 1 1[...]

  • Page 42

    Rev 1.6 Mellanox Technologies 42 T able 14 - SFP+ Connector Pinout Pin Symbol Name Description 1 V eeT T ransmitter Ground (Comm on with Receiver Ground) a a. Circuit ground is internally isolated from chassis ground. 2 T X _ F a u l t T ransmitter Fault. b b. T FA U LT is an open collector/drain out put, which should be p ulled up with a 4.7k – [...]

  • Page 43

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 43 Appendix C: Replacing a T all Bracket With a Short Bracket This section provides instructions on how to remove the tall bracket of a standard Me llanox T ec h- nologies adapter card and replace it with a sh ort one. It includes the following sections: • Removing a brac ket • [...]

  • Page 44

    Rev 1.6 Mellanox Technologies 44 Figure 18: Putting on the Bracket 6. Make sure that the LEDs a re a ligned onto the bracket holes. 7. Use a torque driver to apply up to 2 lbs-in torque on the screws.[...]

  • Page 45

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 45 Appendix D: A vertissements de sécurité d’inst allation (W arnings in French) 1. Instructions d’installation 2. T empérature excessive 3. Orages – dangers électriques 4. Branchement/débranchement des câbles InfiniBand en cuivre 5. Installation du matériel 6. Eliminat[...]

  • Page 46

    Rev 1.6 Mellanox Technologies 46 Appendix E: Sicherheit shinweise (W arnings in German) 1. Installationsanleitungen 2. Übertemperatur 3. Bei Gewitter - Elektrische Gefahr 4. Anschließen/T rennen von InfiniBand-Kupferkabel 5. Geräteinstallation 6. Geräteentsorgung 7. Regionale und nationale elektrische Bestimmungen Lesen Sie alle Installatio nsa[...]

  • Page 47

    ConnectX-2 Ethernet Card User Manual Rev 1.6 Mellanox Technologies 47 Appendix F: Advertencias de seguridad p ara la inst alación (W arnings in S p anish) 1. Instrucciones de instalación 2. Sobreca lentamiento 3. Cuando hay rayos: pelig ro de descar ga eléctrica 4. Conexión y desconexión del cable Copp er InfiniBand 5. Instalación de equipos [...]

  • Page 48

    Rev 1.6 Mellanox Technologies 48 Códigos eléctricos locales y nacionales Este equipo se debe instalar conforme a los códigos eléctricos locales y nacio- nales.[...]