Cisco InfiniBand Cisco 4x manual

Corporate Headquarters Cisco Systems, In c. 170 West Tasman Drive San Jose, CA 951 34-1706 USA http://www.ci sco.com Tel: 408 526-4000 800 553-NETS (638 7) Fax: 408 526-4100 Cisco 4x Inf iniBand S witc h Module f or IBM BladeCenter User Guide Text Part Number: 78 -17838-01[...]

THE SPECIFICATION S AND INFORMAT ION REGARDING TH E PRODUCTS IN THIS MANUAL ARE SUBJECT T O CHANGE WITH OUT NOTICE. A LL STATEMENTS , INFORMATION, AND RECOMMENDATI ONS IN THI S MANUAL ARE BE LIEVED TO BE A CCURATE BUT ARE PRESENTED WI THOUT WARRANTY OF ANY KIND, EX PRESS OR IMPLIED. USERS MUST TAKE FULL RESPO NSIBILITY FOR THEIR APPLICAT ION OF ANY[...]

i Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 CONTENTS Preface v Audience v Organization v Conventi ons v Related Documentation vi Contacting Technical Supp ort vii IBM Support Contact Information vii Before You Call viii Determining the Software Version viii Downloading the Latest Software Relea se viii Cisco Syste[...]

Contents ii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 EQUADOR xv MEXICO xv PARAGUAY xv PERU xv URUGUAY xvi VENEZUELA xvi NORTH AMERICA xvi CANADA xvi UNITED STATES xvi ASIA PACIFIC xvi AUSTRALIA xvi CAMBODIA AND LAOS xvii CAMBODIA, INDONESIA, AND LAOS xvii HONG KONG S.A.R. OF CHINA AND MACAU S.A .R. OF CHINA xvii[...]

Contents iii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Types of Warranty Servic e xxvii EU Country Telephone List xxviii Notices xxix Edition Notice xxx Trademarks xxx Important Notes xxxi Product Recycling and Disposal xxxi Battery Return Program xxxi Electronic Emi ssion Notices xxxii Federal Communications Comm[...]

Contents iv Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Changing the Default Pass word 2-3 Configuring the System Clo ck 2-3 Connecting Beyond BladeCenter 2-5 Connecting InfiniBand Cables 2-5 CHAPTER 3 Specifications 3-1 Electrical Requirements 3-1 Environmental Requirem ents 3-1 I NDEX[...]

v Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Preface Audience The intended audience is the administrat or responsible for installing, conf iguring, an d managing Server Switch equipment. This ad ministrator should ha ve e xperience administering similar netw orking or storage equipment. Organization This publicati [...]

vi Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Preface Related Documentation Notes use the follo wing con ventions: Note Means reader take note . Notes contain helpful suggestions or references to mat erial not cov ered in the publication. Cautions use the following con v entions: Caution Means re a d e r b e c a r[...]

vii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Preface Contacting Technica l Support • Cisco Command Line Interface Refer ence G uide • Cisco Chassis Manager User Guide (W eb GUI) • Cisco Element Manager User Guide Contacting Technical Support Please read this section b efore contacting IBM or Cisco t echnica[...]

viii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Preface Contacting Tec hnical Support Before You Call Be prepared to pro vide the follo wing informat ion to support personn el. General Information • T echnical Support registrati on number , if applicable • Error messages receiv ed • Detailed description o f [...]

ix Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Preface Contacting Technica l Support Cisco Technical Support & Documentation Website The Cisco T echnical Support & Documentation website p rovides online d ocuments and tools for troubleshooting and resolv ing technical issues with Cisco pr oducts and technolo[...]

x Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Preface Contacting Tec hnical Support Se verity 2 (S2)—O peration of an exi sting network is se v erely degraded, o r signif icant aspects of your business operation are negativ ely affected by i nadequate performance of Cisco products. Y ou and Cisco will commit full[...]

xi Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms Part 1 - Gen eral T erms This Statement of Limited W arranty includes Part 1 - General T erms, Part 2 - Co untry-unique T e rms, and Part 3 - W arranty Information. The terms of Part 2 replace[...]

xii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 1 - General Terms What This Warranty Does Not Cover This warranty does not cov er the follo wing: • any softw are programs, whether pre-l oaded or shipped with the Machine, or instal led subsequently; • [...]

xiii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms Exchange of a Machine or Part When the warran ty service in v olv es the exchange of a Machine o r part, the item IBM or your reseller replaces becomes its property and the replaceme nt beco[...]

xiv Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms 2. the amount of an y other actual dir ect damages, up to the char ges (if recurring, 12 mont hs' char ges apply) for the Machine that is subject of the claim. F or purposes of [...]

xv Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms Any litig ation arising from this Statem ent of Limited W arranty will be settled exclusi vely b y the courts of the city of La Paz. BRAZIL Jur is di ct io n : The follow ing is adde d [...]

xvi Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms URUGUAY Jur is di ct io n : The follow ing is adde d after the first senten ce : Any liti gation arising from this St atement of Limited W arranty will be settled e xclusi vely b y t[...]

xvii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms Limitation of Liability : The fo llowing is added to this secti on : Where IBM is in breach of a cond ition or warranty implied b y the T rade Practices Act 1974 or other similar le g[...]

xviii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms INDIA Limitation of Liability : The fo llowing r eplaces items 1 and 2 o f this section : 1. liability for bod ily injury (including death) or damage to real property and tangible [...]

xix Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms PEOPLE'S REPUBLIC OF CHINA (PRC) Govern ing La w : The following r eplaces “laws of the country in whic h you acquir ed t he Machine” in the fir st sentence : laws of the Stat[...]

xx Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms If you purchase a Machine in one of the W estern Eu ropean countries, as defined abo ve, you may obtain warranty service for that Machine in an y of those co unt ries from either (1) [...]

xxi Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms ex ecution of this Statement of Limited W arranty shall be settled by Arbitrat ion Court of Moscow ; 6) in South Africa, Namibia, Lesotho and Swaziland , both of us agree to submit all[...]

xxii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms CONSUMERS Consumers ha ve le gal rights und er applicable nationa l legi slation go verni ng the sale of consumer goods. Such rights are not affected b y the warranties provi ded in[...]

xxiii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms THE FOLLOWING TERMS APPLY TO THE COUNTRY SPECIFIED: AUSTRIA The provisions o f this Statement of Limited W a rranty replace any applic able statutory warranti es. What this W arranty[...]

xxiv Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 2 - Country-unique Terms GERMANY What this W arranty Covers : The f ollowing r eplaces the f irst sent ence of the f irst p aragr aph of this section : The warranty for an IBM Machine c ov ers the functiona[...]

xxv Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 2 - Country-unique Terms This section sets out the e xtent of IBM's liability and yo ur sole remedy . 1. IBM will accept unlimited li ability for death or personal injury caused by the ne gligence of IBM.[...]

xxvi Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 3 - Warranty Information 3. IBM's entire liability for actual damag es for any one Def ault will not in an y e vent, e xcept as provid ed in items 1 and 2 abov e, exceed th e greater of 1) Pounds Sterl[...]

xxvii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 3 - Warranty Information Types of Warranty Service If required, IBM provides repair or e xchange service depending on the type of warranty service specif ied for your Machine in t he abov e table and as desc[...]

xxviii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 IBM Statement of Limited Warranty Z125-4753-08 04/2 004 Part 3 - Warranty Information When a 5, 6 or 7 ty pe of warranty serv ice is listed, IBM will determine which type of warranty service is appropriate f or the r epair . * This type of serv ice is called ThinkP[...]

xxix Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Notices This information w as de veloped for pr oducts and services of fered in the U.S.A. IBM may not offer the products, services, or features discussed in this documen t in other countries. Consult your local IBM representati v e for informati on on the products an[...]

xxx Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Notices Edition Notice Edition Notice © Copyright Intern ational Business Machines Cor poration 2004. All righ ts reserved . U.S. Gov ernment Users Restrict ed Rights — Use, du plication, or disclosure restricted by GSA ADP Schedule Contract wit h IBM Corp. Tradema[...]

xxxi Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Notices Important Notes Other company , product, or service names may be trademarks or service marks of others. Important Notes Processor speeds indicate the i nte rnal clock speed of the microproce ssor; other factors also affect application performance. CD-R OM dri [...]

xxxii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Notices Electronic E mission Notices In the United States, IBM has estab lished a collection process for reuse, rec ycling, or proper disposal of used IBM seal ed lead acid, nick el cadmium, n ickel meta l hydride, and b attery packs from IB M equipment. For in form[...]

xxxiii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Notices Electronic Emission Notices Industry Canada Class A emission compliance statement This Class A digital app aratus complies with Canadian ICES- 003. A vis de conformité à la réglemen tation d'Industrie Canad a Cet appareil numérique de la classe A es[...]

xxxiv Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Notices Electronic E mission Notices Chinese Class A warning statement Japanese Voluntary Control Council for Interference (VCCI) statement[...]

xxxv Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 WEEE Notice Note This mark applies only to countri es within the European Union (EU) and Norw ay . This appliance i s labelled in accordance with European Directi v e 2002/96/EC concerni ng waste electrical and electronic equipment (WEEE). The Dire cti ve determines t[...]

xxxvi Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 WEEE Noti ce[...]

xxxvii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Safety[...]

xxxviii Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Safety[...]

xxxix Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Safety[...]

xl Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Safety[...]

CH A P T E R 1-1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 1 About the Server Switch Module This chapter describes the Ci sco 4x Inf iniBand Sw itch module for IBM BladeCenter . This chapter contains the follo wing sections: • Introduction, page 1-1 • T opologies, page 1-3 • Module Manage ment, page 1-3 • [...]

1-2 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 1 Abo ut the Server Switch Module Introduction Figur e 1 -1 Cisco 4x InfiniBand S witc h Module for IBM BladeCen ter Each Server Switch Module includes 16 4x p orts to the backplane and 8 4 x ports (in the form of 2 4x connectors and 2 12x connectors) on the f[...]

1-3 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Chapter 1 About the Server Switch Module Topologies Topologies W ith the Serv er Switch Mo dule and HCA e xpansion cards, you can create a non-redu ndant, single-switch topology or a redund ant, dual-switc h topology . Note There are four I/O module bays in the switch,[...]

1-4 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 1 Abo ut the Server Switch Module LEDs LEDs The Server Switch Module pro vides m odule status LEDs and port status LEDs. Ta b l e 1-1 provides more information abou t the LEDs. Note External port numbers be gin at 17 because ports 1 through 16 are internal. Ta[...]

1-5 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Chapter 1 About the Server Switch Module External InfiniBand Ports Port Status LEDs The port LEDs on the Server Switch Module indicat e connection and transmission status (see tables belo w). Link LEDs indicate connection st atus . T raf fic LEDs indicate transmission [...]

1-6 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 1 Abo ut the Server Switch Module External InfiniBand Ports[...]

CH A P T E R 2-1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 2 Installing and Configuring the Server Switch This chapter describes install ing and confi gurin g the Serv er Switch Module and conf iguring the password and system clock , Additional conf igurin g includes synchroni zing with NTP and e xtending an Infi [...]

2-2 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 2 Installing and Configuring the Server Switch Configuring th e Server Switch Module Step 5 Click Cisco Inf iniBand Switch Module for IBM BladeCenter to obtain the operat ing system T opspinOS v2.6.0. The software is an ISO image. Step 6 Burn the ISO image ont[...]

2-3 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Chapter 2 Installing a nd Configuring the Server Switch Configuring the Server Switch Module Step 9 Open a T elnet session to the Serv er Switch Module with the I P address that you conf igured in Step 6 . Log in to your Ser ver Swit ch Module, an d then proceed t o ?[...]

2-4 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 2 Installing and Configuring the Server Switch Configuring th e Server Switch Module Step 2 Set the on-board clock b y entering the cl ock set command with th e follo wing v ariables: • the time of day in hh:mm:ss format • the day in dd format • the mont[...]

2-5 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 Chapter 2 Installing a nd Configuring the Server Switch Connecting Beyon d BladeCenter Connecting Beyond BladeCenter T o extend y our Inf iniBand fabric be yond your BladeCenter clu ster , you must conf igure the pri ority of the subnet manager on your Serv er Switch M[...]

2-6 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 2 Installing and Configuring the Server Switch Connecting B eyond BladeCenter Figur e 2-1 Removing a Pull Connect or 154590[...]

CH A P T E R 3-1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 3 Specifications This chapter details th e Server Switch Module specif ications. Electrical Requirements Ta b l e 3-1 DC Supply Requir ements Note External port po wered de vices are limited to 21 w atts total. Environmental Requirements Ta b l e 3-2 Envir[...]

3-2 Cisco 4x InfiniBand Switch Module for IBM Bla deCenter User Guide 78-17838-01 Chapter 3 Specifications Environmental Requir ements Note In the case of non-uniform air flo w , the velocity at an y point must meet or exceed 150 LFM. Operation al V ibration 0.27G at 5Hz to 500Hz for 30 m inutes Storage Shock 50G with a veloci ty change of 3454 mm/[...]

IN-1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 INDEX A about the server switch module 1-1 C cable connection 2-5 change the default password 2-3 clock, setting 2-3 configuration, initial 2-2 configuring the ser v er switch module 2-2 connect IB cables 2-5 connect to Topspi n IB networks 2-5 D documentation convent[...]

Index IN-2 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01 T topology dual-switch 1-3 single-switch 1-3[...]