Sun Microsystems CP3240 manual

Sun Microsystems, Inc. www .sun.com Submit comments about this document at: http://www.sun.com/hwdocs/feedback Sun Netra ™ CP3240 Switch Installation Guide P ar t No . 820-3251-13 September 2009, Revision A[...]

Please Recycle Copyright © 2009 Sun Microsystems, Inc., 4150 Network Cir cle, Santa Clara, California 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual pr operty rights relating to technology embodied in the pr oduct that is described in this document. In particular , and without limitation, these intellectual property rig[...]

Please Recycle[...]

[...]

v Contents Preface xix 1. Getting Started 1–1 1.1 System Requirements 1–2 1.1.1 Connectivity 1–2 1.1.1.1 Hub Connectivity 1–2 1.1.2 Electrical and Environmental 1–2 1.2 Unpacking 1–4 1.3 Handling Switches 1–4 1.4 Connectors 1–5 1.5 Jumper Options 1–5 1.6 Switch and R TM Faceplates 1–6 1.6.1 Base 10/100/1000 Uplink Ports (RJ-45) [...]

vi Sun Netra CP3240 Switch Installation Guide • September 2009 1.8.1.1 Removing a Switch From the Fr ont of the Server 1–1 1 1.8.1.2 Removing a Rear T ransition Module for a Switch 1–13 1.8.2 Installing a Switch Set 1–14 1.8.2.1 Installing the Rear T ransition Module for a Switch 1–14 1.8.2.2 Installing a Switch 1–14 1.9 Switch LEDs 1?[...]

Contents vii 2.3.2 F ASTP A TH Routing 2–1 1 2.3.3 F ASTP A TH Quality of Service 2–12 2.3.3.1 DiffServ 2–12 2.3.3.2 Access Control Lists (ACLs) 2–12 2.3.3.3 Class of Service (CoS) 2–13 2.3.3.4 Quality of Service MIBs 2–13 2.3.4 F ASTP A TH Multicast 2–13 2.3.4.1 Multicast MIBs 2–14 2.3.5 F ASTP A TH IPv6 Routing 2–14 2.3.5.1 IPv6[...]

viii Sun Netra CP3240 Switch Installation Guide • September 2009 3.1.2 P6(1-2) Fabric Zero Reset Conf iguration W ord 3–3 3.1.3 P6(3-4) Base Zero Reset Conf iguration W ord 3–3 3.1.4 P8(1-2) Base W rite Protect 3–3 3.1.5 P8(3-4) Fabric W rite Protect 3–4 3.1.6 P9 IPMC Firmware Pr ogram 3–4 3.1.7 P10(1-2) IPMC Reset 3–4 3.1.8 P10(3-4) [...]

Contents ix 4.6 Boot Sequence 4–10 4.6.1 Boot Utility Menu 4–1 1 4.6.1.1 Load Code Update Package using TFTP/FTP 4–12 4.6.1.2 Erase Current Conf iguration 4–12 4.6.1.3 Erase Permanent Storage 4–12 4.6.1.4 Select Boot Method 4–12 4.6.1.5 Start Diagnostic Application 4–13 4.7 Primary and Backup Flash 4–13 4.8 Dual Firmware Images 4–[...]

x Sun Netra CP3240 Switch Installation Guide • September 2009 4.1 1.2 Firmware Upgrades 4–24 4.12 Fiber Optic R TM Configuration 4–25 4.12.1 Module Support 4–25 4.12.2 Backplane Ports V ersus R TM Optical Ports 4–25 4.12.3 10G Configuration for R TM Optical Port 4–25 4.12.4 1G Configuration for R TM Optical Port 4–26 A. Environment Sp[...]

Contents xi B.14 A TCA Zone 3 R TM Connector (J6-Bottom) B–12 C. Datasheet Reference C–1 D. Agency Certif ications D–1 D.1 CE Certification D–1 D.2 NEBS/ETSI D–1 D.3 Safety D–2 D.4 Emissions T est Regulations D–3 D.4.1 EN 50081-1 Emissions D–3 D.4.2 EN 55024 Immunity D–3 D.5 Regulatory Information D–4 D.5.1 D.4.1FCC (USA) D–4 [...]

xii Sun Netra CP3240 Switch Installation Guide • September 2009[...]

xiii Figur es FIGURE 1-1 Switch and RTM Faceplates 1–6 FIGURE 1-2 Netra CP3240H-RTM-CU Airflow (Pa) 1–8 FIGURE 1-3 Netra CP3240H-RTM-CU Airflow (InchesH20) 1–8 FIGURE 1-4 Netra CP3240H-RTM-OP Airflow (Pa) 1–9 FIGURE 1-5 Netra CP3240H-RTM-OP Airflow (InchesH20) 1–9 FIGURE 1-6 Front Cable Management Bracket in Lower Position 1–11 FIGURE 1[...]

xiv Sun Netra CP3240 Switch Installation Guide • September 2009[...]

xv T ables TABLE 1-1 Power Requirements 1–2 TABLE 1-2 Airflow Requirements 1–7 TABLE 1-3 ATCA Board Status LEDs 1–15 TABLE 1-4 Hot-Swap LED States 1–16 TABLE 2-1 Switch External Ports 2–18 TABLE 2-2 Zone 3 RTM Ports 2–18 TABLE 2-3 Copper RTM External Ports 2–20 TABLE 2-4 Copper RTM Port Restrictions 2–20 TABLE 2-5 Fiber Optic Extern[...]

xvi Sun Netra CP3240 Switch Installation Guide • September 2009 TABLE 3-3 P6 (1-2) Fabric Zero Reset Jumper Settings 3–3 TABLE 3-4 P6 (3-4) Base Zero Reset Jumper Settings 3–3 TABLE 3-5 P8 (1-2) Base Write-Protect Jumper Settings 3–3 TABLE 3-6 P8 (3-4) Fabric Write Protect Jumper Settings 3–4 TABLE 3-7 P9 IPMC Firmware Program Jumper Sett[...]

T ables xvii TABLE B-10 10/100/1000Base-T RJ-45 Connector (J13-3rd and Bottom) B–10 TABLE B-11 10/100Base-TX RJ-45 Connector (J13-2nd) B–10 TABLE B-12 ATCA Zone 3 RTM Connector (J8-Top)) B–11 TABLE B-13 ATCA Zone 3 RTM Connector (J7-Middle) B–11 TABLE B-14 ATCA Zone 3 RTM Connector (J6-Bottom) B–12[...]

xviii Sun Netra CP3240 Switch Installation Guide • September 2009[...]

xix Pr eface The Sun Netra CP3240 Switch Installation Guide describes the installation and configuration of the Sun Netra CP3240 switch. This guide also includes information about software, envir onment specifications, connectors, and certifications. Befor e Y ou Read This Document Obtain and read the following documents: ■ Sun Netra CP3x40 Switc[...]

xx Sun Netra CP3240 Switch Installation Guide • September 2009 How This Document Is Or ganized Chapter 1 provides unpacking instructions and initial setup information for the switch. It provides information and procedur es needed to install and make the Sun Netra CP3240 switch operational. Chapter 2 introduces the key features of the switch. This[...]

Preface xxi T ypographic Conventions Note – Characters display differently depending on br owser settings. If characters do not display correctly , change the character encoding in your br owser to Unicode UTF-8. T ypeface Meaning Examples AaBbCc123 The names of commands, files, and directories; on-scr een computer output Edit your .login file. U[...]

xxii Sun Netra CP3240 Switch Installation Guide • September 2009 Related Documentation The following table lists the documentation for this product. The online documentation is available at: http://docs.sun.com/app/docs/prod/cp3240.switch?l=en#hic The following table lists the documentation that is related to this product. The online documentatio[...]

Preface xxiii . Thir d-Party W eb Sites Sun is not responsible for the availability of third-party web sites mentioned in this document. Sun does not endorse and is not responsible or liable for any content, advertising, products, or other materials that are available on or thr ough such sites or resour ces. Sun will not be responsible or liable fo[...]

xxiv Sun Netra CP3240 Switch Installation Guide • September 2009 http://www.sun.com/hwdocs/feedback Please include the title and part number of your document with your feedback: Sun Netra CP3240 Switch Installation Guide , part number 820-3251-13.[...]

1-1 CHAPTER 1 Getting Started This chapter provides information and procedur es needed to install and make the Sun Netra CP3240 switch operational. This chapter should be read before using the board. In addition to this chapter , r efer to the following safety documentation: ■ Netra CP3X20 Switch Safety and Compliance Manual (820-3505) ■ Import[...]

1-2 Sun Netra CP3240 Switch Installation Guide • September 2009 1.1 System Requir ements The following sections briefly describe the minimum system r equirements and the configurable featur es. Links are pr ovided to other chapters and appendices containing more detailed information. 1.1.1 Connectivity The switch can work in any AdvancedTCA shelf[...]

Chapter 1 Getting Star ted 1-3 The 48VDC has a tolerance of 0VDC to 75VDC without damage. The switch will operate when 48VDC is 36VDC to 72VDC, inclusive. Caution – Any input voltage outside the range of 0VDC to 75VDC can damage the switch. The switch requir es air flow . A minimum of 100 LFM should be kept on the board at all times. If an R TM i[...]

1-4 Sun Netra CP3240 Switch Installation Guide • September 2009 1.2 Unpacking Check the shipping carton for damage. If the shipping carton or contents are damaged, notify the carrier and Sun. Retain the shipping carton and packing material for inspection by the carrier . Obtain authorization before returning any product to Sun. Refer to the Netra[...]

Chapter 1 Getting Star ted 1-5 Caution – The heat sinks on a switch can be damaged by incorrect handling. Do not touch the heat sinks while installing or removing a switch. Hold a switch only by the edges. If a heat sink is loose or broken, obtain a replacement switch. Caution – The heat sinks on a switch can be damaged by improper packaging. W[...]

1-6 Sun Netra CP3240 Switch Installation Guide • September 2009 1.6 Switch and R TM Faceplates The following shows the Sun Netra CP3240 switch and the copper R TM (XCP3240H-R TM-CU-Z) and the fiber optic R TM (XCP3240H-R TM-OP-Z). FIGURE 1-1 Switch and R TM Faceplates Copper R TM Fiber Optic RT M Switch Faceplate[...]

Chapter 1 Getting Star ted 1-7 1.6.1 Base 10/100/1000 Uplink Ports (RJ-45) There ar e two front panel Base 10/100/1000Base-T Ethernet uplink ports on the faceplate. They are ports number 18 and 19 on the Base network. 1.6.2 10/100 Management Port (RJ-45) There is a 10/100 management Ethernet port on the faceplate. This port can be used to manage th[...]

1-8 Sun Netra CP3240 Switch Installation Guide • September 2009 1.7.2 R TM Pr essure Dr op V ersus Airflows (Impedence Curves) The following graphs plot the impedance curves for the RTMs, per PICMG 3.0 requir ements. FIGURE 1-2 Netra CP3240H-R TM-CU Airflow (Pa) FIGURE 1-3 Netra CP3240H-R TM-CU Airflow (InchesH 2 0)[...]

Chapter 1 Getting Star ted 1-9 FIGURE 1-4 Netra CP3240H-R TM-OP Airflow (Pa) FIGURE 1-5 Netra CP3240H-R TM-OP Airflow (InchesH20)[...]

1-10 Sun Netra CP3240 Switch Installation Guide • September 2009 1.8 Removing and Installing Switches This section describes how to remove and install switches. If you are hot-swapping the r ear transition module for a switch, you must remove the switch from the fr ont of the system before r emoving the rear transition module from the r ear of th[...]

Chapter 1 Getting Star ted 1-11 1.8.1.1 Removing a Switch From the Fr ont of the Server 1. Move the front cable management bracket to the lower position ( FIGURE 1-6 ). FIGURE 1-6 Fr ont Cable Management Bracket in Lower Position 2. Disengage the injector/ejector mechanisms at the top and bottom of the board to notify software that the board is abo[...]

1-12 Sun Netra CP3240 Switch Installation Guide • September 2009 FIGURE 1-7 Injector/Ejector on the Switch (Open Position) 3. Disconnect all cables connected to the switch. 4. Loosen the two board retention screws that fasten the board to the enclosure. 5. Open the ejectors fully , rotating the handles outward until the board disengages from the [...]

Chapter 1 Getting Star ted 1-13 Note – As long as the switch is removed fr om the slot, and the system is running with only the single remaining switch, you do not have r edundancy for that component. Both switches must be installed and running to have redundancy for that component. 1.8.1.2 Removing a Rear T ransition Module for a Switch 1. Remov[...]

1-14 Sun Netra CP3240 Switch Installation Guide • September 2009 1.8.2 Installing a Switch Set If you have a rear -access server , you must install the rear transition module befor e installing the front switch. ■ If you want to install the rear transition module, go to Section 1.8.2.1, “Installing the Rear T ransition Module for a Switch” [...]

Chapter 1 Getting Star ted 1-15 4. Prepare the switch by opening its injector/ejector handles ( FIGURE 1-7 ). 5. Carefully align the edges of the board with the guides in the slot. Look into the enclosure to verify correct alignment of the rails in the guides. 6. Keeping the board aligned in the guides, slide the board in until the injector/ejector[...]

1-16 Sun Netra CP3240 Switch Installation Guide • September 2009 Note that both the OOS and MINOR LEDs are lit when the boar d is powered but not booted. This includes all Hot-Swap states M1 through M3. Refer to “Hot-Swap LED” on page 1-16 . 1.9.2 Hot-Swap LED This blue LED communicates the Hot-Swap status of the switch. T ABLE 1-4 shows the [...]

2-1 CHAPTER 2 Overview This chapter introduces the key featur es of the switch and R TMs. This chapter includes a product def inition, a list of product features, and functional block diagrams with brief descriptions. This chapter can be used to compare the features of the switch against the needs of a specific application. This chapter contains th[...]

2-2 Sun Netra CP3240 Switch Installation Guide • September 2009 2.1 Featur es Part of Sun’s A TCA platform, the Sun Netra CP3240 switch complies with PICMG 3.0 R2.0 ECN002 and PCIMG 3.1 Option 1 and Option 9. The Sun Netra CP3240 switch implements two separate switched networks on a single PCB. By separating the Base and Fabric networks, the Su[...]

Chapter 2 Overview 2-3 2.1.2 Base Interface ■ Broadcom BCM56503-based design ■ Full gigabit non-blocking, wire-speed switching/routing ■ Layer 2 switching ■ Layer 3 routing ■ 27 ports: ■ 14 10/100/1000Base-T Ethernet ports for node slots ■ 1 10/100/1000Base-T Ethernet port for redundant switch ■ 2 10/100/1000Base-T Ethernet ports fo[...]

2-4 Sun Netra CP3240 Switch Installation Guide • September 2009 2.1.3 Fabric Gigabit Interface ■ Broadcom BCM56800-based design ■ Full gigabit non-blocking, wire-speed switching/routing ■ Layer 2 switching ■ Layer 3 routing ■ 20 total ports: ■ 14 10GBase-BX4 Ethernet ports for node slots ■ 1 10GBase-BX4 Ethernet port for redundant s[...]

Chapter 2 Overview 2-5 2.1.4 AMC Sites ■ Site 1 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 22 ■ Site 2 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 23 ■ Site 2 10GBase-BX to 3.1 network (AMC port 8-1 1) is ethernet port 16 ■ Site 3 1000Base-BX to 3.0 network (AMC port 0) is ethernet port 24 Site 1: AMC.0 Mid-size, [...]

2-6 Sun Netra CP3240 Switch Installation Guide • September 2009 2.2 Switch Components The following sections list and describe key components of the Sun Netra CP3240 switch. 2.2.1 Br oadcom StrataXGS 3 BCM56503 Ethernet Switch The Sun Netra CP3240 switch uses Broadcom StrataXGS 3 BCM56503 for Base Ethernet switching/routing. This chip is an Ether[...]

Chapter 2 Overview 2-7 2.2.4 Fr eescale PowerQUICC II MPC8247 Communications Pr ocessor The Freescale MPC8247 is a micr oprocessor designed for maximum flexibility . It features a dual cor e architectur e with a PPC G2 LE core and a RISC core controlling the peripherals. Running at 400MHz with only 1 watt of power , the MPC8247 provides high perfor[...]

2-8 Sun Netra CP3240 Switch Installation Guide • September 2009 2.3 Pr otocols, RFCs, and MIBs Support The Sun Netra CP3240 switch features Level7 FastPath 2340 switching software, version 4.4.4 and newer . This softwar e provides layer 2 switching, quality of service, IPv4 routing, IPv6 r outing, and IP multicast. 2.3.1 F ASTP A TH Switching ■[...]

Chapter 2 Overview 2-9 2.3.1.1 Additional Layer 2 Functionality ■ Broadcast/Multicast/Unknown Unicast storm recovery ■ Double VLAN/vMAN tagging ■ DHCP filtering ■ Jumbo Ethernet frames ■ Many-to-one port mirroring (Tx, Rx, both) ■ Static MAC filtering ■ MAC-based VLANs ■ IP subnet-based VLANs ■ Port description ■ Protected ports[...]

2-10 Sun Netra CP3240 Switch Installation Guide • September 2009 ■ RFC 3164—BSD Syslog protocol ■ RFC 3396—Encoding long options in the Dynamic Host Configuration Protocol (DHCPv4) ■ RFC 3580—802.1X RADIUS usage guidelines 2.3.1.3 Switching MIBs ■ RFC 1213 — MIB-II ■ RFC 1493 — Bridge MIB ■ RFC 1643 — Ethernet-like MIB ■[...]

Chapter 2 Overview 2-11 2.3.2 F ASTP A TH Routing ■ RFC 826—Ethernet ARP ■ RFC 894—T ransmission of IP Datagrams over Ethernet networks ■ RFC 896—Congestion control in IP/TCP networks ■ RFC 919—IP Broadcast ■ RFC 922—IP Broadcast in the presence of subnets ■ RFC 950—IP subnetting ■ RFC 1027—Using ARP to implement T ransp[...]

2-12 Sun Netra CP3240 Switch Installation Guide • September 2009 2.3.3 F ASTP A TH Quality of Service 2.3.3.1 DiffServ ■ RFC 2474—Definition of the Dif ferentiated Services f ield (DS field) in the IPv4 and IPv6 headers ■ RFC 2475—Architectur e for differ entiated services ■ RFC 2597—Assured Forwar ding PHB Group ■ RFC 3246—Expedi[...]

Chapter 2 Overview 2-13 2.3.3.3 Class of Service (CoS) Direct user conf iguration of the following: ■ IP DSCP to traff ic-class mapping ■ Interface trust mode: 802.1p, IP DSCP , or untrusted ■ Interface traff ic-shaping rate ■ Minimum and maximum bandwidth per queue ■ Strict priority versus weighted (WRR/WFQ) scheduling per queue ■ T ai[...]

2-14 Sun Netra CP3240 Switch Installation Guide • September 2009 2.3.4.1 Multicast MIBs ■ RFC 2932—IPv4 multicast routing MIB ■ RFC 2933—IGMP MIB ■ RFC 2934—PIM MIB for IPv4 ■ Draft-ietf-magma-mgmd-mib-03.txt—Multicast Group Membership Discovery MIB ■ Draft-ietf-idmr-dvmrp-mib-1 1.txt—DVMRP MIB ■ IANA-R TP-PROTO-MIB ■ F AS[...]

Chapter 2 Overview 2-15 2.3.5.1 IPv6 Routing MIBs ■ RFC 2465—IPv6 MIB ■ RFC 2466—ICMPv6 MIB ■ F ASTP A TH Enterprise MIBs supporting IPv6 features 2.3.6 F ASTP A TH Management ■ RFC 854—T elnet ■ RFC 855—T elnet option specifications ■ RFC 1 155—SMI v1 ■ RFC 1 157—SNMP ■ RFC 1212—Concise MIB definitions ■ RFC 1867—[...]

2-16 Sun Netra CP3240 Switch Installation Guide • September 2009 ■ RFC 3417—T ransport mappings for SNMP ■ RFC 3418—MIB for SNMP ■ Configurable management VLAN 2.3.6.1 SSL 3.0 and TLS 1.0 ■ RFC 2246—TLS protocol, version 1.0 ■ RFC 2346—AES Ciphersuites for transport layer security ■ RFC 2818—HTTP over TLS 2.3.6.2 SSH 1.5 and[...]

Chapter 2 Overview 2-17 2.4 Functional Diagrams and Port Maps 2.4.1 Switch FIGURE 2-1 Switch Functional Block Diagram[...]

2-18 Sun Netra CP3240 Switch Installation Guide • September 2009 2.4.2 Rear T ransition Modules (R TM)s The Sun Netra CP3240 switch supports R TMs through A TCA Zone 3 connectors (see T ABLE 2-2 ). The Sun Netra CP3240 switch was designed to support multiple R TM designs. 4 Fabric 10Gigabyte, 2 Base 10Gigabyte, 2 Base 1Gigabyte, AMC I/O, and mana[...]

Chapter 2 Overview 2-19 2.4.2.1 Copper R TM The XCP3240H-R TM-CU-Z is the Sun Netra CP3240 switch’s copper R TM pairing. This R TM supports all the ports the Sun Netra CP3240 switch provides using 10GBase-CX4 for the 10Gigabyte and 10/100/1000Base-T for the 1Gigabyte ports. FIGURE 2-2 Copper R TM Functional Block Diagram[...]

2-20 Sun Netra CP3240 Switch Installation Guide • September 2009 T ABLE 2-3 Copper RTM External Ports P ort T ype RTM Module Logical Port Comments Base 10GBE XAUI CX4 Base - 27 Base 10GBE XAUI CX4 Base - 26 Base 1GBE 10/100/1000Base T RJ-45 Base - 20 Standar d I/F is copper Base 1GBE 10/100/1000Base T RJ-45 Base - 21 Standar d I/F is copper Manag[...]

Chapter 2 Overview 2-21 2.4.2.2 Fiber Optic R TM The XCP3240H-R TM-OP-Z is the Sun Netra CP3240 switch’s fiber optic R TM pairing. This R TM supports all the ports the Netra CP3240 switch provides using 10GBase-X for the 10Gigabyte and 1000Base-X for the 1Gigabyte ports. FIGURE 2-3 Fiber Optic R TM Functional Block Diagram[...]

2-22 Sun Netra CP3240 Switch Installation Guide • September 2009 Note – For enabling 1G operation of Fabric ports 17-20, see Section 4.12, “Fiber Optic R TM Configuration” on page 4-25 . T ABLE 2-5 Fiber Optic External RTM Ports P ort T ype Fiber Optic RTM Module Logical Port Comments Management 10/100 Base T RJ-45 Not a module, 10/100Base [...]

Chapter 2 Overview 2-23 2.4.3 Base Fabric Switch Subsystem FIGURE 2-4 Base Fabric Switch Subsystem PICMG 3.0 AdvancedTCA defines 10/100/1000Base-T Ethernet over the Base Fabric, called “Base” in this guide. The Base is designed to be the control plane for the shelf. Scaling from 10Mbps to 1000Mbps, the Base interface can accommodate a wide vari[...]

2-24 Sun Netra CP3240 Switch Installation Guide • September 2009 2.4.4 Expansion Fabric Switch Subsystem FIGURE 2-5 Expansion Fabric Switch Subsystem PICMG 3.0 AdvancedTCA provides an agnostic mesh on the backplane called the Expansion Fabric. This interface is the data plane in the shelf. The Expansion Fabric is called “Fabric” in this guide[...]

Chapter 2 Overview 2-25 2.4.5 AdvancedMC Sites PICMG AdvancedMC AMC.0 defines hot-swappable daughter car ds to be used in PICMG architectur es. The Sun Netra CP3240 switch supports three AMC sites so that the functionality of the Sun Netra CP3240 switch can be expanded. AMC.2 (Ethernet) AMCs are supported with dir ect connections to the Base and Fa[...]

2-26 Sun Netra CP3240 Switch Installation Guide • September 2009 2.4.5.1 AMC Port Maps FIGURE 2-6 AMC Port Map Diagram[...]

Chapter 2 Overview 2-27 T ABLE 2-7 AMC Ethernet Port A vailability Slot Location 1G Infrastructure 10G Infrastructure AMC 1 T op 1G Base SGMII None AMC 2 Middle 1G Base SGMII XAUI from BCM56800 Fabric Port 16 AMC 3 Bottom 1G Base SGMII None T ABLE 2-8 AMC Site 1: AMC.0 Mid-size, AMC.2 T ype E1, AMC.2 T ype 5 (Optional), LED Module Support P ort T y[...]

2-28 Sun Netra CP3240 Switch Installation Guide • September 2009 TCLK2 CLK OUT FPGA and Zarlink TCLK3 CLK IN FPGA and Zarlink TCLK4 CLK OUT FPGA and Zarlink FCLK 100MHz 100MHz T ABLE 2-9 AMC Site #2 AMC.0 Mid-Size, AMC.2 T ype E1, AMC.2 T ype 5, Double module support, LED Module Support P ort T ype Connected 0 GbE Base (3.0) switch 1 Any Port 1 o[...]

Chapter 2 Overview 2-29 T ABLE 2-10 AMC Site 3: AMC.0 Mid-size, AMC.2 T ype E1, Master Clock Generator Support, LED Support P ort T ype Connected 0 GB Base (3.0) switch 1 Any Port 1 of AMC #2, usually used for GB 2 Any Port 2 of AMC #2, usually used for SAS/SA T A 3 Any Port 2 of AMC #1, usually used for SAS/SA T A 4 Not connected 5 Not connected 6[...]

2-30 Sun Netra CP3240 Switch Installation Guide • September 2009 2.4.5.2 AMC Module Support by Site 2.4.5.3 AMC Port Restrictions for R TMs The following tables list AMC port restrictions for R TMs T ABLE 2-11 AMC Modules by Site AMC Module Site 1 (top most when the board is vertical) Site 2 (middle) Site 3 (bottom most site when the board is ver[...]

3-1 CHAPTER 3 Conf iguring Jumper Settings This chapter describes jumper settings for configuring the switch. Other configuration options ar e software contr olled. Software conf iguration options are described in Chapter 4 . This chapter contains the following topics: ■ Section 3.1, “Jumper Settings” on page 3-2 ■ Section 3.2, “Jumper Lo[...]

3-2 Sun Netra CP3240 Switch Installation Guide • September 2009 3.1 Jumper Settings The following jumper settings are available for conf iguring the switch. Each of jumper settings is described in the subsections. 3.1.1 P4 Cr oss-Connect Contr ol This jumper is used to control ShMC cr oss connect. ShMC cross connect is the ability to connect to t[...]

Chapter 3 Configur ing Jumper Settings 3-3 3.1.2 P6(1-2) Fabric Zer o Reset Conf iguration W or d This jumper is used to configur e the Fabric Gigabyte CPU to use the default Reset Configuration W ord, or use the one in the flash. 3.1.3 P6(3-4) Base Zer o Reset Conf iguration W or d This jumper is used to configur e the Base CPU to use the default [...]

3-4 Sun Netra CP3240 Switch Installation Guide • September 2009 3.1.5 P8(3-4) Fabric W rite Protect This jumper configur es write protection on the Fabric reset configuration wor d. 3.1.6 P9 IPMC Firmwar e Pr ogram These jumpers configure IPMC f irmware write actions via the debug serial port. 3.1.7 P10(1-2) IPMC Reset This jumper configur es the[...]

Chapter 3 Configur ing Jumper Settings 3-5 3.1.8 P10(3-4) IPMC FWE These are r eserved and should be left configur ed for the default setting. 3.1.9 P1 1(1-2) Forced-Boar d Enable This jumper controls whether the switch is for ced on when it is powered up, or if the IPMI subsystem controls the power up of the switch. For cing power to the board is [...]

3-6 Sun Netra CP3240 Switch Installation Guide • September 2009 3.1.1 1 P12(1-2) and P12 (3-4) EMI Ground to Logic Gr ound The switch and the entire AdvancedTCA shelf separate the ground of the chassis itself from digital gr ound for EMI protection. This jumper conf igures the two grounds. 3.1.12 P13 Serial Dir ection The front panel serial port [...]

Chapter 3 Configur ing Jumper Settings 3-7 3.2 Jumper Locations FIGURE 3-1 shows the locations of the jumper settings. FIGURE 3-1 Switch Jumper Locations[...]

3-8 Sun Netra CP3240 Switch Installation Guide • September 2009[...]

4-1 CHAPTER 4 Conf iguring Switch Software This chapter describes how to configure the switch softwar e. The switch incorporates four main software components: ■ IPMI firmwar e, which runs on a separate microcontr oller . ■ uBoot loads the boot for the system. It can be compared to a BIOS on a node board. ■ Operating system (OS), which uses M[...]

4-2 Sun Netra CP3240 Switch Installation Guide • September 2009 4.1 IPMI Firmwar e Sensors This section describes the IPMI firmware sensors. 4.1.1 State Sensors T ABLE 4-1 IPMI State Sensors Sensor Name T ype Description Discrete States Returned T ABLE 4-2 0 FRU 0 HOT_SWAP State hotswap discr ete state M state, per A TCA spec 1 RTM hotswap State [...]

Chapter 4 Configur ing Switch Software 4-3 *These sensors represent the software state of the switch. The following tables lists the logic that drives these sensors. T ABLE 4-3 IPMI Sensor Logic State Timeout Failover Condition Early Full Good Initial power-on state 5 sec Uboot corrupted or erased 001 uBoot initial state 5 sec Uboot finds no image [...]

4-4 Sun Netra CP3240 Switch Installation Guide • September 2009 4.1.2 Thr eshold Sensors T ABLE 4-4 IPMI Thresold Sensors Sensor Name T ype Description Units Lower Thresholds Upper Thresholds # Minor Major Critical Minor Major Critical 7 +12.0V Thresh +12V main power V olts 1 1.016 10.21 9.02 13.45 13.72 13.77 8 +3.3V Thresh +3.3V line V olts 3.1[...]

Chapter 4 Configur ing Switch Software 4-5 4.2 uBoot uBoot is the boot loader . Much like a BIOS, it brings the system to a usable state for the operating system (OS) to boot. It also performs a POST of the CPU subsystem. It can be used as a recovery console if the firmwar e image becomes corrupt or a firmwar e update fails. Several important envir[...]

4-6 Sun Netra CP3240 Switch Installation Guide • September 2009 There is only a one-second delay to press a key . When the uBoot prompt is displayed, you can then enter any of the following commands. 4.2.2 E-Keying Contr ol in uBoot See the Section 4.3.1, “e-Keying” on page 4-7 for a description of e-Keying, and how it is supported. T o disab[...]

Chapter 4 Configur ing Switch Software 4-7 4.2.3 Serial-Baud Rate Contr ol in uBoot Y ou can change the serial baud rate from within F ASTP A TH, the boot menu, or in uBoot. Changing it in uBoot is the only method that allows it to persist through a reset. Only standar d baud rates are accepted. Note – Change the baud rate on the Fabric first, th[...]

4-8 Sun Netra CP3240 Switch Installation Guide • September 2009 4.3.2 e-Keying Bypass e-Keying can be ignored by cr eating a uBoot environment variable as described in Section 4.2.2, “E-Keying Control in uBoot” on page 4-6 . When e-Keying is bypassed, ports are no longer enabled or disabled based on the e-Keying state. The e-Keying informatio[...]

Chapter 4 Configur ing Switch Software 4-9 4.5 Serial Location The switch supports running its serial port to both the front panel and the R TM. It does not support both the front panel and R TM at the same time. Y ou must configur e the port to go to the location needed. By default, the port is routed to the front panel. 4.5.1 Changing Serial Loca[...]

4-10 Sun Netra CP3240 Switch Installation Guide • September 2009 4.6 Boot Sequence The following is an example of a boot sequence. CODE EXAMPLE 4-3 Boot Sequence Example CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash ### JFFS2 loading 'image1' to 0x400000 Scanning JFFS2 FS: . done. ### JFFS2 load complete: 10026500 bytes lo[...]

Chapter 4 Configur ing Switch Software 4-11 The switch takes approximately 50 seconds to boot to a console and be fully functional. 4.6.1 Boot Utility Menu There is a utility menu that can be used before F ASTP A TH boots. During boot there is a two-option menu displayed for five seconds. This menu allows access to the utility menu if you press num[...]

4-12 Sun Netra CP3240 Switch Installation Guide • September 2009 4.6.1.1 Load Code Update Package using TFTP/FTP F ASTP A TH can be updated from within F ASTP A TH itself, but it can also be updated from this menu. ■ The update image must be on a TFTP server or FTP server . ■ The server must be on the same network as the out-of-band managemen[...]

Chapter 4 Configur ing Switch Software 4-13 4.6.1.5 Start Diagnostic Application This option boots the Broadcom diag shell, curr ently SDK version 5.4.1. This environment is pr ovided as is , with no support. Several commands provided in this shell are not supported and will not work. This shell is provided mainly for debug, testing, and diagnostic[...]

4-14 Sun Netra CP3240 Switch Installation Guide • September 2009 4.8.2 Updating the Non-Active Image All of the update methods supported by the active image (image1 by default) are also supported for the non-active image (image2). 4.8.3 Fabric 1G/10G Auto-negotiation The Fabric networks support both 1Gigabyte and 10Gigabyte operation. No configur[...]

Chapter 4 Configur ing Switch Software 4-15 9. Press number 1 to boot the system. CODE EXAMPLE 4-5 Example Console Output CPU: 400 MHz DRAM: 256 MB FLASH: 64 MB, Base, Primary Flash ### JFFS2 loading 'image1' to 0x400000 Scanning JFFS2 FS: . done. ### JFFS2 load complete: 10026500 bytes loaded to 0x400000 Booting ... Loading Ramdisk to 0f[...]

4-16 Sun Netra CP3240 Switch Installation Guide • September 2009 6 - Retrieve Error Log using TFTP 7 - Retrieve Error Log using XMODEM/YMODEM/ZMODEM 8 - Erase Current Configuration 9 - Erase Permanent Storage 10 - Select Boot Method 11 - Activate Backup Image 12 - Start Diagnostic Application 13 - Reboot Select option (1-13): 10 Current Boot Meth[...]

Chapter 4 Configur ing Switch Software 4-17 10. Use option 8 of the Utility Menu to change the boot method back to f lash, when desired. Select option (1-13): 1 Creating tmpfs filesystem on tmpfs for download...done. Bringing up eth0 interface...done. HOST IPADDR=10.10.2.246 Transferring '/switch.1.0.1.0.stk' from '10.10.3.199' [...]

4-18 Sun Netra CP3240 Switch Installation Guide • September 2009 4.10 F ASTP A TH The switch features L VL7 Systems’ F ASTP A TH 2340 softwar e, version 4.4. F ASTP A TH is a software package providing the r obust management needed to control a modern switch-r outer . This section serves as a quick primer on using F ASTP A TH on the switch. F A[...]

Chapter 4 Configur ing Switch Software 4-19 4.10.2 Basic CLI Commands T ABLE 4-6 Basic CLI Commands Command Function Mode enable Switch to privilege mode. Y ou must be in enabled mode for most options. default show port all Show status of the ports. privilege show interface ethernet 0/x Show detailed statistics on port 0/x. privilege clear counters[...]

4-20 Sun Netra CP3240 Switch Installation Guide • September 2009 4.10.3 Logins and Pr ompts F ASTP A TH supports multiple users with different security levels. By default, there is one admin user with no password. In the CLI, privilege mode is password-pr otected separately from the default mode, but also has a default of no password. 4.10.3.1 CL[...]

Chapter 4 Configur ing Switch Software 4-21 4.10.3.3 SNMP Defaults SNMP is enabled by default. The default read-only community string is public . The default read-write community string is private . 4.10.4 Secur e Remote Access 4.10.4.1 SSH and SSL/TLS Keys The switch supports SSH for a secure CLI console as well as SSL/TLS for secure HTTP . By def[...]

4-22 Sun Netra CP3240 Switch Installation Guide • September 2009 4.10.5 Default Settings The switch comes configur ed with a default configuration. This configuration boots the board to Layer 2 switching. This conf iguration is very basic and should be updated for your environment. The default settings are as follows: ■ every port is in VLAN 1 [...]

Chapter 4 Configur ing Switch Software 4-23 Note – If ports are or dered the same as the A TCA channels, Base and Fabric are always one number off. So, if a board is in slot 6, it is connected to Base channel 3 and Fabric channel 2. 398 476 554 632 7 Netra CP3240 Netra CP3240 821 943 10 6 5 11 8 7 12 10 9 13 12 1 1 14 14 13 15 15 14 16 16 15 ShMC[...]

4-24 Sun Netra CP3240 Switch Installation Guide • September 2009 4.1 1 Firmware Updates All of the firmwar e listed in this section is field upgradeable. Ther e is a single update image that supports the Base firmware, Fabric f irmware, and IPMC firmwar e. This update image must be installed twice though, once on the Base and once on the Fabric. [...]

Chapter 4 Configur ing Switch Software 4-25 4.12 Fiber Optic R TM Configuration For the fiber optic R TM (XCP3240H-R TM-OP-Z) to work correctly , switch firmwar e 1.2.1.0 or newer is requir ed. The following sections describe support and requir ements for configuring the RTM. 4.12.1 Module Support The Fiber Optic R TM was designed to work with any [...]

4-26 Sun Netra CP3240 Switch Installation Guide • September 2009 4.12.4 1G Conf iguration for R TM Optical Port For 1G operation, an SFP module must be used and the port (17-20) must be set to auto-negotiate . When you enable auto-negotiation, 1G operation is automatically enabled and 10G operation is automatically disabled. For all 10G R TM opti[...]

A-1 APPENDIX A Envir onment Specifications This appendix describes the electrical, environmental, and mechanical specifications. It includes connector descriptions and pin outs, as well as illustrations of the board dimensions and connector locations.[...]

A-2 Sun Netra CP3240 Switch Installation Guide • September 2009 A.1 Electrical and Envir onmental The following sections provide tables and illustrations showing the electrical and environmental specif ications. A.1.1 Absolute Maximum Ratings The following values are str ess ratings only . Do not operate at these maximums. See " Section A.1.[...]

Appendix A Environment Specifications A-3 A.2Reliability Reliability prediction was done using Issue 6, Method I, Case 3 of the Bellcore Industrial Reliability program. The pr ediction assumed 25% operating temperature with 100% duty cycle, in a ground benign, controlled environment. ■ MTBF: 122,173 hours ■ 2-year limited warranty A.3 Mechanica[...]

A-4 Sun Netra CP3240 Switch Installation Guide • September 2009 FIGURE A-1 PCB Dimensions[...]

B-1 APPENDIX B Connectors This appendix describes the switch’s connectors you can use to interface to application-specific devices.[...]

B-2 Sun Netra CP3240 Switch Installation Guide • September 2009 B.1 Connector Assignments T ABLE B-1 Connector Assignments Connector Function J13 T op Serial MGMT Port RJ-45 (Base and Fabric) J13 2nd 10/100 Ethernet MGMT Port RJ-45 (Base and Fabric) J13 3rd 10/100/1000 Ethernet Port RJ-45 (Base) J13 Bottom 10/100/1000 Ethernet Port RJ-45 (Base) J[...]

Appendix B Connectors B-3 B.2 Connector Locations (T opside) FIGURE B-1 Connector Locations (T opside) )[...]

B-4 Sun Netra CP3240 Switch Installation Guide • September 2009 FIGURE B-2 A TCA Zone 1 Connector (J9) T ABLE B-2 A TCA Zone 1 Connector (J9) Pins Pin # Pin Name Pin # Pin Name 1N C 1 8 N C 2N C 1 9 N C 3N C 2 0 N C 4N C 2 1 N C 5 HA(0) 22 NC 6 HA(1) 23 NC 7 HA(2) 24 NC 8 HA(3) 25 SHELF_GND 9 HA(4) 26 GND 10 HA(5) 27 NC 1 1 HA(6) 28 VR TN_A 12 HA[...]

Appendix B Connectors B-5 B.3 A TCA Zone 2 P20 ZD Connector (J5) B.4 A TCA Zone 2 P21 ZD Connector (J4) T ABLE B-3 A TCA Zone 2 P20 ZD Connector (J5) ab cdef g h 1 Clks CLK1A+ CLK1A- CLK1B+ CLK1B- CLK2A+ CLK2A- CLK2B+ CLK2B- 2 Upd ate Ch and Clks AMC3 P12_TX+ AMC3 P12_TX- AMC3 P12_RX+ AMC3 P12_RX- CLK3A+ CLK3A- CLK3B+ CLK3B- 3 Base Tx2 10GbE+ Base [...]

B-6 Sun Netra CP3240 Switch Installation Guide • September 2009 5 Fabri cC h 10 Tx2 10GB [10]+ Tx2 10GB [10]- Rx2 10GB [10]+ Rx2 10GB [10]- Tx3 10GB [10]+ Tx3 10GB [10]- Rx3 10GB [10]+ Rx3 10GB [10]- 6 Tx0 10GB [10]+ Tx0 10GB [10]- Rx0 10GB [10]+ Rx0 10GB [10]- Tx1 10GB [10]+ Tx1 10GB [10]- Rx1 10GB [10]+ Rx1 10GB [10]- 7 Fabri cC h 9 Tx2 10GB [9[...]

Appendix B Connectors B-7 B.5 A TCA Zone 2 P22 ZD Connector (J3) B.6 A TCA Zone 2 P23 ZD Connector (J2) T ABLE B-5 A TCA Zone 2 P22 ZD Connector (J3) ab cdef g h 1 Fabri cC h 7 Tx2 10GB [7]+ Tx2 10GB [7]- Rx2 10GB [7]+ Rx2 10GB [7]- Tx3 10GB [7]+ Tx3 10GB [7]- Rx3 10GB [7]+ Rx3 10GB [7]- 2 Tx0 10GB [7]+ Tx0 10GB [7]- Rx0 10GB [7]+ Rx0 10GB [7]- Tx1[...]

B-8 Sun Netra CP3240 Switch Installation Guide • September 2009 5 ShM C Po r t ( s) SHMC BI_A+ SHMC BI_A- SHMC BI_B+ SHMC BI_B- SHMC BI_C+ or SHMC2 BI_A+ SHMC BI_C- or SHMC2 BI_A- SHMC BI_D+ or SHMC2 BI_B+ SHMC BI_D- or SHMC2 BI_B- 6 Base Ch 2 BI_ DA2+ BI_ DA2- BI_ DB2+ BI_ DB2- BI_ DC2+ BI_ DC2- BI_ DD2+ BI_ DD2- 7 Base Ch 3 BI_ DA3+ BI_ DA3- BI[...]

Appendix B Connectors B-9 B.7 A TCA Zone 2 P24 ZD Connector (J1) B.8 Serial RJ-45 Connector (J13 T op) This connector is the Cisco/Intel pinout. T ABLE B-7 A TCA Zone 2 P24 ZD Connector (J1) ab cdef g h 1 Base Ch 7 BI_ DA7+ BI_ DA7- BI_ DB7+ BI_ DB7- BI_ DC7+ BI_ DC7- BI_ DD7+ BI_ DD7- 2 Base Ch 8 BI_ DA8+ BI_ DA8- BI_ DB8+ BI_ DB8- BI_ DC8+ BI_ DC[...]

B-10 Sun Netra CP3240 Switch Installation Guide • September 2009 B.9 Serial Cable The switch uses RJ-45 connectors for its serial ports. T o connect to a PC’s DB9 connector , a special cable or adapter is needed. These cables and adapters are available through various sour ces or can be made easily . The cable must cross over the TXD and RXD si[...]

Appendix B Connectors B-11 B.10 10/100/1000Base-T RJ-45 Connector (J13 3 rd , J13 Bottom) B.1 1 10/100Base-TX RJ-45 Connector (J13 2nd) T ABLE B-10 10/100/1000Base-T RJ-45 Connector (J13-3rd and Bottom) Signal 1 MDI_0+ 2 MDI_0- 3 MDI_1+ 4 MDI_2+ 5 MDI_2- 6 MDI_1- 7 MDI_3+ 8 MDI_3- T ABLE B-11 10/100Base-TX RJ-45 Connector (J13-2nd) Signal 1 Tx+ 2 T[...]

B-12 Sun Netra CP3240 Switch Installation Guide • September 2009 B.12 A TCA Zone 3 R TM Connector T op (J8) T ABLE B-12 A TCA Zone 3 RTM Connector (J8-T op)) B.13 A TCA Zone 3 R TM Connector (J7- Middle) T ABLE B-13 A TCA Zone 3 RTM Connector (J7-Middle) ab cdef g h 1 Fabric MDC Fabric MDIO Base MDCGE Base MDIO GE Base MDC XE Base MDIO XE Base PH[...]

Appendix B Connectors B-13 B.14 A TCA Zone 3 R TM Connector (J6- Bottom) T ABLE B-14 A TCA Zone 3 RTM Connector (J6-Bottom) ab cdef g h 1 Tx2 10GB [19]+ Tx2 10GB [19]- Rx2 10GB [19]+ Rx2 10GB [19]- Tx3 10GB [19]+ Tx3 10GB [19]- Rx3 10GB [19]+ Rx3 10GB [19]- 2 Tx0 10GB [19]+ Tx0 10GB [19]- Rx0 10GB [19]+ Rx0 10GB [19]- Tx1 10GB [19]+ Tx1 10GB [19]- [...]

C-1 APPENDIX C Datasheet Refer ence This appendix provides links to datasheets, standards, and specif ications for the technology designed into the switch. IEEE 802.3-2002 CDMA/CD (Ethernet) and Other IEEE 802.3/802.1 Documents This document defines Ethernet and several of the pr otocols used in Ethernet. Any 802 document can be obtained for free s[...]

C-2 Sun Netra CP3240 Switch Installation Guide • September 2009 Broadcom Datasheets A non-disclosure agr eement (NDA) with Broadcom is r equired to view their datasheets. Contact Broadcom for more information. http://www.broadcom.com/ MPC8272 PowerQUICC II Family Reference Manual This and other Freescale documents about the MPC8247 are available [...]

D-1 APPENDIX D Agency Certif ications This appendix lists standards agencies and the certifications r elated to the switch. D.1 CE Certif ication The Sun Netra CP3240 switch and R TM meets the intent of Directive 89/336/EEC for Electromagnetic Compatibility [EN55024:1998, EN55022:1998] and Low-V oltage Directive 73/23/EEC for Pr oduct Safety [EN609[...]

D-2 Sun Netra CP3240 Switch Installation Guide • September 2009 ■ ETSI EN 300 132-2, September 1996, Equipment Engineering Power Supply Interface At The Input T o T elecommunications Equipment; Part 2: Operated by direct curr ent (dc) ■ ETSI EN 300 753, October 1997, Acoustic Noise Emitted By T elecommunications Equipment Certification is dep[...]

Appendix D Agency Cer tifications D-3 D.4 Emissions T est Regulations ■ FCC Part 15, Subpart B Class A Commercial Equipment ■ Industry Canada ICES-003 Class A Commercial Equipment ■ CISPR 22/EN 55022:1998 Class A Radiated, Power line Conducted ■ T elcordia GR-1089-CORE EMC For Network T elecommunication Equipment Issue 4 Jun 2006 (Designed [...]

D-4 Sun Netra CP3240 Switch Installation Guide • September 2009 ■ IEC 61000-4-3:2003 EMC - Part 4. T esting and measurement techniques - Section 3: Radiated, radio-frequency , electromagnetic f ield immunity test ■ IEC 61000-4-4:2004 EMC - Part 4: T esting and measurement techniques - Section 4: Electrical fast transient/burst immunity test -[...]

Appendix D Agency Cer tifications D-5 D.5.2 Industry Canada (Canada) This Class A digital apparatus meets all requirements of the Canadian Interference Causing Equipment Regulations. Operation is subject to the following two conditions: 1. This device may not cause harmful interference. 2. This device must accept any interference received, includin[...]

D-6 Sun Netra CP3240 Switch Installation Guide • September 2009[...]

Index-1 Index A Airflow requir ements, 1-7 C cable management bracket lowering, 1-1 1, 1-14 H handling cards, cautions, 1-4 I installing switches, 1-14 switches, rear transition car ds, 1-14 L LEDs switches, 1-15 R removing switches, 1-1 1 switches, rear transition car ds, 1-13 S switches installing, 1-14 LEDs, 1-15 removing, 1-1 1 switches, rear [...]

Index-2 Sun Netra CP3240 Switch Installation Guide • September 2009[...]