Sun Microsystems CP3260 manual

Sun Microsystems, Inc. www .sun.com Submit comments about this document at: http://www.sun.com/hwdocs/feedback Sun Netra ™ CP3260 Blade Ser v er User’ s Guide P ar t No . 820-0457-11 April 2009, Revision 01[...]

Please Recycle Copyright © 2009 Sun Microsystems, Inc., 4150 Network Cir cle, Santa Clara, California 95054, U.S.A. All rights reserved. This distribution may include materials developed by third parties. Parts of the product may be derived fr om Berkeley BSD systems, licensed from the University of California. UNIX is a r egistered trademark in t[...]

iii Contents Preface xiii 1. Introduction 1–1 1.1 Overview 1–1 1.2 Features 1–2 1.3 System Configurations 1–8 1.4 Advanced Rear T ransition Modules 1–1 1 1.5 Hot-Swap Support 1–14 1.6 System Requirements and Options 1–14 1.6.1 Hardwar e Requirements and Options 1–14 1.6.2 Software Requir ements 1–15 1.7 T echnical Support and W ar[...]

iv Netra CP3260 Blade Server User’s Guide • April 2009 2.4 Installation Procedur e Summary 2–4 2.5 Configuring On-Boar d Hardware 2–5 2.5.1 Adding or Replacing FB-DIMM Memory Modules 2–5 2.5.1.1 T o Remove FB-DIMM Memory Modules 2–10 2.5.1.2 T o Install FB-DIMM Memory Modules 2–1 1 2.5.2 Adding or Replacing Compact Flash Card 2–13 2[...]

Contents v 4.3 OpenBoot Firmware 4–4 4.3.1 Getting to the ok Prompt 4–5 4.3.2 Auto-Boot Options 4–6 4.3.3 OpenBoot Commands 4–6 4.3.3.1 probe-scsi and probe-scsi-all Commands 4–7 4.3.3.2 probe-ide Command 4–8 4.3.3.3 show-devs Command 4–8 4.3.3.4 Using watch-net and watch-net-all Commands to Check the Network 4–1 1 4.3.4 OpenBoot Co[...]

vi Netra CP3260 Blade Server User’s Guide • April 2009 5.2.1 UltraSP ARC T2 Pr ocessor 5–4 5.2.1.1 Electronic Fuse 5–7 5.2.1.2 Cores 5–8 5.2.1.3 L2 Cache 5–8 5.2.1.4 Memory Controller 5–8 5.2.1.5 I/O Interface 5–8 5.2.2 Memory Subsystem 5–9 5.2.2.1 Memory Capacity 5–9 5.2.2.2 Memory Speed 5–9 5.2.3 I/O Subsystem 5–9 5.2.3.1 [...]

Contents vii 5.2.8.4 A TCA Hot-Swap Latch 5–14 5.2.8.5 LEDs 5–15 5.2.8.6 Power Control 5–15 5.2.8.7 System Monitor (ADM1026) and Thresholds 5–15 5.2.8.8 FRUID PROMs 5–17 5.2.9 I/O Subsystem Resets 5–17 5.2.10 A TCA Power Module ( − 48V to 12V) 5–18 5.2.1 1 TOD Clock Battery 5–18 A. Physical Characteristics A–1 A.1 Form Factor A?[...]

viii Netra CP3260 Blade Server User’s Guide • April 2009 B.3 Solaris OS Graceful Shutdown Commands B–4 B.4 Send Sensor State Command B–5 Index Index–1[...]

ix Figur es FIGURE 1-1 Netra CP3260 Blade Server Front Panel 1–4 FIGURE 1-2 Netra CP3260 Blade Server (Top View) 1–5 FIGURE 1-3 Netra CP3260 Blade Server in an ATCA Shelf Enclosure 1–9 FIGURE 1-4 Netra CP3260 Blade Server, Midplane, and Netra CP32x0 ARTM 1–12 FIGURE 1-5 Netra CP3260 Blade Server Barcode Labeling 1–17 FIGURE 2-1 FB-DIMM Me[...]

x Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE A-5 Power Distribution Connector (Zone 1) P10 A–7 FIGURE A-6 Zone 2 Connectors A–9 FIGURE A-7 Zone 3 Signal Connectors A–10 FIGURE A-8 Zone 3 Power Connector A–12 FIGURE A-9 TOD Battery Location A–14[...]

xi T ables TABLE 1-1 Netra CP3260 Blade Server Feature Summary 1–2 TABLE 1-2 I/O Configurations 1–13 TABLE 2-1 Ethernet Device Names 2–24 TABLE 3-1 Local Network Information 3–7 TABLE 4-1 Ways of Accessing the ok Prompt 4–5 TABLE 4-2 OpenBoot Configuration Variables 4–12 TABLE 4-3 Network Device Aliases 4–18 TABLE 4-4 PICL Frutree Ent[...]

xii Netra CP3260 Blade Server User’s Guide • April 2009 TABLE B-1 Get Version Command Data Bytes B–2 TABLE B-2 Get RTM Status Command Data Bytes B–3 TABLE 1 Solaris OS Graceful Shutdown Parameters in /etc/fsmd.conf B–4 TABLE B-3 Send Sensor State Command Data Bytes B–5[...]

xiii Pr eface The Sun Netra CP3260 Blade Server User ’ s Guide provides information about features, installation, configuration, functional hardwar e components, and physical properties of this blade server . The Sun Netra CP3260 Blade Server User ’ s Guide is written for system integration engineers, field applications and service engineers, a[...]

xiv Sun Netra CP3260 Blade Server User’s Guide • April 2009 Using UNIX Commands This document might not contain information on basic UNIX ® commands and procedur es such as shutting down the system, booting the system, and configuring devices. Refer to the following for this information: ■ Software documentation that you received with your s[...]

Preface xv T ypographic Conventions Note – Characters display differently depending on br owser settings. If characters do not display correctly , change the character encoding in your browser to Unicode UTF-8. Related Documentation The Netra CP3260 blade server documentation is listed in the following table. Except for the Important Safety Infor[...]

xvi Sun Netra CP3260 Blade Server User’s Guide • April 2009 Documentation, Support, and T raining Thir d-Party W eb Sites Sun is not responsible for the availability of third-party web sites mentioned in this document. Sun does not endorse and is not responsible or liable for any content, advertising, products, or other materials that are avail[...]

Preface xvii or resour ces. Sun will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content, goods, or services that are available on or through such sites or r esources. Sun W elcomes Y our Comments Sun is interested in impr oving its documentation and welcom[...]

xviii Sun Netra CP3260 Blade Server User’s Guide • April 2009[...]

1-1 CHAPTER 1 - CHAPTE R Intr oduction This chapter contains the following sections: ■ Section 1.1, “Overview” on page 1- 1 ■ Section 1.2, “Features” on page 1-2 ■ Section 1.3, “System Configurations” on page 1-8 ■ Section 1.5, “Hot-Swap Support” on page 1- 14 ■ Section 1.6, “System Requirements and Options” on page 1-[...]

1-2 Netra CP3260 Blade Server User’s Guide • April 2009 ■ Elimination of PCI connectivity between the blade servers in the system and reallocation of connectivity to serial interconnects, eliminating single points of failure ■ Mandatory use of Intelligent Platform Management Interface (IPMI) management interfaces ■ Flexible user I/O ■ P[...]

Chapter 1 Introduction 1-3 PICMG compliance • Single-wide A TCA module • A TCA 8U form factor • Single-wide A TCA module • 6-HP slot width • PICMG 3.0 R1.0 • PICMG 3.1 R1.0 Node board support Functions as a CPU node board with the Solaris operating system and software packages Operating system Solaris 10 8/07 OS and subsequent compatibl[...]

1-4 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 1-1 Netra CP3260 Blade Server Fr ont Panel[...]

Chapter 1 Introduction 1-5 FIGURE 1-2 Netra CP3260 Blade Server (T op V iew)[...]

1-6 Netra CP3260 Blade Server User’s Guide • April 2009 ① ② ③ ④ ⑤ ⑦[...]

Chapter 1 Introduction 1-7 Figure Legend 1 T op latch 2 Out of Service LED (y ellow) 3 Ethernet management por t (RJ-45) 4 OK LED (green) 5 Dual USB P or ts (USB 2.0) 6 Serial (console) por t (RJ-45, ttya) 7 Hot-Swap LED (b lue) 8 Hot-Swap s witch and bottom latch Figure Legend 1 Zone 1 power connector 2 Zone 2 signal connectors 3 Zone 3 RTM signal[...]

1-8 Netra CP3260 Blade Server User’s Guide • April 2009 1.3 System Conf igurations Netra CP3260 blade servers can be installed in an A TCA shelf (or chassis), as shown in FIGURE 1-3 . The blade servers can be deployed in various electrical configurations to suit each end-user requir ement. For example, the blade server can be configured to boot[...]

Chapter 1 Introduction 1-9 FIGURE 1-3 Netra CP3260 Blade Server in an A TCA Shelf Enclosure[...]

1-10 Netra CP3260 Blade Server User’s Guide • April 2009 Ether net Diskless client that boots through network from a remote ser v er ③④ ① ② ⑤ ⑥⑦[...]

Chapter 1 Introduction 1-11 1.4 Advanced Rear T ransition Modules An optional Sun Netra CP32X0 Advanced Rear T ransition Module (ARTM) can be installed into the rear of the A TCA enclosure, opposite the Netra CP3260 blade server ( FIGURE 1-3 ). The Netra CP32x0 AR TM connects to the Netra CP3260 blade server ’s Zone 3 rear I/O connectors ( FIGURE[...]

1-12 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 1-4 Netra CP3260 Blade Server , Midplane, and Netra CP32x0 AR TM ① ② ③[...]

Chapter 1 Introduction 1-13 Note – When a Netra CP32x0 AR TM is used with the Netra CP3260 blade server , shielded cables are r equired for serial I/O ports. Unshielded cables can be used on Ethernet ports to satisfy EMI compliance standards. The shields for all shielded cables must be terminated on both ends. The customer can order a Netra CP32x[...]

1-14 Netra CP3260 Blade Server User’s Guide • April 2009 1.5 Hot-Swap Support There ar e three hot-swap models described in the PICMG A TCA specification: basic hot-swap, full hot-swap, and high-availability (HA) hot-swap. Refer to the PICMG A TCA Specification, which pr ovides a detailed description of this subject. The Netra CP3260 blade serv[...]

Chapter 1 Introduction 1-15 ■ Serial terminal or terminal emulation for console output. ■ Cables for terminal and network connections. ■ Optional AMC disks and PCI-E cards. The following are A TCA and other minimum requir ements met by the Netra CP3260 blade server . ■ A TCA system enclosure for 8U boards (includes chassis, midplane, power [...]

1-16 Netra CP3260 Blade Server User’s Guide • April 2009 1.7.1 Part Number , Serial Number , and Revision Number Identif ication The Netra CP3260 blade server part number , serial number , and revision can be found on labels located on the card ( FIGURE 1-5 ). The Sun barcode labels pr ovide the following information: ■ SunSN – Sun serial n[...]

Chapter 1 Introduction 1-17 FIGURE 1-5 Netra CP3260 Blade Server Bar code Labeling Note – Y ou might find the labels shown in FIGURE 1-5 on other locations on your blade server . Y our particular blade server configuration might also appear differ ent than the illustration.[...]

1-18 Netra CP3260 Blade Server User’s Guide • April 2009[...]

2-1 CHAPTER 2 - CHAPTE R Har dware Installation This chapter describes the hardwar e installation procedur es for the Netra CP3260 blade server , and contains the following sections: ■ Section 2.1, “Equipment and Operator Safety” on page 2-1 ■ Section 2.2, “Materials and T ools Required” on page 2-3 ■ Section 2.3, “Preparing for the[...]

2-2 Netra CP3260 Blade Server User’s Guide • April 2009 The installer must be familiar with commonly accepted procedur es for integrating electronic systems and with the general practice of Sun systems integration and administration. Although parts of these systems are designed for hot-swap operation, other components must not be subjected to s[...]

Chapter 2 Hardware Installation 2-3 2.2 Materials and T ools Requir ed This section provides information on the materials and tools requir ed to perform installation. The minimum tools required to perform installation ar e: ■ Phillips screwdrivers, No. 1, No. 2 (optional) ■ Antistatic wrist strap ■ T erminal console See Section 1.6.1, “Hard[...]

2-4 Netra CP3260 Blade Server User’s Guide • April 2009 ■ Facility power loading specifications can support the rack or enclosure requir ements. ■ Y our enclosure specif ications support the cooling airflow requir ements. The Netra CP3260 blade server fits a standar d A TCA shelf or chassis. If your installation requir ements are differ ent[...]

Chapter 2 Hardware Installation 2-5 2.5 Conf iguring On-Board Har dwar e This section provides the pr ocedures for adding or r eplacing the on-board har dware components such as memory modules, a Compact Flash card, and the time-of day (TOD) battery . Read and perform the procedur es, as necessary , before installing the Netra CP3260 blade server i[...]

2-6 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 2-1 FB-DIMM Memory Locations[...]

Chapter 2 Hardware Installation 2-7[...]

2-8 Netra CP3260 Blade Server User’s Guide • April 2009 ①② ③ ④ ⑤ ⑥ ⑦ ⑧[...]

Chapter 2 Hardware Installation 2-9 Figure Legend 1 FB0B DIMM Pair 0 5 FB2A DIMM Pair 2 2 FB0A 6 FB2B 3 FB1B DIMM Pair 1 7 FB3A DIMM Pair 3 4 FB1A 8 FB3B[...]

2-10 Netra CP3260 Blade Server User’s Guide • April 2009 2.5.1.1 T o Remove FB-DIMM Memory Modules Y ou might need to r emove a FB-DIMM module from the Netra CP3260 blade server if you are r eturning the FB-DIMM module or the blade server for service, or if you are r eplacing a module with another FB-DIMM module. Note – Safely store the origi[...]

Chapter 2 Hardware Installation 2-11 FIGURE 2-2 Removing an FB-DIMM Memory Module 6. If you are replacing the module you removed with a new FB-DIMM, install it as described in Section 2.5.1.2, “T o Install FB-DIMM Memory Modules” on page 2-1 1 . 2.5.1.2 T o Install FB-DIMM Memory Modules The following procedur e provides a general guide for ins[...]

2-12 Netra CP3260 Blade Server User’s Guide • April 2009 1. Locate the FB-DIMM connectors on the Netra CP3260 blade server . Select the connectors where you will install the memory module ( FIGURE 2-1 ). If you need to replace an existing memory module with a new module, see Section 2.5.1.1, “T o Remove FB-DIMM Memory Modules” on page 2-10 [...]

Chapter 2 Hardware Installation 2-13 4. Press the top edge of the FB-DIMM toward the blade server until the retainer clips click into place (see ➋ in FIGURE 2-3 ). The small retainer clips on each side of the FB-DIMM slot click into place in the notches on the sides of the FB-DIMM. 2.5.2 Adding or Replacing Compact Flash Car d Y ou can install an[...]

2-14 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 2-4 Compact Flash Car d Location 2.5.3 Adding or Replacing T OD Clock Battery FIGURE 2-5 shows the location of the TOD clock battery and holder . The optional TOD battery must be type CR1632, with a minimum of 4ma abnormal charging curr ent rating (for example; a Renata CR1632). Fi[...]

Chapter 2 Hardware Installation 2-15 Caution – Risk of explosion if the battery is replaced by an incorrect type. Dispose of batteries properly in accordance with manufactur er ’s instructions and local regulations. T o install the battery , perform the following steps: 1. Remove the FB-DIMM closest to the front panel. T o do this, see Section [...]

2-16 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 2-5 T OD Battery Location[...]

Chapter 2 Hardware Installation 2-17 ① ② ③[...]

2-18 Netra CP3260 Blade Server User’s Guide • April 2009 2.6 Installing an Advanced Rear T ransition Module (Optional) For rear I/O access, a compatible R TM must be used with the Netra CP3260 blade server . Note – If you are using a Netra CP32x0 AR TM, refer to the appropriate Netra CP32x0 AR TM User ’s Guide for specific installation inst[...]

Chapter 2 Hardware Installation 2-19 FIGURE 2-6 Installing a Netra CP32X0 AR TM Figure Legend 1 Netra CP3260 blade server 2 Netra CP32X0 AR TM 3 A TCA chassis midplane ① ② ③[...]

2-20 Netra CP3260 Blade Server User’s Guide • April 2009 2.7 Installing the Netra CP3260 Blade Server Caution – The Netra CT 900 server fan tray upgrade kit (594-4953) must be installed in the chassis before the Netra CP3260 blade server is installed. This fan tray upgrade is requir ed to provide adequate cooling and to prevent the system fro[...]

Chapter 2 Hardware Installation 2-21 FIGURE 2-7 Installing Blade Server Into Chassis Slot 6. Push the blade server slightly into the midplane connectors, and then close the latches to seat the blade server in the connectors ( FIGURE 2-8 ). 7. When the lower latch is closed, the blue Hot-Swap LED blinks while the blade server is initializing. The bl[...]

2-22 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 2-8 Netra CP3260 Blade Server Latches and Locking Scr ews 2.8 Connecting External I/O Cables External I/O cables are connected to the Netra CP3260 blade server , or to the Netra CP32x0 AR TM when a rear transition module is used.[...]

Chapter 2 Hardware Installation 2-23 Note – Shielded cables are r equired for serial I/O ports. Unshielded cables can be used on Ethernet ports to satisfy EMI compliance standards. The shields for all shielded cables must be terminated on both ends. Information on connecting each of these cables follows: ■ For Ethernet connections , category 5e[...]

2-24 Netra CP3260 Blade Server User’s Guide • April 2009 2.9 Hot-Swapping the Netra CP3260 Blade Server The Netra CP3260 blade server supports hot-swapping at the blade server level. Note – Before hot-swapping the blade server , the system administrator should gracefully shutdown the applications and operating system, and deactivate the blade[...]

Chapter 2 Hardware Installation 2-25 FIGURE 2-9 Hot-Swap Latch and Hot-Swap LED[...]

2-26 Netra CP3260 Blade Server User’s Guide • April 2009[...]

3-1 CHAPTER 3 - CHAPTE R Softwar e Installation This chapter contains the following sections: ■ Section 3.1, “Operating Systems and Patches” on page 3-1 ■ Section 3.2, “Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3” on page 3-2 ■ Section 3.3, “Installing as a Diskless Clients” on page 3-4 ■ Section 3.4, “[...]

3-2 Netra CP3260 Blade Server User’s Guide • April 2009 Depending the blade server ’s hardware conf iguration, the Sun Netra CP3260 blade server can be used as a diskless client with the Solaris OS installed on a boot server (see Section 3.3, “Installing as a Diskless Clients” on page 3-4 . The OS can also be downloaded to a boot device l[...]

Chapter 3 Software Installation 3-3 CODE EXAMPLE 3-3 Zone 2 and Zone 3 Connectivity: First NIU Port ( nxge0 ) to Zone 2 (at 10Gbps Speed) and the Second NIU Port ( nxge1 ) to Zone3 name = "SUNW,niusl" parent = "/niu@80" unit-address = "0" phy-type = "xgsd"; name = "SUNW,niusl" parent = "/niu@80[...]

3-4 Netra CP3260 Blade Server User’s Guide • April 2009 3.3 Installing as a Diskless Clients The following procedur es describe how to create a boot server for diskless clients and how to add new diskless clients to the patched boot server . For additional instructions on installing diskless clients, refer to the appropriate Solaris Documentati[...]

Chapter 3 Software Installation 3-5 ■ locale is the locale that you want to use ■ (backslash) is a line-continuation character indicating that the command is continued on the next line. Refer to the smosservice (1M) man page for more information and options. For example: Where: ■ root_password = root_password ■ image_directory = /export/i[...]

3-6 Netra CP3260 Blade Server User’s Guide • April 2009 4. Change directories to the /usr/sadm/bin directory . 5. Set up the diskless clients. For each diskless client, type the following command as superuser: Where: ■ ip_address is the client’s IP address ■ ethernet_address is the client’s Ethernet address ■ host_name is the client?[...]

Chapter 3 Software Installation 3-7 ■ time_zone = US/Pacific ■ locale = en_US ■ name_service = NIS ■ name_server = nameserver_host ■ (backslash) is a line continuation character indicating that the command is continued on the next line. Y ou must type your superuser password again after typing this command. The installation process shou[...]

3-8 Netra CP3260 Blade Server User’s Guide • April 2009 Y ou might need the MAC (Ethernet) addresses of the local hosts to make nameserver database entries. The MAC address can be seen in the console output while booting to the ok prompt. It can also be derived fr om the host ID seen on the barcode label (see Section 1.7.1, “Part Number , Ser[...]

Chapter 3 Software Installation 3-9 The NIU (Network Interface Unit) driver configuration file, nxge.conf , might need to be modified to employ the pr oper port configurations. The nxge.conf f ile is located in the /platform/sun4v/kernel/drv directory . The following examples show the proper entries for each switch conf iguration. These entries are[...]

3-10 Netra CP3260 Blade Server User’s Guide • April 2009 SunVTS software is used to validate a system during development, production, inspection, troubleshooting, periodic maintenance, and system or subsystem stressing. SunVTS software can be tailor ed to run on machines ranging from desktops to servers with modifiable test instances and proces[...]

4-1 CHAPTER 4 - CHAPTE R Firmwar e and Blade Server Management This chapter contains the following sections: ■ Section 4.1, “System Firmware” on page 4-2 ■ Section 4.2, “Power-On Self-T est Diagnostics” on page 4-3 ■ Section 4.3, “OpenBoot Firmware” on page 4-4 ■ Section 4.4, “Error Handling Summary” on page 4-15 ■ Section[...]

4-2 Netra CP3260 Blade Server User’s Guide • April 2009 4.1 System Firmwar e The Netra CP3260 blade server contains a modular firmware ar chitecture that gives you latitude in controlling boot initialization. Y ou can customize the initialization, test the firmwar e, and even enable the installation of a custom operating system. This platform a[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-3 4.2 Power -On Self-T est Diagnostics Power-on self-test (POST) is a f irmware pr ogram that helps determine whether a portion of the system has failed. POST verifies the cor e functionality of the system, including the CPU modules, motherboard, memory , and some on-board I/O devices. The software[...]

4-4 Netra CP3260 Blade Server User’s Guide • April 2009 4.2.2 POST Diagnostic and Err or Message Format POST diagnostic and error messages are displayed on a console. The format of the these messages is the following: The following is an example of a POST error message 4.3 OpenBoot Firmwar e The Solaris OS installed operates at different run le[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-5 ■ When the operating system crashes, the system reverts to OpenBoot f irmware control. ■ During the boot process, when there is a serious har dware pr oblem that prevents the operating system from running, the system r everts to OpenBoot firmwar e control. ■ When a serious hardwar e problem[...]

4-6 Netra CP3260 Blade Server User’s Guide • April 2009 4.3.2 Auto-Boot Options The system firmwar e stores a conf iguration variable called auto-boot? , which controls whether the f irmware will automatically boot the operating system after each reset. The default setting for Sun platforms is true . Normally , if a system fails power-on diagno[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-7 4.3.3.1 probe-scsi and probe-scsi-all Commands The probe-scsi and probe-scsi-all commands diagnose problems with the SCSI devices. Caution – If you used the halt command or the Stop-A key sequence to reach the ok prompt, issuing the probe-scsi or probe-scsi-all command can hang the system. The [...]

4-8 Netra CP3260 Blade Server User’s Guide • April 2009 4.3.3.2 probe-ide Command The probe-ide command communicates with all Integrated Drive Electronics (IDE) devices connected to the IDE bus. This is the internal system bus for media devices such as the DVD drive. Caution – If you used the halt command or the Stop-A key sequence to reach t[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-9 /cpu@37 /cpu@36 /cpu@35 /cpu@34 /cpu@33 /cpu@32 /cpu@31 /cpu@30 /cpu@2f /cpu@2e /cpu@2d /cpu@2c /cpu@2b /cpu@2a /cpu@29 /cpu@28 /cpu@27 /cpu@26 /cpu@25 /cpu@24 /cpu@23 /cpu@22 /cpu@21 /cpu@20 /cpu@1f /cpu@1e /cpu@1d /cpu@1c /cpu@1b /cpu@1a /cpu@19 /cpu@18 /cpu@17 /cpu@16 /cpu@15 /cpu@14 /cpu@13 /[...]

4-10 Netra CP3260 Blade Server User’s Guide • April 2009 /cpu@8 /cpu@7 /cpu@6 /cpu@5 /cpu@4 /cpu@3 /cpu@2 /cpu@1 /cpu@0 /virtual-devices@100 /virtual-memory /memory@m0,8000000 /aliases /options /openprom /chosen /packages /ebus@c0/serial@0,ca0000 /pci@0/pci@0 /pci@0/pci@0/pci@a /pci@0/pci@0/pci@9 /pci@0/pci@0/pci@8 /pci@0/pci@0/pci@2 /pci@0/pci[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-11 4.3.3.4 Using watch-net and watch-net-all Commands to Check the Network The watch-net diagnostics test monitors Ethernet packets on the primary network interface. The watch-net-all diagnostics test monitors Ethernet packets on the primary network interface and on any additional network interface[...]

4-12 Netra CP3260 Blade Server User’s Guide • April 2009 4.3.4 OpenBoot Conf iguration V ariables The OpenBoot configuration variables are stor ed in the OBP flash PROM and determine how and when OpenBoot tests ar e performed. This section explains how to access and modify OpenBoot configuration variables. For a list of important OpenBoot confi[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-13 error-reset-recovery boot, none, sync boot Specifies r ecovery action after an error r eset. Default is boot . • none – No recovery action. • boot – System attempts to boot. • sync – Firmware attempts to execute a Solaris sync callback routine. fcode-debug? true, false false If true [...]

4-14 Netra CP3260 Blade Server User’s Guide • April 2009 4.3.4.1 V iewing and Setting OpenBoot Configuration V ariables ● Halt the server to display the ok prompt. ■ T o display the current values of all OpenBoot conf iguration variables, use the printenv command. The following example shows a short excerpt of this command’s output. secur[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-15 ■ T o set or change the value of an OpenBoot configuration variable, use the setenv command: ■ T o set OpenBoot configuration variables that accept multiple keywor ds, separate keywords with a space. 4.4 Err or Handling Summary Error handling during the power-on sequence falls into one of th[...]

4-16 Netra CP3260 Blade Server User’s Guide • April 2009 ■ Any CPU failed ■ All logical memory banks failed ■ Flash RAM cyclical redundancy check (CRC) failure ■ Critical field-r eplaceable unit (FRU) PROM configuration data failure ■ Critical application-specific integrated circuit (ASIC) failur e 4.5 Automatic System Recovery Automa[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-17 4.5.1 Enabling and Disabling Automatic System Recovery The automatic system recovery (ASR) feature is not activated until you enable it. Enabling ASR requir es changing configuration variables in OpenBoot. 4.5.1.1 T o Enable Automatic System Recovery 1. At the ok prompt, type: 2. T o cause the p[...]

4-18 Netra CP3260 Blade Server User’s Guide • April 2009 After you disable the automatic system recovery (ASR) feature, it is not activated again until you re-enable it. 4.6 Network Device Aliases A device alias is a shorthand representation of a device path . The Solaris OS pr ovides some predef ined device aliases for the network devices so t[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-19 4.7 Retrieving Device Information Y ou can use the Solaris platform information and control library (PICL) framework for obtaining the state and condition of the Netra CP3260 blade server . The PICL framework provides information about the system configuration that it maintains in the PICL tree.[...]

4-20 Netra CP3260 Blade Server User’s Guide • April 2009 virtual-channel-client (obp-device, 6f0000011d) n2cp (obp-device, 6f00000127) ncp (obp-device, 6f00000134) random-number-generator (obp-device, 6f00000141) flashprom (flashprom, 6f0000014d) console (serial, 6f00000154) rtc (obp-device, 6f00000161) cpu (cpu, 6f00000169) cpu (cpu, 6f0000017[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-21 cpu (cpu, 6f00000399) cpu (cpu, 6f000003a7) cpu (cpu, 6f000003b5) cpu (cpu, 6f000003c3) cpu (cpu, 6f000003d1) cpu (cpu, 6f000003df) cpu (cpu, 6f000003ed) cpu (cpu, 6f000003fb) cpu (cpu, 6f00000409) cpu (cpu, 6f00000417) cpu (cpu, 6f00000425) cpu (cpu, 6f00000433) cpu (cpu, 6f00000441) cpu (cpu, [...]

4-22 Netra CP3260 Blade Server User’s Guide • April 2009 network (network, 6f00000804) network (network, 6f00000814) os-io (console, 6f00000824) pseudo (devctl, 6f00000843) zconsnex (devctl, 6f0000084c) obp (picl, 6f0000001f) packages (packages, 6f00000034) SUNW,builtin-drivers (SUNW,builtin-drivers, 6f0000003a) deblocker (deblocker, 6f00000040[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-23 L2_BANK7 (other, 6f00000cda) BR0 (other, 6f00000ce0) CH0 (other, 6f00000ce6) D0 (module, 6f00000cec) R0 (unknown, 6f00000cf7) R1 (unknown, 6f00000cfc) SEEPROM (other, 6f00000d01) PRSNT (presence-indicator, 6f00000d07) T_AMB (temperature-sensor, 6f00000d0f) CH1 (other, 6f00000d1f) D0 (module, 6f0[...]

4-24 Netra CP3260 Blade Server User’s Guide • April 2009 CH0 (other, 6f00000e4e) D0 (module, 6f00000e54) R0 (unknown, 6f00000e5f) R1 (unknown, 6f00000e64) SEEPROM (other, 6f00000e69) PRSNT (presence-indicator, 6f00000e6f) T_AMB (temperature-sensor, 6f00000e77) CH1 (other, 6f00000e87) D0 (module, 6f00000e8d) R0 (unknown, 6f00000e98) R1 (unknown,[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-25 IMMU (other, 6f00000f86) DMMU (other, 6f00000f8c) CORE5 (other, 6f00000f92) SPU (other, 6f00000f98) FPU (other, 6f00000f9e) ICACHE (other, 6f00000fa4) DCACHE (other, 6f00000faa) IMMU (other, 6f00000fb0) DMMU (other, 6f00000fb6) CORE6 (other, 6f00000fbc) SPU (other, 6f00000fc2) FPU (other, 6f0000[...]

4-26 Netra CP3260 Blade Server User’s Guide • April 2009 For more information on the PICL framework, refer to the picld (1M) man page. P24 (other, 6f000010a0) P25 (other, 6f000010a6) P26 (other, 6f000010ac) P27 (other, 6f000010b2) P28 (other, 6f000010b8) P29 (other, 6f000010be) P30 (other, 6f000010c4) P31 (other, 6f000010ca) P32 (other, 6f00001[...]

Chapter 4 Fir mware and Blade Ser ver Management 4-27 4.8 Multiplexing to Zones 2 and 3 A multiplexer (MUX) controller and ShMM conf iguration is available for use on Netra CP3260 blade servers to multiplex 10GbE network interface unit (NIU) ports to Zone 2 (backplane) and/or to Zone 3 (ARTM). ■ MUX control conf igurations take effect during the [...]

4-28 Netra CP3260 Blade Server User’s Guide • April 2009 Be aware of the following possible issues when multiplexing zones: ■ MUX and payload driver configuration could go out of sync, r equiring a system administrator to make sure they ar e set to the same configuration. ■ Midplane FRUID recor d could be corrupted when updating with the MU[...]

5-1 CHAPTER 5 Har dware Functional Descriptions This chapter contains the following sections: ■ Section 5.1, “Hardwar e Architectur e” on page 5-1 ■ Section 5.2, “Hardwar e Modules” on page 5-4 5.1 Har dware Ar chitectur e The Netra CP3260 blade server is an A TCA node board based on the UltraSP ARC T2 processor . It is hot-swappable to[...]

5-2 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE 5-1 Netra CP3260 Blade Server Block Diagram 5.1.1 UltraSP ARC T2 Pr ocessor and Memory The UltraSP ARC T2 processor has eight SP ARC physical processor cores. Each SP ARC physical processor cor e has full hardware support for eight strands, two integer execution pipelines, one float[...]

Chapter 5 Hardware Functional Descriptions 5-3 There ar e four on-chip memory controllers that interface directly to FB-DIMM memory and include eight FB-DIMM slots (one per channel, or two per memory controller). In addition, there ar e an on-chip PCI-Express I/O interface and two 10-Gb Ethernet ports. The UltraSP ARC T2 processor is a highly integ[...]

5-4 Netra CP3260 Blade Server User’s Guide • April 2009 5.1.6 AR TM Support AR TM support is routed to the AR TM through the Zone 3 connectors. T ypical ARTM support includes: ■ One RS-232 serial console port ■ One 10/100/100BASE-T Ethernet management port Because Netra CP32x0 Advanced R TMs (AR TM) provide additional functionality , the fo[...]

Chapter 5 Hardware Functional Descriptions 5-5 FIGURE 5-2 UltraSP ARC T2 Multicore Processor Block Diagram[...]

5-6 Netra CP3260 Blade Server User’s Guide • April 2009[...]

Chapter 5 Hardware Functional Descriptions 5-7 5.2.1.1 Electronic Fuse The Electronic Fuse (Efuse) block within the UltraSP ARC T2 processor contains configuration information that is electronically burned in manufacturing. UltraSP ARC T2 SP ARC core SP ARC core SP ARC core SP ARC core SP ARC core SP ARC core SP ARC core SP ARC core L2 Bank 0 L2 Ba[...]

5-8 Netra CP3260 Blade Server User’s Guide • April 2009 5.2.1.2 Cores The UltraSP ARC T2 processor pr ovides 8 physical SP ARC processor cor es and each physical core is capable of supporting 8 threads for a total of 64 thr eads. 5.2.1.3 L2 Cache The UltraSP ARC T2 provides a total of 4 Mbytes of L2 cache banked 8 ways. 5.2.1.4 Memory Controlle[...]

Chapter 5 Hardware Functional Descriptions 5-9 The XAUI interface is the Fabric interface to the A TCA midplane. The XAUI interface is routed to the Zone 2 connector on the midplane (see Section A.4, “Connectors and Pinout” on page A-3 for details on board connector pinout assignments. 5.2.2 Memory Subsystem The heart of the Memory subsystem is[...]

5-10 Netra CP3260 Blade Server User’s Guide • April 2009 5.2.3.1 PCI Express Switch The Netra CP3260 uses the PLX PCI switch that encompasses a 32-lane PCI Express switch with six configurable ports (x1, x2, x4, x8, x16). 5.2.3.2 Base Interface The Netra CP3260 provides dual r edundant 10/100/1000BASE-T Ethernet links to connect to the midplane[...]

Chapter 5 Hardware Functional Descriptions 5-11 5.2.4 Other AR TM Interfaces In addition to the AMC-type interface to the ARTM, additional interfaces ar e routed to the AR TM to provide for rear access to the Netra CP3260 I/O. 5.2.4.1 Serial Ports The console port from the PPC is r outed to the ARTM for r ear access. When a serial device is connect[...]

5-12 Netra CP3260 Blade Server User’s Guide • April 2009 Note – A serial port is also provided on the Netra CP32x0 AR TM. Y ou can use either the serial port on the front panel or the serial port on Netra CP32x0 AR TM, but only one of the ports should be used at one time. 5.2.5.3 Dual USB Ports Dual USB ports are 2.0 compliant and r outed fro[...]

Chapter 5 Hardware Functional Descriptions 5-13 ■ H8 interface providing IPMC support. ■ MPC interface. ■ SSI Interface (UltraSP ARC T2). ■ BUS Interface and Arbiter (Round Robin). ■ Bus Clock Control ■ Power sequence control of DC/DCs on board. ■ Interrupts. ■ Data Channel/Fast Mailbox Control. The FPGA configuration is performed a[...]

5-14 Netra CP3260 Blade Server User’s Guide • April 2009 ■ Environmental monitoring ■ Access to all environmental I 2 C devices ■ Access to all I 2 C devices when MPC is reset 5.2.8.1 Intelligent Platform Management Bus The BMR-H8S provides dual buffer ed Intelligent Platform Management Bus (IPMB) interfaces to the IPMB-0 bus on the PICMG[...]

Chapter 5 Hardware Functional Descriptions 5-15 5.2.8.5 LEDs The Netra CP3260 blade server supports three LEDs compliant with A TCA specification: ■ Green LED – Healthy status ■ Amber LED – Fault condition ■ Blue LED – Indicates that blade server is safe for removal when on and hot swapping in progr ess when blinking. The LEDs are contr[...]

5-16 Netra CP3260 Blade Server User’s Guide • April 2009 T emperature Monitoring The ADM1026 also monitors the CPU diode temperature on the Netra CP3260 blade server . The ADM1206 and H8 firmware r eports a minor , major , or critical alarm when the temperature of the CPU goes beyond the thresholds listed in T ABLE 5-2 . Also, if the temperatur[...]

Chapter 5 Hardware Functional Descriptions 5-17 The H8 temperature alarm equivalents are: Minor Alarm = Upper Non Critical (UNC) Major Alarm = Upper Critical (UC) Critical Alarm = Upper Non Recoverable (UNR) Emergency H8 Shutdown (EMR) = UNR Caution – These voltage and temperature thr esholds should not be changed under normal operating condition[...]

5-18 Netra CP3260 Blade Server User’s Guide • April 2009 A reset mask r egister is provided in the FPGA to allow the masking of resets to individual I/O components. For example, a PEX_RESET_L reset from the UltraSP ARC T2 processor r esets only those I/O subcomponents not masked by the reset mask r egister . 5.2.10 A TCA Power Module ( − 48V [...]

A-1 APPENDIX A Physical Characteristics Specifications for the Netra CP3260 blade server are pr ovided in the following sections: ■ Section A.1, “Form Factor” on page A-1 ■ Section A.2, “Layout” on page A-1 ■ Section A.3, “Front Panel” on page A-3 ■ Section A.4, “Connectors and Pinout” on page A-3 A.1 Form Factor The Netra C[...]

A-2 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE A-1 Netra CP3260 Blade Server Layout Figure Legend 1 Ethernet por t connector (RJ-45) 5 Zone 1 po wer connector 2 Dual USB por t connector (USB 2.0) 6 Zone 2 I/O connectors 3 Serial por t connector 7 Zone 3 RTM I/O connectors 4 Compact flash connector 8 Zone 3 RTM pow er connector ?[...]

Appendix A Physical Characteristics A-3 A.3 Fr ont Panel The single-slot-wide, 8U front panel was designed to meet PICMG 3.0 R 1.0 and other specifications. A.3.1 V isual Indicators The Netra CP3260 blade server has the following indicators on the front panel: ■ Green LED – Boar d’s healthy status or user programmable (ACTIVE). ■ Amber LED [...]

A-4 Netra CP3260 Blade Server User’s Guide • April 2009 A.4.1.1 Ethernet Port The Ethernet connector is a RJ-45 connector . The controller autonegotiates to either 10BASE-T , 100BASE-T , or 1000BASE-T . The Ethernet connector pin numbering is shown in FIGURE A-2 . FIGURE A-2 Ethernet RJ-45 Connector T ABLE A-1 shows the Ethernet connector pin a[...]

Appendix A Physical Characteristics A-5 FIGURE A-3 Dual USB Connector For USB connector signals, see T ABLE A-2 . A.4.1.3 Serial Port FIGURE A-4 contains the connector pin assignments for the front panel serial port. T ABLE A-2 USB Connector Pin Assignments Pin Signal Description Pin Signal Description A1 + 5 V (fused) B1 + 5 V (fused) A2 USB2 − [...]

A-6 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE A-4 Front Panel Serial Port Diagram T ABLE A-3 shows the serial port connector pin assignments. A.4.2 Compact Flash Connector The Compact Flash connector is a type I/II connector . A.4.3 Midplane Power Connector (Zone 1) The Netra CP3260 blade server uses a Zone 1 power connector . [...]

Appendix A Physical Characteristics A-7 FIGURE A-5 shows the pin assignments. FIGURE A-5 Power Distribution Connector (Zone 1) P10 T ABLE A-4 lists the power connector pin assignments. T ABLE A-4 Power Distribution Connector Pin Assignments Pin Number Name Description 1 Reserved Reserved 2 Reserved Reserved 3 Reserved Reserved 4 Reserved Reserved 5[...]

A-8 Netra CP3260 Blade Server User’s Guide • April 2009 A.4.4 Data T ransport Connector (Zone 2) The data transport connector consists of one 120-pin HM-Zd connector , labeled P23, with 40 differ ential pairs. This is called the Zone 2 connector . The Zone 2 connector provides the following signals: ■ T wo 1000BASE-T Ethernet Base interface c[...]

Appendix A Physical Characteristics A-9 FIGURE A-6 Zone 2 Connectors T ABLE A-5 gives the Zone 2 J23 connector pin assignments. T ABLE A-6 gives the Zone 2 J20 connector pin assignments. T ABLE A-5 Zone 2 J23 Connector Pin Assignments Row A B C D E F G H 1 P23_F2_TX2 + P23_F2_TX2 − P23_F2_RX2 + P23_F2_RX2 − P23_F2_TX3 + P23_F2_TX3 − P23_F2_RX[...]

A-10 Netra CP3260 Blade Server User’s Guide • April 2009 For information about multiplexing to Zone 2 or Zone 3, see Section 4.8, “Multiplexing to Zones 2 and 3” on page 4-27 . A.4.5 Sun Netra AR TM Connectors (Zone 3) The Netra CP3260 blade server provides all the I/O and power connections for rear access through the Zone 3 R TM connectors[...]

Appendix A Physical Characteristics A-11 T ABLE A-7 gives the Zone 3 J31 connector signals and pin assignments. T ABLE A-8 gives the Zone 3 J32 connector signals and pin assignments T ABLE A-8 Zone 3 J32 Connector Pin Assignments T ABLE A-7 Zone 3 J31 Connector Pin Assignments R o w A BC DE F 1 Z3_F1_TX1+ Z3_F1_TX1 − Z3_F1_RX0 + Z3_F1_RX0 − Z3_[...]

A-12 Netra CP3260 Blade Server User’s Guide • April 2009 T ABLE A-9 gives the Zone 3 J33 connector signals and pin assignments. FIGURE A-8 Zone 3 Power Connector T ABLE A-10 gives the Zone 3 power connector signals and pin assignments. T ABLE A-9 Zone 3 J33 Connector Pin Assignments R o w A BC DE F 1 R TM_TX0 + RTM_TX0 − R TM_RX0 + R TM_RX0 ?[...]

Appendix A Physical Characteristics A-13 For information about multiplexing to Zone 2 or 3, see Section 4.8, “Multiplexing to Zones 2 and 3” on page 4-27 . A.4.6 T OD Clock Battery Holder FIGURE A-9 shows the location of the TOD clock battery and holder . The TOD battery must be type CR1632, with a minimum of 4ma abnormal charging current ratin[...]

A-14 Netra CP3260 Blade Server User’s Guide • April 2009 FIGURE A-9 TOD Battery Location Figure Legend 1 T OD batter y location 2 T OD clock batter y location with FB-DIMM remov ed ① ②[...]

B-1 APPENDIX B Sun OEM IPMI Commands This appendix contains the following sections: ■ Section B.1, “ Get Version Command” on page B-2 ■ Section B.2, “ Get RTM Status Command” on page B-3 ■ Section B.3, “Solaris OS Graceful Shutdown Commands” on page B-4 ■ Section B.4, “ Send Sensor State Command” on page B-5 The commands des[...]

B-2 Netra CP3260 Blade Server User’s Guide • April 2009 B.1 Get Version Command Get Version returns the IPM contr oller (IPMC) firmware version and Standby CPLD version. Bytes 8, 9, and A are r eserved for future use. Command NetFn Opcode Reference Get _ Version 0x2E (OEM) 0x80 - - T ABLE B-1 Get Version Command Data Bytes T ype Byte Data Field[...]

Appendix B Sun OEM IPMI Commands B-3 Example (T erminal Mode): [B8 00 80 00 00 2A] <------- Request [BC 00 80 00 00 00 2A 02 02 00 00 00 00] <----Response ■ The IPMC version is read as: lower nibble of REV1 . high nibble of REV2 . low nibble of REV2 In the preceding example, the IPMC version is 2.0.0. ■ The CPLD version is read as: -> [...]

B-4 Netra CP3260 Blade Server User’s Guide • April 2009 Example (T erminal Mode): [B8 00 88 00 00 2A] <------ Request [BC 00 88 00 00 00 2A 01] <------Response B.3 Solaris OS Graceful Shutdown Commands Users can change the graceful shutdown time out of Solaris OS with an IPMI command executed from ShMM or a user application. Note that thi[...]

Appendix B Sun OEM IPMI Commands B-5 Note – As of the R3U2 release, Netra CP3260 sysfw 7.2.1 has graceful shutdown support. Note – T o use the graceful shutdown effectively , users must determine a component’s shutdown time “cost,” which depends on hardware, softwar e, and configuration. For example, the default 120 seconds could be too s[...]

B-6 Netra CP3260 Blade Server User’s Guide • April 2009 Byte4 Sensor type byte Indicates event class or type of sensor that generates the event message. (Refer to IPMI specification for sensor type codes.) Byte5 Sensor # (optional); W rite to 0 Unique number repr esenting the sensor within the management controller that generates the event mess[...]

Appendix B Sun OEM IPMI Commands B-7 Byte12 Reserved for futur e use. (W rite zero) Response data Byte1 Completion code: 00 = OK C1 = Command not supported CC = Invalid data in request (Refer to IPMI specification for mor e completion codes) Byte2 00 Byte3 00 Byte4 2A T ABLE B-3 Send Sensor State Command Data Bytes (Continued) T ype Byte Data Field[...]

B-8 Netra CP3260 Blade Server User’s Guide • April 2009[...]

Index-1 Index A Advanced Rear T ransition Module (AR TM), 1-1 1 Advanced T elecommunications Computing Architectur e (A TCA), 1-1 AR TM, 1-1 1, 2-18, 5-4, 5-10 installing, 2-19 A TCA (AdvancedTCA), 1-1 A TCA midplane, 2-20 auto-boot (OpenBoot variable), 4-6 automatic system recovery (ASR) disabling, 4-17 enabling, 4-17 obtaining recovery informatio[...]

Index-2 Netra CP3260 Blade Server User’ s Guide • April 2009 fabric interface, 5-1 management port, 5-4, 5-1 1 RJ-45, A-4 SERDES, 5-1 F fabric interface, 5-1, 5-3, 5-9, 5-10 fan tray upgrade kit, 1-14 FB-DIMMs, 2-5, 5-8 description, 2-5 installing, 2-1 1 location, 2-6 removing, 2-10 requir ements, 2-5 features, 1-2 CPU, 1-2 hot-swap, 1-3 IPMI s[...]

Index-3 M MAC address, 3-8 label, 1-16 management port Ethernet, 5-1 1 manual system reset, 4-5 memory, 1-2, 5-8 FB-DIMMs, 2-5, 5-9 N netfunction (NetFn), B-1 Netra CP32X0 AR TM, 1-8, 1-1 1, 1-13, 1-14, 2-20 network boot, 1-8 network device aliases, 4-18 O OpenBoot ok prompt, 4-5 OpenBoot commands probe-ide , 4-8 probe-scsi , 4-7 probe-scsi-all , 4[...]

Index-4 Netra CP3260 Blade Server User’ s Guide • April 2009 Solaris commands init , 4-5 shutdown , 4-5 SP ARC, 5-2 subsystems, 5-1 CPU and memory, 5-1 I/O, 5-1 IPMI, 5-1 power, 5-1 service processor, 5-1 Sun Download Center (SDLC), 3-1 Sun OEM IPMI commands Get R TM Status, B-3 Sun Services, 1-15 SunVTS software, 3-9 support, 1-15 switch con?[...]