Sun Microsystems 3.2 manual

Sun Microsystems, Inc. www .sun.com Submit comments about this document at: http://www.sun.com/hwdocs/feedback Sun StorEdge ™ A v ailability Suite 3.2 Softw are T roub leshooting Guide P ar t No . 817-3752-10 December 2003, Revision 51[...]

Please Recycle Copyright© 2003 Sun Microsystems, Inc., 4150 Network Cir cle, Santa Clara, California 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual pr operty rights relating to technology embodied in this pr oduct. In particular , and without limitation, these intellectual property rights may include one or mor e of the[...]

iii Contents Preface v 1. Point-in-T ime Copy Software T roubleshooting T ips 1 Troubleshooting Checklist 1 Checking Log Files 2 Improving Performance 2 Safeguarding the VTOC Information 3 2. Remote Mirror Software T roubleshooting T ips 5 Troubleshooting Checklist 6 Troubleshooting Log Files and Services 6 Checking Log Files 7 Checking the /etc/ns[...]

iv Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Correcting Common User Errors 13 Enabled Software on Only One Host 13 V olumes Ar e Inaccessible 13 W rong V olume Set Name Specif ied 14 Accommodating Memory Requirements 16 3. Error Messages 19[...]

v Pr eface Sun StorEdge A vailability Suite 3.2 Software T r oubleshooting Guide helps users solve common problems that might arise when using the Sun StorEdge™ A vailability Suite 3.2 software. Befor e Y ou Read This Book T o use the information in this document, you must have thorough knowledge of the topics discussed in these books: ■ Sun St[...]

vi Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 How This Book Is Or ganized This book includes the following chapters: Chapter 1 helps to solve problems associated with the point-in-time copy software. Chapter 2 helps to solve problems associated with the remote mirr or software. Chapter 3 provides an alphabe[...]

Preface vii Shell Pr ompts T ypographic Conventions Shell Prompt C shell machine-name % C shell superuser machine-name # Bourne shell and Korn shell $ Bourne shell and Korn shell superuser # T ypeface 1 1 The settings on your browser might differ from these settings. Meaning Examples AaBbCc123 The names of commands, files, and directories; on-scr e[...]

viii Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Related Documentation Application Title Part Number Man pages sndradm iiadm dsstat kstat svadm N/A Latest release information Sun StorEdge A vailability Suite 3.2 Software Release Notes 817-2782 Sun Cluster 3.0 and Sun StorEdge Software Release Note Supplement[...]

Preface ix Accessing Sun Documentation Y ou can view , print, or purchase a broad selection of Sun documentation, including localized versions, at: http://www.sun.com/documentation Contacting Sun T echnical Support If you have technical questions about this product that are not answered in this document, go to: http://www.sun.com/service/contacting[...]

x Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003[...]

1 CHAPTER 1 Point-in-T ime Copy Softwar e T roubleshooting T ips This chapter describes how to avoid or troubleshoot problems that might occur when using the point-in-time copy software. The chapter includes the following topics: ■ “T roubleshooting Checklist” on page 1 ■ “Checking Log Files” on page 2 ■ “Improving Performance” on[...]

2 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Checking Log Files Y ou can check the status of the point-in-time copy software by examining two system log files: ■ /var/opt/SUNWesm/ds.log The /var/opt/SUNWesm/ds.log file contains timestamped messages about the point-in-time copy software, including error me[...]

Chapter 1 P oint-in-Time Copy Software T roubleshooting Tips 3 Safeguar ding the VTOC Information Caution – When creating shadow volume sets, do not cr eate shadow or bitmap volumes using partitions that include cylinder 0. Data loss might occur . The Solaris system administrator must be knowledgable about the virtual table of contents (VTOC) tha[...]

4 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003[...]

5 CHAPTER 2 Remote Mirr or Software T roubleshooting T ips This section describes how to avoid or troubleshoot any pr oblems might occur when using the remote mirr or software. The following topics are described. ■ “T roubleshooting Checklist” on page 6 ■ “T roubleshooting Log Files and Services” on page 6 ■ “Checking the Integrity [...]

6 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 T roubleshooting Checklist This table shows the troubleshooting checklist and related sections. T roubleshooting Log Files and Services The remote mirr or software is client-server software that is bidir ectional. The primary and secondary hosts each act as a cli[...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 7 Checking Log Files Check the following files to tr oubleshoot problems: ■ /var/opt/SUNWesm/ds.log The /var/opt/SUNWesm/ds.log file contains timestamped messages about the software. For example: ■ /var/adm/messages Make sure that the rdc service is active when the remote mirr or software [...]

8 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Checking the /etc/nsswitch.conf File If entries in the /etc/nsswitch.conf are not conf igured corr ectly , you might encounter these problems: ■ If the hosts: entry is incorrect, volume sets not r esume after a reboot. ■ If the services: entry is incorrect, t[...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 9 where: ■ -T tcp specifies the transport that the service uses. ■ hostname is the name of the machine where the service is running. If the service is not running, this message is displayed: If you see this message, it is possible that the /etc/nsswitch.conf services: entry is incorrectly [...]

10 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 ■ The /etc/name_to_major file is missing an entry for the /dev/rdc pseudo-link. This example shows a valid entry (the number following rdc can be any number): ■ The /usr/kernel/drv/rdc.conf file is incomplete. This example shows a valid entry: Checking the I[...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 11 T esting with ifconfig Use the ifconfig command to make sure that the network interface is configur ed and running correctly . This example output shows all the interfaces that are configur ed and running: T esting with ping Use the ping command to verify that the network interfaces can com[...]

12 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 In the first example, the command is issued fr om the primary host nws822 to the secondary host nws350 . The network interface is hme0 and the port used by the rdc service is reported. In the second example, the link is A TM so the atmsnoop utility is appr opria[...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 13 Corr ecting Common User Errors This section describes user errors encountered often when using the softwar e. ■ “Enabled Software on Only One Host” on page 13 ■ “V olumes Are Inaccessible” on page 13 ■ “W r ong V olume Set Name Specified” on page 14 Enabled Softwar e on On[...]

14 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 This example shows the newfs -N command completing successfully: This example shows a typical error caused when the secondary host is not enabled or a disk or volume is inaccessible: W rong V olume Set Name Specified When you first enable a set, the remote mirr [...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 15 For example, this command updates the volume on the secondary host calamari from the primary host volume: T o correctly display the volume set name, use the sndradm -p command on the primary host. See “T o Find the V olume Set Name” on page 16 . Using the dsstat Command Incorrectly An a[...]

16 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 ▼ T o Find the V olume Set Name 1. If you are unsure of the volume set name, type the following command from the primary host: Running Startup Script Out of Order The scripts to configure the network interface must run befor e the A vailability Suite’s start[...]

Chapter 2 Remote Mirror Software T roub leshooting Tips 17 The order of write operations must be maintained within a group. Ther efore, these out of order r equests must be stored in memory on the secondary host until the missing request comes in and completes. The secondary host can store up to the har d-coded limit of 64 requests per gr oup. Exce[...]

18 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003[...]

19 CHAPTER 3 Err or Messages Error messages during the installation pr ocess are described in the Sun StorEdge A vailability Suite 3.2 Software Installation Guide. Solaris err or messages related to the Sun StorEdge A vailability Suite software are described in .... T ABLE 3-1 lists Sun StorEdge A vailability Suite 3.2 error messages in alphabetica[...]

20 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Abort failed PITC iiadm could not abort a copy or update operation on a set. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_EEMPTY : No set was specif ied. DS[...]

Chapter 3 Error Messages 21 Bitmap reconfig failed %s:%s Kernel A request to reconf igure the bitmap on the local host has failed. This can happen for two reasons: • The old bitmap cannot be read fr om to obtain needed information. • The new bitmap cannot be reserved because the volume is not accessible or is already in use. V erify that the ne[...]

22 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 cannot find SNDR set <shost>:<svol> in config RM Remote mirror set cannot be found in the conf iguration database. The set is not configur ed. Check the entry for errors. Cannot reconfig %s:%s to %s:%s, Must be in logging mode Kernel An operation has[...]

Chapter 3 Error Messages 23 Cannot enable %s:%s ==> %s:%s, secondary in use in another set Kernel A set being enabled or resumed has a secondary volume that is already in use as a secondary volume for another remote mirr or set. A volume cannot be enabled as a secondary volume if it is already in use as a secondary volume by another remote mirro[...]

24 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Copy failed PITC A copy or update operation could not be initiated. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_EEMPTY : No set was specif ied on the comma[...]

Chapter 3 Error Messages 25 Create overflow failed PITC An overflow volume couldn’t be initialized. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_EEMPTY : No overflow volume was specif ied. DSW_EINUSE : The volume that was specified is alr ead[...]

26 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Disable pending on diskq %s, try again later Kernel A request to disable the disk queue is already in progr ess. V erify that the previous request has completed successfully . If it has, this r equest is no longer valid. If it has not, wait for it to complete un[...]

Chapter 3 Error Messages 27 disk queue volume <vol> must not match any primary SNDR volume or bitmap RM The disk queue volume specified for the reconf iguration operation is already in use by the remote mirr or software as a data volume or bitmap volume. don't understand shadow type PITC The iiadm -e command expected dep or ind . DSWIOC_[...]

28 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Enable failed PITC Could not enable volume. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_ESHUTDOWN : The kernel module is in the process of shutting down th[...]

Chapter 3 Error Messages 29 Failed to detach overflow volume PITC iiadm had a problem detaching the overflow volume from a set. Possible err ors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_EEMPTY : No set was specif ied to detach from. DSW_ENOTFOUND : The set[...]

30 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 hostname tag exceeds CFG_MAX_BUF PITC Because CFG_MAX_BUF is 1k, this message is not expected to be reported. Import failed PITC Could not import shadow volume. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm EN[...]

Chapter 3 Error Messages 31 Join failed PITC Could not join shadow volume back to the set. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . DSW_EEMPTY : A volume was missing on the command line. DSW_ENOTFOUND : The set could not be found in the kernel.[...]

32 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Memory allocation failure PITC iiadm ran out of memory . Must be super-user to execute Kernel The user issued a remote mirror command but does not have superuser privileges. All r emote mirror commands requir e superuser privileges. must specify full set details[...]

Chapter 3 Error Messages 33 Overflow list access failure PITC iiadm could not get a list of overflow volumes fr om the kernel. Possible errors: EF AUL T : The kernel module tried to read out-of- bound. File a bug against iiadm. ENOMEM: The kernel module ran out of memory . Overflow volume is already in an InstantImage group PITC During an attach op[...]

34 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Reverse sync needed, cannot sync %s:%s ==> %s:%s Kernel The user requested a forward sync operation for a remote mirr or set which needs a reverse sync. This occurs when a previous r everse sync does not complete successfully or because the primary volume was[...]

Chapter 3 Error Messages 35 Shadow group %s is suspended PITC The user attempted to perform a copy or update operation on a group with one or more suspended sets. The %s parameter identifies the f irst set found in the group that is suspended. Shadow group suspended PITC The user attempted to perform a copy or update operation on a suspended set. S[...]

36 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 SNDR set does not have a disk queue RM Set does not have a disk queue attached when attempting either a queue r emove operation or a queue replace operation. SNDR: The volume ’<vol2>’ has been configured previously as ’<vol1>’. Re-enter comma[...]

Chapter 3 Error Messages 37 The volume %s is already in use Kernel The data volume in the remote mirror set is alr eady in use as a bitmap volume or a disk queue volume. Use a differ ent data volume. Too many volumes given for update PITC iiadm ran out of memory . Unable to access bitmap PITC During an enable operation, iiadm tried to validate the [...]

38 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 unable to determine hostname: <host> RM Could not determine the host name of the system. unable to determine IP addresses for either host <phost> or host <shost> RM The IP address for either the primary host or the secondary host could not be d[...]

Chapter 3 Error Messages 39 unable to obtain unique set id for <shost>:<svol> RM Lookup of the set ID in the configuration database for this set has failed. Unable to open bitmap file <vol> RM The volume specified for the bitmap could not be opened. The volume might not exist or is already in use by another program. Unable to open[...]

40 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Update failed PITC One or more volumes in a group copy or update command failed. Possible errors: EFAULT : The kernel module tried to read out-of- bounds. File a bug against iiadm. ENOMEM : The kernel module ran out of memory . EINVAL : User is performing a shad[...]

Chapter 3 Error Messages 41 Volumes are not in same disk group PITC iiadm detected that the master , shadow , and bitmap volumes are not all in the same cluster device group, as requir ed by the point-in-time copy software. volume "<vol>" is not part of a disk group, please specify resource ctag RM The volume <vol> is not bein[...]

42 Sun StorEdge Availability Suite 3.2 Software Troubleshooting Guide • December 2003 Related Error Messages The Solaris configuration administration utility , cfgadm, reports an err or when it is used on systems where the Sun StorEdge A vailability Suite software is installed. The error occurs because a pr ocess does not suspend properly so that[...]