Symantec HP-UX manual

N184 86G Veritas Storage Foundation ™ Cluster File System Installation Guide HP-UX 5.0[...]

Veritas Storage Foundation Cluster File System Installation Guide Copy right © 2006 Symantec Corporation. All rights reserved. SFCFS 5. 0 Symantec, the Symantec logo, Veritas, and Veritas Storage Foun dation Cluster File System are trademarks or registered tra demarks of Symantec Corporation or its aff iliates in the U.S. and other countries. Othe[...]

Third-pa rty legal notices Third-party sof tware may be recommended, distributed, embedded, or bundled with this Veritas prod uct. Such third-party software i s licensed separately by i ts copyright holder. All third -party copyrigh ts associated with thi s product are listed in the a ccompany ing rele ase notes. HP-UX is a registered trademark of [...]

[...]

Contents Chapter 1 Installing and c onfiguring the pr oduct Hardware overview ........................................................................................ ...... 10 Shared storage ........................................................................................ ...... 11 Fibre channel swi tch .....................................[...]

6 Chapter 2 Upgrading the product Preparing to upgrade the product ............................................... ...................... 36 Planning the upgr ade ........................................ ........................................ .. 36 Upgrade paths ........................................................................ ............[...]

Chapter 1 Installing and configuring the product This c hapter describes ho w to inst all the Veri tas Storage Foundati on Clu ster Fil e Sys tem ( SFCF S). SFCFS requires several Veritas software packages to configure a cluster and to pr ovide messaging services. T hese packages include the Veritas Cluster Serv er (VCS) to mo nitor systems and app[...]

10 Installi ng and co nfiguri ng the produ ct Hardwa re overview Hardware overview VxFS clus ter functi onalit y runs optimal ly on a Fibre C hannel fabric . Fibre Channel technology provi des the fastest, most reliable, and highest band width connectivity currently available. By employing Fibre Channel technology, SFCFS can be used in conjunction [...]

11 Instal ling an d con figuring the pr oduct Hard ware over view Shared storage Shared storage can be one or more sh ared disks or a disk array connected either directly to the nodes of th e cluster or through a Fibre C hannel Switch. Nodes c an also have non -shared or local devi ces on a local I/O channel. It is advisable to have / , /usr , /var[...]

12 Installi ng and co nfiguri ng the produ ct Soft wa re com pon ents Software components Storage Foundation for Cluster File System i s the name of th e Veritas Cluster File System product and its supporting software packages. Storage Foundation Cluster File System HA (SF CFS HA) provides support for application failover functio nality of Veritas [...]

13 Instal ling an d con figuring the pr oduct Sof twa re com pon ents VRTSvcs V eri tas Clu ster Serve r VRTSacclib V eritas ACC L ibrary VRTSvcsag V eri tas Clu ster Serve r B un dl ed Age nts VRTSvcsmg V eri tas Clu ster Serve r Mes sa ge Cat alog s VRTSjre V eritas J ava Runtime Envir onment Redistribution VRTSjre15 V eritas J ava Runtime Envir [...]

14 Installi ng and co nfiguri ng the produ ct Soft wa re com pon ents Optiona l packages for SFCFS and SFCFS H A Note: S FCFS 5.0 operates only on HP-UX 11 i 64-bit operating system with the Septem ber 2004 HP-UX 11 i Versi on 2.0 or la ter. All cl uster nod es must be runnin g this OS vers ion. For cluster fi le system functi onality to work re li[...]

15 Instal ling an d con figuring the pr oduct Req uir ed HP- UX patc he s Required HP-UX pat ches HP-UX required patches include the following: HP-UX Patch ID Description PHCO_32385 Enables fscat (1M). PHCO_32387 Enables getext (1M). PHCO_32388 Enables setext (1M). PHCO_32389 Enables vxdump (1M). PHCO_32390 Enables vxrestore (1M) . PHCO_32391 Enabl[...]

16 Installi ng and co nfiguri ng the produ ct Required HP-UX patches In addition to the ab ove patches the EnableVXFS bundle needs to be installed before instal ling the SFCFS 5.0. Thi s bundle is a HP bundle and cont ains enhancements to various commands to understand the new disk layout Version 6 and later. The EnableVXFS bu ndle contain s the fo[...]

17 Instal ling an d con figuring the pr oduct Preins tallation Preinstallation Release No tes Read the Release Notes for all products included with this product. Portable Docum ent Fo rmat (.p df) ve rsions of the Release Notes are included on the software disc in the storage_foundation_cluster_file_system /release_notes directory and on the docume[...]

18 Installi ng and co nfiguri ng the produ ct Prei nstal latio n Also, you can get th e patches from Hewl ett-Packard’s Patch Database off ered under the Maintenance and Support section of the HP Services & Support - IT Resource Center. HP’s Patc h Database provides f ast, accura te searches f or the latest recommended and superseded patche[...]

19 Instal ling an d con figuring the pr oduct Prerequi sites Veritas Enterprise Administrator The Ver itas Ente rprise Adm inistra tor (VEA ) client can be insta lled and r un on any machine that supports the Java R untime Environment. VEA is required to ac cess the graphical user interface (GUI) for Veritas Storage Foundation. You can use the GUI [...]

20 Installi ng and co nfiguri ng the produ ct Installin g the pro duct ■ The host n ames of the c luster nodes. ■ The de vice names of the networ k interf ace c ards (NICs ) used f or the priv ate networks among nodes. ■ Esta blishi ng c ommu nicat ion betw een n odes is r equ ir ed t o insta ll V eritas softw ar e fr om a r emote sy stem, or[...]

21 Instal ling an d con figuring the pr oduct Install ing the product T o install the product 1 Log in as superuser . 2 Insert the appr opriate media disc int o your s yst em ’ s D VD-ROM driv e con necte d to yo ur sys tem. 3 Determine the block de vice f ile f or the DVD dr ive: # ioscan -fnC disk Make a note of the device file as i t applies t[...]

22 Installi ng and co nfiguri ng the produ ct Conf igur ing the Co mpon ents Configuring the Components This s ections descr ibes the conf igurat ion of SFCFS compon ents. T o configure the component s 1 Log in as superuser . 2 Run t h e installer comman d t o install the SFCFS . F or example: # cd /cdrom # ./installer 3 F r om the Installation men[...]

23 Instal ling an d con figuring the pr oduct Configurin g the Compone nts 7 Answer the pr o mpts t o co nfigur e V CS f or SFCFS. Y ou are pr ompted to confi gur e SFCFS to use V eritas Security Ser vic es. Would you like to configure SFCFS to use Veritas Security Services? [y,n,q] (n) 8 Ent er y or n to c onf igur e SFCFS t o use V erita s Securi[...]

24 Installi ng and co nfiguri ng the produ ct Using th e log files 16 Ent er y or n if the V xVM default disk gr oup inf ormation is cor rec t. Y ou ar e pr ompted t o enable c entr aliz ed managem ent. Enable Centralized Management? [y,n,q] (y) n 17 Ent er y or n t o enable centr alized manag ement. Y ou are pr o mpted t o verif y the f ully q ual[...]

25 Instal ling an d con figuring the pr oduct V erifyin g the configurat ion fi les Verifying the configuration files You can inspect the contents of the configuration files that were installed and modified after a successful install ation process. These files reflect the config urat ion ba sed on the i nformati on y ou sup plied. T o verify the co[...]

26 Installi ng and co nfiguri ng the produ ct V erifyin g the co nfigura tion fil es Checking Low Latency Tra nsport operation Use the lltstat command to verify tha t links are active for LLT. This command returns informat ion about the links for LLT for the system on whic h it is typed. See the lltstat (1M) manual page. In the following example, l[...]

27 Instal ling an d con figuring the pr oduct V erifyin g the configurat ion fi les . . . 31 CONNWAIT lan1 DOWN lan2 DOWN Note: The output lists 3 2 nodes. It reports on the two cluster nodes, system01 and system02, plus non-existent nodes. For each correctly configured system, the information shows a state o f OPE N , a status for each link of UP [...]

28 Installi ng and co nfiguri ng the produ ct V erifyin g the co nfigura tion fil es Group Membership and Atomic B roadcast configuration files The following files a re required by th e VCS communication services for Group Memb ershi p and Atom ic Br oadc ast (G AB). /e t c /g a b t a b After installation , the file /etc/gabtab c ontai ns a gabconf[...]

29 Instal ling an d con figuring the pr oduct V erifyin g the configurat ion fi les Checkin g clus ter operat ion This section descr ibes how to check cl uster operation. T o check cluster operation 1 Ent er th e f o llo wing c omma nd on an y s ys tem : # hastatus -summary The output for a n SFCFS HA installation resembles: -- SYSTEM STATE -- Syst[...]

30 Installi ng and co nfiguri ng the produ ct V erifyin g the co nfigura tion fil es system01 ConfigDiskState CURRENT system01 ConfigFile /etc/VRTSvcs/conf/config system01 ConfigInfoCnt 0 system01 ConfigModDate Tues June 25 23:00:00 2006 system01 CurrentLimits system01 DiskHbStatus system01 DynamicLoad 0 system01 Frozen 0 system01 GUIIPAddr system0[...]

31 Instal ling an d con figuring the pr oduct V erifyin g age nt config uration Verifying agent configuration This section descr ibes how to verify the a gent configuration. T o verify the agent configuration ■ Ent er the c luste r stat us com mand fr o m any node in the c lust er: # cfscluster status Output resembles: Node : system01 Cluster Man[...]

32 Installi ng and co nfiguri ng the produ ct Conf igur ing VC S In a VCS cluster, the first system to be brought online rea ds the configuratio n file and creates an internal (in-memory) representation of the configuration. Systems brough t online after the first system derive their info rmatio n from systems running in the cluster. You mu st stop[...]

33 Instal ling an d con figuring the pr oduct VCS applic ation fail over s ervi ces ) CVMVxconfigd cvm_vxconfigd ( Critical = 0 CVMVxconfigdArgs = { syslog } ) cvm_clus requires cvm_vxconfigd vxfsckd requires cvm_clus // resource dependency tree // // group cvm // { // CFSfsckd vxfsckd // { // CVMCluster // cvm_clus // { // CVMVxconfigd // cvm_vxco[...]

34 Installi ng and co nfiguri ng the produ ct VCS applic ation fail over s ervi ces[...]

Chapter 2 Upgrading the product If you are running an ea rlier release of Veritas Storage Foundation C luster File System, you can upgrade your product using the procedures described in this chapt er. Topic s covered in this chap ter incl ude: ■ Preparing to upgrade th e product ■ Upgrade Overview ■ Upgr ading fro m 3.5 to 5. 0 ■ Upgr ading[...]

36 Upgrading the product Preparin g to upgrade th e product Preparing to upgrade the product This section prepares you for the Veritas Storage Foun dation Cluster File System upgrade. Planning the up grade Complete the f ollowing tasks in advanc e of upgr ading: ■ Review t he Veritas Storag e Foundat ion Cluster File Sy stem Release Notes for any[...]

37 Upgrading the product Prepari ng to upgr ade the product Stor ag e F oundation Clust er Fi le Sys tem 3.5 Upd ate 3 (form erl y known as, SANPoint F ound ation Suite 3.5 Upd ate 3) Stor age F oundation Cluste r File System 5. 0 Pr oceed to “ Upgrading from 3.5 to 5.0 ” on page 39. Stor ag e F oundation Clust er F ile Sy stem 4.1 Stor age F o[...]

38 Upgrading the product Upgrade Ove rview Upgrade Overview There are two ways to upgrade cluster nodes to the latest version of Storage Foundation Cluster File System: phased a nd full. Phased upgrade A phased upgrade minimizes downtime by upgrading portions of the cluster, one at a time. Although the entire c luster is offline for a shorter perio[...]

39 Upgrading the product Upgra ding fro m 3.5 to 5.0 Upgradin g from 3 .5 to 5.0 SFCFS can be upgraded from 3 .5 to 5.0 using phased or full upgrade procedure. Phased upgrade Following procedure as sumes a 4 n ode cluster system01, system02, system03, system04 where system01 a nd system02 ar e initially upgraded and rest of the cluster is brought u[...]

40 Upgrading the product Upgra ding from 3.5 t o 5. 0 9 Unin stall VCS 3. 5 from system 01 a nd syste m02. Run the followi ng c omma nds f r om one of t he n odes. See the Veritas Cluster Server In stallation Guide . # cd /opt/VRTSvcs/install # ./uninstallvcs Note: I gnore any errors from the uninstallvcs script and proceed with the unins tall of V[...]

41 Upgrading the product Upgra ding fro m 3.5 to 5.0 15 Change the configuration files by running the f ollowing commands on one of the upgr aded nodes, sa y sy stem01. # /opt/VRTS/bin/hastart # /opt/VRTS/bin/haconf -makerw # hagrp -unfreeze cvm -persistent # hagrp -unfreeze service_group -persistent # /opt/VRTS/bin/hatype -add CVMVxconfigd # /opt/[...]

42 Upgrading the product Upgra ding from 3.5 t o 5. 0 18 Conf igure SFCFS on syst em01 and system 02. See “ U sing the log fil es ”. Note: VCS configurati on files are not c hanged duri ng this config uration . 19 Upgr ade f ile sy stems t o pr oper disk la y out ver sion as ment ioned in “ Upgradin g the disk layout version s ” on pa ge 50[...]

43 Upgrading the product Upgra ding fro m 3.5 to 5.0 8 Unins tall V CS 3.5 fr om all the n odes. R un the f ollo wing c ommands fr om one of the nodes. See the Veritas Cluster Server In stallation Guide . # cd /opt/VRTSvcs/install # ./uninstallvcs Note: Ign ore any er rors fr om the uninstallvcs script and proceed wi th the uninstall of VCS. 9 Upgr[...]

44 Upgrading the product Upgra ding from 3.5 t o 5. 0 13 Change the configu ration files by runn ing the following commands from one of the nodes. # /opt/VRTS/bin/hastart # /opt/VRTS/bin/haconf -makerw # /opt/VRTS/bin/hatype -add CVMVxconfigd # /opt/VRTS/bin/hares -add cvm_vxconfigd CVMVxconfigd cvm # /opt/VRTS/bin/hares -modify cvm_vxconfigd Enabl[...]

45 Upgrading the product Upgra ding fro m 4.1 to 5.0 16 Conf igure SFCFS on syst em01 and system 02. See “ U sing the log file s ”. Note: VCS confi guration f iles ar e not ch anged du ring t his co nfigur ation. 17 Upgr ade f ile sy stems t o pr oper disk lay o ut v ersion as mentioned in “ Upgradin g the disk layout version s ” on pa ge 5[...]

46 Upgrading the product Upgra ding from 4.1 t o 5. 0 # mv /etc/llthosts /etc/llthosts.bak b Install all the pr er equisite pat c hes and reboot the machines. c Move /etc/llthosts to /etc/llthosts.bak on al l the node s t o be upgraded . # mv /etc/llthosts.bak /etc/llthosts 8 Offline all SFCFS reso urc es on nodes sel ected in step 2 by r unning th[...]

47 Upgrading the product Upgra ding fro m 4.1 to 5.0 12 Change the configuration files by running the f ollowing commands on one of the upgra ded nodes. F or exampl e, sy stem01. # /opt/VRTS/bin/hastart # /opt/VRTS/bin/haconf -makerw # hagrp -unfreeze cvm -persistent # hagrp -unfreeze service_group -persistent # /opt/VRTS/bin/hares -delete qlogckd [...]

48 Upgrading the product Upgra ding from 4.1 t o 5. 0 16 Conf igure SFCFS on syst em01 and system 02. See “ U sing the log fil es ”. Note: VCS configurati on files are not c hanged duri ng this config uration . 17 Upgr ade f ile sy stems t o pr oper disk la y out ver sion as ment ioned in “ Upgradin g the disk layout version s ” on pa ge 50[...]

49 Upgrading the product Upgra ding fro m 4.1 to 5.0 8 Inst all SFCFS 5.0 a nd reboot all the nodes. See “ Installing the product ” on page 19. Note: Do not configure SFCFS after reboot. 9 Start vxfen on all the nodes. vxfen can be started either in disable or enable mode. F or starting vxfen i n disabled mode, ru n the f ollo wing c ommands: #[...]

50 Upgrading the product Upgradin g the disk layout versio ns 12 V erify the s yntax of the /etc/VRTSvcs/conf/config/main.cf file by running the following comma nds on system01: # cd /etc/VRTSvcs/conf/config # /opt/VRTS/bin/hacf -verify . 13 R un the f ollow ing c ommand o n all the node s t o st art V CS. # /opt/VRTS/bin/hastart 14 Conf igure SFCF[...]

51 Upgrading the product Upgrad ing th e disk layout v ersions 4 On the node select ed in step 1 , after the disk lay o ut has been su cc essfully upgraded , unmount t he file sy stem. # umount /mnt1 5 Thi s file sy stem can be mount ed on all no des of th e c luster using cfsmount .[...]

52 Upgrading the product Upgradin g the disk layout versio ns[...]

Chapter 3 Adding and removing a node This cha pter prov ides info rmatio n on how to ad d a node to an exis ting clus ter and re moving a node from a c luster . Topics incl ude: ■ Adding a no de t o a clu ster ■ Configu ring SFCFS an d CVM a gents on the new no de ■ Removin g a no de fro m a cl uster[...]

54 Adding and remo ving a node Adding a no de to a cluste r Adding a node to a cluster If yo u wa nt to a dd a ne w nod e to a m ulti -n ode c lus ter, fir st pre pa re the new system hardware. Physically con nect the new system to the cluster using private networks and attach to any shared st orage. Then install the required OS software. Install a[...]

55 Adding an d remov ing a nod e Adding a n ode to a cl uster 11 Ent er y or n f or another lic ense k e y . Y ou ar e pr ompted t o pr ess R eturn t o con ti nu e . Do you want to enter another license key for system03? [y,n,q,?] (n) 12 Ent er 1 or 2 to be installed o n all sy stem s. Select the packages to be installed on all systems? [1-2,q,?] ([...]

56 Adding and remo ving a node Conf igur ing SFC FS a nd CVM age nts on t he new nod e Configuring SFCFS and CVM agents on the new node You must configure t he SFCFS and CVM agents, after rebooting the new sy stem. T o configure SFCFS and CVM agents on the new node 1 Start the VCS server and vxfen on sys tem 03. a Use hastart on sy stem03 f or star[...]

57 Adding an d remov ing a nod e Removing a nod e from a clus ter # haconf —dump -makero 9 Put the CVM r esour ces bac k online, in the follo wing or der: # hagrp -online cvm -sys system01 # hagrp -online cvm -sys system02 # hagrp -online cvm -sys system03 10 Check the s yst em stat us to see whether the ne w node is online: # hastatus —sum -- [...]

58 Adding and remo ving a node Removing a n ode from a cluste r 4 Remo v e syst em03 fr o m the s ystem li st at tribute of the CVM and SFCFS servic e gr oups: # hagrp -modify service_group SystemList -delete system03 # hagrp -modify cvm SystemListf -delete system03 where service_group i s the command that displays the service groups by hagrp -dep [...]

59 Adding an d remov ing a nod e Removing a nod e from a clus ter 12 From t h e scripts direct ory , run the uninstallsfcfs scr ipt and r emo ve SFCFS on system 03: # ./uninstallsfcfs If you do not want to remove the Veritas Clu ster Server software, enter n when prompted to uninstall VCS. See the Veritas Cluster Server In stallation Guide , on th [...]

60 Adding and remo ving a node Removing a n ode from a cluste r[...]

Chapter 4 Uninstalling the product If you need to uninstall SFCFS software. Us e the uninstallsfcfs script. T o uninstall SFCFS HA 1 Log in as superuser . Note: Do not use the hastop -force command to sto p VCS. 2 Change directory to /opt/VRTS/install : # cd /opt/VRTS/install 3 Run t h e uninstallsfcfs c ommand t o uninstall SFCF S. T he uninstalls[...]

62 Uninstalling the p roduct[...]

Appendix A Troubleshooting and recovery Installation issues If you encounter any issues installing SF CFS, refer to the following paragraphs for typical problems and their solutions. Incorrect permissions for root on remote system The permissions are inappropriate. M ake sure you have remote root access permission on each system to which you are in[...]

64 Storage F oun dation Clust er File Syst em probl ems Resource tempo rarily unavailable If the installation f ails with the fo llowing error message on the console: fork() failed: Resource temporarily unavailable The value of nkthr ead tunable parameter nay not be large enough. Th e nkthread tunable req uires a minimum val ue of 600 on al l syste[...]

65 Sto rag e Foun dati on C lus ter File Sy stem probl ems Unmount fa ilu res The umount command ca n fai l if a referen ce is being h eld by a n NFS server. Unshare the mount point and try the unmount agai n. Moun t failure s Mounti ng a file system can fail for the followi ng reason s: ■ The f ile s ys tem is not using disk lay out V ersio n 6 [...]

66 Storage F oun dation Clust er File Syst em probl ems ■ If this err or messag e displays: mount: slow The node may be in the process of joining the cluster. ■ If you try to mount a file sy stem that is already mounted without –o cluster option (that is, not in shared mode ) on another cluster node, # mount -F vxfs /dev/vx/dsk/share/vol01 /v[...]

67 Sto rag e Foun dati on C lus ter File Sy stem probl ems High availability issues Network pa rtition/jeopardy Networ k partit ion (o r split brain ) is a con dition where a netw ork fa ilure can be misinterpreted as a failure of one or more nodes i n a cluster. If one system in the cluster incor rectly assumes th at another system f ailed, it may[...]

68 Storage F oun dation Clust er File Syst em probl ems Low memo ry Under h eavy lo ads, softwar e that ma na ges heartbeat communication links may not be ab le to all ocate ke rnel memory . If thi s occurs , a no de halts to avoi d any chance of network partitioning. Reduce the load on the node if this happens frequent ly. A similar situation may [...]