LINUX and SOLARIS: NFS

HA NFS solution using Veritas Cluster server on Solaris

1.Architecture

2.Create disk group and volume

bash-3.00# vxdg init datadg cds=off c1t4d0s2 c1t5d0s2 c1t6d0s2 c1t8d0s2 c1t9d0s2 c1t10d0s2 c1t11d0s2 c1t12d0s2 c1t13d0s2

VxVM vxdg ERROR V-5-1-2349 Device c1t4d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t5d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t6d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t8d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t9d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t10d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t11d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t12d0s2 appears to be owned by disk group datadg.

VxVM vxdg ERROR V-5-1-2349 Device c1t13d0s2 appears to be owned by disk group datadg.

bash-3.00# vxdg import datadg

bash-3.00# vxdisk list

DEVICE TYPE DISK GROUP STATUS

c1t0d0s2 auto:none – – online invalid

c1t1d0s2 auto:cdsdisk c1t1d0 vxfencoorddg online

c1t4d0s2 auto:sliced datadg01 datadg online

c1t5d0s2 auto:sliced datadg02 datadg online

c1t6d0s2 auto:sliced datadg03 datadg online

c1t8d0s2 auto:sliced datadg04 datadg online

c1t9d0s2 auto:sliced datadg05 datadg online

c1t10d0s2 auto:sliced datadg06 datadg online

c1t11d0s2 auto:sliced datadg07 datadg online

c1t12d0s2 auto:sliced datadg08 datadg online

c1t13d0s2 auto:sliced datadg09 datadg online spare

c2t0d0s2 auto:cdsdisk c2t0d0 vxfencoorddg online

c3t0d0s2 auto:cdsdisk c3t0d0 vxfencoorddg online

bash-3.00# vxassist -g datadg make vol1 100m layout=nolog

on both nodes:

mkdir /nfsshare

bash-3.00# newfs /dev/vx/dsk/datadg/vol1

/dev/vx/rdsk/datadg/vol1: Unable to find Media type. Proceeding with system determined parameters.

newfs: construct a new file system /dev/vx/rdsk/datadg/vol1: (y/n)? y

/dev/vx/rdsk/datadg/vol1: 204800 sectors in 100 cylinders of 32 tracks, 64 sectors

100.0MB in 7 cyl groups (16 c/g, 16.00MB/g, 7680 i/g)

super-block backups (for fsck -F ufs -o b=#) at:

32, 32864, 65696, 98528, 131360, 164192, 197024,

3.On both nodes, disable system NFS

bash-3.00# svccfg delete -f svc:/network/nfs/server:default

svccfg: Pattern ‘svc:/network/nfs/server:default’ doesn’t match any instances or services

bash-3.00# svccfg delete -f svc:/network/nfs/mapid:default

4.check cluster status

bash-3.00# lltstat -l

LLT link information:

link 0 e1000g2 on etherfp hipri

mtu 1500, sap 0xcafe, broadcast FF:FF:FF:FF:FF:FF, addrlen 6

txpkts 29754 txbytes 2079067

rxpkts 77210 rxbytes 5029877

latehb 13 badcksum 0 errors 0

link 1 e1000g3 on etherfp hipri

mtu 1500, sap 0xcafe, broadcast FF:FF:FF:FF:FF:FF, addrlen 6

txpkts 29758 txbytes 2065160

rxpkts 82794 rxbytes 6109424

latehb 16 badcksum 0 errors 0

bash-3.00# modinfo|grep gab

226 fffffffff719d000 48850 229 1 gab (GAB device 5.0)

bash-3.00# gabconfig -a

GAB Port Memberships

===============================================================

Port a gen 813c01 membership 01

Port h gen 813c03 membership 01

bash-3.00# hacf -verify /etc/VRTSvcs/conf/config

bash-3.00# hauser -display

UserName : Privilege

——————–

admin : ClusterAdministrator

bash-3.00# haclus -value EngineVersion

5.0.00.0

5.Create NFS resources

bash-3.00# hares -modify nfsIP Address 192.168.47.239

VCS WARNING V-16-1-11309 Configuration must be ReadWrite

bash-3.00# haconf -makerw

bash-3.00# hares -modify nfsIP Address 192.168.47.239

bash-3.00# haconf -dump –makero

420 hagrp -modify hanfs SystemList solarisA 1 solarisB 2

421 hagrp -autoenable hanfs -sys solarisA

422 hares -add nfsNIC NIC hanfs

423 hares -modify nfsNIC Enabled 1

424 hares -modify nfsNIC Device e1000g0

425 hares -modify nfsIP Enabled 1

426 hares -modify nfsIP Device e1000g0

427 hares -modify nfsIP Address 192.168.47.133

428 hares -modify nfsIP IfconfigTwice 1

429 hares -add nfsDG DiskGroup hanfs

430 hares -modify nfsDG Enabled 1

431 hares -modify nfsDG DiskGroup datadg

432 hares -modify nfsDG StartVolumes 0

433 hares -add nfsVOL Volume hanfs

434 hares -modify nfsVOL Enabled 1

435 hares -modify nfsVOL Volume vol01

436 hares -modify nfsVOL DiskGroup datadg

437 hares -add nfsMOUNT Mount hanfs

438 hares -modify nfsMOUNT Enabled 1

439 hares -modify nfsMOUNT MountPoint /nfsshare

440 hares -modify nfsMOUNT BlockDevice /dev/vx/dsk/datadg/vol01

441 hares -modify nfsMOUNT FSType ufs

442 hares -modify nfsMOUNT FsckOpt %-n

443 hares -add nfsNFS NFS hanfs

444 hares -modify nfsNFS Enabled 1

445 hares -modify nfsNFS Nservers 24

446 hares -add nfsSHARE Share hanfs

447 hares -modify nfsSHARE Enabled 1

448 hares -modify nfsSHARE PathName /nfsshare

449 hares -modify nfsSHARE Options rw

6.Link resources relationship

450 hares -link nfsIP nfsNIC

451 hares -link nfsVOL nfsDG

452 hares -link nfsMOUNT nfsVOL

453 hares -link nfsSHARE nfsIP

454 hares -link nfsSHARE nfsMOUNT

455 hares -link nfsSHARE nfsNFS

7.Online the resource group

bash-3.00# hagrp -online hanfs -sys solarisA
bash-3.00# hastatus
attempting to connect….connected
group resource system message
————— ——————– ——————– ——————–
solarisA RUNNING
solarisB RUNNING
hafile solarisA OFFLINE
hafile solarisB OFFLINE
————————————————————————-
hanfs solarisA ONLINE

hanfs solarisB OFFLINE
fileon solarisA ONLINE
fileon solarisB ONLINE
nfsDG solarisA ONLINE
————————————————————————-
nfsDG solarisB OFFLINE
nfsIP solarisA ONLINE
nfsIP solarisB OFFLINE
nfsMOUNT solarisA ONLINE
nfsMOUNT solarisB OFFLINE
————————————————————————-
nfsNFS solarisA ONLINE
nfsNFS solarisB OFFLINE
nfsNIC solarisA ONLINE
nfsNIC solarisB ONLINE
nfsSHARE solarisA ONLINE
————————————————————————-
nfsSHARE solarisB OFFLINE
nfsVOL solarisA ONLINE
nfsVOL solarisB OFFLINE
^C

a new IP after the hanfs group is online

bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.47.131 netmask ffffff00 broadcast 192.168.47.255
ether 0:c:29:59:47:8f
e1000g0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2

inet 192.168.47.239 netmask ffffff00 broadcast 192.168.47.255

e1000g4: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 192.168.47.211 netmask ffffff00 broadcast 192.168.47.255
ether 0:c:29:59:47:b7
on the redhat vm:
mount 192.168.47.239:/nfsshare /hanfs
copy some files to /nfsshare
bash-3.00# cp -r /etc/inet/hosts .
now we see the files under /hanfs on the redhat
hagrp -switch hanfs -to solarisB
bash-3.00# hares -display nfsVOL
#Resource Attribute System Value
nfsVOL Group global hanfs
nfsVOL Type global Volume
nfsVOL AutoStart global 1

add autostartlist to the service group

bash-3.00# hagrp -display hanfs
#Group Attribute System Value
hanfs AdministratorGroups global
hanfs Administrators global
hanfs Authority global 0
hanfs AutoFailOver global 1
hanfs AutoRestart global 1
hanfs AutoStart global 1
hanfs AutoStartIfPartial global 1
hanfs AutoStartList global
bash-3.00# hagrp -modify hanfs AutoStartList -add solarisA
VCS WARNING V-16-1-11309 Configuration must be ReadWrite
bash-3.00# haconf -makerw
bash-3.00# hagrp -modify hanfs AutoStartList -add solarisA

update cluster admin password

bash-3.00# hauser -display
UserName : Privilege
——————–
admin : ClusterAdministrator
bash-3.00# hauser -list
admin
bash-3.00# haconf -makerw
bash-3.00# hauser -update admin
Enter New Password:
Enter Again:

Issue fixing

RG is partial/stopping

a resource is fault

bash-3.00# hares -clear nfsVOL
bash-3.00# hagrp -online hanfs -sys solarisB
hares –online mysqlMOUNT –sys solarisB
./bin/mysqld_safe&

nfs service group in VCS conflict with SMF

Posted on December 8, 2009by jeanwan

If there is a nfs service group in VCS, the nfs service group will not get online becase of the conflict with solaris10 SMF.
Logs:
2009/12/8 12:35:07 VCS ERROR V-16-1-7012 (tys12app1) NFS:/opt/VRTSvcs/bin/NFS/monitor:???:Service Management Facility monitoring is not disabled for NFS nfsmapid daemon!! Returning Unknown
Solution:
Disable SMF for nfsd and mountd:
# svccfg delete -f svc:/network/nfs/server:default
Disable SMF for nfsmapid:
# svccfg delete -f svc:/network/nfs/mapid:default
Then the nfs service group could online.
When you want to import the two service, follow the steps:
(1) cd /var/svc/manifest/network/nfs
(2) svccfg import mapid.xml
(3) svccfg import server.xml
(4) reboot server

bash-3.00# svccfg list | grep nfs
network/nfs/cbd
network/nfs/client
network/nfs/mapid
network/nfs/nlockmgr
network/nfs/status
network/nfs/rquota
network/nfs/server

reference

https://www4.symantec.com/Vrt/offer?a_id=89446

agents: https://sort.symantec.com/agents

LINUX and SOLARIS

Sunday, March 10, 2019

NFS_VCS