Tuesday, April 13, 2010

Adding multipath disks without rebooting

this is done on a CEntOS 5.3 x86_64 Box with HP EVA 8000 LUNs.
HBA is Emulex.

after disks have been presented on the OS, they can be "discovered"
by scanning the hba ports.

doing a scsi bus rescan or reset causes connectivity problems
and should be avoided if you can't stop I/O activities on  the
server.

i'm doing this on a CEntOs 5.3 machine with several existing
HP EVA 8000 LUNs with multipath as shown below:
mpath2 (3600508b40005459d0000c00001120000) dm-10 HP,HSV210
[size=200G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=110][active]                      
 \_ 1:0:0:2 sdc 8:32  [active][ready]                    
 \_ 2:0:0:2 sdh 8:112 [active][ready]                    
 \_ 2:0:1:2 sdm 8:192 [active][ready]                    
mpath1 (3600508b40005459d0000c000010f0000) dm-9 HP,HSV210
[size=200G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=110][active]                      
 \_ 1:0:0:1 sdb 8:16  [active][ready]                    
 \_ 2:0:0:1 sdg 8:96  [active][ready]                    
 \_ 2:0:1:1 sdl 8:176 [active][ready]                    
mpath5 (3600508b40005459d0000e00000450000) dm-13 HP,HSV210
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw] 
\_ round-robin 0 [prio=70][active]                        
 \_ 1:0:0:5 sdf 8:80  [active][ready]                     
 \_ 2:0:0:5 sdk 8:160 [active][ready]                     
 \_ 2:0:1:5 sdp 8:240 [active][ready]                     
mpath4 (3600508b40005459d0000c000011a0000) dm-12 HP,HSV210
[size=200G][features=1 queue_if_no_path][hwhandler=0][rw] 
\_ round-robin 0 [prio=110][active]                       
 \_ 1:0:0:4 sde 8:64  [active][ready]                     
 \_ 2:0:0:4 sdj 8:144 [active][ready]                     
 \_ 2:0:1:4 sdo 8:224 [active][ready]                     
mpath3 (3600508b40005459d0000c00001150000) dm-11 HP,HSV210
[size=200G][features=1 queue_if_no_path][hwhandler=0][rw] 
\_ round-robin 0 [prio=110][active]                       
 \_ 1:0:0:3 sdd 8:48  [active][ready]                     
 \_ 2:0:0:3 sdi 8:128 [active][ready]                     
 \_ 2:0:1:3 sdn 8:208 [active][ready] 


what's needed:
1. current LUN IDs of the server (via multipath -ll  or lsscsi)
2. WWN of the storage server
3. the number and channel of HBA

current LUN IDs are shown in the multipath -ll output above.
or, via lsscsi, its in the leftmost column, the 4th number:


[2:0:1:1]    disk    HP       HSV210           5000  /dev/sdl
[2:0:1:2]    disk    HP       HSV210           5000  /dev/sdm
[2:0:1:3]    disk    HP       HSV210           5000  /dev/sdn
[2:0:1:4]    disk    HP       HSV210           5000  /dev/sdo
[2:0:1:5]    disk    HP       HSV210           5000  /dev/sdp

the current LUN ids are 1,2,3,4 and 5.

0x50001fe1500a7340  is the WWN of the storage server
(from /sys/class/fc_transport/target1\:0\:0/node_name)

to list all existing hba number/channel, run:

grep 0x50001fe1500a7340 /sys/class/fc_transport/*/node_name

/sys/class/fc_transport/target1:0:0/node_name:0x50001fe1500a7340  -> 1st path
/sys/class/fc_transport/target2:0:0/node_name:0x50001fe1500a7340  -> 2nd path
/sys/class/fc_transport/target2:0:1/node_name:0x50001fe1500a7340  -> 3rd path

now let the OS know of the new storage device(s):

echo "c t l" > /sys/class/scsi_host/hosth/scan

h is the HBA number
c is the channel on the HBA
t is the SCSI target ID,   
l is the LUN.              

in my case, i need to use the following commands:
 
echo "0 0 06" > /sys/class/scsi_host/host1/scan
echo "0 0 06" > /sys/class/scsi_host/host2/scan
echo "0 1 06" > /sys/class/scsi_host/host2/scan

from dmesg, the following is for the first echo command:


and for the next two echo commands:


now we have 3 new disk: sdq, sdr and sds (these are not persistent
and may change during reboots).

to verify, run scs_id on each new disks to see
if they have the same UID:                   


now i run fdisk on /dev/sdq to create one partition wity type 8e
(Linux LVM):


and then to re-enable multipath for the new drive, run:

[root@LXI*****60 ~]#multipath

mpath6 is then created:

[root@LXI*****60 ~]# multipath -ll
mpath6 (3600508b40005459d0000e00001370000) dm-22 HP,HSV210
[size=750G][features=1 queue_if_no_path][hwhandler=0][rw] 
\_ round-robin 0 [prio=70][active]                        
 \_ 1:0:0:6 sdq 65:0  [active][ready]                     
 \_ 2:0:0:6 sdr 65:16 [active][ready]                     
 \_ 2:0:1:6 sds 65:32 [active][ready] 


Adding to existing Volume Group

run kpartx on the new mpath to list available mapping

[root@LXI*****60 ~]#kpartx -l /dev/mapper/mpath6
mpath6p1 : 0 1572859827 /dev/mapper/mpath6 63

now create the partition mappings:

[root@LXI*****60 ~]# kpartx -a /dev/mapper/mpath6

[root@LXI*****60 ~]# ls /dev/mapper/mpath6*
 
/dev/mapper/mpath6  /dev/mapper/mpath6p1   

[root@LXISAP0060 ~]# ls -ltr /dev/mapper/mpath6*
brw-rw---- 1 root disk 253, 22 Apr  7 12:00 /dev/mapper/mpath6
brw-rw---- 1 root disk 253, 23 Apr  7 13:43 /dev/mapper/mpath6p1

initiliaze the device for LVM use:

[root@LXI*****60 ~]# pvcreate /dev/mapper/mpath6p1

returns:
Physical volume "/dev/mapper/mpath6p1" successfully created

listing via pvs:


/dev/dm-23 is the one to be used:

[root@LXI*****60 ~]# vgextend lx0060apps00vg /dev/dm-23
  Volume group "lx0060apps00vg" successfully extended


Resizing logical volume and the FS

[root@LXI*****60 ~]# lvresize -i1 -L +200G /dev/mapper/lx0060apps00vg-lx0060data00LV
  Extending logical volume lx0060data00LV to 930.00 GB
  Logical volume lx0060data00LV successfully resized

and then for the fs:

[root@LXI*****60 ~]# resize2fs /dev/mapper/lx0060apps00vg-lx0060data00LV


till next time. :)

0 comments:

Post a Comment