Persistent naming with UDEV and EMC's Powerpath

[Roberto Fratelli]

Hello everyone !

I'm trying to set up a quorum disk in a 2-node redhat cluster (RHEL 4.7). Both nodes are attached to a SAN Clariion over qlogic HBA's and i'm using EMC's powerpath.

The problem i'm facing is that the disks's device names are different across the nodes. For example, the lun i elected to be my quorum disk is called '/dev/emcpowerf' at node-a and '/dev/emcpowerd' at node-b, so i can't use the device name (/dev/emcpowerXX) in my cluster.conf file. In a attempt to overcome this i've tried to use the disk label instead :

<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677" interval="2" tko="10" votes="1"/>

But did not work also, because this way it finds '/dev/sdXX' before '/dev/emcpowerXX':

***@node-a:~ # cman_tool nodes
Node Votes Exp Sts Name
0 1 0 M /dev/sdf1
1 1 2 M node-a
2 1 2 M node-b
***@node-a:~ #


Since this '/dev/sdf' is just one of the paths i have under powerpath (it has 4 paths per HBA, 8 paths per lun total) i can't keep it this way because in a event of lun trespass (clariion does it very often) i would loose quorum. My understanding is that i need some kind of "persistent binding" so i can use Powerpath's pseudo-device '/dev/emcpowerXX', so i tried to create a UDEV rule like this:

[***@node-a RSM]# cat 75-quorum.rules
KERNEL=="emcpower[a-z]1, SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677", SYMLINK+="discodequorum%n"
[***@node-a RSM]#


Which did not worked, i think because UDEV starts way before powerpath. Does anyone here faced this problem before ? how can i keep the device names persistent across the nodes (at least for the quorum disk) ?


Below some more info on my quorum's disk:


***@node-a:~ # mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

***@node-a:~ #



[***@node-b ~]# mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerd1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

[***@node-b ~]#

[Stevan Colaco]

Hello,

There are different ways one may write the persistent naming udev rules.

Instead of SYSFS{serial}, it would be better if you can use scsi_id

first try to get the scsi id of the quorum device

#/sbin/scsi_id -g -s /block/emcpowerf

you should receive a lengthy number.

Then you can use the rule something like below

[***@node-a RSM]# cat 75-quorum.rules
KERNEL=="emcpower[a-z]1, PROGRAM=="/sbin/scsi_id -g -s /block/emcpowerf", RESULT=="Paste the output of scsi_id here" SYMLINK+="discodequorum"


regards,
-Steve

[***@node-a RSM]#

----- Original Message -----
From: "Roberto Fratelli" <***@yahoo.com>
To: linux-***@redhat.com
Sent: Monday, April 27, 2009 4:17:47 PM GMT +03:00 Iraq
Subject: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath


Hello everyone !

I'm trying to set up a quorum disk in a 2-node redhat cluster (RHEL 4.7). Both nodes are attached to a SAN Clariion over qlogic HBA's and i'm using EMC's powerpath.

The problem i'm facing is that the disks's device names are different across the nodes. For example, the lun i elected to be my quorum disk is called '/dev/emcpowerf' at node-a and '/dev/emcpowerd' at node-b, so i can't use the device name (/dev/emcpowerXX) in my cluster.conf file. In a attempt to overcome this i've tried to use the disk label instead :

<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677" interval="2" tko="10" votes="1"/>

But did not work also, because this way it finds '/dev/sdXX' before '/dev/emcpowerXX':

***@node-a:~ # cman_tool nodes
Node Votes Exp Sts Name
0 1 0 M /dev/sdf1
1 1 2 M node-a
2 1 2 M node-b
***@node-a:~ #


Since this '/dev/sdf' is just one of the paths i have under powerpath (it has 4 paths per HBA, 8 paths per lun total) i can't keep it this way because in a event of lun trespass (clariion does it very often) i would loose quorum. My understanding is that i need some kind of "persistent binding" so i can use Powerpath's pseudo-device '/dev/emcpowerXX', so i tried to create a UDEV rule like this:

[***@node-a RSM]# cat 75-quorum.rules
KERNEL=="emcpower[a-z]1, SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677", SYMLINK+="discodequorum%n"
[***@node-a RSM]#


Which did not worked, i think because UDEV starts way before powerpath. Does anyone here faced this problem before ? how can i keep the device names persistent across the nodes (at least for the quorum disk) ?


Below some more info on my quorum's disk:


***@node-a:~ # mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

***@node-a:~ #



[***@node-b ~]# mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerd1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

[***@node-b ~]#

[Robert Hurst]

This post might be inappropriate. Click to display it.

[b***@emc.com]

Hi Roberto.

You can use the emcpadm() command to rename the device on each
node so they are all the same. The syntax is "emcpadm rename[pseudo] -s
src pseudodevice -t tgt pseudodevice". For example, if on node A the
device is emcpowerg and you wish it to be be emcpowerq you would issue:

# emcpadm rename -s emcpowerg -t emcpowerq

For each node run the "powermt display dev=all" and review the
Logical device ID that is displayed for each LUN on each node. Find your
quorom disk on each - for example, Logical device
ID=6006016018C80800B5069570B8A9D911

# powermt display dev=emcpowerg

Pseudo name=emcpowerg
CLARiiON ID=WRE00100100415
Logical device ID=6006016018C80800B5069570B8A9D911
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A
======================================================================
------------ Host ----------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
======================================================================
3 lpfc sdaa SP A0 active alive 0 0
2 lpfc sdj SP A1 active alive 0 0

Then rename it....

# emcpadm rename -s emcpowerg -t emcpowerq

...and your in business.

Regards,
Wayne.

-----Original Message-----
From: linux-cluster-***@redhat.com
[mailto:linux-cluster-***@redhat.com] On Behalf Of Roberto Fratelli
Sent: Monday, April 27, 2009 9:18 AM
To: linux-***@redhat.com
Subject: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath


Hello everyone !

I'm trying to set up a quorum disk in a 2-node redhat cluster (RHEL
4.7). Both nodes are attached to a SAN Clariion over qlogic HBA's and
i'm using EMC's powerpath.

The problem i'm facing is that the disks's device names are different
across the nodes. For example, the lun i elected to be my quorum disk is
called '/dev/emcpowerf' at node-a and '/dev/emcpowerd' at node-b, so i
can't use the device name (/dev/emcpowerXX) in my cluster.conf file. In
a attempt to overcome this i've tried to use the disk label instead :

<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677" interval="2"
tko="10" votes="1"/>

But did not work also, because this way it finds '/dev/sdXX' before
'/dev/emcpowerXX':

***@node-a:~ # cman_tool nodes
Node Votes Exp Sts Name
0 1 0 M /dev/sdf1
1 1 2 M node-a
2 1 2 M node-b
***@node-a:~ #


Since this '/dev/sdf' is just one of the paths i have under powerpath
(it has 4 paths per HBA, 8 paths per lun total) i can't keep it this way
because in a event of lun trespass (clariion does it very often) i would
loose quorum. My understanding is that i need some kind of "persistent
binding" so i can use Powerpath's pseudo-device '/dev/emcpowerXX', so i
tried to create a UDEV rule like this:

[***@node-a RSM]# cat 75-quorum.rules
KERNEL=="emcpower[a-z]1,
SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677",
SYMLINK+="discodequorum%n"
[***@node-a RSM]#


Which did not worked, i think because UDEV starts way before powerpath.
Does anyone here faced this problem before ? how can i keep the device
names persistent across the nodes (at least for the quorum disk) ?


Below some more info on my quorum's disk:


***@node-a:~ # mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

***@node-a:~ #



[***@node-b ~]# mkqdisk -L
mkqdisk v0.5.2
/dev/sdf1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdo1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdx1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdag1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdap1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sday1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbh1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/sdbq1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

/dev/emcpowerd1:
Magic: eb7a62c2
Label: 97196ef0-da2f-4b79-a5c5-999dce70d677
Created: Mon Dec 24 11:47:11 2007
Host: node-a
Kernel Sector Size: 512

[***@node-b ~]#

[Roberto Fratelli]

Excellent ! thanks a million ! this emcpadm command was unknown to me. I just tested it and works great !

Just another question: Is it necessary to issue 'powermt save' after renaming the pseudo-devices ?

Regards,

Roberto

Subject: RE: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath
Date: Monday, April 27, 2009, 9:43 AM
Hi Roberto.
     You can use the emcpadm()
command to rename the device on each
node so they are all the same. The syntax is "emcpadm
rename[pseudo] -s
src pseudodevice -t tgt pseudodevice". For example, if on
node A the
device is emcpowerg and you wish it to be be emcpowerq you
# emcpadm rename -s emcpowerg -t emcpowerq
    For each node run the "powermt display
dev=all" and review the
Logical device ID that is displayed for each LUN on each
node. Find your
quorom disk on each - for example, Logical device
ID=6006016018C80800B5069570B8A9D911
# powermt display dev=emcpowerg
Pseudo name=emcpowerg
CLARiiON ID=WRE00100100415
Logical device ID=6006016018C80800B5069570B8A9D911
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A
======================================================================
------------ Host ----------- - Stor - -- I/O Path - --
Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
======================================================================
3 lpfc sdaa SP A0 active alive 0 0
2 lpfc sdj SP A1 active alive 0 0
Then rename it....
# emcpadm rename -s emcpowerg -t emcpowerq
...and your in business.
Regards,
Wayne.
-----Original Message-----
On Behalf Of Roberto Fratelli
Sent: Monday, April 27, 2009 9:18 AM
Subject: [Linux-cluster] Persistent naming with UDEV and
EMC's Powerpath
Hello everyone !
I'm trying to set up a quorum disk in a 2-node redhat
cluster (RHEL
4.7). Both nodes are attached to a SAN Clariion over qlogic
HBA's and
i'm using EMC's powerpath.
The problem i'm facing is that the disks's device names are
different
across the nodes. For example, the lun i elected to be my
quorum disk is
called '/dev/emcpowerf' at node-a and '/dev/emcpowerd' at
node-b, so i
can't use the device name (/dev/emcpowerXX) in my
cluster.conf file. In
a attempt to overcome this i've tried to use the disk label
<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677"
interval="2"
tko="10" votes="1"/>
But did not work also, because this way it finds
'/dev/sdXX' before
Node  Votes Exp Sts  Name
   0    1   
0   M   /dev/sdf1
   1    1   
2   M   node-a
   2    1   
2   M   node-b
Since this '/dev/sdf' is just one of the paths i have under
powerpath
(it has 4 paths per HBA, 8 paths per lun total) i can't
keep it this way
because in a event of lun trespass (clariion does it very
often) i would
loose quorum. My understanding is that i need some kind of
"persistent
binding" so i can use Powerpath's pseudo-device
'/dev/emcpowerXX', so i
KERNEL=="emcpower[a-z]1,
SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677",
SYMLINK+="discodequorum%n"
Which did not worked, i think because UDEV starts way
before powerpath.
Does anyone here faced this problem before ? how can i keep
the device
names persistent across the nodes (at least for the quorum
disk) ?
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
     
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster

[b***@emc.com]

Hi Roberto.

It won't hurt to do it and would be advisable.

Regards,
Wayne.

-----Original Message-----
From: linux-cluster-***@redhat.com [mailto:linux-cluster-***@redhat.com] On Behalf Of Roberto Fratelli
Sent: Monday, April 27, 2009 1:00 PM
To: linux clustering
Subject: RE: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath


Excellent ! thanks a million ! this emcpadm command was unknown to me. I just tested it and works great !

Just another question: Is it necessary to issue 'powermt save' after renaming the pseudo-devices ?

Regards,

Roberto

Subject: RE: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath
Date: Monday, April 27, 2009, 9:43 AM
Hi Roberto.
     You can use the emcpadm()
command to rename the device on each
node so they are all the same. The syntax is "emcpadm
rename[pseudo] -s
src pseudodevice -t tgt pseudodevice". For example, if on
node A the
device is emcpowerg and you wish it to be be emcpowerq you
# emcpadm rename -s emcpowerg -t emcpowerq
    For each node run the "powermt display
dev=all" and review the
Logical device ID that is displayed for each LUN on each
node. Find your
quorom disk on each - for example, Logical device
ID=6006016018C80800B5069570B8A9D911
# powermt display dev=emcpowerg
Pseudo name=emcpowerg
CLARiiON ID=WRE00100100415
Logical device ID=6006016018C80800B5069570B8A9D911
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A
======================================================================
------------ Host ----------- - Stor - -- I/O Path - --
Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
======================================================================
3 lpfc sdaa SP A0 active alive 0 0
2 lpfc sdj SP A1 active alive 0 0
Then rename it....
# emcpadm rename -s emcpowerg -t emcpowerq
...and your in business.
Regards,
Wayne.
-----Original Message-----
On Behalf Of Roberto Fratelli
Sent: Monday, April 27, 2009 9:18 AM
Subject: [Linux-cluster] Persistent naming with UDEV and
EMC's Powerpath
Hello everyone !
I'm trying to set up a quorum disk in a 2-node redhat
cluster (RHEL
4.7). Both nodes are attached to a SAN Clariion over qlogic
HBA's and
i'm using EMC's powerpath.
The problem i'm facing is that the disks's device names are
different
across the nodes. For example, the lun i elected to be my
quorum disk is
called '/dev/emcpowerf' at node-a and '/dev/emcpowerd' at
node-b, so i
can't use the device name (/dev/emcpowerXX) in my
cluster.conf file. In
a attempt to overcome this i've tried to use the disk label
<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677"
interval="2"
tko="10" votes="1"/>
But did not work also, because this way it finds
'/dev/sdXX' before
Node  Votes Exp Sts  Name
   0    1   
0   M   /dev/sdf1
   1    1   
2   M   node-a
   2    1   
2   M   node-b
Since this '/dev/sdf' is just one of the paths i have under
powerpath
(it has 4 paths per HBA, 8 paths per lun total) i can't
keep it this way
because in a event of lun trespass (clariion does it very
often) i would
loose quorum. My understanding is that i need some kind of
"persistent
binding" so i can use Powerpath's pseudo-device
'/dev/emcpowerXX', so i
KERNEL=="emcpower[a-z]1,
SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677",
SYMLINK+="discodequorum%n"
Which did not worked, i think because UDEV starts way
before powerpath.
Does anyone here faced this problem before ? how can i keep
the device
names persistent across the nodes (at least for the quorum
disk) ?
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
     
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster

[Roberto Fratelli]

Hi Stevan,

I saw this in some examples, but somehow it just don't work for me :

# /sbin/scsi_id -g -s /block/emcpowerf
Cannot find sysfs device associated with /sys/block/emcpowerf
#



Weird.. A guy from EMC just sent me something that will fit just perfect. I'll share with the entire list.

Regards,

Subject: Re: [Linux-cluster] Persistent naming with UDEV and EMC's Powerpath
Date: Monday, April 27, 2009, 8:52 AM
Hello,
There are different ways one may write the persistent
naming udev rules.
Instead of SYSFS{serial}, it would be better if you can use
scsi_id
first try to get the scsi id of the quorum device
#/sbin/scsi_id -g -s /block/emcpowerf
you should receive a lengthy number.
Then you can use the rule something like below
KERNEL=="emcpower[a-z]1, PROGRAM=="/sbin/scsi_id -g -s
/block/emcpowerf", RESULT=="Paste the output of scsi_id
here" SYMLINK+="discodequorum"
regards,
-Steve
----- Original Message -----
Sent: Monday, April 27, 2009 4:17:47 PM GMT +03:00 Iraq
Subject: [Linux-cluster] Persistent naming with UDEV and
EMC's Powerpath
Hello everyone !
I'm trying to set up a quorum disk in a 2-node redhat
cluster (RHEL 4.7). Both nodes are attached to a SAN
Clariion over qlogic HBA's and i'm using EMC's powerpath.
The problem i'm facing is that the disks's device names are
different across the nodes. For example, the lun i elected
to be my quorum disk is called '/dev/emcpowerf' at node-a
and '/dev/emcpowerd' at node-b, so i can't use the device
name (/dev/emcpowerXX) in my cluster.conf file. In a attempt
<quorumd label="97196ef0-da2f-4b79-a5c5-999dce70d677"
interval="2" tko="10" votes="1"/>
But did not work also, because this way it finds
Node  Votes Exp Sts  Name
   0    1   
0   M   /dev/sdf1
   1    1   
2   M   node-a
   2    1   
2   M   node-b
Since this '/dev/sdf' is just one of the paths i have under
powerpath (it has 4 paths per HBA, 8 paths per lun total) i
can't keep it this way because in a event of lun trespass
(clariion does it very often) i would loose quorum. My
understanding is that i need some kind of "persistent
binding" so i can use Powerpath's pseudo-device
'/dev/emcpowerXX', so i tried to create a UDEV rule like
KERNEL=="emcpower[a-z]1,
SYSFS{label}=="97196ef0-da2f-4b79-a5c5-999dce70d677",
SYMLINK+="discodequorum%n"
Which did not worked, i think because UDEV starts way
before powerpath. Does anyone here faced this problem before
? how can i keep the device names persistent across the
nodes (at least for the quorum disk) ?
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
mkqdisk v0.5.2
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
        Magic:     
          eb7a62c2
        Label:     
         
97196ef0-da2f-4b79-a5c5-999dce70d677
        Created:     
        Mon Dec 24 11:47:11 2007
        Host:     
           node-a
        Kernel Sector
Size:   512
     
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster
--
Linux-cluster mailing list
https://www.redhat.com/mailman/listinfo/linux-cluster