[ILUG] Re: Small LVM issue - the solution

[Conor Wynne]

Conor Wynne wrote:
> Hi all,
>
> I've an LVM missing UUID issue.
>
> Situation:
> Added a new LUN and pvcreate /dev/sdd on it. Then added it to a VG and
> extended an lv.
> Then realised I needed to use half for one volgroup and the other half
> for another, so did:
>
> lvreduce -l -254 vgname etc
> Which is fine as I had only added 254 extents to the VG in the first
> place, and had not extended the filesystem since reiserfs doesn't
> support it online.
>
> I then did a pvremove -ff /dev/sdd and went about adding /dev/sdd1 and
> /dev/sdd2 to the new lvm set.
>
> Now I have this :
> /proc/partitions :
>    8    49   97667136 sdd1
>    8    50   45062325 sdd2
>
>   /dev/sdd1  [       93.14 GB] LVM physical volume
>   /dev/sdd2  [       42.97 GB] LVM physical volume
> (which are just empty partitions from the newly created LUN, no data on
> them yet, so no worries)
>
>   /dev/sdb   [      136.12 GB] LVM physical volume
> sdb has the database, which we have not touched yet, but need to extend
> by 20GB, not an issue, but I need to fix the missing UUID first.
>
> server:/etc/lvm/backup # vgscan
>   Reading all physical volumes.  This may take a while...
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>
> This is normal as /dev/sdd is no longer part of any LVM group. As sdd1
> and sdd2 are.
>
>   Couldn't find all physical volumes for volume group database.
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>   Couldn't find all physical volumes for volume group database.
>   Volume group "database" not found
>   Found volume group "system" using metadata type lvm2
> server:/etc/lvm/backup #
> server:/etc/lvm/backup #
> server:/etc/lvm/backup #
> server:/etc/lvm/backup #
> server:/etc/lvm/backup #
> server:/etc/lvm/backup # pvscan
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>   PV /dev/sdb         VG database   lvm2 [136.12 GB / 21.12 GB free]
>   PV unknown device   VG database   lvm2 [136.12 GB / 136.12 GB free]
>
> It says unknown device because I commented out the UUID for missing
> device from the lvm backup configs.
>
>   PV /dev/sda6        VG system     lvm2 [125.80 GB / 0    free]
>   PV /dev/sda2        VG system     lvm2 [8.00 GB / 4.80 GB free]
>   PV /dev/sdd1                      lvm2 [93.14 GB]
>   PV /dev/sdd2                      lvm2 [42.97 GB]
>   Total: 6 [542.15 GB] / in use: 4 [406.04 GB] / in no VG: 2 [136.12 GB]
> server:/etc/lvm/backup # lvscan
>
>   /dev/sdc: open failed: No medium found
> (I know, I need to black list this, but it's not an issue)
>
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>   Couldn't find all physical volumes for volume group database.
>   Couldn't find device with uuid 'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'.
>   Couldn't find all physical volumes for volume group database.
>   Volume group "database" not found
>
> Which is not actually true, it just says this because not all members
> are there.
> As you can see:
>
>   ACTIVE            '/dev/system/opt_lv' [3.00 GB] inherit
>   ACTIVE            '/dev/system/root_lv' [4.00 GB] inherit
>   ACTIVE            '/dev/system/srv_lv' [3.00 GB] inherit
>   ACTIVE            '/dev/system/tmp_lv' [3.00 GB] inherit
>   ACTIVE            '/dev/system/usr_lv' [7.00 GB] inherit
>   ACTIVE            '/dev/system/var_lv' [3.00 GB] inherit
>   ACTIVE            '/dev/system/home_lv' [8.00 GB] inherit
>   ACTIVE            '/dev/system/backup_lv' [80.00 GB] inherit
>   ACTIVE            '/dev/system/informix_lv' [4.00 GB] inherit
>   ACTIVE            '/dev/system/parteor_lv' [10.00 GB] inherit
>   ACTIVE            '/dev/system/public_lv' [4.00 GB] inherit
>
> So, how can I remove theis issue without harming the dataset?
> Its not that big an issue, but I'd need to be careful all the same.
>
> Regards
> Conor.
>
>   

Well lads,

Since noone was able to help me, I had to use my own brain to sort it
out. This is how I proceeded:

1. Hexdump'd both LUNs to get UUID and other info.

parteor1:/etc/lvm/backup # hexdump -C /dev/sdb |less
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
|................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00 
|LABELONE........|
00000210  32 c7 66 c7 20 00 00 00  4c 56 4d 32 20 30 30 31  |2.f.
...LVM2 001|
00000220  63 36 44 71 75 78 56 6f  53 4d 72 6d 39 4f 4a 31 
|c6DquxVoSMrm9OJ1|


2. dd backup of each volume - just in case, although I was absolutely
sure it wouldn't be necessary

3. Removed the /dev/sdd1 and /dev/sdd2 as PV devices (they were not yet
placed in a VG remember)

4.fdisk /dev/sdd
(removed the partitions)

5. pvcreate'd it again with the UUID it was complaining about!

parteor1:/etc/lvm/backup # pvcreate /dev/sdd --uuid

'BZwaqZ-zKZM-rXNo-MVuk-JfTM-MUHj-4bkTuK'
  Physical volume "/dev/sdd" successfully created

And there you have it. Back to square one and now can happly vgextend & lvextend to my hearts content. 
I hope someone else finds this useful, all ops were made on a live database server, albeit it well backed up just in case. 

Conor.