WARNING: The example scripts and commands used here are destructive and will destroy your drive/s unless you know what you are doing. For proper interpretation, I am running:
- six SATA HDDs, which appear in my bare-metal Ubuntu LTS environment as
/dev/sd[a-f] (meaning sda, sdb, ..., sdf)
- there was only one partition on each of these HDDs i.e.
/dev/sd[a-f]1, and each was (and still is) the RAIDZ2 raidz=type vdev (i.e. part of the RAIDZ2 pool)
- there was (and still is) one SLOG using partition #5 on my NVMe drive
Having moved my entire RAIDZ2 cluster to a bigger drives quite a while ago, I never had the need to expand the entire volume size (yet). But since I was to embark on creating a new RAIDZ3 for some other reason, I decided to first expand the part1s across all six disks (where by RAIDZ2 was sitting astride):
Gather all “before” data:
zpool get autoexpand <POOL NAME>zpool list <POOL NAME>sfdisk -l /dev/sd[a-f]
Export the zpool:
Backup the GPT tables:
mkdir -p /root/gpt-backup
for d in /dev/sd[a-f]; do
sgdisk --backup="/root/gpt-backup/$(basename "$d").gpt" "$d";
done
Then resize part1 across all 6 disks, based on your expanded ending sector (use ‘0‘ if you want to extend to maximum allowable, leaving correct end-disk spacing for backup GPT and the like):
for d in /dev/sd[a-f]; do
sgdisk --delete=1 "$d";
sgdisk --new=1:2048:20835092399 "$d";
sgdisk --typecode=1:BF01 "$d";
sgdisk --change-name=1:<POOL NAME> "$d";
done
Don’t worry: sgdisk re-creating the partition 1 information with the same starting sector and later ending sector simply means your data is still there, but the partition is expanded.
You can now check the “new” boundaries:
And you can also check that the ZFS labels still exist:
for d in /dev/sd[a-f]1; do
echo "===== $d ====="
zdb -l "$d" | grep -E 'name:|pool_guid:|top_guid:|guid:|txg:'
done
You can then proceed to re-import the pool, either with a blanket command below, or using specific IDs:
zpool import -d /dev/disk/by-id <POOL NAME>
Then check:
zpool status -P <POOL NAME>
Ensure the pool is set to auto-expand:
zpool set autoexpand=on <POOL NAME>
And then explicitly set each member vdev to expand:
# P.S. READ TO THE END BEFORE YOU TRY THIS
for d in /dev/sd[a-f]1; do
zpool online -e <POOL NAME> "$d"
done
And this where all the trouble started…
No matter how many times I did that last two commands, or even exported and reimported, ZFS refused to expand the individual vdevs, despite the “extra space” being acknowledged (by the EXDPANDSZ parameter showing as such)…
# zpool sync
# zpool list <POOL>
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
<POOL> 32.5T 30.7T 1.85T - 25.2T 18% 94% 1.00x ONLINE -
# zpool list -v <POOL>
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
<POOL> 32.5T 30.7T 1.85T - 25.2T 18% 94% 1.00x ONLINE -
raidz2-0 32.5T 30.7T 1.85T - 25.2T 18% 94.3% - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
ata-<redacted>-part1 9.70T - - - - - - - ONLINE
logs - - - - - - - - -
nvme-<redacted>-part5 4G 536K 3.97G - - 0% 0.01% - ONLINE
Continue reading →
