FreeBSD下ZFS mirror升降级、硬盘替换和在线/离线扩容实践

创建一个非冗余的ZFS pool
test# zpool create zfspool da1
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 110K 7.94G 0% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
da1 ONLINE 0 0 0

errors: No known data errors

增加一个盘,升级为双路mirror
test# zpool attach zfspool da1 da2
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 112K 7.94G 0% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 21:24:27 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0

errors: No known data errors

如果想直接建立一个双路mirror的ZFS pool用
test# zpool create zfspool mirror da1 da2

再增加一个盘,升级为三路mirror
test# zpool attach zfspool da1 da3
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 112K 7.94G 0% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 21:24:40 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0

errors: No known data errors

删除一个盘,降级为双路mirror
test# zpool detach zfspool da3
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 111K 7.94G 0% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 21:24:40 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0

errors: No known data errors

直接拔掉一个硬盘
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using ‘zpool online’.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver completed with 0 errors on Tue Jul 21 21:24:40 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 UNAVAIL 0 0 0 cannot open

errors: No known data errors
ZFS认为硬盘da2离线,强制上线一下
test# zpool online zfspool da2
Bringing device da2 online
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% DEGRADED –
test# zpool status
pool: zfspool
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using ‘zpool online’.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver completed with 0 errors on Tue Jul 21 21:51:48 2009
config:

NAME STATE READ WRITE CKSUM
zfspool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
da1 ONLINE 0 0 0
da2 UNAVAIL 0 0 0 cannot open

errors: No known data errors

把拔掉的盘清空再插回去
test# zpool online zfspool da2
Bringing device da2 online
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using ‘zpool replace’.
see: http://www.sun.com/msg/ZFS-8000-4J
scrub: resilver completed with 0 errors on Tue Jul 21 21:59:39 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 UNAVAIL 0 0 0 corrupted data

errors: No known data errors
应该直接使用zpool replace来替换坏掉的盘
test# zpool replace zfspool da2
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% DEGRADED –
test# zpool status
pool: zfspool
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 2.06% done, 0h9m to go
config:

NAME STATE READ WRITE CKSUM
zfspool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
da1 ONLINE 0 0 0
replacing DEGRADED 0 0 0
da2/old UNAVAIL 0 0 0 cannot open
da2 ONLINE 0 0 0

errors: No known data errors
等待数据恢复完成,pool就恢复正常了
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 22:11:41 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0

errors: No known data errors

用更大的da3、da4替换小的da1、da2
test# zpool replace zfspool da1 da3
test# zpool replace zfspool da2 da4
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 2.50% done, 0h10m to go
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
replacing ONLINE 0 0 0
da1 ONLINE 0 0 0
da3 ONLINE 0 0 0
replacing ONLINE 0 0 0
da2 ONLINE 0 0 0
da4 ONLINE 0 0 0

errors: No known data errors
因替换硬盘,所以需将数据从旧硬盘复制到新硬盘上,完成后容量没有增加
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 3.97G 3.97G 49% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 22:36:43 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0

errors: No known data errors
需要导入导出pool才能查看增加的容量
test# zpool export -f zfspool
test# zpool import zfspool
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 11.9G 3.97G 7.97G 33% ONLINE –

那么如何才能在线扩容呢?需要首先detach到非mirror,replace实现在线扩容,然后在attach到mirror
删除zfspool,重新建立做一次
test# zpool destroy zfspool
test# zpool create zfspool mirror da1 da2
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 454M 7.49G 5% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0

errors: No known data errors

降级并替换
test# zpool detach zfspool da2
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 454M 7.49G 5% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
da1 ONLINE 0 0 0

errors: No known data errors
test# zpool replace zfspool da1 da3
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 7.94G 454M 7.49G 5% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 48.54% done, 0h0m to go
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
replacing ONLINE 0 0 0
da1 ONLINE 0 0 0
da3 ONLINE 0 0 0

errors: No known data errors

替换完成后容量增大
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 11.9G 454M 11.5G 3% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 23:26:01 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
da3 ONLINE 0 0 0

errors: No known data errors

升级到mirror
test# zpool attach zfspool da3 da4
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 11.9G 454M 11.5G 3% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 45.43% done, 0h0m to go
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0

errors: No known data errors

升级完成
test# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
zfspool 11.9G 454M 11.5G 3% ONLINE –
test# zpool status
pool: zfspool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Jul 21 23:29:13 2009
config:

NAME STATE READ WRITE CKSUM
zfspool ONLINE 0 0 0
mirror ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0

errors: No known data errors

如果attache的硬盘小是不会成功的
test# zpool attach zfspool da3 da1
cannot attach da1 to da3: device is too small

发表评论

电子邮件地址不会被公开。 必填项已用*标注

*

code

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据