Its set up roughly like this.
I
want to add one new fresh Gluster Server to the Pool and update the two
volumes. I want the Replicated to be
Distributed Replicated volume and the Dispersed to remain the same type but add
one more brick on the new server to it. see https://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/#types-of-volumes [1] for details on volume types.
Also see this diagram for what I am trying to move to.
Create a new Machine from scratch
I am not going to go into any detail on the first creation of the server itself. The server is an ubuntu 22.04 with a secondary drive attached to it and mounted. Now I just need to install glusterfs like I did in this post http://www.whiteboardcoder.com/2023/02/installing-glusterfs-103-on-ubuntu-2204.html [2]
So let’s get it set up
OK run the following commands to trust this and get it updated
|
> sudo
add-apt-repository ppa:gluster/glusterfs-10 |
Now update
|
> sudo apt
update |
Check version
|
> apt show
glusterfs-server |
10.4 I will take that!
Now install glusterfs
|
> sudo apt
install glusterfs-server |
Start GlusterFS up
Start it up
|
> sudo systemctl
start glusterd |
Check its status
|
> sudo systemctl
status glusterd |
Now enable it so it starts after a reboot (You will notice in the above image it is currently disabled)
|
> sudo systemctl
enable glusterd |
Check the status
|
> sudo systemctl
status glusterd |
It is enabled, now reboot
|
> sudo reboot |
Check if its running after a reboot
|
> sudo systemctl
status glusterd |
And we are happy now.
Add new GlusterFS server to pool and check status
My new server is at 192.168.0.203 so run this command from on of my existing glusterfs servers
|
> sudo gluster
peer probe 192.168.0.203 |
Then check
|
> sudo gluster
peer status |
Now checking out the current volume data
|
> sudo gluster
volume info |
Expand the Dispersed Volume
Some good notes on this can be found at https://docs.gluster.org/en/v3/Administrator%20Guide/Managing%20Volumes/#expanding-volumes [3]
volume-two is my dispersed volume run this command to add an extra brick to it
|
> sudo gluster volume
add-brick volume-two
192.168.0.203:/data/brick2 |
|
volume add-brick: failed: Incorrect number of bricks supplied 1
with count 3 |
OK that failed
Why?
Here is the original command I used to create this volume
|
> sudo gluster volume
create volume-two
disperse 3 redundancy 1 192.168.0.200:/data/brick2
192.168.0.201:/data/brick2 192.168.0.202:/data/brick2 |
I guess in the end I want it to be a disperse 4 with a
redundancy 1.
So how do I do that?
Another try.
|
> sudo gluster volume
add-brick volume-two disperse 4 redundancy 1 192.168.0.203:/data/brick2 |
No go on that either
Hmm doing some research
|
Note When expanding distributed replicated and distributed dispersed
volumes, you need to add a number of bricks that is a multiple of the replica
or disperse count. For example, to expand a distributed replicated volume
with a replica count of 2, you need to add bricks in multiples of 2 (such as
4, 6, 8, etc.). |
So I could expand it but I would have to add 3 bricks to it at a time since the original had 3 bricks.
I don’t want to do that so what choices do I have?
I think the only real viable answer is to create the new volume with a new name and have it set up how I want it. Then mount it and copy the files over I want from the original volume. I may have to delete files after I copy them if I don’t have enough overall space since the bricks of both are on the same servers. Then once all that is done stop the first volume and rename it. Then stop the new volume and rename it to the original volumes name. Then go around and make sure everyone who has mounted the original volume remounts the new volume. This is all doable but would have some downtimes. I just wish I could add a brick and have it adjust itself… Oh well it can’t … in the manner I want it to…
Create New Volume
|
> sudo gluster volume
create volume-three
disperse 4 redundancy 1 192.168.0.200:/data/brick3
192.168.0.201:/data/brick3 192.168.0.202:/data/brick3 192.168.0.203:/data/brick3 |
Now let me mount this volume from my client server
|
> sudo vi /etc/fstab |
And place the following in it
|
192.168.0.200:/volume-three /volume_three_client glusterfs
defaults,_netdev 0 0 |
Make the mount point folder
|
> sudo mkdir /volume_three_client |
Now mount them
|
> sudo mount -a |
And check them out
|
> df -h |
Change the ownership of the mounts to my user
|
> sudo chown
$USER:$USER /volume_three_client/ |
Now check the volume status from one of the gluster servers
|
> sudo gluster volume
info volume-three |
Copy Files
Now I do not have much data so I can easily copy everything from volume-two to volume-three. I can just use this command
|
> sudo rsync -a
/volume_two_client/ /volume_three_client |
This may take some time if there are lots of files. Keep that in mind.
Change Volume names
|
> sudo gluster volume stop volume-two |
But we get a failure….
Poking around it looks like this CLI feature was removed at some point.
|
> sudo gluster volume
stop volume-two |
|
> sudo gluster volume info |
OK that did not really work…
After poking around for a long time, I did not find an
answer that worked.
Clean up
Now to clean up we need to remove the original volume-two.
|
> sudo gluster volume
delete volume-two |
|
> sudo gluster volume
info |
|
Volume Name: volume-one Volume Name: volume-three |
Here we can see the volume is no longer there
The folders on the glusterfs servers are still there
We need to remove the /data/brick2 from every gluster server that had it.
|
> sudo rm -r
/data/brick2 |
Expand the Replicated
I think we will be able to change the Replicate volume to a
Distributed Replicated volume like this.
Run this command to see my before.
|
> sudo gluster volume
info volume-one |
|
> sudo gluster volume
add-brick volume-one
192.168.0.202:/data/brick1
192.168.0.203:/data/brick1 |
Hey that worked
|
> sudo gluster volume
info volume-one |
Hey that looks good.
Doing a few quick checks it has expanded and it looks all good 😊
So I guess I can expand a replicate volume easy enough, if I do it in 2x with my set up.
References
[1] Replicated GlusterFS Volume
https://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/#types-of-volumes
Accessed 04/2023
[2] Installing GlusterFS 10.3 on
Ubuntu 22.04 and get it working
http://www.whiteboardcoder.com/2023/02/installing-glusterfs-103-on-ubuntu-2204.html
Accessed 04/2023
[3] Expanding Volumes
https://docs.gluster.org/en/v3/Administrator%20Guide/Managing%20Volumes/#expanding-volumes
Accessed 04/2023
No comments:
Post a Comment