Saturday, April 8, 2023

GlusterFS mount if Server goes down




I have a basic GlusterFS running with 3 GlusterFS machines. 

Its set up roughly like this.



See http://www.whiteboardcoder.com/2023/03/glusterfs-etcfstab-mount-options.html [1] for details on how I set up the mount in /etc/fstab

 

So in /etc/fstab I have the following defined.

 

 

#defaults = rw, suid, dev, exec, auto, nouser, and async.

192.168.0.200:/volume-one /volume_one_client glusterfs defaults,_netdev 0 0

192.168.0.200:/volume-two /volume_two_client glusterfs defaults,_netdev 0 0

 

 

I have 3 Gluster FS servers located at

192.168.0.200
192.168.0.201
192.168.0.202


The I mount the drive via 192.168.0.200
but I could do it on any of the three servers.

OK let me first just test this out to see what happens.
I have the mount points set up and working and now I am going to shut off glusterfs on 192.168.0.200.


Check the status

 

 > sudo systemctl status glusterd

 

 

Now stop it.

 

 > sudo systemctl stop glusterd

 

And recheck status

 

 > sudo systemctl status glusterd

 

 

OK now after that is off what do I see from my client…

 

 > df -h

 

 

I still see the drives let me see if I can read/write still to them

 

 > tree /volume_one_client/ /volume_two_client/

 

 

 

OK I can still see everything but can I write a new file…

 

 > echo "THIS IS FILE STUFF" >> /volume_one_client/new_file.txt
 > echo "THIS IS FILE STUFF" >> /volume_two_client/new_file.txt

 

OK now test

 

 > tree /volume_one_client/ /volume_two_client/

 

Yep we are good.

Looks like this is the built in nature.  See https://unix.stackexchange.com/questions/213705/glusterfs-how-to-failover-smartly-if-a-mounted-server-is-failed [2]

 

You are initially connecting to one of the nodes that make up the Gluster volume, but the Gluster Native Client (which is FUSE-based) receives information about the other nodes from gluster-server-01. Since the client now knows about the other nodes it can gracefully handle a failover scenario.

 

 

Nice..

OK I am going to go the full route and shut the machine off.

I am going to reboot my client server that has the mount points

 

 > sudo reboot now

 


 

 > df -h

 

 

Nope it cannot mount it if 192.168.0.200 is down.
That makes sense… I guess I can update /etc/fstab with 201 and have it work.

So… good news is if there is a server down in the glusterFS we do not lose our connection. But if we have to restart and the mount server is down we will not be able to reconnect to it without some changes to the /etc/fstab file.

As a test let me update /etc/fstab while 192.168.0.200 is still down.

 

 

 > sudo vi /etc/fstab

 

 

 

 > sudo mount -a
 > tree /volume_one_client/ /volume_two_client/

 

 

 And its back just fine.

 

 

 

References 

[1]       GlusterFS /etc/fstab mount options GlusterFS/NFS testing in Ubuntu 22.04
            http://www.whiteboardcoder.com/2023/03/glusterfs-etcfstab-mount-options.html
            Accessed 03/2023
[2]       GlusterFS how to failover (smartly) if a mounted Server is failed
              
https://unix.stackexchange.com/questions/213705/glusterfs-how-to-failover-smartly-if-a-mounted-server-is-failed
            Accessed 03/2023

 

 

 


No comments:

Post a Comment