Create a bucket
Log into your console and go to the amazon s3 tool and click
create bucket.
Give it a unique name and click next
For me I am going to leave the defaults here and click next
The default settings are good for me on this page as
well. I want my aws user to have
read/write to this bucket, even though I am not going to use that user to mount
the S3 bucket. I also want to make the
bucket private so “Do not grant public read access” . Click Next
Click Create Bucket!
There is the bucket!
Create a user with credentials
Now I want to create a user with credentials who has
permissions to read/write to this new bucket.
From the AWS console open up the IAM tool and click on users
Click add user.
Enter in a user name, select Programmatic Access and click
Next Permissions.
Select Attach existing policies and search for s3.
Click on AmazonS3FullAccess and click on JSON
This JSON policy
|
{
"Version":
"2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
}
]
}
|
Is close to what I want.
It gives read/write access, but to all my s3 buckets. I want to tweak this slightly to limit it to
a single bucket.
Here is a policy that will do what we want.
|
{
"Version":
"2012-10-17",
"Statement":
[
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": ["arn:aws:s3:::a-test-bucket-124568d"]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::a-test-bucket-124568d/*"]
}
]
}
|
Click on Create Policy
Click on JSON and paste the policy in.
Click Review Policy
Give it a name and click Create Policy
The policy has been created!
From the create user page select “Customer Managed” from the
pull down menu.
Then click Refresh
Select the policy you just made and click Next: Review
Click Create User.
Record your Access Key ID and show your secret access key.
In my case
Access Key: AKIAJETFXNV4NYVV64EA
Secret: Gcdpi2aETNuLsBxB1DKKU9g44qhpaAl6Eoviqreo
(don’t worry I am deleting this bucket and user after
writing this how to)
Click Close you are done with this part.
Install goofys
I am installing all this on Ubuntu 16.04 server.
First You need to install go and get it set up. I am going to install go 1.8. Run the following commands to install go 1.8
|
> sudo curl -O
https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz
> sudo tar
-xvf go1.8.linux-amd64.tar.gz
> sudo mv go
/usr/local
> sudo ln -s /usr/local/go/bin/go
/usr/bin/go
|
Now check the version
|
> go version
|
Now install goofys (build it from source
|
> export GOPATH=$HOME/work
> go get
github.com/kahing/goofys
> go install
github.com/kahing/goofys
> sudo cp work/bin/goofys
/usr/bin/
|
Now check the version
|
> goofys
--version
|
Switch to the root user and create a credentials files
|
> sudo su -
root
> mkdir ~/.aws
> vi
~/.aws/credentials
|
In this file put in the
Access Key ID: Secret Access Key ID Here is an example (using my credentials)
Access Key ID: Secret Access Key ID Here is an example (using my credentials)
|
[default]
aws_access_key_id
= AKIAJETFXNV4NYVV64EA
aws_secret_access_key
= Gcdpi2aETNuLsBxB1DKKU9g44qhpaAl6Eoviqreo
|
Mount it!
Create a mount point and mount it!
|
> sudo mkdir
-p /s3/bucket-test
> sudo goofys
a-test-bucket-124568d /s3/bucket-test
|
Got a fatal error
2017/11/27 20:49:49.984443
main.FATAL Unable to mount file system, see syslog for details
Let me switch to the root user and see if it will mount as
the root user
|
> sudo su -
root
> goofys
a-test-bucket-124568d /s3/bucket-test
|
OK that worked but only the root user can see it.
Let me tweak the command….
First unmount the drive
|
> umount
/s3/bucket-test
> goofys -o allow_other
a-test-bucket-124568d /s3/bucket-test
|
Now a regular user can see it. Let me see if they can upload to it
Let me use /dev/urandom to create a 100MiB file with random
data in it in a /tmp folder
|
> cd /tmp
> dd if=/dev/urandom
of=random.txt count=1048576 bs=100
|
Now copy it over to the s3 bucket
|
> cp
/tmp/random.txt /s3/bucket-test/
|
Permission denied!
I think I have an issue with /etc/fuse.conf file…
|
> sudo vi
/etc/fuse.conf
|
Uncomment out the user_allow_other line
Unmount the s3 drive and remount it.
|
> sudo su -
root
> umount
/s3/bucket-test
> goofys -o
allow_other a-test-bucket-124568d /s3/bucket-test
|
Now copy it over to the s3 bucket
|
> cp
/tmp/random.txt /s3/bucket-test/
|
Still denied.
Let me try another tweak to the command
|
> sudo su -
root
> umount
/s3/bucket-test
> goofys -o
allow_other --file-mode=0777
--dir-mode=0777 a-test-bucket-124568d /s3/bucket-test
|
Now copy it over to the s3 bucket (as a non-root user)
|
> cp
/tmp/random.txt /s3/bucket-test/
|
Hey that worked J
A bit overkill as now anyone on my server can write and
execute any file mounted from this s3 bucket.
Next up../ getting it to automount via /etc/fstab!
Mount using /etc/fstab
First let me unmount the s3 bucket
|
> sudo umount
/s3/bucket-test
|
Open and edit /etc/fstab
|
> sudo vi
/etc/fstab
|
And append the following line to the bottom of the file
|
goofys#a-test-bucket-124568d /s3/bucket-test fuse
_netdev,allow_other,--file-mode=0666
,--dir-mode=0777,--profile=default
0 0
|
Now mount it
|
> sudo mount
/s3/bucket-test
|
Run a quick test
|
> cp /tmp/random.txt
/s3/bucket-test/random2.txt
|
That worked!
Let me reboot and see if it automounts or not
|
> sudo reboot
now
|
Wahoo it automounted
Also I do not think I need the user_allow_other in the fuse.conf file
|
> sudo vi
/etc/fuse.conf
|
Comment out the
user_allow_other line
I am just going to reboot at this point and see if it works
with the new fuse.conf file and also if it automounts
|
> sudo reboot
now
|
Wahoo it is all working J
One last test let me copy a file over again as a non-root
user. (After I remake my random file of
course )
|
> dd if=/dev/urandom
of=/tmp/random.txt count=1048576 bs=100
> cp
/tmp/random.txt /s3/bucket-test/random3.txt
|
That worked J
So that’s how you mount an S3 bucket as a drive in Ubuntu
16.04 using goofys.
References
[1] goofys github repo
No comments:
Post a Comment