How to mount s3 using goofys

Total views:74,593 / Applauds for blog:1
Welcome to my page. I am an adminisrator of this site.
If you are this db's user, please contact with me by private mail. If not, please contact with me by email or twitter or facebook
Access record[Graph / PV Info.(Past 1 day / Past 1 week) / Access from outside (Yesterday / Past 1 week) / Vistors's list]
ProfilePmail(Mail)
Inbox   /Send   /Sent
Reviews(List   /Limit)
Poll   /Agree:Got   /Sent
Fan
Works/Music
Blog
[Write]
Links
My Play List
<=Newer article Introduction of AWS in 2017 and how to use it
=>Older article SERPOSCOPE

1.
2017/02/25 (Updated 2018/06/24) "AWS > How to mount s3 using goofys"
[Show only this article / Modify / Delete / Send trackback / Add to the shared category]

1. What is goofys?
2. Preparation on AWS web console
3. Installation of necessary files
4. Mount s3 bucket
5. Execute automatically when instance restarts
6. How to change s3's permission later

1. What is goofys?

https://github.com/kahing/goofys
You can access files on s3 through directory path.
2. Preparation on AWS web console

You have to prepare S3 Bucket beforehand and have to give S3 full privilege to some role and associate the role with ec2 instance.
Following procedure is in the case of AmazonLinux.
3. Installation of necessary files

yum update -y;
yum install golang fuse -y;
export GOPATH=/usr/local/go/bin;
go get github.com/kahing/goofys
go install github.com/kahing/goofys

If something doesn't go well, please download the latest version from official site and install it following the instruction.
wget https://redirector.gvt1.com/edgedl/go/go1.8.5.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.8.5.linux-386.tar.gz

4. Mount s3 bucket

get apache user's group id and user id
grep apache /etc/passwd
As example, apache user's group id and user id are 48.
Without doing this, only root can write file, so apache cannot create or delete files.

gid=48; #Example
uid=48; #Example
DIRMODE=0775;
FILEMODE=0666;
S3_BUCKET_NAME=firstclass-sakuhindb-uploadedimg; #Example
S3_MOUNT_NAME=/s3/${S3_BUCKET_NAME}; #Example
sudo mkdir -p ${S3_MOUNT_NAME};
sudo umount ${S3_MOUNT_NAME} ;sudo /usr/local/bin/goofys ${S3_BUCKET_NAME} ${S3_MOUNT_NAME} -o allow_other,--uid=$uid,--gid=$gid,--dir-mode=$DIRMODE,--file-mode=$FILEMODE,--use-content-type,--acl=public-read

gid=48; #Example
uid=48; #Example
DIRMODE=0775;
FILEMODE=0666;
S3_BUCKET_NAME=minakoe-index; #Example considering that making files public through https://stat.image.sakuhindb.com/ <= You have to combine with CloudFront to make it accessible through https
S3_MOUNT_NAME=/s3/${S3_BUCKET_NAME}; #Example
mkdir -p ${S3_MOUNT_NAME};
umount ${S3_MOUNT_NAME} ;sudo /usr/local/go/bin/bin/goofys ${S3_BUCKET_NAME} ${S3_MOUNT_NAME},--uid=$uid,--gid=$gid,--dir-mode=$DIRMODE,--file-mode=$FILEMODE

--user-content-type=Set user content type based on file's extension
--acl=public-read=This is necessary when you want to make this content accessible through web to users

verify
df -kh

Result example
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.9G 64K 7.9G 1% /dev
tmpfs 7.9G 0 7.9G 0% /dev/shm
/dev/xvda1 99G 59G 41G 60% /
fs-4e65c607.efs.us-east-1.amazonaws.com:/ 8.0E 46G 8.0E 1% /efs
firstclass-weblogs 1.0P 0 1.0P 0% /s3/firstclass-weblogs

5. Execute automatically when instance restarts

sudo vi /etc/rc.local;

gid=48;
uid=48;
S3_BUCKET_NAME=firstclass-weblogs; #Example
S3_MOUNT_NAME=/s3/${S3_BUCKET_NAME};
/usr/local/bin/goofys ${S3_BUCKET_NAME} ${S3_MOUNT_NAME} -o allow_other,--uid=$uid,--gid=$gid,--dir-mode=$DIRMODE,--file-mode=$FILEMODE,--use-content-type,--acl=public-read

Restart the server and verify that s3 will be mounted automatically when the instance restarts
6. How to change s3's permission later

Install s3cmd
sudo yum install s3cmd --enablerepo=epel

Ex. Make files under s3://firstclass-sakuhindb-uploadedimg bucket public to users through web
s3cmd setacl -r --acl-public s3://firstclass-sakuhindb-uploadedimg


Add comment to this article


[Read other articles]
<=Newer article Introduction of AWS in 2017 and how to use it
=>Older article SERPOSCOPE


Articles categorized as "AWS by this user"
All articles of this user
Subscribe to RSS
RSS
Term
Category
All
1.Japan
2.Atlassian's products
3.Self
4.Development of this site
5.Japanese comics
6.Japanese anime
7.Weekly hot news of Japanese culture
8.OP/ED/PV
9.Japanese game
10.Ranking
11.Japanese Comics (Manga)
12.Search Engine
13.Japanese drama
14.Japanese otaku culture
15.Programming
16.Ineternet world
17.Movie
18.C/C++
19.BerkeleyDB
20.Apache programming
21.Spam
22.Meteor
23.Marketing
24.Python
25.Scrum
26.JIRA
27.Git
28.CI
29.Jenkins
30.AWS
31.Operation
32.Singapore
33.Cloud
34.Mysql Cluster
35.DevOps
36.Bitbucket
37.Xamarin
38.Good and new
39.Monitoring
40.JavaScript(node.js)
41.React
42.Phillipines
43.Hiring
44.Python
45.SEO
46.Malaysia
47.Mongodb
48.Perl
49.Docker
50.Life hack
51.Dance
52.QA
53.Mysql
54.Digital Life Hack
55.Project management
56.Django
57.Gatsby
58.Administrator
Sayings from S-Cry-Ed

Rule in this world is speed. Even stupid person can write cool novel if he can spend 20 years for it.

If someone helped me, I will help him in return, which is my rule.

To become stronger, consider what is cowardliest thinking. And rebel against the thinking, which will make you stronger.



I am Japanese but working for some English sites.

Doctor Job Career
Nurse Job Career
↑Top