Sync a Linux VPS with Object Storage (Rclone)

Rclone is a program that allows you to sync your server's files and directories with Hostwinds Object Storage. It can be mounted locally so you can write directly to remote Object Storage. Commands like rclone sync can sync your local directories with directories in remote Object Storage.

This guide will walk through installing, configuring, syncing, and mounting Rclone remotes with a Linux VPS.

Install Rclone

Start by installing Rclone with the script below:

curl https://rclone.org/install.sh | sudo bash

Now you can configure your remote storage after the installation script ends:

rclone config

You will enter the configuration wizard. To set up a new remote, enter n at the first prompt:

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

Next, enter the name of your Rclone remote.

After that, enter 4 for S3 compliant storage (Hostwinds Object Storage):

Type of storage to configure.
Enter a string value. Press Enter for the default.
Choose a number from below, or type in your own value
1 / 1Fichier
"fichier"
2 / Alias for an existing remote
"alias"
3 / Amazon Drive
"amazon cloud drive"
4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital
Ocean, Dreamhost, IBM COS, Minio, etc.)
"s3"
...
Storage> 4

Enter 11 to choose Any other S3 compatible provider:

Choose your S3 provider.
Enter a string value. Press Enter for the default.
Choose a number from below, or type in your own value
1 / Amazon Web Services (AWS) S3
"AWS"
...
11 / Any other S3 compatible provider
"Other"
provider> 11

Enter 1 to enter the credentials in the next step:

Get AWS credentials from runtime (environment variables or EC2/ECS meta
data if no env vars).
It only applies if access_key_id and secret_access_key are blank.
Enter a boolean value (true or false). Press Enter for the default
("false").
Choose a number from below, or type in your own value
1 / Enter AWS credentials in the next step
"false"
2 / Get AWS credentials from the environment (env vars or IAM)
"true"
env_auth> 1

Now you need the Access Key, Secret Key, and API Endpoint from your Hostwinds account. Navigate here to find these credentials. Select Object API Info in the upper-right corner, and copy:

Access Key
Secret Key
API Endpoint

of the location, you want to use. It is recommended to use the same location as your Hostwinds VPS. You will need to use these for the access_key_id and secret_access_key options.

Enter 1 to opt-out of selecting a region:

Region to connect to.
Leave blank if you are using an S3 clone and you don't have a region.
Enter a string value. Press Enter for the default.
Choose a number from below, or type in your own value
1 / Use this if unsure. We will use v4 signatures and an empty region.
\ ""
2 / Use this only if v4 signatures don't work, eg pre Jewel/v10 CEPH.
\ "other-v2-signature"
region> 1

Endpoint for S3 API: enter the API Endpoint for the Object Storage location you want to use.

Location constraint - must be set to match the Region. Used when creating buckets only: leave blank, click ENTER.

Canned ACL used when creating buckets and/or storing objects in S3: the default choice is 1, but you can choose the access rights you wish to give.

Edit advanced config? (y/n): enter n, and verify the values for your remote storage at the next prompt.

Finally, enter q to quit the config:

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Rclone Sync

First, you need to make a directory for your new remote. Syncing before making a directory will cause synced data to be in unreadable buckets. Enter the name of your remote and new bucket/directory like below:

rclone mkdir remote_name:remote_directory

Now navigate to your Object Storage here. Click Refresh Containers, and you should see your new bucket like below:

To sync a directory with your new bucket, use the command below with the path to your local directory, your remote name, and bucket/remote directory name. This will only modify the destination, your Rclone remote directory:

rclone sync /path/to/local/directory remote_name:remote_directory

Rclone Mount

Create a mount point first:

mkdir /path/to/local/mount

Similar to syncing, you should work directly with a remote directory in your Rclone remote. To mount a remote directory locally, run the command below with your remote name, directory, and mount point:

rclone mount remote_name:remote_directory /path/to/local/mount --allow-non-empty --vfs-cache-mode writes

Then open a new shell, and write directly to your mount point.

To end the mount, ensure you are not in the mount point and CTRL + C from the shell where the mount is running. If there is an issue unmounting, manually unmount with the command below:

fusermount -u /path/to/local/mount

Conclusion

Now you have successfully configured your Rclone remote and created your first remote directory. You can find all Rclone commands by visiting their documentation for commands like cat, move, copy, serve, and more.