Andes is currently the only HPC with rclone already installed (8/2024). The wording on the steps may vary depending on the version of rclone installed on the machines.
Interactive guide - Google
The following guide walks through setting up rclone on the Research Computing environment and using it to interactive with Google Drive.
Create a GUI session into andes and launch a terminal
Configure an rclone integration for Google Drive
rclone config
n/s/q> n
name> gdrive
Storage> drive
client_id> <enter>
client_secret> <enter>
scope> 1
root_folder_id> <enter>
service_account_file> <enter>
y/n> <enter>
Browser opens, fill in email "elijah.w.gagne@dartmouth.edu" and click Next
Complete the Dartmouth Single Sign On process
Click "Allow" to grant rclone access to your Google Drive
y/n> <enter>
y/e/d> <enter>
e/n/d/r/c/s/q> q
Test
The following does a listing of data in your Google Drive
[d92495j@andes ~]$ rclone lsf gdrive: --max-depth 1
Code/
Documents/
Presentations/
_Archived/
The following uploads a file
[d92495j@andes ~]$ rclone copy slurm_test.sh gdrive:
[d92495j@andes ~]$ rclone lsf gdrive: --max-depth 1
Code/
Documents/
Presentations/
_Archived/
slurm_test.sh
Remote guide (no GUI) - Dropbox
The following guide walks through setting up rclone on the Research Computing environment and using it to interactive with Dropbox. You will need to have rclone installed on both your remote and local machines to complete the authentication process.
Remote machine
Start the configuration process by running
rclone config
Create a new connection
e/n/d/r/c/s/q> n
Give it a name of your choosing
Enter name for new remote.
name> dropbox
Select the type of storage
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
Storage> dropbox
Leave the client id and secret blank unless you already have Dropbox app credentials.
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n
Since we are doing this on a machine without a web browser, select No to the question below.
Use web browser to automatically authenticate rclone with remote?
* Say Y if the machine running rclone has a web browser you can use
* Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes (default)
n) No
y/n> n
You'll be prompted for a config token, swap to your local machine.
Local Machine (with a web browser)
You'll need your config token, the easiest way is to install rclone on your local machine then run
rclone authorize "dropbox"
This opens a web browser window where an OAuth2 flow will authorize rclone to access your Dropbox.
Back in your terminal window, copy the config token.
Remote Machine
Paste the config token you copied from your local machine.
Confirm the details and select yes.
[dropbox]
type = dropbox
token = {<Redacted>}
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
Quit Rclone
e/n/d/r/c/s/q> q
See the rclone Dropbox reference for examples of how to find the directory path for Dropbox.
Copy content to your Dropbox account. The -P option will show progress, remove if not needed. The example below copies the content from rclone-sync in my DartFS home to a directory called dropbox-rclone-sync in my Dropbox account.
rclone copy -P rclone-sync/ dropbox:dropbox-rclone-sync
References