> For the complete documentation index, see [llms.txt](https://netmind-power.gitbook.io/netmind-power-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://netmind-power.gitbook.io/netmind-power-documentation/rent-gpus/container/cloud-sync.md).

# Cloud Sync

In the examples we provided below, we use [rclone](https://rclone.org) to synchronize the data with [Google Drive](#google-drive) and [Dropbox](#dropbox), and we use CLI to setup cloud sync with [AWS](#aws). 

## Google Drive

{% hint style="info" %}
Prerequisite: Active Google Drive account

NetMind operates at the account level. Users are advised to have a dedicated Google Drive account for their NetMind-related activities, as opposed to using personal accounts. This separation aids in maintaining a clear distinction and enhanced security for NetMind operations.
{% endhint %}

### Install and setup rclone on your rental GPU

Follow the instruction [here](https://rclone.org/install/) to install rclone on the SSH instance you rent.

Here we show an example of how to install rclone v1.65.2 to your SSH instance. 

<pre class="language-bash"><code class="lang-bash"><strong>// Download rclone
</strong><strong>wget https://github.com/rclone/rclone/releases/download/v1.65.2/rclone-v1.65.2-linux-amd64.deb
</strong></code></pre>

```bash
// Install rclone
apt install ./rclone-v1.65.2-linux-amd64.deb
```

```bash
// Config rclone in "Rent GPU" instance
root@702760ea4881:~/workspace/test# rclone config
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

Enter name for new remote.
name> remote

Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
...
18 / Google Drive
   \ (drive)
...
Storage> drive

Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id> 

Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret> 

Option scope.
Comma separated list of scopes that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / Full access all files, excluding Application Data Folder.
   \ (drive)
 2 / Read-only access to file metadata and file contents.
   \ (drive.readonly)
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ (drive.file)
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ (drive.appfolder)
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ (drive.metadata.readonly)
scope> 

Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file> 

Edit advanced config?
y) Yes
n) No (default)
y/n> 

Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.

y) Yes (default)
n) No
y/n> n

Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
        rclone authorize "drive"
Then paste the result.
Enter a value.
config_token> 
```

### Authorization on Google Drive

Install rclone to your own machine that has a browser and do Google Drive authentication.

{% hint style="info" %}
We are only using Google Drive as an example. You can use a cloud storage service that is more convenient for you to synchronize data in the SSH instance. For more tools, please refer [here](https://rclone.org/overview/).
{% endhint %}

1. Execute "authorize" command in terminal

```sh
(base) xxxxxx@xxxxxxxxx rclone-v1.65.2-osx-arm64 % ./rclone authorize "drive"
```

2. Allow rclone in "System Setting" (if your are using Mac)

<figure><img src="https://2171129615-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fzj4sv2NjN42aoPb00bj8%2Fuploads%2F5V0c0iBv8ng7Ntpaoexx%2Fimage.png?alt=media&amp;token=c4a927a5-b18b-4a6d-b105-ca26771413eb" alt=""><figcaption></figcaption></figure>

3. Sign in your account in browser

<figure><img src="https://2171129615-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fzj4sv2NjN42aoPb00bj8%2Fuploads%2Fdxm1zzft16OCMqFJaojr%2Fimage.png?alt=media&amp;token=c2ab89cc-b29b-45e8-8f8b-866dfaa2af79" alt=""><figcaption></figcaption></figure>

### Paste the authorize code into SSH instance

<figure><img src="https://2171129615-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fzj4sv2NjN42aoPb00bj8%2Fuploads%2FbM87ZJ23XB7684ZXGCU2%2Fimage.png?alt=media&amp;token=f6e2b848-e5e2-4ff4-84bd-8d4656f097ee" alt=""><figcaption></figcaption></figure>

```bash
config_token> eyJ0...SJ9
```

### Sync files to Google Drive

```bash
// Sync folder from Google Drive
root@702760ea4881:~/workspace/test# rclone sync remote:test /root/workspace/test
```

```bash
// Sync folder to Google Drive
root@702760ea4881:~/workspace/test# rclone sync remote:test /root/workspace/test
```

***

## AWS

{% hint style="info" %}
Prerequisitie: Active AWS account

For NetMind, we advise against using an existing IAM user. As NetMind connects at a user level, it is preferable to create a new IAM user with specific authorizations tailored for the data you plan to store on NetMind servers. This approach ensures more precise and secure data management.
{% endhint %}

### Install Cloud Sync with CLI

```bash
root@xxxxxxxxxx:~/workspace/test# pip3 uninstall awscli
...
root@xxxxxxxxxx:~/workspace/test# aws configure
AWS Access Key ID [None]: xxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxxxxxxxxx
Default region name [None]: 
Default output format [None]:
```

### Sync Files to AWS

```bash
// Sync files from S3
aws s3 sync s3://test0131/ /root/workspace/test
```

```bash
// Sync files to S3
aws s3 sync s3://test0131/ /root/workspace/test
```