如果您在 TWSC COS 儲存體使用上有遇到以下情況，請您參考下方步驟透過 Rclone 轉移資料至另一 儲存體 (同專案或跨專案皆可)：
1. 儲存體儲存的檔案資料量龐大 (約三千萬個檔案)，希望減少原儲存體中的檔案數量，也需保留所有檔案。
2. 需將儲存體檔案複製一份至另一儲存體使用。
前置作業
1. 於 TWSC 建立 Linux VCS 主機進行 Rclone 操作
2. 取得來源及目標 COS 的 endpoint、Access Key、 Secret key、來源及目標儲存體名稱
操作步驟
Step 1. 安裝 Rclone
1. 登入 Linux VCS 主機
2. 輸入指令進行 Rclone 安裝
curl https://rclone.org/install.sh | sudo bash
3. 確認安裝成功
rclone --version
Step 2. 設定 COS 連線資訊
1. 輸入指令進行連線設定
rclone config
2. 選擇新增 remote，輸入n
Current remotes: Name Type ==== ==== E1 s3 G1 s3 e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> n
3. COS 顯示名稱：請自訂，可分辨連線目標即可
name> sourceCOS
4. 連線類型：輸入s3，選擇 s3 連線類型
Option Storage.
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
1 / 1Fichier
\ "fichier"
2 / Alias for an existing remote
\ "alias"
3 / Amazon Drive
\ "amazon cloud drive"
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS
\ "s3"
5 / Backblaze B2
．
．
．
45 / seafile
\ "seafile"
Storage> s3
5. S3 服務提供者：因 TWCC COS 並無在目前列表上，故選擇 Other。
Option provider.
Choose your S3 provider.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
．
．
．
14 / Any other S3 compatible provider
\ "Other"
provider> Other
6. 連線認證方式：輸入 false，選擇 Enter AWS credentials in the next step
Option env_auth.
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value.
1 / Enter AWS credentials in the next step.
\ "false"
2 / Get AWS credentials from the environment (env vars or IAM).
\ "true"
env_auth> false
7. access key：輸入目標 COS 的 access key
Option access_key_id.
AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
access_key_id> TWCC_COS_access_key_id
8. secret_access_key：輸入目標 COS 的 secret_key
Option secret_access_key.
AWS Secret Access Key (password).
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
secret_access_key>TWCC_COS_secret_access_key
9. region：按 Enter 使用預設值
Option region.
Region to connect to.
Leave blank if you are using an S3 clone and you don't have a region.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
/ Use this if unsure.
1 | Will use v4 signatures and an empty region.
\ ""
/ Use this only if v4 signatures don't work.
2 | E.g. pre Jewel/v10 CEPH.
\ "other-v2-signature"
region>
10. endpoint，輸入 cos.twcc.ai 連線節點
Option endpoint.
Endpoint for S3 API.
Required when using an S3 clone.
Enter a string value. Press Enter for the default ("").
endpoint>cos.twcc.ai
11. location_constraint，按 Enter 使用預設值
Option location_constraint.
Location constraint - must be set to match the Region.
Leave blank if not sure. Used when creating buckets only.
Enter a string value. Press Enter for the default ("").
location_constraint>
12. acl，按 Enter 使用預設值
Option acl.
Canned ACL used when creating buckets and storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
Note that this ACL is applied when server-side copying objects as S3
doesn't copy the ACL from the source but rather writes a fresh one.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
/ Owner gets FULL_CONTROL.
1 | No one else has access rights (default).
\ "private"
/ Owner gets FULL_CONTROL.
2 | The AllUsers group gets READ access.
\ "public-read"
/ Owner gets FULL_CONTROL.
3 | The AllUsers group gets READ and WRITE access.
| Granting this on a bucket is generally not recommended.
\ "public-read-write"
/ Owner gets FULL_CONTROL.
4 | The AuthenticatedUsers group gets READ access.
\ "authenticated-read"
/ Object owner gets FULL_CONTROL.
5 | Bucket owner gets READ access.
| If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
\ "bucket-owner-read"
/ Both the object owner and the bucket owner get FULL_CONTROL over the object.
6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
\ "bucket-owner-full-control"
acl>private
13. 是否要編輯細節，按 Enter 使用預設值
Edit advanced config? y) Yes n) No (default) y/n>
14. 確認輸入內容，確認無誤後輸入 y 進行存檔
-------------------- [sourceCOS] type = s3 provider = Other access_key_id = <access_key_id> secret_access_key = <secret_access_key> endpoint = cos.twcc.ai -------------------- y) Yes this is OK (default) e) Edit this remote d) Delete this remote y/e/d> y
15. 回到起始畫面，重複此步驟，建立目標 COS 連線 targetCOS，再輸入 q 離開此畫面。
Current remotes: Name Type ==== ==== E1 s3 G1 s3 sourceCOS s3 targetCOS s3 e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q>q
Step 3. 進行資料轉移
1. 確認連線設定，已建立 sourceCOS 及 targetCOS
ubuntu@awsclitest02-2451501-iaas:~$ rclone config Current remotes: Name Type ==== ==== E1 s3 G1 s3 sourceCOS s3 targetCOS s3 e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
2. 完成設定後，即可執行將資料從來源儲存體轉移至目標儲存體。
完成後，後續即可刪除來源儲存體不需要的資料。
rclone sync sourceCOS:<source_bucket_name> targetCOS:<target_bucket_name>