Home > Storage > ObjectScale and ECS > Product Documentation > Dell ECS: Data Movement (Copy to Cloud) > Data movement policy
A data movement policy defines which objects in a source bucket should be copied to a target bucket. An IAM account admin user can use the ECS Portal (GUI) or the management REST API to define a data movement policy.
The policy definitions are stored in the resource table. Only one data movement policy can be associated with any bucket. A data movement policy can be removed from a source bucket by deleting the policy, using the management REST API. When a data movement policy is deleted, the policy definitions in the resource table will be cleaned up. After deleting a data movement policy, the associated source bucket remains on ECS.
A data movement policy scan job scans the associated bucket to collect objects for data movement that meet the policy criteria. Data movement policy scan jobs are automatically triggered by ECS to scan objects. Then, the data movement policy batch copy job copies candidate objects automatically and immediately from ECS to the target bucket after its associated scan job completes. The copy job supports batching up to 10K objects or a total size of 10GB. The default data movement policy scan interval value is one hour. The interval is configurable. Contact your Dell representative if you need to change the interval for data movement policies.
You can pause or resume a data movement policy at any time. When you pause a data movement policy, the associated scan job or copy job is paused but the data movement policy is not removed. When you resume a data movement policy, the associated scan job or copy job is resumed from the checkpoint. Copy to Cloud does not support manually starting a policy scan job or a batch copy job.
The policy criteria defines which objects should be copied to a target bucket. The policy criteria include:
Copy to Cloud uses the watermark lag to allow a window clock skew across ECS nodes. The default watermark lag value is two hours, which is system real-time minus the minimum watermark of the data movement policy.
Note: Contact your Dell representative if you need to change the watermark lag for data movement policies.
The Data Mobility feature allows you to set up automated copying of bucket data to a target bucket. This target bucket can be in the same ECS cluster, or an external ECS cluster, or Amazon S3.
Data Mobility can only be enabled on the bucket that has MD Search enabled with the LastModified field indexed. After Data Mobility is enabled, you can start configuring and creating a data movement policy on the source bucket, as shown in Figure 3.
Note: The Copy to Cloud feature only supports IAM buckets.
Table 2 shows the management REST API for managing data movement policies.
API | Description |
POST /object/bucket/test-policy | Validates a data movement policy |
POST /object/bucket/test-policy-edit | Validates a data movement edit operation |
GET /object/bucket/{bucketName}/copypolicy | Returns a data movement policy for the specified bucket |
PUT /object/bucket/{bucketName}/copypolicy | Updates a data movement policy for the specified bucket |
DELETE /object/bucket/{bucketName}/copypolicy | Deletes a data movement policy for the specified bucket |
GET /object/bucket/copypolicy | Returns a list of all data movement policies for the specified namespace |
For more information about how to create or edit data movement policies, see ECS Administration Guide and ECS REST API Guide.