This use case is another vital functionality aspect for a large environment deployment to validate that it performs as expected. We have validated that data can be transferred reliably from a bus to the depot. The following testing shows that since the data cannot reside at the depot indefinitely due to size constraints for IT equipment, a replication mechanism can be connected to a reliable central location where additional analysis or training of future AI models can occur.
Test Specification
To test this in our lab, we created a VM with a simple NFS mount. In production scenarios, we would recommend using replication of a Dell PowerScale storage appliance. Dell PowerScale is typically deployed as part of our CV platform that has been validated for many industries and use cases in the past.
It is necessary to set up a method to replicate data from a bus to the data center mount point from the depot VM. A typical lsyncd configuration is shown below:
sync {
default.rsync,
source = "/var/nfs/data",
target = "/var/nfs/datacenter",
delete = false
}
The testing procedure followed these four steps:
- Ensure connectivity from depot to data center
- Setup an NFS mount point on the depot VM that maps to the data center
- Configure lsyncd to replicate incoming bus data to the data center mount point
- Stream video offload traffic through the system and verify it is saved in the data center
Test results
The most straight forward way to see the data transfer pattern is to look first at the profile of the network data sent from the depot:
This is an unusual load that has a spike when offload begins after 4 hours. The data sent from the depot to the data center is the blue line in the above graph.
Next, we need to look at the data received at the data center side:
Only one depot is configured to transfer data to the data center for this test setup. The red line above shows the data rate coming into the data center from this single source. The red line aligns well on the time axis with the blue data sent in the previous graph.
In addition to the visual comparison, we compared the data difference between the source bus (/var/nfs/data) and the data center NFS (/var/nfs/datacenter). This test showed that all video files from the source were replicated intact to the destination. The destination data center has additional files, which is expected since it retains video data until the retention period has been reached. The retention is user-defined and can use as much space as is available on the customer PowerScale.
Findings
- The visual representation shows an exact replica of data sent versus data received.
- Inspection of the underlying video files shows that all files from the bus are available in the data center.
- Historic data can be retained in the data center long after it is overwritten in the bus or depot.