Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Virtualized Computer Vision for Smart Transportation with Milestone
Introduction Architecture concepts and requirements Solution Architecture Validation
Validation overview
Performance validation
Performance results and findings
Performance summary
High availability validation
High availability results and findings
High availability summary
Sizing the solution Backup and restore operations Monitoring the solution Troubleshooting recommendations Summary and conclusions References

Performance validation

Performance validation

  • The goal of our performance validation was to assess whether our proposed design could produce reliable planning input metrics for sizing a range of small, medium, and large deployment options. Most VMS and CV applications are sized by scaling out infrastructure based on per application service building blocks, e.g., video recorder service, database service, alert processing service, etc. We configured application service VMs to offer a good balance of performance and high availability. We monitored all key OS and application-specific performance metrics to see if we could detect any signs of resource exhaustion or process blocking. Our mixed workloads included processing a large simulated camera network with a Milestone XProtect VMS while simultaneously running BreifCam and Ipostek real-time alert processing for a subset of the total camera streaming workload.

    VM placement with DRS

    A typical muli-vendor solution for a VMS/CV consists of a total VM count that is too difficult to manage with static rules and guidelines. VMware provides the DRS tool that will intelligently assess the layout of the application VMs on the VxRail solution to ensure even and reliable performance for the system. In addition to performance, it is also important to consider the impacts of high availability protection when defining DRS rules.

    Placement constraints

    The rules on placement of VMs are critical for small or medium size platforms due to the limited count of nodes in the system. This is less of a concern in a large system, but the rules must still be followed. Failure to follow the rules can result in some nodes that are over-allocated and not in a HA configuration.

    The following application constraints were considered when configuring DRS for VM placement prior to the testing performed to produce this guide:
    • A Maximum of 3 Milestone XProtect Recorders per VxRail Host
    • A Maximum of 2 CV processing nodes per VxRail Host (assuming 2 GPUs available)
    • Primary and secondary VMs for a system must not be on the same VxRail Host:
      • Ipsotek Management VMs cannot be on the same Host.
      • Ipsotek Database nodes cannot be on the same Host.
      • BriefCam Management nodes cannot be on the same Host.
      • BriefCam Database nodes cannot be on the same Host.
      • Milestone XProtect Management nodes cannot be on the same Host.
      • Milestone XProtect Database nodes cannot be on the same Host.
      • Milestone XProtect Hot and Cold Recorders cannot be on the same Host as the Primary recorders.

    DRS Rule Details

    1. No host or VM groups were needed for this design. The use of dynamic hosts that can get placed into maintenance mode results in static rules not working.
    2. Briefcam- PS : A rule that ensures that all BriefCam processing nodes are kept on separate hosts
    3. Mil Management: A rule that ensures that the two XProtect Management VMs are placed on separate hosts.
    4. Ipso-ps : A placeholder rule for completeness when vGPU support is added to Ipsotek.
    5. Max 3 Recorders per Host: One recorder from each group of three was chosen and forced to be placed on separate hosts.
    6. Group Rec 1-3 : A rule that says Keep Virtual Machines Together. Creates a group that can run on any host.
    7. Group Rec 4-6: see 6 above
    8. Group Rec 7-9: see 6 above
    9. Group Rec 10-12: see 6 above

    The following screenshot shows the rules when configured with the vSphere web UI.

    Figure 9. DRS rules

    VM placement for validation

    After enabling VMWare DRS and starting all the VMs for the solution we observed the following distribution of VMs across the VxRail cluster:

    Partner Node 1 Node 2 Node 3 Node 4 Node 5
    VxRail/vCenter
    • VxRail Manager
    • VMware Center Server Appliance
    Milestone XProtect
    • Backup Mgmt Server
    • Recorder 1
    • Recorder 2
    • Recorder 3
    • Recorder 4
    • Recorder 5
    • Recorder 6
    • Main Mgmt Server
    • Recorder 10
    • Recorder 11
    • Recorder 12
    • Mgmt SQL Server Database 2
    • Recorder 7
    • Recorder 8
    • Recorder 9
    • Mgmt SQL Server Database 2
    • Cold Standby 1
    • Cold Standby 2
    • Hot Standby 1
    • Hot Standby 2
    BriefCam
    • Processing Server (with GPU)
    • Processing Server (with GPU)
    • Processing Server (with GPU)
    • Web Server
    • Processing Server (with GPU)
    • Database Server
    • VS Server
    Ipsotek
    • Processing Server (with GPU)
    • Processing Server (with GPU)
    • Processing Server (with GPU)
    • Processing Server (with GPU)
    • Ipsotek Management Server
    • Datastore Server
    • Processing Server (with GPU)
    Note: This is only one possible solution that could result from the application of DRS rules for placement.

    Video workload

    The testing workload was generated by 840 camera simulators using two different source videos. The details of the videos used are as follows:

    Type CV workload Resolution FPS Bit rate
    Busy Hallway Face Recognition 1080 30 3.2 Mb/s
    Isolated Stairwell Restricted Zone 1080 20 2.8 Mb/s

    Camera mapping to Milestone

    The camera simulators ran on hardware outside of the VxRail cluster that hosted the VMS and CV processing services. The following table shows how the mapping of camera simulators to Milestone XProtect recorder VMs:

    Camera Simulator Milestone Recorder Camera Count
    Simulator-01 Recorder-01 70
    Simulator-01 Recorder-02 70
    Simulator-01 Recorder-03 70
    Simulator-02 Recorder-04 70
    Simulator-02 Recorder-05 70
    Simulator-02 Recorder-06 70
    Simulator-03 Recorder-07 70
    Simulator-03 Recorder-08 70
    Simulator-03 Recorder-09 70
    Simulator-04 Recorder-10 70
    Simulator-04 Recorder-11 70
    Simulator-04 Recorder-12 70

    Total 840