Underlying HCI infrastructure architecture considerations

As many organizations embrace digital transformation and the application modernization journey that is involved in this process, Dell Technologies and VMware supporting customers by providing them with modern cloud infrastructure and storage solutions that support the demands of this new set of cloud native applications.

Dell ObjectScale, VMware vSphere with Tanzu, and the vSAN Data Persistence Platform (vDPp) are all examples of next generation cloud native technologies that deliver simple, scalable, and enterprise grade Kubernetes native S3 compatible object storage services on a Kubernetes runtime built into the vSphere hypervisor. To learn more about the details of this powerful set of technologies, check out these great blog posts from my colleagues over at VMware here and here. A recently published reference architecture white paper also walks through the steps of deploying these technologies together. 

Now let’s get into our primary topic for this blog, which is the underlying HCI infrastructure architecture considerations for running ObjectScale on vSphere with Tanzu. 

Setting the stage

Cloud infrastructure administrators have a lot of flexibility in terms of what and how to configure the infrastructure on which Dell ObjectScale runs. These options not only come at the underlying HCI infrastructure implementation layer but also at the VMware SDDC layer. This gives administrators choices on mixing the right combination of the two layers that best meet their business and operational requirements.

So, what are the layers that make up these options? For this discussion we will break it down as follows:

HCI Infrastructure Layer Options

• Construct – Dell vSAN Ready Nodes
• Consume – Dell VxRail HCI Integrated Systems

VMware SDDC Software Layer Options

• 
  

  Construct - VMware vSphere with Tanzu + VMware NSX-T
• Consume - VMware Cloud Foundation (VCF) with Tanzu

After we review these options, we will highlight how they can be used to align to your ObjectScale architecture design and workload requirements.

Construct HCI and Construct VMware SDDC – Dell ObjectScale on Dell vSAN Ready Nodes with VMware vSphere with Tanzu + VMware NSX-T

This option involves deploying ObjectScale on vSphere with Tanzu enabled Dell vSAN Ready Node clusters and then manually deploying and configuring the rest of the required VMware SDDC software stack including NSX-T. This is essentially the builder’s approach to implementing the HCI infrastructure stack and the VMware SDDC stack. This gives infrastructure administrators the most control over their infrastructure configuration and components. The tradeoff, however, is that it adds a bit more complexity and more manual steps to get to an outcome that is ObjectScale ready.

Consume HCI and Construct VMware SDDC – Dell ObjectScale on Dell VxRail with VMware vSphere with Tanzu + VMware NSX-T

 With this approach, infrastructure administrators can take advantage of consuming pre-validated and co-engineered Dell VxRail HCI integrated systems, enabling vSphere with Tanzu on them, and then manually deploying the NSX-T components of the solution. This speeds up and simplifies the HCI infrastructure management and operations portion of the stack while still delivering on the required SDDC infrastructure foundations needed for ObjectScale to run.

Construct HCI and Consume VMware SDDC – Dell ObjectScale on VMware Cloud Foundation with Tanzu on Dell vSAN Ready Nodes

 This option delivers infrastructure administrators with granular control in constructing the underlying HCI HW components while simplifying the VMware SDDC layer and consuming it as a full cloud platform using VMware Cloud Foundation. This helps streamline the VMware SDDC to include NSX-T out of the box and can automate the deployment and configuration of the VMware SDDC components that are required to enable vSphere with Tanzu and run ObjectScale.

Consume HCI and Consume VMware SDDC – Dell ObjectScale on VMware Cloud Foundation with Tanzu on Dell VxRail

 This option provides a true full stack turnkey cloud infrastructure platform for infrastructure administrators to consume. This co-engineered solution between VMware and Dell Technologies delivers the fastest path to hybrid cloud and Kubernetes. Administrators gain the operational and feature benefits of VxRail, the only HCI system with deep VMware Cloud Foundation integration, with the out of the box simplicity and automation of the VMware Cloud Foundation SDDC cloud platform. From an ObjectScale use case perspective, infrastructure administators can accelerate getting all the needed underlying cloud infrastructure up and running so that ObjectScale can be deployed quickly and easily at scale and with a standardized cloud infrastructure architecture built in.

Choosing the right ObjectScale deployment infrastructure architecture

All these options deliver the necessary infrastructure prerequisites required to deploy and run ObjectScale, just through different implementation approaches that align to an organization’s operating model. ObjectScale, however, can also be deployed in several different ways, which can affect the implementation of your underlying infrastructure. 

 Let’s review what these options are, how our infrastructure can support these deployment models, and when would be the best time to choose one over the other.

 First, let’s call out the ObjectScale deployment architecture options available:

1. Co-locate ObjectScale data services on the same clusters where user application workloads run
2. Run ObjectScale data services on dedicated cluster infrastructure separate from user application workloads

How an infrastructure administrator would configure the underlying HCI and VMware SDDC stack based on these options will ultimately depend on which SDDC deployment method was used, vSphere with Tanzu + NSX-T or VCF with Tanzu.

The infrastructure implementation design details vary slightly since VCF implements a prescriptive cloud architecture using the concept of workload domains. This means that cloud infrastructure administrators must consider how to deploy vSphere with Tanzu enabled clusters to run ObjectScale within the context of this VCF’s workload domain architecture. On the other hand, if administrators were using the build approach of deploying individual vSphere with Tanzu enabled clusters, architecture design decisions are a bit more open ended. Either way, both implementation methods support both ObjectScale deployment architecture models of co-located and dedicated and can be run on both Dell vSAN Ready Nodes and Dell VxRail HCI Integrated Systems.

So, what would the first option look like when co-locating ObjectScale data services on the same cluster as where user application workloads are run?

The following figure provides a visual depiction of what this option may look like in a VCF on VxRail deployment using a single VI workload domain with a single vSphere with Tanzu enabled VxRail cluster in it. In this example, we would deploy ObjectScale to the Supervisor Cluster running on this WLD cluster. Application teams would then have their user application workloads running on the same cluster infrastructure and share the underlying physical HCI compute, network, and storage resources. 

 Figure 1: VCF on VxRail – ObjectScale co-location cluster deployment

This approach has advantages in terms of minimizing the infrastructure footprint required to run both workload types. It can also help drive improved resource utilization of the HCI infrastructure that has been deployed. This can also be a great fit for minimizing licensing costs if you have containerized user workloads and VM-based workloads that need to consume ObjectScale storage since there is only one cluster you need to enable vSphere with Tanzu on and vSphere can support running containers and VMs on the same vSphere cluster. However, there are possible downsides. These include resource contention for user workloads since you are sharing the same infrastructure to run ObjectScale data services and lack of independent scalability and right sizing of infrastructure resources for ObjectScale and the user applications.  

Option 2, running ObjectScale data services on dedicated cluster infrastructure separate from user application workloads, eliminates the resource contention by running ObjectScale on its own dedicated cluster infrastructure separate from user workloads. In a VCF on VxRail deployment, this may be implemented in a couple of ways. The first is to create a single VI WLD with two or more VxRail clusters in it. One cluster would have vSphere with Tanzu enabled on it and is where ObjectScale would be deployed. The other cluster, depending on the types of workloads running (whether they be VM-based only or a mix of containers and VMs) may not require vSphere with Tanzu be enabled on it and can just be used to run user application workloads.

By running ObjectScale on its own workload domain cluster resources, we now have physical resource isolation for both ObjectScale and user application workloads. This avoids resource contention between the two and now have the flexibility to independently scale resources for both as needed. Using this VCF workload domain organizational model may be helpful if your organization is aligning ObjectScale storage and the workloads that consume it as part of a single business unit and you may want to keep all of that together and managed within a single managed pool of cloud infrastructure resources. The following diagram provides an illustration of how this would look. 

 Figure 2: VCF on VxRail – ObjectScale dedicated cluster deployment with single VI WLD

The other VCF workload domain design approach is to deploy two VI workload domains. One would contain one or more VxRail clusters with vSphere with Tanzu enabled on them and ObjectScale would be deployed on top. The other VI workload domain would contain one or more VxRail clusters that may or may not have vSphere with Tanzu enabled on them and would run user application workloads only. This method still gets you separation of physical resources to avoid resource contention as well as independent scaling for both workload types, but organizationally we have deployed workload domains based on infrastructure service function. 

Deploying ObjectScale into its own dedicated workload domain provides the possibility of maximum scale of how many clusters we can deploy into a single domain that can be used solely for running ObjectScale data services. We can also help simplify the networking for those clusters since we only need to accommodate for the networking needs of ObjectScale and not also for user applications workloads, too. 

The following example uses dedicated NSX-T instances for each VI workload domain. In VCF, it is possible to share an NSX-T instance across multiple VI workload domains. If we would have done this, we wouldn’t have to deploy another cluster of NSX Edge appliances and could have just used the NSX Edge appliance deployed in VI Workload Domain 2 to meet the requirements that are needed when enabling vSphere with Tanzu on vSphere clusters. But since we are using separate dedicated NSX-T instances, each VI workload domain will require NSX Edge appliances to meet these vSphere with Tanzu and ObjectScale minimum requirements for the clusters contained within them. The following figure shows an illustration of what this multi-workload domain organizational model would look like. 

Figure 3: VCF on VxRail – ObjectScale dedicated cluster deployment with two VI WLDs

It is important to call out that these same co-located and dedicated cluster ObjectScale architecture models can be used in vSphere with Tanzu + NSX-T on Dell vSAN Ready Nodes/VxRail deployment options as well and are not tied to just the VCF on VxRail examples shown here. The same overall ObjectScale logical and physical layout considerations would apply. Administrators who choose to approach running ObjectScale in this way would be responsible for determining where the NSX-T Manager VM’s, Edge appliances, and vCenter components would run as there would be no Management Domain construct defined as part of a cloud platform architecture like VCF has.

This is not the end, it’s just the beginning…

I hope you have found this information helpful as you work through your ObjectScale adoption journey. This is not the end of your journey, however. For more information about VxRail and ObjectScale, check out the links at the bottom of this post.

Author: Jason Marques

Twitter: @vWhipperSnapper

Additional Resources

ObjectScale page on InfoHub

VxRail page on DellTechnologies.com

VxRail page on InfoHub

VCF on VxRail Interactive Demos

VxRail Videos

With each turn of the calendar, as winter dissipates and the warmer spring weather brings new life back into the world, a certain rite of passage comes along with it: Spring Cleaning! As much as we all hate to do it, it is necessary to ensure that we keep everything operating in tip top shape. Whether it be errands like cleaning inside your home or repairing the lawn mower to be able to cut the grass, we all have them, and we all recognize they are important, no matter how much we try to avoid it. 

The VMware Cloud Foundation (VCF) on Dell VxRail team also believes in applying a spring cleaning mindset when it comes to your VCF on Dell VxRail cloud environment. This will allow your cloud environment to keep running in an optimal state and better serve you and your consumers. 

So, in the spirit of the spring season, Dell is happy to announce the release of Cloud Foundation 4.4.1 on VxRail 7.0.371. Beginning on May 25, 2022, existing VCF on VxRail customers will be able to LCM to this latest version while support for new deployments will be made available beginning June 2, 2022.

This new release introduces the following “spring cleaning” enhancements:

• New component software version updates
• New VxRail LCM logic improvements
• New VxRail serviceability enhancements
• VCF and VxRail software security bug fixes
• VCF on VxRail with VMware Validated Solution Enhancements

VCF on VxRail life cycle management enhancements

New VxRail prechecks and vSAN resync timeout improvements

Starting with this release, the VxRail LCM logic has been modified to address scenarios when the cluster update process may fail to put a node into Maintenance Mode. This LCM logic enhancement is leveraged in addition to similar SDDC Manager prechecks that already exist. All VxRail prechecks are used when SDDC Manager calls on VxRail to run its precheck workflow prior to an LCM update. SDDC Manager does this by using its integration with the VxRail Health Check API. SDDC Manager also calls on these prechecks during an LCM update using its integration with the VxRail LCM API. So, VCF on VxRail customers benefit from this VxRail enhancement seamlessly. 

Failing to enter Maintenance Mode can cause VxRail cluster updates to fail. Finding ways to mitigate this type of failure will significantly enhance the LCM reliability experience for many VCF on VxRail customers. 

Figure 1: VCF on VxRail LCM

The following list describes scenarios in which a VxRail node could fail to enter maintenance mode, but are improved with the latest enhancements:

• If VMtools are mounted to customer VM workloads: VxRail LCM precheck now checks for this state to detect if VMtools are mounted. If this exists, it is the administrator’s responsibility to address the issue in their environment before initiating a VxRail cluster update.
• If VMs are pinned to specific hosts:  VxRail LCM precheck will now detect whether there is host pinning configured for VMs.   If this exists, it is the administrator’s responsibility to address the configuration in their environment before initiating a cluster update.
• vSAN Resync Time Timeout: During the cluster update process, a VxRail node can fail if vSAN resync takes too long. When the system waits before the node is put into Maintenance Mode, it causes a timeout. To prevent this from happening, the VxRail vSAN Resync timeout value has been increased by 2x while the cluster update waits for the vSAN resync to finish.

VCF on VxRail serviceability enhancements

Support for next generation Dell secure remote service connectivity agent and gateway

VxRail systems will now use the next generation secure remote service connectivity agent and the Secure Connect Gateway to connect to the Dell cloud for dial home serviceability. This new connectivity agent running within VxRail will also be used on all Dell infrastructure products.  

Figure 2: Next Generation Dell Secure remote connectivity agent and gateway architecture

The Secure Connect Gateway is the 5th generation gateway that acts as a centralization point for Dell products in the customer environment to manage the connection to the Dell cloud.  This remote connectivity enables a bi-directional communication between the product and Dell cloud.  Products can send telemetry data and event information to the Dell cloud which can be used to facilitate remote support by Dell services as well as to deliver cloud services such as CloudIQ, MyService360, Licensing Portal, and Service Link.

The latest generation remote service connector is intended to provide a uniform telemetry experience across all Dell ISG products.   By providing standardization, customers can reduce redundant infrastructure used to provide remote services for all their Dell products. The connectivity agent also introduces a simpler setup experience by streamlining and automating setup process of the secure remote service for new VxRail cluster deployments.

Figure 3: Enabling secure remote gateway connectivity

For existing VxRail clusters running an earlier version than VCF 4.4.1 on VxRail 7.0.371 in a VCF on VxRail deployment, the migration effort to adopt the new secure connect gateway requires the administrator to first upgrade their older generation dell serviceability gateways in their environment (whether it’s the 3rd generation Secure Remote Service gateway or the 4th generation Dell SupportAssist Enterprise gateway).  

Once the gateways are upgraded to the latest 5th generation Dell Secure Connect Gateway, the VCF on VxRail administrator can perfrom the VxRail cluster update for the migration, as part of the standard VCF on VxRail LCM process. The built-in VxRail LCM precheck steps will inform the administrator to upgrade their gateways if necessary. The VxRail cluster update will now retrieve the gateway configuration for the connectivity agent and convert the device or access key to a unique connectivity key for remote connection authentication. Administrators should be aware that this additional migration work may add a one time 15 minutes or so time increase to the total cluster update time.

New nodes that are shipped with VxRail 7.0.350 or higher will also now include a unique connectivity key for the secure remote gateway. Dell manufacturing will embed this key into the iDRAC of the VxRail nodes. So, instead of a user logging onto the Dell support portal to retrieve the access key to enable secure remote services, the enablement process will automatically retrieve this unique connectivity key from iDRAC for the connectivity agent to enable the connection. This feature is designed to simplify and streamline the secure connect gateway serviceability setup experience.

Customers can also have a direct connection to Dell cloud bypassing having a gateway deployed.  This option is available for any clusters running VxRail 7.0.350 and higher.

VxRail dial home payload improvements

VxRail dial home payload improvements have been introduced to help provide Dell support with additional key cluster information in the dial home payload itself and capture more system error conditions to help further improve VCF on VxRail serviceability and reduce time to resolution of any VxRail related issues. 

 Additional payload information now includes: 

• Smart Logs: Smart logging automatically collects the logs on the node of the call-home event, which provides additional information to the Support team when necessary. Starting with VCF 4.4.1 on VxRail 7.0.371, smart logging functionality has been redesigned to achieve the following tasks: 
  1. Adapt smart logging workflow to the new secure remote gateway architecture
  2. Associate smart log with Dell Service Request (SR) such that the smart log file can be included in the SR as a link.
• Sub-component details: These include information such as the part number and slot number for CRU/FRU items such as disk drives and memory DIMMs for more efficient auto-dispatch of these failed components.
• VxRail cluster personality identifier information: To help making the troubleshooting experience more efficient, this cluster metadata information allows Dell Support to know that the VxRail clusters are deployed within a VCF on VxRail environment.

 Also included are additional error conditions that are now captured to bring VxRail events in parity with existing PowerEdge events and additional ADC error states. And finally, to reduce the cost of service and improve the customer experience by avoiding a deluge of unnecessary event information, some events are no longer being reported.

VxRail physical view UI update now includes Fiber Channel HBA hardware view

New support for FC HBA Physical HW views have been introduced as part of the VxRail Manager vCenter Plugin Physical View UI for E560F, P570F, and V570F VxRail nodes that support externally attached storage.

 Supported FC HBAs include the following Emulex and QLogic models:

• Emulex LPE 35002 Dual Port 32 Gb HBA
• Emulex LPE 31002 Dual Port 16 Gb HBA
• QLogic 2772 Dual Port 32 Gb HBA
• QLogic 2692 Dual Port 16 Gb HBA

Figure 4: Fiber Channel HBA physical hardware view in VxRail Manager vCenter Plugin – firmware

 This new functionality provides a similar UI viewing experience to what administrators are already used to seeing, regarding physical NICs and NIC ports. This new FC HBA view will include port link status and firmware/driver version information. An example of the firmware/driver views is shown in Figure 4.

VCF on VxRail security enhancements

VCF and VxRail software security vulnerability fixes

This release includes several security vulnerabilities fixes for both VxRail and VCF software components.

 VxRail Software 7.0.371 contains fixes that resolve multiple security vulnerabilities. Some of these include:

• DSA-2022-084
• DSA-2022-056
• DSA-2021-255
• iDRAC8 Updates 

 For more information, see iDRAC8 2.82.82.82 Release Notes 

 For more details on the DSAs, see the Dell Security Advisory (DSA) portal and search for DSA IDs.

 VCF 4.4.1 Software: This contains fixes that resolve issues in NSX-T by introducing support for NSX-T 3.1.7.3.2. For more information about these issues, see the VMware KB Article.

 vRealize Suite Software: In the last VCF 4.4 on VxRail 7.0.320 release we introduced vRealize Flexible Upgrades. Read more about it here. As a result, the vRealize Suite components (other than vRealize Suite Lifecycle Manager) are no longer a part of the VCF core software package. So if there are security vulnerabilities that are discovered and relevant patches that need to be applied, the process of doing so has changed. No longer will those vRealize component software updates be delivered and applied through VCF software update bundles. Administrators now must apply them independently using vRSLCM starting from the VCF 4.4 on VxRail 7.0.320 release.

 I bring this up because there has been some vRealize Suite component security patches that have also been released that are relevant to VCF 4.4.1 on VxRail 7.0.371 deployments. See this blog post, written by my peers on the VMware team, describing the issue related to VMSA-2022-0011 and how to apply the fixes for it.

 VCF on VxRail with VMware Validated Solution enhancements

New VCF on VxRail qualification with VMware Validated Solutions

For those of you who aren’t aware, VMware Validated Solutions are technical validated implementations built and tested by VMware and VMware Partners. These solutions are designed to help customers solve common business problems using VMware Cloud Foundation as the foundational infrastructure. Types of solutions include Site Protection and Disaster Recovery for VMware Cloud Foundation using multi-site VCF deployments with stretched NSX-T networks and Advanced Load Balancing for VMware Cloud Foundation using VMware NSX Advanced Load Balancer for workloads on VCF. These validated solution designs have been enhanced over time to include VMware developed automation scripts to help customers further simplify and accelerate getting these implemented. You can learn more about them here.

 Although this solution is not directly tied to this latest VCF 4.4.1 on VxRail 7.0.371 release as a release feature itself, VMware and Dell can now qualify the VMware Validated Solutions on VCF on VxRail. All VVS solutions that are qualified will be marked with a VxRail tag. 

Figure 5: VMware Validated Solutions Portal

These solutions get updated asynchronously from VCF releases. Be sure to check the VMware VVS portal for the latest updates on existing solutions or to see when new solutions are added.

That’s a wrap

Thanks for taking the time learn more about VMware Cloud Foundation on Dell VxRail. For even more solution information, see the Additional Resources links at the bottom of this post. I don’t know about you, but I feel squeaky clean already! Can’t say the same about my outdoor landscaping though...I should probably go address that…

Author: Jason Marques

Twitter: @vWhipperSnapper

Additional Resources

VMware Cloud Foundation on Dell VxRail Release Notes

VxRail page on DellTechnologies.com

VxRail page on InfoHub

VCF on VxRail Interactive Demos

VxRail Videos