Azure Arc-enabled data services meets VxRail
Tue, 29 Jun 2021 14:00:38 -0000
|Read Time: 0 minutes
On June 29th, 2021, Microsoft and Dell Technologies announced the launch of Azure Arc-enabled data services as part of the Azure Hybrid and Multicloud Digital Event. This launch benefits several Dell Technologies product lines, including Dell EMC VxRail, Powerflex, PowerMax, and PowerStore. While my colleague, Robert Sonders, discusses the technical details of Azure Arc-enabled data services in his blog, I want to highlight the value that VxRail delivers for customers when running Azure Arc-enabled data services on VMware Tanzu Kubernetes Grid (TKG).
The figure above shows the stack transition from VxRail up through TKG, and Azure Arc-enabled data services. These lines don’t separate into hard lines between products as when you add SQL Server to a bare metal server. VxRail, specifically VMware vSphere, has to integrate at some point to TKG. This is defined in what is now called the “Holistic” model used to communicate between data formats or models. For more information, see Why Canonicalization Should Be a Core Component of Your SQL Server Modernization (Part 2). The vSphere software has key factors in running TKG, as does TKG to run the control plane with vSphere. TKG and Kubernetes (K8s) has a similar requirement when it comes to controlling Azure Arc in Kubernetes.
Azure Arc-enabled data services is an application data services layer available to developers that now provides either a SQL Managed Instance or a PostgreSQL Hyperscale Instance. With Azure Arc-enabled data services, these instances receive updates on a frequent basis, including servicing patches and new features -- similar to the experience in Azure. In addition, updates from the Microsoft Container Registry are provided to you, and you set deployment cadences in line with your own policies. Again, Rob's blog goes into much more detail about Azure Arc-enabled data services.
VMware Tanzu Kubernetes Grid (TKG) addresses challenges in the enterprise Kubernetes runtimes, allowing Kubernetes-orchestrated containers across multiple cloud infrastructures. TKG is also tested, signed, and supported by VMware. This includes signed and supported open-source applications: shared Kubernetes services that a production Kubernetes environment requires.
As of v1.3.0, TKG supports Photon OS and now Ubuntu 20.04. Adding Ubuntu 20.04 to TKG’s list of supported operating systems allows for the “bring your own node image” experience by providing different OS options when deploying consistent Kubernetes environments and containers.
VxRail provides the only fully integrated, pre-configured, and pre-tested VMware hyperconverged integrated system optimized for VMware solutions. VxRail transforms HCI networking and simplifies VMware cloud adoption while meeting any HCI use cases, including support for many of the most demanding workloads and applications. Powered by Dell EMC PowerEdge server platforms and VxRail HCI System software, VxRail features next-generation technology to future-proof your infrastructure and enable deep integration across the VMware ecosystem. The advanced VMware hybrid cloud integration and automation simplifies the deployment of a secure VxRail cloud infrastructure.
VxRail helps make this layer of the stack more efficient by simplifying management and operations. Although there is no direct VxRail integration with TKG or with Azure Arc-enabled data services applications, infrastructure admins can expand dynamically and add lifecycle infrastructure to meet the resource needs of next-generation applications developed with Azure Arc that run on TKG. With VxRail, we deliver automated Lifecycle Management (LCM), automated cluster right sizing with cluster expansions, and, as clusters may shrink to quickly deliver the right infrastructure resources for next-generation apps when needed, bringing agility to on-prem infrastructure as well as to cloud native app architectures.
Conclusion
Dell EMC VxRail provides the secure, scalable, and proven platform to build Azure Arc-enabled data services on TKG. VxRail allows enterprises to effectively execute across edge, private, public, hybrid, and multi-clouds while now adding Azure Arc-enabled data services. VxRail is the solid foundation for running TKG, which provides a consistent, upstream-compatible implementation of Kubernetes, all while delivering the Azure Arc-enabled data services as a solution.
Dell Technologies focuses on innovation and enhancing solutions, and providing validated solutions like Azure Arc-enabled data services. This solution helps your organization focus on what matters to you.
Additional Resources
VxRail white papers: https://infohub.delltechnologies.com/t/white-papers-40/
Robert Sonders’ InFocus blogs: https://infocus.delltechnologies.com/author/robert_sonders/
Azure Arc validation program: https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/validation-program
TKG 1.3.1 docs: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.3.1/rn/VMware-Tanzu-Kubernetes-Grid-131-Release-Notes.html#k8s-versions
Blog: Fuel Azure hybrid cloud with a validated infrastructure: https://www.delltechnologies.com/en-us/blog/fuel-azure-hybrid-cloud-with-a-validated-infrastructure/
Blog: Let’s get technical, and fuel your Azure hybrid cloud with Azure Arc: https://infocus.delltechnologies.com/robert_sonders/lets-get-technical-and-fuel-your-azure-hybrid-cloud-with-azure-arc/
Related Blog Posts
Microsoft SQL Server Big Data Clusters on Tanzu Kubernetes Grid on Dell EMC VxRail
Thu, 19 Aug 2021 15:25:13 -0000
|Read Time: 0 minutes
A recently created reference architecture, running Microsoft SQL Server Big Data Clusters (BDC) on Tanzu Kubernetes Grid (TKG) on Dell EMC VxRail, demonstrates a fast and simple way to get started with big data workloads running on Kubernetes. It also shows how the containerized workloads ran using VxRail.
SQL BDC on TKG on VxRail enables simplified servicing for cloud native workloads, and is designed to scale with business needs. Administrators can implement the policies for namespaces and manage access and quota allocation for application-focused management. All of this helps build a developer ready infrastructure with enterprise-grade Kubernetes with advanced governance, reliability, and security.
This reference architecture also validated SQL BDC with Spark SQL TPC-DS benchmark optimized parameters. The test results showed that Tanzu Kubernetes Grid on VxRail provides linear scalability (for complex TPC-DS-like decision support workloads that use different query types) with predictable query response time and high throughput.
In the business value section for using SQL BDC and TKG on VxRail, based on the five measurements below. It's covered in more detail within the reference architecture.
- Simplified installation of Kubernetes
- Automated multi-cluster operations
- Integrated platform services
- Open source alignment
- Production Ready
Cross-functional teams from Dell EMC VxRail, VMware, and Microsoft have reviewed the reference architecture for content and supportability. This can provide comfort for those wanting to run on Tanzu. Some notes from Microsoft Release notes from Cumulative Update 12 (CU12) of BDC):
SQL Server Big Data Clusters is supported as a workload. Microsoft provides support for the software components on the containers installed and configured by SQL Server Big Data Clusters only. Kubernetes itself, and other containers that may influence SQL Server Big Data Clusters behavior, are not supported by the (Microsoft) support team. For Kubernetes support please contact your certified Kubernetes distribution provider.
Note: This reference architecture provides general design and deployment guidelines of running Microsoft SQL Server Big Data Clusters on VMware Tanzu™ Kubernetes Grid™ on Dell EMC VxRail. The reference architecture also applies to any compatible hardware platforms running VMware Tanzu Kubernetes Grid on vSAN™.
To wrap up, VxRail provides SQL BDC on Tanzu as a scalable and secure platform to deliver key business outcomes. This reference architecture highlights one of the first known support solutions built on Tanzu Kubernetes Grid to manage Kubernetes. The paper covers the spectrum on the build, testing, and expected performance on VxRail.
Resources:
- Running Microsoft SQL Server Big Data Clusters on VMware Tanzu Kubernetes Grid
- SQL Server Big Data Clusters platform release notes - SQL Server Big Data Clusters | Microsoft Docs It's reformulated release notes, now simpler and focused on tested configurations and documenting known issues.
- SQL Server Big Data Clusters CU12 release notes - SQL Server Big Data Clusters | Microsoft Docs For every release starting with CU12, we will provide a dedicated article containing many details about what's new, fixes, and most importantly, an extensive list of what's under the hood.
- SQL Server Big Data Clusters cumulative updates history - SQL Server Big Data Clusters | Microsoft Docs In this article, we will keep track of all previous update details.
Author: Vic Dery – Linkedin
Big Solutions on Dell EMC VxRail with SQL 2019 Big Data Cluster
Mon, 17 Aug 2020 18:31:31 -0000
|Read Time: 0 minutes
The amount of data and different formats organizations must manage, ingest, and analyze has been the driving force behind Microsoft SQL 2019 Big Data Clusters (BDC). SQL Server 2019 BDC demonstrates the deployment of scalable clusters of SQL Server, Spark, and containerized HDFS (Hadoop Distributed File System) running on Kubernetes.
We recently deployed and tested SQL Server 2019 BDC on Dell EMC VxRail hyperconverged infrastructure to demonstrate how VxRail delivers the performance, scalability, and flexibility needed to bring these multiple workloads together.
The Dell EMC VxRail platform was selected for its ability to incorporate compute, storage, virtualization, and management in one platform offering. The key feature of the VxRail HCI is the integration of vSphere, vSAN, and VxRail HCI System Software for an efficient and reliable deployment and operations experience. The use of VxRail with SQL Server 2019 BDC makes it easy to unite relational data with big data.
The testing demonstrates the advantages of using VxRail with SQL Server 2019 BDC for analytic application development. This also demonstrates how Docker, Kubernetes, and the vSphere Container Storage Interface (CSI) driver accelerate the application development life cycle when they are used with VxRail. The lab environment for development and testing used four VxRail E560F nodes supported by the vSphere CSI driver. With this solution, developers can provision SQL Server BDC in containerized environments without the complexities of traditional methods for installing databases and provisioning storage.
Our white paper, Microsoft SQL Server 2019 Big Data Cluster on Dell EMC VxRail shows the power of implementing SQL Server 2019 BDC technologies on VxRail. Integrating SQL Server 2019 RDBMS, SQL Server BDC, MongoDB, and Oracle RDBMS helps to create a unified data analytics application. Using VxRail enhances the ability of SQL Server 2019 to scale out storage and compute clusters while embracing the virtualization techniques from VMware. This SQL Server 2019 BDC solution also benefits from the simplicity of a complete yet flexible validated Dell EMC VxRail with Kubernetes management and storage integration.
The solution demonstrates the combined value of the following technologies:
- VxRail E560F – All-flash performance
- Large tables stored on a scaled-out HDFS storage cluster that is hosted by BDC
- Smaller related data tables that are hosted on SQL Server, MongoDB, and Oracle databases
- Distributed queries that are enabled by the PolyBase capability in SQL Server 2019 to process Transact-SQL queries that access external data in SQL Server, Oracle, Teradata, and MongoDB.
- Red Hat Enterprise Linux
Big Data Cluster Services
This diagram shows how the pools are built. It provides details of the benefits for Kubernetes features for container orchestration at scale, including:
- Autoscaling, replication, and recovery of containers
- Intracontainer communication, such as IP sharing
- A single entity—a pod—for creating and managing multiple containers
- A container resource usage and performance analysis agent, cAdvisor
- Network pluggable architecture
- Load balancing
- Health check service
This white paper, Microsoft SQL Server 2019 Big Data Cluster on Dell EMC VxRail, addresses big data storage, the tools for handling big data, and the details around testing with TPC-H. When we tested data virtualization with PolyBase, the queries were successful, running without error and returning the results that joined all four data sources.
Because data virtualization does not involve physically copying and moving the data (so that the data is available to business users in real-time), BDC simplifies and centralizes access to and analysis of the organization’s data sphere. It enables IT to manage the solution by consolidating big data and data virtualization on one platform with a proven set of tools.
Success starts with the right foundation:
SQL Server 2019 BDC is a compelling new way to utilize SQL Server to bring high-value relational data and high-volume big data together on a unified, scalable data platform. All of this can be deployed with VxRail, enabling enterprises to experience the power of PolyBase to virtualize their data stores, create data lakes, and create scalable data marts in a unified, secure environment without needing to implement slow and costly Extract, Transform, and Load (ETL) pipelines. This makes data-driven applications and analysis more responsive and productive. SQL Server 2019 BDC and Dell EMC VxRail provide a complete unified data platform to deliver intelligent applications that can help make any organization more successful.
Read the full paper to learn more about how Dell EMC VxRail with SQL 2019 Big Data Clusters can:
- Bring high-value relational data and high-volume big data together on a single, scalable platform.
- Incorporates intelligent features and gets insights from more of your data—including data stored beyond SQL Server in Hadoop, Oracle, Teradata, and MongoDB.
- Supports and enhances your database management and data-driven apps with advanced analytics using Hadoop and Spark.
Additional VxRail & SQL resources:
Microsoft SQL Server 2019 Big Data Cluster on Dell EMC VxRail
Microsoft SQL Server on VMware Cloud Foundation on Dell EMC VxRail
Key Benefits of Running Microsoft SQL Server on Dell EMC hyperconverged infrastructure (HCI) - Whitepaper
Key benefits of running Microsoft SQL Server on Dell EMC Hyperconverged Infrastructure (HCI) - Infographic
Architecting Microsoft SQL Server on VMware vSphere
Author: Vic Dery, Senior Principal Engineer, VxRail Technical Marketing