Part 2 – The ‘What’ - Introducing Dell Container Storage Modules (CSM)
Fri, 19 Nov 2021 14:17:13 -0000
|Read Time: 0 minutes
In the first post of the series, which you can read all about here, I discussed some of the challenges that are associated with managing the storage / Data Protection aspects of Kubernetes. Now, let’s discuss our solutions:
Enter CSM, or Introduction to Container Storage Modules
Remember the 2019 session and the in-depth thinking we had gone through about our customers’ real world needs? The Kubernetes ecosystem is growing rapidly and when it comes to storage integration, CSI plugins offer a way to expose block and file storage systems to containerized workloads on Container Orchestration systems (COs) like Kubernetes.
Container Storage Modules (CSM) improves the observability, usability, and data mobility for stateful applications using Dell Technologies storage portfolio. It also extends Kubernetes storage features beyond what is available in the Container Storage Interface (CSI) specification. CSM and the underlying CSI plugins are pioneering application-aware/application consistent backup and recovery solutions from the most comprehensive enterprise-grade storage and data protection for Kubernetes.
CSM extends enterprise storage capabilities to Kubernetes. It reduces management complexity so developers can independently consume enterprise storage with ease and automate daily operations such as provisioning, snapshotting, replication, observability, authorization, and resiliency. CSM is open-source and freely available from https://github.com/dell/csm.
Dell EMC Container Storage Modules (CSM) brings powerful enterprise storage features and functionality to Kubernetes for easier adoption of cloud-native workloads, improved productivity, and scalable operations. This release delivers software modules for storage management that provide developers with access to build automation for enhanced IT needs and other critical enterprise storage features. These include data replication across data centers, role-based access control (RBAC) authorization, observability, and resiliency for disaster recovery and avoidance. Improved resource utilization enables automated access to any of our portfolio storage systems into K8s environments and:
- Gives the flexibility to choose whatever in the back end allows them to provision and leverage the strengths of the individual system
- Flexible + simple = powerful
- You have storage that isn’t 100% utilized
This enables the K8 environment manager to directly allocate storage and services, and it:
- Reduces time
- Gives them the pot to pull things out of and then lets them go handle it
- Frees up the developer to develop
Extend Enterprise Storage to Kubernetes – by accelerating adoption of cloud-native workloads with proven enterprise storage with proven enterprise storage:
- Dell EMC Container Storage Modules (CSM) enables a high-performing and resilient enterprise storage foundation for Kubernetes.
- CSM delivers a full stack of enterprise capabilities such as industry-leading replication, authorization, failure recovery, and management. These capabilities accelerate deployment testing, resulting in a faster application deployment life cycle.
- CSM allows developers and storage admins to take advantage of the unique benefits of Dell EMC storage systems, such as PowerMax Metro smart DR and the PowerFlex software-defined storage architecture.
- Dell Technologies has purpose-built platforms for streaming data, IoT, and Edge computing use cases designed with container-based architecture and management. These capabilities accelerate deployment testing, resulting in a faster application deployment life cycle.
Empower Developers – Improve productivity by reducing development life cycles
- CSM reduces storage management complexity with observability modules so developers can consume enterprise storage with ease.
- It also provides a complete K8s solution stack that delivers an integrated experience for developers and storage admins.
- Customers will be able to take advantage of consistent monitoring, management, and policy enforcement across enterprise storage and DevOps environments.
Automate storage operations – Integrate enterprise storage with existing Kubernetes toolsets for scalable operations
- CSM allows customers to realize the promise of infrastructure as code for frictionless data collection and consumption
- CSM observability empowers customers to create storage pools across multiple storage arrays for minimal storage management
- CSM delivers an integrated experience that bridges the gap between Kubernetes admins/developers and the traditional IT admins, furthering solidifying enterprise storage’s role as a viable alternative to public cloud while eliminating silos and shadow IT.
The modules are separated into these six specific capabilities:
Observability – Delivers a single pane to view the whole CSM environment for the K8s/container administrator, using Grafana and Prometheus dashboards that K8s admins are familiar with in monitoring persistent storage performance.
Replication – Enables array replication capabilities to K8s users with support for stretched and replica K8s clusters.
Authorization – Provides storage and provides Kubernetes administrators the ability to apply RBAC and usage rules for our CSI Drivers.
Resiliency – Enables K8s node failover by monitoring persistent volume health, designed to make Kubernetes Applications, including those that use persistent storage, more resilient to node failures. The module is focused on detecting node failures (power failure), K8s control plane network failures, and Array I/O network failures, and to move the protected pods to hardware that is functioning correctly.
Volume Placement – Intelligent volume placement for Kubernetes workloads, optimized based on available capacity.
Snapshots - CSI based snapshots for operational recovery and data repurposing. The Snapshots feature is part of the CSI plugins of the different Dell EMC arrays and takes advantage of the state-of-the-art snapshot technology to protect and repurpose data. In addition to point-in-time recovery, these snapshots are writable and can be mounted for test and dev and analytics use cases without impacting the production volumes. These modules are planned for RTS, but there is a rolling release prioritized based upon customer demand by storage platform – applicable to PowerScale, PowerStore, PowerMax, PowerFlex, and Unity XT. Available on RTS:
- Authorization Module
- PowerScale
- PowerMax
- PowerFlex
- Resiliency Module
- PowerFlex
- Unity XT
- Observability Module
- PowerFlex
- PowerStore
- Replication Module
- PowerMax Metro/Async
- One Installer
The publicly accessible repository for CSM is available at https://github.com/dell/csm. For a complete set of material on CSM, see the documentation at https://dell.github.io/csm-docs/.
Here is an overview demo of CSM:
Watched it? Awesome, now let’s go deeper into the modules:
Observability
CSM for Observability is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell EMC products. It is an OpenTelemetry agent that collects array-level metrics for Dell EMC storage so they can be scraped into a Prometheus database. With CSM for Observability, you will gain visibility not only on the capacity of the volumes/file shares you manage with Dell CSM CSI (Container Storage Interface) drivers but also their performance in terms of bandwidth, IOPS, and response time. Thanks to pre-packaged Grafana dashboards, you will be able to go through these metrics’ history and see the topology between a Kubernetes PV (Persistent Volume) and its translation as a LUN or file share in the backend array. This module also allows Kubernetes admins to collect array level metrics to check the overall capacity and performance directly from the Prometheus/Grafana tools rather than interfacing directly with the storage system itself. Metrics data is collected and pushed to the OpenTelemetry Collector, so it can be processed and exported in a format consumable by Prometheus.
CSM for Observability currently supports PowerFlex and PowerStore. Its key high-level features are:
- Collect and expose Volume Metrics via the OpenTelemetry Collector
- Collect and expose File System Metrics via the OpenTelemetry Collector
- Collect and expose export (K8s) node metrics via the OpenTelemetry Collector
- Collect and expose filesystem capacity metrics via the OpenTelemetry Collector
- Collect and expose block storage capacity metrics via the OpenTelemetry Collector
- Non-disruptive config changes
- Non-disruptive log level changes
- Grafana Dashboards for displaying metrics and topology data
Below, you can see the module, working with PowerStore:
And PowerFlex:
The publicly accessible repository is available at https://github.com/dell/csm-observability.
See documentation for a complete set of material on CSM Observability: https://dell.github.io/csm-docs/docs/observability/.
Replication
CSM for Replication is the module that allows provisioning of replicated volumes using Dell storage. CSM for Replication currently supports PowerMax and PowerStore.
Key High-Level Features:
- Replication of PersistentVolumes (PV) across Kubernetes clusters Multi/single cluster topologies
- Replication action execution (planned/unplanned failover, sync, pause, resume)
- Async/Sync/Metro configurations support (PowerStore only supports Async)
- repctl – CLI tool that helps with replication related procedures across multiple K8s clusters
The publicly accessible repository for CSM is available at https://github.com/dell/csm-replication.
See the documentation for a complete set of material on CSM Replication: https://dell.github.io/csm-docs/docs/replication/.
The following video includes an Introduction and the Architecture (using PowerMax as the example):
Below, you can see end-to-end demos on how to configure CSM replication for PowerStore, and how to perform failover & failback operations of WordPress and MySQL DB, using PowerStore Async replication.
Installing:
Performing Failover & Failback (Reprotect):
Using PowerMax?
- The following video shows synchronous replication using CSM Replication for PowerMax SRDF Sync Replication with File I/O being generated.
- This video shows Active-Active High-Availability using CSM Replication for PowerMax SRDF Metro Volumes with PostgreSQL:
Authorization
- CSM for Authorization is part of the CSM (Container Storage Modules) open-source suite of Kubernetes storage enablers for Dell EMC products. CSM for Authorization provides storage and Kubernetes administrators the ability to apply RBAC for CSI Drivers. It does this by deploying a proxy between the CSI driver and the storage system to enforce role-based access and usage rules.
- Storage administrators of compatible storage platforms will be able to apply quota and RBAC rules that instantly and automatically restrict cluster tenants’ usage of storage resources. Users of storage through CSM for Authorization do not need to have storage admin root credentials to access the storage system.
- Kubernetes administrators will have an interface to create, delete, and manage roles/groups to which storage rules may be applied. Administrators and/or users can then generate authentication tokens that can be used by tenants to use storage with proper access policies being automatically enforced.
- CSM for Authorization currently supports PowerFlex, PowerMax, and PowerScale.
Its key high-level features are:
- Ability to set storage quota limits to ensure K8s tenants are not over consuming storage
- Ability to create access control policies to ensure K8s tenant clusters are not accessing storage that does not belong to them
- Ability to shield storage credentials from Kubernetes administrators, ensuring that credentials are only handled by storage admins
The publicly accessible repository is available at https://github.com/dell/csm-authorization.
See the documentation for a complete set of material on CSM Authorization: https://dell.github.io/csm-docs/docs/authorization/.
Below, you can see the Authorization module for PowerFlex:
Resiliency
User applications can have problems if you want their Pods to be resilient to node failure. This is especially true of those deployed with StatefulSets that use PersistentVolumeClaims. Kubernetes guarantees that there will never be two copies of the same StatefulSet Pod running at the same time and accessing storage. Therefore, it does not clean up StatefulSet Pods if the node executing them fails.
CSM for Resiliency currently supports PowerFlex and Unity.
Key High-Level Features:
- Detect pod failures for the following failure types – Node failure, K8s Control Plane Network failure, Array I/O Network failure
- Cleanup pod artifacts from failed nodes
- Revoke PV access from failed nodes
Below, you can see a demo of the Resiliency module for PowerFlex:
The publicly accessible repo is available at https://github.com/dell/karavi-authorization.
See the documentation for a complete set of material on CSM Resiliency: https://dell.github.io/csm-docs/docs/authorization/.
The Snapshots feature is part of the CSI plugins of the different Dell EMC arrays and takes advantage of the state-of-the-art snapshot technology to protect and repurpose data. In addition to point-in-time recovery, these snapshots are writable and can be mounted for test and dev and analytics use cases without impacting the production volumes.
See the following demo about volume groups snapshots for PowerFlex:
No man (or a customer) is an island and Kubernetes comes in many flavors. Here at Dell Technologies, we offer a wide variety of solutions for the customer, starting from just storage arrays for every need (from PowerStore to PowerFlex to PowerMax to PowerScale and ECS) to turnkey solutions like VxRail with/without VCF, deep integration with our storage arrays to anything from upstream Kubernetes to RedHat Openshift, with deep integration to the OpenShift Operator, or vSphere with Tanzu, just so we can meet you where you are today AND tomorrow.
With Dell Technologies’ broad portfolio designed for modern and flexible IT growth, customers can employ end-to-end storage, data protection, compute, and open networking solutions that support rapid container adoption. Developers can create and integrate modern data applications by relying on accessible open-source integrated frameworks and tools across bare metal, virtual, and containerized platforms. Dell enables support for organizational autonomy and real-time benefits for container and Kubernetes platforms with and adherence to IT best practices based on an organization’s own design needs.
In the next post, we will be covering the ‘How’ to install the new CSI 2.0 Common installer and the CSM modules.
Related Blog Posts
Introducing Dell Container Storage Modules (CSM), Part 1 - The 'Why'
Fri, 19 Nov 2021 14:17:13 -0000
|Read Time: 0 minutes
Dell Tech World 2019, yea, the days of actual in-person conferences, Michael Dell is on stage and during his keynote, he says “we are fully embracing Kubernetes”. My session is the next one where I explain our upcoming integration of storage arrays with the Kubernetes CSI (Container Storage Interface) API. Now, don’t get me wrong, CSI is awesome! But at the end of my session, I’m getting a lot of people coming to me and ask very similar questions, the theme was around ‘how do I still keeping track of what’s going to happen in the storage array’, you see, CSI doesn’t have role-based access to the storage array, not to even mention things like quota management. At a very high level, think about storage admins that want to embrace Kubernetes but are afraid to lose control of their storage arrays. If ‘CSI’ feels like a name of a TV show, I encourage you to stop here and go ahead and have some previous reads in my blog about it: https://volumes.blog/?s=csi Back to 2019. Post my session, I gathered a team of product managers and we started to think about upcoming customer’s needs, we didn’t have to use a crystal ball but rather, as the largest storage company in the world, started to interview customers about their upcoming needs re K8s. Now, let’s take a step back and discuss the emergence of cloud-native apps and Kubernetes.
In the past, companies would rely on Waterfall development and ITIL change management operational practices. This meant organizations had to plan for:
- Long Development cycles before handing an application to ops
- IT ops often resisting change and slow innovation
Now companies want to take advantage of a new development cycle called Agile along with DevOps operational practices. This new foundation for IT accelerates innovation through:
- Rapid iteration and quick releases
- Collaboration via involving the IT ops teams throughout the process
Operational practices aren’t the only evolving element in today’s enterprises; application architectures are quickly changing as well. For years, monolithic architectures were the standard for application architectures. These types of applications had great power and efficiency and run on virtual machines. However, these applications have proven costly to reconfigure, update, and take a long time to load. In cloud-native applications, components of the app are segmented into microservices, which are then bundled and deployed via containers. This container/microservice relationship allows cloud-native apps to be updated and scaled independently. To manage these containerized workloads, organizations use an open-source management platform called Kubernetes. To give a real-world example, imagine monolithic apps like a freight train – there is a lot of power and capacity but it takes a long time to load and is not easy to reconfigure. Whereas cloud-native apps function more like a fleet of delivery vehicles with reduced capacity but resilient and flexible in changing the payload or adapting capacity as needed. A fleet of delivery vehicles needs a conductor to schedule and coordinate the service, and that is the role that Kubernetes plays for containers in a cloud-native environment. Both approaches are present in today’s modern apps but the speed and flexibility of cloud-native apps shifting priorities everywhere.
Let’s dig more into this shift in software development and delivery. Leading this shift is the use of microservices, which are loosely coupled components that are self-contained, highly available, and easily deployable, and with containers that provide these microservices with lightweight packages capable of resource utilization efficiencies, enable those microservices patterns. They provide a ‘build once, run anywhere flexibility with the scale that developers are embracing. Then came Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It has become the industry “go-to” for more service discovery, load balancing, storage orchestration. With agile development comes the need for speed and continuous delivery which, with the right tools and infrastructure can create the right business outcomes as demands increase. With the advent of flexible cloud-native applications; DevOps teams formed and created their own agile frameworks that in addition to increasing delivery of code with less dysfunction and overhead of traditional models whereby intentionally or unintentionally bypassing IT Operations’ best practices and the opportunity to build modern IT infrastructures to support their development initiatives, as well as enhance them.
As traditional models for software development evolve, so does the infrastructure that supports it. IT Operations’ best practices can be applied to these new models through the Enterprise level data management tools that Dell Technologies’ provides. DevOps teams require seamless, non-disruptive, and reliable mechanisms to continue to meet business demands with agility and scale. With Dell Technologies” broad portfolio designed for modern and flexible IT growth, customers can employ end-to-end storage, data protection, compute and open networking solutions that support accelerated container adoption. Developers can create and integrate modern applications by relying on accessible open-source integrated frameworks and tools across bare metal, virtual, and containerized platforms. Dell enables support for DevOps elasticity and real-time benefits for container and Kubernetes platforms’ applying best practices based on their own design and needs.
Dell Technologies aligns developers and IT operations, empowering them to design and operate cloud-native organizations while achieving business demands and increasing quality outputs. With the support of industry standards built on containers such as Containers’ storage interfaces, Plug-ins with container storage modules, PowerProtect data manager can Availability is the most important aspect of data that customers and different levels of business ultimately care about from about every angle; especially securely accessed data whether it be on-premises, in the cloud. Though developers seem to claim they understand Kubernetes inside and out, they miss out on features at the IT operations level that we can provide. With a big portfolio such as ours, we must understand what maturity level the customer is in. For the storage administrator, they will defer using their PowerMax or VxRail; if they want to continue to purchase these products, they would appreciate built-in containers/Kubernetes support that is easy to onboard without disrupting their developers. At the application layer, you may be employing Kubernetes or OpenShift well into the software-defined journey and PowerFlex would be an optional choice. GitHub CSI downloads exceed 1 million downloads. Kubernetes developers know nothing about storage except local storage servers and drives; whereby their operational partners care about resiliency, snapshot, restore, replication, compression, and security. With the variety of storage solutions, having CSI plug-ins and Container Storage Modules simplifies deployment choices, emphasis on applying operational best practices.
Build:
- Cloud-Native Computing Foundation (CNCF) SIG contributor
- Complete E2E integrated industry-standard APIs
- Self-service CSI driver workflows
- GitHub repository for developers
- Partner integrations with VMware Tanzu, Red Hat OpenShift, Google Anthos, Rancher, others
- DevOps and IaC integration with Ansible, Terraform, Puppet, ServiceNow, vRO, Python, Powershell, etc.
- Kubernetes Certified Service Provider (KCSP) Consultant Services
Automate & Manage:
- Container storage modules (CSM)
- Data replication across data centers
- RBAC authorization
- Observability
- Resiliency (disaster recovery & avoidance)
- Single platform, Kubernetes & application-aware data protection
- Application consistent backups
- MySQL, MongoDB, Cassandra, Postgres, etc.
- Infrastructure Automation & Lifecycle Management
- API driven software-defined infrastructure with automated lifecycle management
- Policy-based protection
- Replication, retention, tiering to S3-compatible object storage, SLA reporting
- Provide in-cloud options for developers with support for AWS, Azure backup policies
Scale & Secure:
- Provisioning and automating policies
- Extract data value in real-time through open networking and server/compute
- Deploy data protection backup and restores via PowerProtect Data Manager
- Integrated Systems; VxBlock, VxRail, PowerFlex, Azure Stack
- Manage Kubernetes with PowerScale in multi-cloud environments
- Accelerate with edge / bare metal via Kubespray / Streaming Data Platform (SDP) w/ Ready Stack for Red Hat OpenShift platforms
- Obtain seamless security and secure critical data via CloudLink
Ok, let’s talk Kubernetes.
Kubernetes is really starting to pick up, as you can see in the above graphs, by 2025, it is expected that up to 70% of the enterprises out there, will be using Kubernetes AND that, 54% will be deployed primarily in their production environments! Yep, that means, we are way beyond the ‘Kicking the tires’ phase. A few weeks ago, I talked with my manager about these trends which you can see below.
BUT, it’s not all rosy, Kubernetes provides a lot of challenges, to name a few:
Lack of internal alignment…shadow IT results… which leads to a harder job for the IT admins with lack of visibility and monitoring, and meeting security and compliance requirements. Kubernetes also cannot automatically guarantee that resources are properly allocated between different workloads running in a cluster. To set that up, you need to set up resource quotas manually. The opportunity is to align developers and IT operations by empowering them to design and operate cloud-native organizations while achieving business demands and increasing quality outputs.
In the next post, I will share the ‘What’ are we releasing to tackle these challenges...
Looking Ahead: Dell Container Storage Modules 1.2
Mon, 29 Apr 2024 18:11:07 -0000
|Read Time: 0 minutes
The quarterly update for Dell CSI Drivers & Dell Container Storage Modules (CSM) is here! Here’s what we’re planning.
CSM Features
New CSM Operator!
Dell Container Storage Modules (CSM) add data services and features that are not in the scope of the CSI specification today. The new CSM Operator simplifies the deployment of CSMs. With an ever-growing ecosystem and added features, deploying a driver and its affiliated modules need to be carefully studied before beginning the deployment.
The new CSM Operator:
- Serves as a one-stop-shop for deploying all Dell CSI driver and Container Storage Modules
- Simplifies the install and upgrade operations
- Leverages the Operator framework to give a clear status of the deployment of the resources
- Is certified by Red Hat OpenShift
In the short/middle term, the CSM Operator will deprecate the experimental CSM Installer.
Replication support with PowerScale
For disaster recovery protection, PowerScale implements data replication between appliances by means of the the SyncIQ feature. SyncIQ replicates the data between two sites, where one is read-write while the other is read-only, similar to Dell storage backends with async or sync replication.
The role of the CSM replication module and underlying CSI driver is to provision the volume within Kubernetes clusters and prepare the export configurations, quotas, and so on.
CSM Replication for PowerScale has been designed and implemented in such a way that it won’t collide with your existing Superna Eyeglass DR utility.
A live-action demo will be posted in the coming weeks on our VP YouTube channel: https://www.youtube.com/user/itzikreich/.
CSI features
Across the portfolio
In this release, each CSI driver:
- Supports OpenShift 4.9
- Supports Kubernetes 1.23
- Supports the CSI Spec 1.5
- Updates the latest UBI-minimal image
- Supports fsGroupPolicy
fsGroupPolicy support
Kubernetes v1.19 introduced the fsGroupPolicy to give more control to the CSI driver over the permission sets in the securityContext.
There are three possible options:
- None -- which means that the fsGroup directive from the securityContext will be ignored
- File -- which means that the fsGroup directive will be applied on the volume. This is the default setting for NAS systems such as PowerScale or Unity-File.
- ReadWriteOnceWithFSType -- which means that the fsGroup directive will be applied on the volume if it has fsType defined and is ReadWriteOnce. This is the default setting for block systems such as PowerMax and PowerStore-Block.
In all cases, Dell CSI drivers let kubelet perform the change ownership operations and do not do it at the driver level.
Standalone Helm install
Drivers for PowerFlex and Unity can now be installed with the help of the install scripts we provide under the dell-csi-installer directory.
A standalone Helm chart helps to easily integrate the driver installation with the agent for Continuous Deployment like Flux or Argo CD.
Note: To ensure that you install the driver on a supported Kubernetes version, the Helm charts take advantage of the kubeVersion field. Some Kubernetes distributions use labels in kubectl version (such as v1.21.3-mirantis-1 and v1.20.7-eks-1-20-7) that require manual editing.
Volume Health Monitoring support
Drivers for PowerFlex and Unity implement Volume Health Monitoring.
This feature is currently in alpha in Kubernetes (in Q1-2022), and is disabled with a default installation.
Once enabled, the drivers will expose the standard storage metrics, such as capacity usage and inode usage through the Kubernetes /metrics endpoint. The metrics will flow natively in popular dashboards like the ones built-in OpenShift Monitoring:
Pave the way for full open source!
All Dell drivers and dependencies like gopowerstore, gobrick, and more are now on Github and will be fully open-sourced. The umbrella project is and remains https://github.com/dell/csm, from which you can open tickets and see the roadmap.
Google Anthos 1.9
The Dell partnership with Google continues, and the latest CSI drivers for PowerScale and PowerStore support Anthos v1.9.
NFSv4 POSIX and ACL support
Both CSI PowerScale and PowerStore now allow setting the default permissions for the newly created volume. To do this, you can use POSIX octal notation or ACL.
- In PowerScale, you can use plain ACL or built-in values such as private_read, private, public_read, public_read_write, public or custom ones;
- In PowerStore, you can use the custom ones such as A::OWNER@:RWX, A::GROUP@:RWX, and A::OWNER@:rxtncy.
Useful links
For more details you can:
- Watch these great CSM demos on our VP YouTube channel: https://www.youtube.com/user/itzikreich/
- Subscribe to Github notification and be informed of the latest releases on: https://github.com/dell/csm
- Ask for help or chat with us on Slack
Author: Florian Coulombel