Introducing the Latest Release of Dell ECS 3.8.1
Tue, 02 Apr 2024 18:32:26 -0000
|Read Time: 0 minutes
IDC predicts that the Global Datasphere will grow to 221 zettabytes by 2026, more than 90% of which is unstructured in nature. ECS, the leading enterprise-class object storage platform from Dell Technologies, has been engineered to support both traditional and next-generation workloads. ECS delivers the capabilities of the public cloud with the command and control of a private cloud infrastructure as an S3-compatible, globally scalable object store.
New Features in ECS 3.8.1
The latest release of ECS 3.8.1 introduces a range of innovative features and enhancements that make significant strides in advancing the field of enterprise object storage solutions.
Azure AD OBO Support
Today more customers are moving to Azure AD, and more apps are using OIDC (OpenID Connect), so that they can talk to a service provider like ECS that supports SAML (Security Assertion Markup Language). Apps in this environment are using an Azure AD On Behalf Of (OBO) workflow to exchange their OIDC token for a SAML assertion. With the support of this new workflow, our customers can integrate their S3 applications to authenticate identity.
The OAuth 2.0 On-Behalf-Of flow (OBO) is ideal for use cases where an application invokes a service/web API and needs to call another service/web API. The idea is to propagate the delegated user identity and permissions through the request chain. For the middle-tier service to make authenticated requests to the downstream service, it needs to secure an access token from the Microsoft identity platform, on behalf of the user.
ECS IAM features S3 work with SAML identity providers to handle authentication and SAML Assertion generation. It provides the following for applications:
- Authenticate with an identity provider and if successful receive a SAML Assertion
- Use the SAML Assertion to make a call to the ECS Secure Token Service (STS) API (AssumeRoleWithSAML Method) to retrieve a temporary set of credentials thus allowing the caller to assume a role
- Perform ECS API calls that the role allows using the temporary credentials.
Note: In the SAML model there are two main roles for a participant: Identify Provider and Service Provider. Based on the ECS IAM SAML design ECS acts as the Service Provider and Azure AD acts as the Identity Provider and generates SAML Assertions.
HDFS Deprecated in ECS
Starting from ECS 3.8.1, we will remove the HDFS support, because customers have moved to ECS for Hadoop by S3a. We made the below changes:
- Disabled HDFS head port (9040) on ECS
- Removed CMF (Configuration Framework) support for port enable/disable with HDFS
- Discontinued Datahead service on port 9040
- Removed HDFS client jar file from downloads
Note: When ECS is upgraded to 3.8.1, HDFS will stop working if it is being used.
New CAS Consistency Option
ECS uses strong consistency, and object concurrent conflicts are resolved by redirecting all object operations to the object owner. However, operations might experience additional latency if an object or bucket owner is at a remote VDC. The is a problem for some CAS applications that are very sensitive to read latency.
Starting from ECS 3.8.1, we introduce a new feature which allows customers to set a CAS bucket with a new consistency mode. The greatest benefit to customers is improving read latency when an object or bucket owner is at a remote VDC (virtual Data Center). With the new feature enabled, read performance should improve, because ECS will no longer check the source bucket or object owner that strong consistency typically requires.
Below is the UI configuration page to create a CAS bucket with the CAS consistency mode. The new CAS consistency only supports the CAS buckets which enable ADO RW. The create, update, and delete operations are still redirected to object owner zone.
Simplified Bucket Deletion
The task of deleting a bucket is simplified by incorporating the object deletion process. Customers no longer need to empty a bucket prior to requesting a bucket deletion. Through the user interface, a customer may delete a bucket in ECS, even if it is not empty.
A new UI is introduced about deleting bucket dialog as below. During S3 bucket deleting,
- Bucket access is set to read only.
- No property changes on bucket are allowed.
- MPUs are aborted.
- Object/versions are removed.
- User permissions, object lock, governance, compliance, retention, and legal hold are honored during delete.
- Once all objects are deleted, the bucket will be removed.
- If objects cannot be deleted, the bucket will be put back into writable state.
A filesystem enabled bucket is also supported in the simplified bucket deletion feature. NFS exports must be removed before deleting. During the deletion process, NFS access will not be allowed (read or write).
Fabric Improvements with Mixed Memory Cluster
The ECS fabric improvements with mixed memory cluster features resolves the problem where 192 GB nodes get set to a 64 GB profile. This occurred when certain service procedures were run like node expansions, node replacement or software upgrades. With this improvement, service procedures leave the allocated memory profile aligned to what's available physically on the node.
Conclusion
Dell ECS offers a sophisticated solution for deploying and managing enterprise-grade object storage. With its well-designed architecture and robust protective features, it presents a compelling option for organizations in pursuit of flexibility, scalability, performance, and security in their object storage solutions.
Please refer to the ECS 3.8.1 release note for more information about the new features.