Dell Technologies tested the Delta Lake modern data stack that was deployed on the Red Hat OpenShift Container Platform. Dell Technologies also tested integrating Dell PowerScale, Dell ECS, and Dell ObjectScale software-defined storage with Delta Lake. This platform enables the running of various data analytics applications such as Spark, Kafka, and Starburst on OpenShift Container Platform with different storage systems.
Dell Technologies used Delta Lake 2.4.0 and 3.0.0rc1 with Spark 3.4.1 to validate all modern data stack functionality. All the features of Delta Lake with Spark were validated, including:
- Atomicity, Consistency, Isolation, and Durability (ACID) transactions
- Scalable metadata handling
- Updates, deletes, and merges
- Audit and time travel
- Schema enforcement and evolution
- Unified batch and streaming