It is useful to understand what features and components are net new in CDP Private Cloud Base. It is also helpful to know what is new and changed when coming from the legacy CDH or HDP releases.

While this topic describes what is in the release, more information about the journey to CDP Private Cloud Base, including the upgrade and migration pathways, is described in The journey to CDP Private Cloud Base.

There are several new features and capabilities that have been included for the first time in CDP as compared to the former CDH and HDP products. These features are over and above what was in prior CDH and HDP releases, and therefore are new to all users who deploy CDP Private Cloud Base. These new features include:

Atlas 2.0

Includes advanced data discovery, metadata catalog and search, data lineage and chain of custody, metadata audit, and support for improved security. Also includes support for Spark.

Enhanced security capabilities

Includes encryption with Ranger KMS-Key Trustee integration, and Navigator Encrypt (Navencrypt) for secure data at rest.

Streaming Services

Introduced with the addition of Kafka and related components, includes cluster management and replication for Kafka clusters, storage and schema through the schema registry service, and the ability to rebalance clusters with Cruise Control. Also includes support for Kafka Connect, which enables you to connect HDFS, Amazon S3, and Kafka Streams.

Ozone object storage

Ozone is a next-generation file system that bridges object store and HDFS, and supports billions of objects.

Capabilities that are new to prior users of CDH include:

Ranger security

Provides full dynamic capability to set up policies and authorizations, with fine-grained access control, dynamic row filtering, dynamic column masking, and attribute-based access control. With Impala as part of the distribution, Impala-Ranger integration is available, so any policy can be propagated to Impala, Hive, and Kudu.

Hive 3 data warehouse software

Includes Atomicity, Consistency, Isolation, and Durability (ACID) support for better ETL performance, and comprehensive ANSI SQL 2016 coverage.

Hive on Tez

Integrates Hive with Tez, an extensible framework for building high-performance batch and interactive data processing applications, providing better ETL performance at petabyte scale.

Capabilities that are new to prior users of HDP include:

Virtual Private Clusters

Virtual Private Clusters simplify deployment of applications and enable workloads running in different clusters to securely and flexibly share data.

Hue

Hue is a web-based interactive query editor for interacting with databases and data warehouses. It delivers an integrated SQL editor with autocompletions, visualizations, and connection to Hive and Impala to run SQL queries seamlessly.

Kudu

Kudu is a columnar storage manager for fast analytics on fast data. It supports variable character field (varchar) and datatype column, Ranger Authz integration, and fast changing of updatable data for better performance.

Impala

Impala is a SQL query engine for massively parallel processing (MPP) queries. It is ideal for Data Mart migration, interactive SQL, and Business Intelligence (BI) style queries like access reports or dashboards, through Tableau or other BI tools.

Cloudera Manager

Cloudera Manager is a web application for managing multiple clusters. It is a change from Apache Ambari in HDP, and includes automated wire encryption setup, fine-grained role-based access control (RBAC) for administrators, and streamlined maintenance workflows.