Over several generations, benchmarks have contributed significantly to the advancement of data processing systems, hardware, and software by enabling users to objectively evaluate and compare systems from different perspectives, such as functionality, performance, efficiency, reliability, and compatibility. Benchmarks have evolved into a powerful tool that can be applied in a wide range of different situations, from systems' design and development, to helping organizations make educated purchasing decisions. The relevance of performance benchmarks lies in their ability to provide a simplified yet representative workload that resembles real-world use cases.
TPCx-BB is a benchmark released by the TPC to measure the performance of systems when running big data analytics tasks. The TPC provides the benchmark as ready to run on a set of preselected platforms. It defines 30 business use cases that represent realistic decision support workloads in the form of queries and machine learning tasks that can be performed on large-scale datasets ranging from 1 Gigabyte to Petabytes of data.
Microbenchmarks often focus on the performance evaluation of a single system component. In contrast, each TPCx-BB query makes the system-under-test (SUT) components stress differently in terms of CPU, memory, or disk and network access, thus providing a more realistic and holistic view of the performance that can be expected from the SUT.