Dell Digital has developed an internal test suite with simple, medium, and complex queries running on a static, benchmarked data. This dataset is created by Dell and is used across all new Greenplum environments as a gold standard to validate a new environment.
- Data and queries are all static, which allows measuring and comparing performance of the cluster with previous runs.
- Static data is a range of small, medium, large, and even extra-large tables.
- Queries range from simple to complex.
- Queries return their individual run time.
- Queries are a mixed workload to simulate a production-like environment with the ability to repeatedly test new environments.
- Queries were designed to simulate mix of workload on the cluster which would impact CPU, memory, IO resources.
- Queries can run sequential or parallel test use-cases.
- Query test suite has two types of workloads. One with 19 queries (smaller suite) and the other with 91 queries (complex suite), both allowing for sequential or parallel runs.
- Query test suite can be scheduled through cron, can be run multiple times, and can be scaled up to support additional stress testing (such as running 2 x 91 queries simultaneously, for a total of 182 parallel queries).
- Once the query test suite is performed, query runtime can be analyzed against different configurations and different clusters to assess performance gains or losses.