Three broad categories of tests were conducted:

• Performance tests: The scope of these tests was to observe the read, write, and delete performance of “Partner” Storage under the following heavy workloads originating from the Kafka broker related to tiered storage:

• Background writes workload to archive stream data from Broker’s local disk (or page cache) to “Partner” Storage
• Streaming read (fetch) workload to serve historical fetch requests from consumers
• Background deletion workload that deletes stream data in “Partner” Storage when data retention has expired

• Correctness tests: The scope of these tests was as follows:

• Check the correctness of the subset of Amazon S3 APIs exercised by the Kafka broker on “Partner” Storage.
• Check the integrity of the end-to-end Kafka event stream under a high produce and consume workload, with the consumer fetch requests served largely from “Partner” Storage (instead of Broker’s local disk or page cache).

• Object store node failure test

Test the robustness of object storage by bringing down one of the nodes of storage.