Home > Storage > PowerFlex > White Papers > Replication using Container Storage Module with Dell PowerFlex > Failover to remote site
This section describes the failover test that was performed to validate the replication while the PostgreSQL database workload was active on the source PowerFlex cluster. The pgbench tool was used for database workload. An additional database test table was also created and updated manually at certain times in the test and inspected later, at the replication target, to demonstrate what records were protected. This helps to check the values in the database before and after replication on the replication target, Site-B.
The following actions are run on the primary cluster, Site-A, before the failover:
# createdb -h localhost -U postgres mydb
# sudo pgbench -h localhost -U postgres -d mydb -p 5432 -i -s 9000
creating tables...
100000 of 900000000 tuples (0%) done (elapsed 0.02 s, remaining 173.15 s)
200000 of 900000000 tuples (0%) done (elapsed 0.04 s, remaining 180.86 s)
……………………………………..
899900000 of 900000000 tuples (99%) done (elapsed 670.19 s, remaining 0.07 s)
900000000 of 900000000 tuples (100%) done (elapsed 670.21 s, remaining 0.00 s)
vacuum...
set primary keys... done.
mydb=# SELECT * FROM pgbench_accounts LIMIT 10;
aid | bid | abalance | filler
-----------+------+----------+--------------------------------------------------------------------------------------
102094481 | 1021 | 0 |
102094482 | 1021 | 0 |
102094483 | 1021 | 0 |
102094484 | 1021 | 0 |
102094485 | 1021 | 0 |
102094486 | 1021 | 0 |
102094487 | 1021 | 0 |
mydb3=# SELECT * FROM employees;
employee_id | first_name | last_name | hire_date
-------------+------------+-----------+------------
1 | John | Doe | 2023-09-28
2 | John | Doe | 2023-09-28
3 | John | Doe | 2023-09-28
4 | John | Doe | 2023-09-28
……………………..
………………..
149 | John | Doe | 2023-09-28
150 | John | Doe | 2023-09-28
151 | John | Doe | 2023-09-28
#./repctl --rg rg-bd285081-0c9c-41d2-8254-dfffae27a803 failover --target secondaryocp
INFO RG (rg-bd285081-0c9c-41d2-8254-dfffae27a803), successfully updated with action: failover
#[root@infraocp repctl]# ./repctl get rg
[2023-10-04 07:23:14] INFO listing replication groups
[2023-10-04 07:23:15] INFO Cluster: primaryocp
+-----+
| RG |
+-----+
Name State rClusterID Driver RemoteRG IsSource LinkState
rg-bd285081-0c9c-41d2-8254-dfffae27a803 Ready secondaryocp csi-vxflexos.dellemc.com rg-bd285081-0c9c-41d2-8254-dfffae27a803 true FAILEDOVER
[2023-10-04 07:23:15] INFO
[2023-10-04 07:23:15] INFO Cluster: secondaryocp
+-----+
| RG |
+-----+
Name State rClusterID Driver RemoteRG IsSource LinkState
rg-bd285081-0c9c-41d2-8254-dfffae27a803 Ready primaryocp csi-vxflexos.dellemc.com rg-bd285081-0c9c-41d2-8254-dfffae27a803 false FAILEDOVER
The following steps are performed by the CSM replication to accomplish a failover.
The link state in the repctl command confirms that the failover is successful, and we can now bring up the PostgreSQL application on the target side. The persistent volume on the secondary cluster which has the replicated data, is shown as available. We can then initiate an install of PostgreSQL that binds to the available PV.
# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
k8s-29d85baf31 504Gi RWO Delete Available vxflexos-replication 15m
# helm install my-release oci://registry-1.docker.io/bitnamicharts/postgresql --set global.storageClass=vxflexos-replication --set global.postgresql.auth.postgresPassword=password --set primary.persistentVolumeClaimRetentionPolicy.enabled=true
The PostgreSQL pod on site-B comes up successfully. Then log in to the database to verify that the data that was written on the primary cluster, site-A, was replicated to the secondary cluster, site-B, successfully. Mydb and mydb3 are the databases that were created on the primary cluster and now exist in site-B. Verify that the contents are replicated, and the secondary cluster PV is write enabled.
mydb=# SELECT * FROM pgbench_accounts LIMIT 5;
aid | bid | abalance | filler
-----------+------+----------+--------------------------------------------------------------------------------------
102094481 | 1021 | 0 |
102094482 | 1021 | 0 |
102094483 | 1021 | 0 |
102094484 | 1021 | 0 |
102094485 | 1021 | 0 |
(5 rows)
mydb3=# SELECT * FROM employees;
employee_id | first_name | last_name | hire_date
-------------+------------+-----------+------------
1 | John | Doe | 2023-09-28
2 | John | Doe | 2023-09-28
3 | John | Doe | 2023-09-28
4 | John | Doe | 2023-09-28
……………………..
………………..
149 | John | Doe | 2023-09-28
150 | John | Doe | 2023-09-28