Home > Workload Solutions > Oracle > Guides > Design Guide — MySQL InnoDB Cluster on Dell EMC PowerStore T > Generate files
For bulk load data testing, we used HammerDB datagen to generate the large flat files. Bulk loading is done in MySQL using the bulk load insert command:
Ex: Bulk load insert into load data infile '/hammer/new_order_1.tbl' INTO table new_order fields terminated by '|';
Group Replication is limited during data loading. Here is an example of the error message that might appear if the limitation is exceeded:
Group Replication Error MY-011608 : Adjust group_replication_transaction_size_limit Option
For information about Group Replication limitations, refer to Known Limitations on the MySQL documentation portal.
Our workaround for the known limitations is to create a custom script to split the large files into small chunks, and then load the data using the bulk load insert command.
Script: if [ -z $1 ]
then
echo "Usage:$0 <customer>"
exit 1
fi
file_type=$1
lines=0
cd /hammer
ls -1 ${file_type}*|while read line
do
lines=`expr $lines + 1`
split -l 1000000 $line ${file_type}split$lines
echo "splitting file $line"
done
ls -1 ${file_type}split*|while read line
do
echo "load data infile '/hammer/$line' INTO table $file_type fields terminated by '|';" >>/tmp/mysql_load${file_type}.$$.sql
done
cat /tmp/mysql_load${file_type}.$$.sql
echo
echo
echo "loading the data into $file_type table"
mysql -u root -p Password1 -d tpcc "source /tmp/mysql_load${file_type}.$$.sql"
echo
echo "load is completed"
echo
echo "removing splitted files"
rm -rf /hammer/${file_type}split*