There are two angles to measure the sizing results, which can show if you are over-sizing or under-sizing the ICAP servers:
The following sections explain both methods in detail. Dell Technologies recommends taking both guidelines into consideration when measuring the sizing results.
In general, the workload for ICAP servers is CPU intensive. If the CPU utilization of the ICAP servers is over 95%, it is recommended to either add more CPU to the ICAP servers or add more ICAP servers in the PowerScale cluster.
The following shows two key metrics which can be leveraged to check if the ICAP server is too busy:
These two metrics are from the PowerScale cluster and indicate whether ICAP servers are too busy to handle further requests. The following sections provide a detailed explanation for both metrics.
PowerScale internally keeps a list of the status of ICAP servers connected to isi_avscan_d. Dump the status list using the following command:
kill -USR2 <PID of isi_avscan_d>
Once the above command has been run, all state information of isi_avscan_d including the status for the ICAP servers is recorded in the file /var/log/isi_avscan_d.log. To view the dumped information, use the following command:
The following figure shows an example of the dumped ICAP server status:
If the too_busy state in Figure 7 is set to true, this means an ICAP server is busy and not able to respond with the expected reply. Typically, this indicates that there are not enough ICAP servers for the workload. Add more ICAP servers until the too_busy state is false for all ICAP servers.
To get the fail-to-scan ratio, use the following CLI command:
The following shows example output from the above command:
In the above example, the failed to scan ratio is as follows:
In this case, a 41% failed-to-scan ratio is too high. This can be caused by various reasons like ICAP socket timeout or a poor network condition, but this typically means there are not enough ICAP servers to catch up with the speed of the workload, especially when using scan on close. In this case, add more ICAP servers and check if the fail-to-scan ratio is reduced.
To obtain a more accurate fail-to-scan ratio, Dell Technologies recommends running the following command to clear the statistics before running the test workload: