Home > Storage > ObjectScale and ECS > Product Documentation > Dell ECS with NGINX (OpenResty) > OpenResty configuration files
The nginx.conf file at each site has an upstream context defined with the IPs of ECS nodes at each site. For this example, the upstream context named ecs_9020 in the nginx.conf for OpenResty servers at each site has been modified as follows:
As mentioned in the previous OpenResty single deployment example, a separate file can be created to contain the upstream servers and an “include” directive in the nginx.conf file can be added within the http context. Thus, for the global OpenResty server, an upstreams.conf file was created with the OpenResty DNS names of each site and an “include upstreams.conf” directive is defined in the nginx.conf before it is referenced in the server context as shown in Figure 71.
The nginx.conf file for the global OpenResty server has similar directives found in the previous example deployments such as events and http contexts, directives for optimizations, logging, and nested server and location contexts for proxying request to backend servers, and Lua directives. Since most of these directives were explained previously, only the directives that differ are highlighted in this section. The nginx.conf used in this example is attached to this white paper and a snippet is shown in Figure 72.
For health checks of upstream servers, an init_by_lua_block uses the healthcheck.lua script to run health checks on the upstream servers and ECS nodes. This init block is started when the nginx.conf file is loaded. The healthcheck.lua in the attached nginx-sample.zip has comments that describe each of its functions and variables. Sample functions inside the healthcheck.lua script include:
The reader is encouraged to read the healthcheck.lua script for additional details which contains self-explanatory comments. Check the error.log file in /usr/local/openresty/nginx/logs for errors in the spawning of this health check.
The server context listens on the 9020 (HTTP) and 9021 (HTTPS) ports for S3 requests. For HTTPS, SSL is terminated at the global load balancing server and forwarded to the upstream server using HTTP. Since the name of the global load balancer has been modified to be os.ecstme.org, certificates generated in previous examples are copied to the global load balancer. A Lua script, gslb.lua, is embedded within the load balancer configuration file to implement the geo-pinning algorithm. The geo-pinning script is explained in the next section. SSL termination can also occur at the site level load balancer instead of at the global level, if desired.
The gslb.lua implements a hash algorithm (modulo to “n” sites) referred to as “geo-pinning” to distribute the creation of objects across sites for storage efficiency and direct reads and updates to site owning object for performance. In ECS architecture, the site that originated the creation of an object is the site owner of the object. In order to maintain strong consistency a read request first checks with the site owning the object for consistency of data at the requesting site. If a request comes to a site that does not own the object, response times are prolonged in order to check with the owning site first and fetch data if data is not current in the cache of the remote site. With geo- pinning, the read goes directly to the site owner of the object to process requests and thus improving read performance. The geo-pinning logic is as follows:
It has several dependent Lua scripts and modules that include:
Reader is encouraged to read through the gslb.lua script and dependent scripts such as s3.urllib and config attached to this white paper. The scripts contain self-explanatory comments.
For metric collection, as shown in Figure 73, the init_by_lua_block and log_by_lua are defined to register and log the metrics collected using the Prometheus library as was explained in the previous section.
The healthcheck.lua is also used for providing the status page for upstream servers (Figure 74).
Sample views of the kind of information provided by the monitoring directives are shown in Figure 75, Figure 76, and Figure 77.