Both Layer 2 and Layer 3 support vSAN connectivity between data nodes. Layer 2 does not require a static route, but Layer 3 does.
Table 5. Inter-Site Communication
Site to Site | Default | Layer 2 | Not Required |
Layer 3 | Static Routes Are Required* |
Site to Witness | Default | Layer 3 | Static Routes Are Required* |
| Witness Traffic Separation | Layer 3 | Static Routes Are Required* when using an interface other than the Management (vmk0) interface |
Layer 2 for 2 Node | Static Routes Are Not Required |
Note: Static routes are required because vSAN uses the Default TCP stack. A VMkernel specific gateway is not supported. The default gateway that is used by the Management VMkernel interface would be used by any other VMkernel interfaces using the same TCP stack. It is recommended that the vSAN network be isolated from normal infrastructure traffic on a dedicated network. As a result, the vSAN network cannot use the default gateway, and therefore static routes must be used to connect with Layer 3 addresses. While this is typically for site-witness communication over Layer 3, it can also include site-site communication when using Layer 3 for data node addresses.
Using Layer 3 is recommended for connectivity between the data sites and the witness. Witness Traffic Separation (WTS) can be used with Stretched Cluster configurations under the following conditions:
- If a VMkernel interface other than the Management VMkernel interface is tagged with "witness" traffic, static routes are required to communicate with the vSAN witness host VMkernel interface tagged for vSAN traffic.
- If the Management VMkernel interface is tagged with "witness" traffic, static routes are not required if the host can already communicate with the vSAN witness host VMkernel interface using the default gateway.
MTU can be different. For instance, an administrator could configure an MTU of 9000 between data nodes, but a smaller MTU of 1500 between data nodes and the witness.
