An interesting discussion found in Linkedin:
How critical for RAC to use stand-alone, physically separate, dedicated switch for interconnection?
We are building our new production 5-nodes RAC DB. Network/Unix team insists that give us a shared but dedicated VLAN 10GB switch for interconnection. They believe 10GB shared switch perform better than our 1GB dedicate switch, and also save space and power etc.
How critical for RAC to use stand-alone, physically separate, dedicated switch as Oracle recommended for best practice, or has this recommendation is out of date for the new technoledges?
said:
Here are some guidelines that we recommend when dealing with this issue. It is a frequent subject of discussion at many customer sites, so we have put down standards, so that there are fewer debates on this topic...:)
Here it is:
1. A physical database server must have at least 2 physical NICs with at least 2 ports on each NIC to avoid having a SPOF on any one NIC.
2. A physical database server must have at least 2 physical HBAs with at least 2 ports on each HBA if SAN is being used for data storage.
3. If data traffic and cluster inter-node communication are combined on the same NICs, then:
a. Multiple VLANs must be created
b. VLAN tagging and Quality of Service (QoS) must both be used.
c. Cluster inter-node communication must be separated from public traffic on different VLANs.
4. Automated failover (such as Etherchannel, 802.3ad), must be used for primary/secondary failover of each NIC.
5. Independent network switches must be configured so that during failover from the primary to secondary NIC, a different network switch will also be used.
6. Network link aggregation may be used to combine network ports into a larger network channel, but this link itself must not be a SPOF. For example, if using 2 dual port NICs, then port A on NIC 1 must be aggregated to port B on the same NIC. NIC 2 must be configured to support failover of both port A and port B from NIC 1.
7. The server and networking infrastructure must be configured to allow multicast communications and large IP packets.
