best practice on Cluster network

Question

best practice on Cluster network

John JY 221

Hi all,

We built windows 2016/2019 cluster with only one NIC
on two virtual VMs. Is this best practice?
should we have two NICs (one for data, one for private
heartbeat)?

Thank you!

Answer accepted by question author

4 additional answers

Your answer

Answer 1

Leon Laude 86,076

Hi @John JY ,

I'm not sure if there are best practices for this, however there certainly are good practices.
From experience in the field it would be better with at least two (2) NICs, but this may depend entirely on the cluster, the purpose of the cluster and cost.

I would really recommend for you to go through John's very detailed post about Failover Clustering and networking basics/fundamentals in the post below:
https://techcommunity.microsoft.com/t5/failover-clustering/failover-clustering-networking-basics-and-fundamentals/ba-p/1706005

The above post should give you the essential information and some good practices.

----------

If the reply was helpful please don't forget to upvote and/or accept as answer, thank you!

Best regards,
Leon

John JY 221 Reputation points

2021-09-09T13:06:27.063+00:00

Thank you so much for your insights. BTW, the two VMs mentioned are located in different site (one in production site and another at DR site). The cluster constantly lost heartbeats and failovered which impact SQL AG. In this situation at two different site, heartbeat NIC still recommended?

Thank you!
Leon Laude 86,076 Reputation points

2021-09-09T15:22:32.033+00:00

The heartbeat communication is time sensitive, but it's a very lightweight packet, a single packet of 134 bytes so it does not require a lot of bandwidth.
If you have too much latency between your sites, you may want to consider leaving out the heartbeat.
John JY 221 Reputation points

2021-09-09T20:46:12.44+00:00

So from your saying, like in my situation, one NIC is fine, right? tried to get network team involved to see why caused heartbeats lost. the network team said no issue with switches .But, these are VMware VMs. The failover happened once per week. plus, SQL AG is configured to allow failback immediately which introduces more downtime.

another way I thought is to increase crossubnetthrehold from 10 to15. the failover cluster network threshholds have been fine turned with Windows 2016 OS.

Thank you for your help!
John JY 221 Reputation points

2021-09-13T23:40:16.17+00:00

@Leon Laude , thanks for your continuing support. still one more question to ask, in terms of heartbeats lost, the cluster failovered.
Does it mean that the cluster lost majority vote already in this case? (two nodes and one file share witness) Thank you!
Leon Laude 86,076 Reputation points

2021-09-14T09:07:40.483+00:00

Please read through this post:
https://techcommunity.microsoft.com/t5/failover-clustering/understanding-quorum-in-a-failover-cluster/ba-p/371678
John JY 221 Reputation points

2021-09-14T14:52:28.903+00:00

@Leon Laude , thank you for your continuing help. In the image, why the first scenario will be running (since it only has 2 votes)?

Answer 2

Limitless Technology 40,031

Hello John,

As reference you can read this article: https://techcommunity.microsoft.com/t5/failover-clustering/no-such-thing-as-a-heartbeat-network/ba-p/388121

But all in all with one NIC you are ready to go with a Cluster service. Basically you can run the Cluster with 1 network, in the case that the NIC in one node fails, heartbeat will fail, this node will be removed from the membership and all roles moved to other available node. However, in case the virtual switch fails, it will fail for all members and the cluster will be down. The main recommendation is to create a separate network for heartbeat, as it is lightweight but important and you would like to avoid at all cost any heartbeat package lost that may bring down the cluster. However, if your network bandwidth is solid and the network usage is low for the cluster, one network is still an option.

Everything depends on the usage or purpose of the cluster itself.

Hope this helps with your query,
Best regards,

John JY 221 Reputation points

2021-09-10T16:13:33.673+00:00

HI LimitlessTechnology-2700,

the two VMs mentioned are located in different site (one in production site and another at DR site). The cluster constantly lost heartbeats and failovered which impact SQL AG. In this situation at two different site, heartbeat NIC still recommended?

Thank you very much!
Leon Laude 86,076 Reputation points

2021-09-10T16:17:39.937+00:00

If it impacts production, go without heartbeat, it's main purpose is to monitor the cluster nodes' status to and maintaining the operation of the cluster.
As long as you monitor your cluster nodes any other way, you'll be fine.
John JY 221 Reputation points

2021-09-10T17:26:48.357+00:00

Hi LeonLaude,

Thanks for your continuing help.

Can I ask if windows core cluster fails, will SQL AG role fail? (AG and cluster group are on separate cluster group)technically, SQL AG role should not failover if windows cluster group fails, right?

Thank you!
Leon Laude 86,076 Reputation points

2021-09-10T17:33:16.01+00:00

The always on availability groups are not dependent of the SQL Failover Clustering.

Always On availability groups relies on the Windows Server Failover Cluster (WSFC) to monitor and manage the current roles of the availability replicas that belong to a given availability group and to determine how a failover event affects the availability replicas.

The overall health of a WSFC is determined by the votes of quorum of nodes in the cluster. If the WSFC goes offline because of an unplanned disaster, or due to a persistent hardware or communications failure, manual administrative intervention is required.
John JY 221 Reputation points

2021-09-10T18:25:42.377+00:00

Hi LeonLaude,

Thanks for your insights,

Checked cluster log and we constantly get the following and cluster failed:(SQL AG failed also)

Microsoft Failover Cluster Virtual Adapter (NetFT) has missed more than 40 percent of consecutive heartbeats.

But, network team says no issue found on switches.

Can I ask your recommendations?

Thank you!

Answer 3

John JY 221

@Limitless Technology

From the link https://techcommunity.microsoft.com/t5/failover-clustering/no-such-thing-as-a-heartbeat-network/ba-p/388121 and see the picture on stretch site, I did not see a dedicated heartbeat network created?

can you explain on this? Thank you!

Answer 4

John JY 221

Hi all,

after reading https://techcommunity.microsoft.com/t5/failover-clustering/no-such-thing-as-a-heartbeat-network/ba-p/388121, it seems that there is no need to set up a dedicated network for heartbeats.

Can anyone help out whether my view of this article is right or not?

Thank you!

Leon Laude 86,076 Reputation points

2021-09-13T15:55:24.73+00:00

Yes this is correct, there is no requirement for a dedicated/separate heartbeat network.

If the reply was helpful please don't forget to upvote and/or accept as answer, thank you!
John JY 221 Reputation points

2021-09-13T18:16:39.443+00:00

@Leon Laude So, if for the better practice, better to add one separate heartbeat network? I am getting lost now.

Thank you!
Leon Laude 86,076 Reputation points

2021-09-13T18:19:32.14+00:00

In general (also as mentioned in the above link) if you're using heartbeat for your cluster, you would configure have a separate network adapter and a separate heartbeat network for your cluster, but as I already also mentioned, the heartbeat network adapter or heartbeat network is not required at all to run a cluster successfully.
John JY 221 Reputation points

2021-09-16T20:18:23.233+00:00

@Leon Laude , certainly, we can add one NIC on a separate network for heartbeat and select "Allow Cluster network communication on this network" to ensure heartbeat runs on this network, Right?

Thank you!

Answer 5

Hi John JY,

Speaking from experience, you can have failover cluster running on a single network, but it's inherently risky and has potential to lead to VM corruption.

In the past I've worked on a setup with 2x Nodes connected to SAN via SPF+ and all together via a single Switch with no redundant power. When the switch connecting these 3 devices lost power, the NODEs started fighting for and in turn corrupting VMs.

This was possible because the NODEs could connect to SAN independently of switch.

Super painful to fix later on and often lead to having do full backup restores and rebuilding DCs.

So yeah, while there is no need for dedicated heartbeat network, it sure saves a headache in the future to have it.

All the best.

Share via

best practice on Cluster network

4 additional answers

Your answer