Unable to Access Service Fabric Cluster Endpoint After Certificate Renewal

Question

Unable to Access Service Fabric Cluster Endpoint After Certificate Renewal

Buchczyk, Kornelia 40

Hello,

I am experiencing an issue accessing a Service Fabric Cluster endpoint after renewing the cluster certificate.

Steps performed during certificate renewal:

Generated and stored a new self-signed certificate in Azure Key Vault
Updated the Cluster and VMSS with the new certificate as a secondary
Promoted the secondary certificate to primary
Removed the old certificate
Configured Microsoft Entra ID for client authentication to access the Service Fabric dashboard

Environment details: I have three clusters configured as follows

ClusterA – 5 nodes
ClusterB – 5 nodes
ClusterC – 3 nodes

The above steps completed successfully for all clusters.

For ClusterA and ClusterB I can access the Service Fabric dashboard and connect using PowerShell (Connect-ServiceFabricCluster @ConnectArgs)
For ClusterC unfortunately I cannot access the dashboard or connect using PowerShell

Observed behavior on ClusterC:

When attempting to open the dashboard, I receive an ERR_TIMED_OUT error.
When connecting via PowerShell- the following warnings and error appear

WARNING: Failed to contact Naming Service. Attempting to contact Failover Manager Service...

WARNING: Failed to contact Failover Manager Service, Attempting to contact FMM...

False

WARNING: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond <masked privateIP>:19000

Connect-ServiceFabricCluster : No cluster endpoint is reachable, please check if there is connectivity/firewall/DNS issue.

At line:1 char:1

+ Connect-ServiceFabricCluster @ConnectArgs

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : InvalidOperation: (:) [Connect-ServiceFabricCluster], FabricException

+ FullyQualifiedErrorId : TestClusterConnectionErrorId,Microsoft.ServiceFabric.Powershell.ConnectCluster

Additional information:

Applications and services are still running correctly within ClusterC.

The issue only affects access to the cluster endpoint and dashboard.

I would appreciate any hints or troubleshooting steps to resolve this issue ;-)

Thank you!!

Answer accepted by question author

0 additional answers

Your answer

Answer 1

Hi

Thank you for reaching out and providing detailed information about the steps you performed during the certificate renewal process. Based on the details and the behavior observed, it appears that ClusterC is facing a certificate binding or connectivity issue following the certificate update.

It mostly likely due to the new certificate may not be fully propagated or correctly bound to the cluster configuration on all nodes. This can prevent client authentication and dashboard access.

Could you please check the below details:

Ensure the certificate thumbprint in the Cluster configuration matches the new certificate.
Ensure ports 19000 and 19080 are open in the Network Security Group (NSG) and Load Balancer. From your management VM, run:

Test-NetConnection <ClusterC-IP> -Port 19000

Test-NetConnection <ClusterC-IP> -Port 19080

If these tests fail, it’s likely a network or firewall configuration issue.

3.Since Clusters A and B are working fine, please compare Cluster C’s certificate thumbprint, VMSS settings, and NSG rules with those clusters to make sure nothing is missing.

Run the following PowerShell command to verify the health state:

Get-ServiceFabricClusterHealth

If any node or subsystem shows as unhealthy, review the event logs or the Service Fabric trace files for certificate or connectivity-related errors.

Reference: https://free.blessedness.top/en-us/azure/service-fabric/service-fabric-cluster-security-update-certs-azure

https://docs.azure.cn/en-us/service-fabric/cluster-security-certificate-management

https://free.blessedness.top/en-us/azure/service-fabric/service-fabric-cluster-security#security-recommendations

Please let me know if you have any queries.

Buchczyk, Kornelia 40 Reputation points

2025-10-23T09:10:15.8166667+00:00

Hlelo!
Thank you for all info andyour guidance. Step 3 helped me to investigate the NSG and VMSS configuration. During the review I discovered that the NSG (which contained all inbound and outbound rules) had been unassigned from the VMSS for some reason—I'm not sure why this happened.

After creating a new NSG with the required rules and assigning it to the VMSS, the cluster became accessible again, and everything is now working as expected. The old NSG has been removed.

Appreciate your assistance in troubleshooting this issue;)
Jilakara Hemalatha 3,115 Reputation points Microsoft External Staff Moderator

2025-10-23T09:26:19.31+00:00

Thanks for update Buchczyk, Kornelia, I'm glad that your issue was resolved.

Could you please accept the answer and upvote it, so it is helpful to community members as well.

Thanks.

Share via

Unable to Access Service Fabric Cluster Endpoint After Certificate Renewal

0 additional answers

Your answer