Internal communication failure - Redundant VR 4.16

[clevervinicius]

ISSUE TYPE

• Bug Report

COMPONENT NAME

VR, VPC, Redundant

CLOUDSTACK VERSION

4.16

CONFIGURATION

VPC with Redundant VR

OS / ENVIRONMENT

KVM Hosts

SUMMARY

Our environment has a VM that loses TCP communication with its own internal IP. This issue has been noticed while we have redundant VR on VPC. We also noticed the message: Failed to start Conntrack Deamon, this message is in other issues on github, but not affecting it in the same way.

When restarting the network with "Cleanup", the problem is temporarily solved, but it occurs again in a few minutes. We deactivate the redundancy and it doesn't repeat itself.

STEPS TO REPRODUCE

EXPECTED RESULTS

Free communication between VM internal ips

ACTUAL RESULTS

closed connection

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[weizhouapache]

@clevervinicius
what about other vms in the same VPC ?
what's the redundant state of the VPC VRs ?

[clevervinicius]

#what about other vms in the same VPC ?
The other VM in the same VPC keeps connecting.

what's the redundant state of the VPC VRs ?
Primary - Running - Backup - Running

My issue scenario: (example ips)

Redundant Router (10.1.0.1)
VM1 (10.1.0.20) internal (33.33.33.33) public - Static NAT
VM2 (database (10.1.0.21)

On normal state:
The VM1 connect to internal API on VM1 per public IP - VM1 IP Internal > Router > VM1 IP public > Router > VM1 IP internal > Connect OK

On problem state:
The VM1 try connect to internal API on VM1 per public IP - VM1 IP Internal > Router > VM1 IP public X(?) > Router X(?) VM1 IP internal > Not connected

There is an application inside the VM itself that needs to connect with itself, but when the router has a problem, it interrupts the communication. External communication is still normal, I can connect directly to this application from a connection outside the VPC.

Another explanation approach:

An API inside VM1 connects to VM1 itself, but through public IP set in static NAT, and goes back to VM1 itself, in some moments this connection stops happening and that's exactly where my problem is.

Monitoring the VM1 lan IP or even VM2 lan IP (database) there are no connection problems. The problem is only when my client's API accesses the VM itself through the public ip.

The client accesses the public IP because he uses the domain of his website in his application to consume his own API.

[weizhouapache]

@clevervinicius
thanks for the details.

can you share the result of iptables-save commad in the VPC VRs ?
(1) VPC with redundant VRs
(1) VPC with single VR

[clevervinicius]

router redundant.txt
router single.txt

[weizhouapache]

added to 4.18.1.0 milestone

[weizhouapache]

@clevervinicius
I checked the iptables rules in primary and backup VRs, they look same.

from my experience, this seems to be a issue caused by network misconfiguration.
some vrrp packets (between redundant VRs) might be dropped in the network