We are having trouble with a pair of LoadMaster running in a HA-cluster. The usual pattern is that at some random point in time the cluster desynchronizes, with the primary node showing status green-grey and the secondary showing status blue-green. It seems that at this point in time the load balancers start fighting over the shared IP address, bringing all services virtually down.
The only (known) cure is to restart the secondary node. Usually everything is ok afterwards.
We noticed also a kind of strange behavior after such an incident. After lb2 was already up and running and the cluster had consistent green-green status, the services stopped for a few seconds. The log from lb1 (which was the master) shows:
eth3 is a direct connection between the two machines, used for HA checking (there is also HA checking on other interfaces). I'm wondering why lb1 has decided to go to BACKUP state and taking down its external interface eth0 when it discovered the HA link does not work?
Regards,
Jarek
We noticed also a kind of strange behavior after such an incident. After lb2 was already up and running and the cluster had consistent green-green status, the services stopped for a few seconds. The log from lb1 (which was the master) shows:
Feb 12 15:37:52 lb1 ucarp[1473]: [CRITICAL] Link eth3 is **DOWN**
Feb 12 15:37:52 lb1 ucarp[1473]: [WARNING] Switching to state: BACKUP
Feb 12 15:37:52 lb1 ucarp[1473]: [WARNING] Interface eth0 taken down
Feb 12 15:37:52 lb1 ucarp[1473]: [WARNING] Spawning [/etc/ha.d/down down]
Feb 12 15:37:53 lb1 ucarp[1473]: [WARNING] Switching to state: BACKUP
Feb 12 15:37:53 lb1 ucarp[1473]: [WARNING] Interface bnd1 taken down
Feb 12 15:37:53 lb1 ucarp[1473]: [WARNING] Switching to state: BACKUP
Feb 12 15:37:53 lb1 ucarp[1473]: [WARNING] Interface eth3 taken down
Feb 12 15:37:55 lb1 ucarp[1473]: [WARNING] Link eth3 is up
Feb 12 15:37:57 lb1 ucarp[1473]: [WARNING] Interface eth0 taken up
Feb 12 15:37:57 lb1 ucarp[1473]: [WARNING] Interface bnd1 taken up
Feb 12 15:37:57 lb1 ucarp[1473]: [WARNING] Interface eth3 taken up
Feb 12 15:37:57 lb1 ucarp[1473]: [WARNING] Switching to state: MASTER
Feb 12 15:37:57 lb1 ucarp[1473]: [WARNING] Spawning [/etc/ha.d/up up]
eth3 is a direct connection between the two machines, used for HA checking (there is also HA checking on other interfaces). I'm wondering why lb1 has decided to go to BACKUP state and taking down its external interface eth0 when it discovered the HA link does not work?
Regards,
Jarek