When clustering database servers, why do I need to place them on different locations?


Databases in a cluster have one primary server and several replicas. The Call Bridge needs to communicate with the primary server. 

If a primary server stops working or is disconnected, another database server can promote itself to primary. However, a database server will only promote itself, if it can see more than half of the database servers in the cluster, and if no other database server is a primary.

Example: 3 databases on 2 locations

In the example below, we have one callbridge (192.168.6.77) and three database servers in a cluster. Databaser server (172.16.5.80) is running as database primary in datacenter A. The other two (192.168.4.81 and 192.168.3.80) are in datacenter B. 

On 172.16.5.80, it is in good condition now.

cms> database cluster status
Nodes:
172.16.5.80 (me) : Connected Primary
192.168.4.81 : Connected Replica ( In Sync )
192.168.3.80 : Connected Replica ( In Sync )
Node in use : 172.16.5.80

If datacenter B has a network connection issue and get isolated, database server (172.16.5.80) and callbridge(192.168.6.77) will lose connection to the two database servers (192.168.4.81 and 192.168.3.80) at datacenter B.
This will result in database server (172.16.5.80) running in replica mode without a primary database, as it cannot see any other database server at datacenter B in the cluster.

cms> database cluster status
Status : Enabled
Nodes:
172.16.5.80 (me) : Connected Replica
192.168.4.81 : Disconnected
192.168.3.80 : Disconnected
Node in use : 172.16.5.80

The two database servers (192.168.4.81 and 192.168.3.80) are isolated and they both can only see each other.

On 192.168.4.81,
cms> database cluster status
Status : Enabled
Nodes:
172.16.5.80 : Disconnected
192.168.4.81 (me) : Connected Primary
192.168.3.80 : Connected Replica ( In Sync )
Node in use : 192.168.4.81

On 192.168.3.80,
cms> database cluster status
Status : Enabled
Nodes:
172.16.5.80 : Disconnected
192.168.4.81 : Connected Primary
192.168.3.80 (me) : Connected Replica ( In Sync )
Node in use : 192.168.4.81

The Callbridge (192.168.6.77) can only see database server (172.16.5.80) which is running in replica mode.

cms> database cluster status
Status : Enabled

Nodes:
172.16.5.80 : Connected Replica
192.168.4.81 : Disconnected
192.168.3.80 : Disconnected
Node in use : 172.16.5.80

When site A gets isolated, all Call Bridges can only see database #3 which is in read only mode. So even though they have 3 nodes, it works as a 2-node cluster. 

Last update:
08-Sep-2020
FAQ ID:
1237