Cluster Mode Deployment
In highly available (HA) deployments with multiple CoreDNS-GSLB instances, executing health checks independently from every instance can put redundant load on backends and lead to temporary DNS answer inconsistencies.
CoreDNS-GSLB features a Cluster Mode backed by a shared Redis database to solve this.
Startup Sequence
When a CoreDNS-GSLB instance starts up, it aligns its in-memory status with the shared Redis state immediately to avoid cold-start issues or temporary routing discrepancies.
sequenceDiagram
autonumber
participant CoreDNS as CoreDNS-GSLB Instance
participant YAML as Local YAML Zone Files
participant Redis as Redis Cache
participant Backend as Backend Pool
Note over CoreDNS: Startup / Initialization
CoreDNS->>YAML: 1. Load zone records and backend definitions
YAML-->>CoreDNS: Return records in-memory
CoreDNS->>Redis: 2. Connect & Subscribe to "gslb:health:updates"
loop For each Backend in loaded records
CoreDNS->>Redis: 3. GetRedisHealth(zone, fqdn, IP)
alt Health status exists in Redis
Redis-->>CoreDNS: Return status (alive/dead)
CoreDNS->>CoreDNS: Set initial backend state (SetAlive)
else Key does not exist / Redis unreachable
CoreDNS->>CoreDNS: Default to offline/local initial state
end
end
CoreDNS->>CoreDNS: 4. Initialize zone-level health status (updateRecordHealthStatus)
CoreDNS->>CoreDNS: 5. Start background scraper loop
Runtime Check Execution (Distributed Lock Mode)
At each scraping interval, CoreDNS-GSLB instances coordinate check execution via distributed locks (SETNX) to ensure only one instance executes a health check per cycle.
flowchart TD
Start([Scraper Loop Triggered]) --> CheckLock{1. Acquire Lock via SETNX?}
CheckLock -- "Yes (Winner)" --> RunCheck[2. Run raw health check locally]
RunCheck --> ApplyLocal[3. Apply raw status locally]
ApplyLocal --> WriteRedis[4. Write raw status to Redis\nwith 2x ScrapeInterval TTL]
WriteRedis --> PubUpdate[5. Publish raw status update\nto Pub/Sub channel]
PubUpdate --> End([End Scrape Cycle])
CheckLock -- "No (Runner-up)" --> ReadRedis[2. Read raw status from Redis]
ReadRedis -- "Success" --> ApplyRemote[3. Apply raw status locally]
ReadRedis -- "Redis Down / Key Missing" --> RunCheckLocal[Fallback: Run raw check locally]
RunCheckLocal --> ApplyFallback[Apply raw status locally]
ApplyFallback --> End
ApplyRemote --> End
State & Threshold Synchronization (Pub/Sub)
To guarantee that all instances apply rise/fall state machine transitions consistently, instances publish and consume raw health check results (before applying threshold logic). Each instance then runs the threshold state machine locally on every update.
flowchart TD
subgraph Publisher ["Active Checker (Node A)"]
A_Check[1. Perform Check] --> A_Apply[2. Apply local threshold]
A_Apply --> A_Pub[3. Publish raw status to updates channel]
end
subgraph SubscriberB ["Standby Node (Node B)"]
B_Sub[1. Receive update via Pub/Sub] --> B_Apply[2. Apply raw status locally]
B_Apply --> B_Thresh[3. Re-evaluate rise/fall thresholds]
end
subgraph SubscriberC ["Standby Node (Node C)"]
C_Sub[1. Receive update via Pub/Sub] --> C_Apply[2. Apply raw status locally]
C_Apply --> C_Thresh[3. Re-evaluate rise/fall thresholds]
end
A_Pub -.->|"Pub/Sub Channel: gslb:health:updates"| B_Sub
A_Pub -.->|"Pub/Sub Channel: gslb:health:updates"| C_Sub
Configuration
To enable Cluster Mode, configure the Redis options inside the gslb block of your Corefile.
For a comprehensive description of all configuration options, refer to the Corefile Reference guide.
Corefile Example
gslb {
# ... other options ...
# Enable Redis sync (default: false)
redis_enable true
# Redis address (default: "127.0.0.1:6379")
redis_addr "redis-server.local:6379"
# Redis password (default: "")
redis_password "securepass"
# Redis database number (default: 0)
redis_db 0
# Key prefix to namespace Redis keys (default: "gslb:")
redis_key_prefix "gslb:"
# Sync mode (default: "lock"). Use "none" to disable distributed locking
redis_sync_mode "lock"
}
Resilience & Failover Modes
- Redis Unreachable: If the connection to Redis fails during startup or execution, the GSLB plugin logs the warning and automatically switches to independent local check mode.
- Lock Owner Death: Locks are acquired with a short Time-To-Live (TTL) equal to
scrapeInterval / 2(minimum 2s). If an instance dies mid-check, the lock naturally expires and another instance will automatically assume check execution during the next cycle.