PowerDNS Cluster Deployment
Overview
The panel writes DNS records over the PowerDNS HTTP API. Two features depend on it:
- Reverse DNS (rDNS) writes PTR records for public IPs. See Private DNS and Reverse DNS.
- System DNS gives every provisioned resource a resolvable name automatically. See System DNS.
Both need somewhere to write to. This page covers deploying that: one primary that the panel talks to, and any number of secondaries that answer queries from the internet.
A deployment kit lives in your Master installation at deploy/powerdns/. It contains Docker Compose files, a bootstrap script, and a fleet setup script. Everything below assumes you are working from that folder.
How replication works
The panel only ever talks to the primary. Secondaries copy zones from it using standard DNS zone transfer (AXFR), triggered by a NOTIFY whenever a zone changes. Transfers are signed with a TSIG key, so a stranger who finds your primary cannot pull your zones.
Secondaries run with autosecondary enabled. When the panel creates a new zone, the primary notifies each secondary, and the secondary provisions that zone by itself. You never log into a secondary to add a zone. This is what makes adding the fifth or tenth nameserver no harder than adding the second.
If you would rather replicate at the database layer (all nodes reading a replicated MySQL backend), you do not need AXFR or TSIG at all. That trade-off is covered under Alternative: database replication.
Quickstart
On the machine that will be your primary:
cd deploy/powerdns
./bootstrap.sh
The script generates an API key and a TSIG key, starts the primary, waits for it to become healthy, and prints:
- the exact values to enter in the panel under DNS > Providers
- the records to create at your registrar
- the command to run for each secondary
Nothing is overwritten if you run it twice. Existing secrets are kept.
Multi-node setup
You do not have to log into each secondary. Authorize SSH from the primary to each node, list the nodes, and one script converges the whole fleet.
First, from the primary, give yourself key-based SSH to each secondary:
ssh-copy-id [email protected]
ssh-copy-id [email protected]
Then list the nodes and run the setup:
cp nodes.conf.example nodes.conf
$EDITOR nodes.conf # one user@host[:port] per line
./cluster-setup.sh --dry-run
./cluster-setup.sh
The script starts with a preflight table. It changes nothing until every node passes. Each check that fails prints a link back to the matching section on this page:
| Check | What it means |
|---|---|
| SSH | The primary can log into the node with the key you provided |
| ROOT | That login can act as root, directly or through sudo |
| DOCKER | Docker and the Compose plugin are installed |
| PORT53 | Nothing else is holding UDP/TCP port 53 |
| DISK | At least 2 GB free |
After converging the nodes it runs a verification pass, querying each node directly for the canary zone and printing PASS or FAIL per node. If any node fails, the script exits non-zero. A green run means replication is genuinely working, not merely configured.
Re-run the script whenever you add a node. It is idempotent.
Readiness failures
SSH access
The primary could not open an SSH session to the node.
Check, from the primary:
ssh -i ~/.ssh/id_rsa root@NODE_IP 'echo ok'
Common causes:
- The primary's public key is not in the node's
~/.ssh/authorized_keys. Fix withssh-copy-id root@NODE_IP. - The node listens on a non-standard port. Write it in
nodes.confasroot@NODE_IP:2222. - A firewall between the two hosts blocks port 22.
- The key you meant to use is not the default. Pass
--ssh-key /path/to/key.
Root access
The SSH login succeeded but cannot act as root. PowerDNS binds port 53, which is privileged, and the kit installs a systemd-level container stack.
Either log in as root, or give the user passwordless sudo on the node:
yourname ALL=(ALL) NOPASSWD:ALL
A sudo that prompts for a password will fail, because the script runs non-interactively.
Docker requirement
Docker or the Compose plugin is missing on the node.
By default the script installs Docker for you. It will refuse to do so in two cases:
- You passed
--no-docker-install. Install Docker yourself, then re-run. - The node has no systemd, so the official installer cannot register the service. Install Docker by hand for that distribution, then re-run with
--no-docker-install.
To install manually:
curl -fsSL https://get.docker.com | sh
docker compose version
Freeing port 53
Something already holds port 53 on the node. Find out what:
ss -lnup | grep ':53 '
ss -lntp | grep ':53 '
If it is systemd-resolved (the usual case on Ubuntu and Debian), the script frees it for you unless you passed --no-port53-fix. To do it by hand:
mkdir -p /etc/systemd/resolved.conf.d
printf '[Resolve]\nDNSStubListener=no\n' > /etc/systemd/resolved.conf.d/no-stub.conf
systemctl restart systemd-resolved
ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
The stub listener is what binds 127.0.0.53:53. Disabling it does not break name resolution on the node, because /etc/resolv.conf then points at the real resolver.
If it is another nameserver (BIND, dnsmasq, Unbound, an existing PowerDNS outside this kit), the script will not touch it. Decide deliberately whether to stop that service or use a different host.
Disk space
Less than 2 GB free on /. The container image and the zone database are small, but a nameserver that fills its disk stops answering. Free space or use a larger host.
Replication not working
A node passed preflight and converged, but the verification pass could not confirm the canary zone. Either the zone never arrived, or its serial or record does not match the primary.
Work through this in order.
1. Is the secondary reachable on 53 at all?
dig @NODE_IP hypervisor-canary.invalid SOA
2. Did the primary try to notify it? On the primary:
docker compose -f docker-compose.primary.yml logs --tail=100 pdns | grep -i notify
No NOTIFY lines at all usually means the primary is not running as a primary. Confirm --primary=yes is present in docker-compose.primary.yml.
3. Did the secondary refuse the transfer? On the node:
docker compose -f docker-compose.secondary.yml logs --tail=100 pdns | grep -iE 'notify|axfr|refus'
Received unsigned NOTIFY from potential autoprimary. Refusing. means the zone has no TSIG metadata attached on the primary. Run ./sync-zone-tsig.sh on the primary, then notify again. See Zones created by the panel.
Signature ... failed to validate means the two nodes hold different TSIG secrets. Re-run ./cluster-setup.sh, which pushes the current key to every node.
4. Is the node authorized? On the primary, the node's IP must appear in allow-axfr-ips. ./add-secondary.sh NODE_IP adds it and is safe to re-run.
5. Is the zone new? PowerDNS caches the list of zones it serves. A zone created outside the API is invisible to the running server until that cache refreshes, which by default takes several minutes. Force it:
docker compose -f docker-compose.primary.yml exec pdns pdns_control rediscover
docker compose -f docker-compose.primary.yml exec pdns pdns_control notify ZONE
Zones created by the panel
TSIG permission is stored per zone, as zone metadata. The panel creates zones over the API and has no knowledge of your TSIG key, so a zone it creates starts with no transfer permission attached and will not replicate.
sync-zone-tsig.sh attaches the key to every zone that is missing it. Run it on the primary from cron:
*/5 * * * * cd /path/to/deploy/powerdns && ./sync-zone-tsig.sh >/dev/null 2>&1
The bootstrap and add-secondary scripts run it too, so this only matters for zones created later by the panel.
Wiring it into the panel
In the panel, go to DNS > Providers and add a provider:
| Field | Value |
|---|---|
| Provider Type | PowerDNS |
| Name | Any label, for example primary-dns |
| Hostname | http://PRIMARY_IP (the scheme is required) |
| Port | The API port, 8081 by default |
| API Key | The key printed by bootstrap.sh |
| Enabled | On |
The API is bound to loopback and private ranges by default. If the panel runs on a different host, set PRIMARY_API_ALLOW_FROM and PRIMARY_API_BIND in .env to admit exactly that host, and put the API behind TLS or a private network. Never expose it to the internet.
Delegating at the registrar
Create, at whoever hosts the parent domain:
- An A record for each nameserver hostname, pointing at that node's public IP. For example
ns1.example.comto203.0.113.10. - NS records for the zone you are delegating, pointing at those hostnames.
If the parent zone is on Cloudflare, both records must be DNS only (grey cloud). Proxying breaks DNS service.
Confirm the delegation is live before expecting the panel to verify it:
dig +short NS your-subdomain.example.com @1.1.1.1
Alternative: database replication
You do not have to replicate with AXFR. If every node runs the gmysql backend against a replicated MySQL, all nodes simply read the same data and there is no zone transfer, no NOTIFY, and no TSIG. The per-zone metadata problem described above disappears with it.
The trade-offs:
| AXFR with TSIG (this kit) | Replicated database | |
|---|---|---|
| What crosses the network | DNS zone transfers on port 53 | MySQL replication |
| Between sites you expose | Nothing new | A database port, needing its own TLS and credentials |
| Mixed nameservers | Any AXFR-capable secondary works, including BIND or a third-party DNS provider | Every node must run your MySQL topology |
| New zone propagation | Immediate, on NOTIFY | Waits for the zone cache to refresh, several minutes by default |
| Failure mode | A refused transfer is visible in the logs | Replication lag or breakage is a database problem |
AXFR is the default here because it keeps the blast radius on port 53 and lets you put a secondary anywhere, including at a third party. Choose database replication if you already operate MySQL replication between the same sites and would rather manage one replication system than two. If you do, set every node to the same gmysql credentials, leave primary and secondary off, and lower zone-cache-refresh-interval so newly created zones appear promptly.
Backup and restore
The zone data lives in a Docker volume on the primary. Back it up:
docker compose -f docker-compose.primary.yml exec -T pdns \
sqlite3 /var/lib/powerdns/pdns.sqlite3 .dump > pdns-backup-$(date +%F).sql
Keep .env somewhere safe as well. It holds the API key and the TSIG secret, and without them the panel and the secondaries cannot talk to a rebuilt primary.
Secondaries hold no unique state. A lost secondary is rebuilt by re-running ./cluster-setup.sh.
Upgrading
Change PDNS_IMAGE in .env, then upgrade secondaries first, one at a time, confirming each still answers before moving on. Upgrade the primary last.
Pin a specific version rather than tracking a floating tag. Check which major you are pinning: the newest tag is sometimes a pre-release, and 4.4 is end of life.