[Question] How to pihole+dhcp+DoH over docker swarm

[bluefangs]

Hello,

I'm a novice to docker/swarm and portainer. I've managed to run portainer in a standalone mode an have been successful at installing pihole + cloudflared DoH in standalone docker-compose. So, I began to delve into the world of swarms and am looking to see if I can move the above setup into a swarm - something that is herculean to me. After wandering the web looking for a solution for a week, I stumbled upon your version of installing pi + cloudflared over a docker swarm.

Existing setup for pihole-dhcp-dns-doh:

version: "2"
services:
    pihole:
        container_name: pihole
        image: pihole/pihole:latest
        restart: unless-stopped
        ports:
            - "53:53/tcp"
            - "53:53/udp"
            - "67:67/udp"
            - "80:80/tcp"
            - "443:443/tcp"
        environment:
            - ServerIP=192.168.0.100
            - TZ='Asia/Kolkata'
            - DNS1='10.0.0.2#5054'
            - DNS2=''
            - IPv6=false
            - DNSMASQ_LISTENING=all
            - WEBPASSWORD=some_password   #................................ set a secure password here or it will be random
        # Volumes store your data between container upgrades
        volumes:
        - '/usr/local/etc/pihole/pihole-data/:/etc/pihole/'
        - '/usr/local/etc/pihole/pihole-data/dnsmasq.d/:/etc/dnsmasq.d/'
        dns:
            - 127.0.0.1
            - 1.1.1.1
        # Recommended but not required (DHCP needs NET_ADMIN)
        #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
        cap_add:
            - NET_ADMIN
        network_mode: host

    cloudflared:
        container_name: cloudflared
        # image: fardog/secureoperator:latest # google dns
        image: visibilityspots/cloudflared:latest # cloudflare dns
        restart: unless-stopped
        dns:
            - 1.1.1.1
            - 1.0.0.1
        networks:
            pihole_net:
                ipv4_address: 10.0.0.2
networks:
    pihole_net:
        driver: bridge
        ipam:
            config:
                - subnet: 10.0.0.0/29   #..................................... /29 can accomodate 6 hosts

I brought down the above setup, set up portainer as a swarm as per the original documentation and then tried to add the below (based off your implementation - albeit using a different cloudflared image (visibilityspots/cloudflared).

Swarm config:

version: "3.7"
services:
  pihole:
    image: pihole/pihole:latest
    hostname: "pihole"    
    deploy:
      replicas: 1
      placement: 
          constraints: [node.role == manager]
      update_config:
        delay: 30s
    # For DHCP it is recommended to remove these ports and instead add: network_mode: "host"
    ports:		# <<<<<< as per the above comments, if I remove the ports, I'm unable to access pihole web interface
      - target: 53
        published: 53
        protocol: tcp
      - target: 53
        published: 53
        protocol: udp
      - target: 67
        published: 67
        protocol: udp
      - target: 80
        published: 80
        protocol: tcp
      - target: 443
        published: 443
        protocol: tcp  
    network_mode: "host" # <<<<<< as per the above comments, I've enabled 'host' network_mode.
    environment:
      TZ: "Asia/Kolkata"
      WEBPASSWORD: "some_password"
      DNS1: "172.23.0.1#5053" # <<<<<< This is my docker_gwbridge gateway IP
      # DNS2: "172.18.0.1#5053"
      DNS_FQDN_REQUIRED: "true"
      DNS_BOGUS_PRIV: "true"
      DNSSEC: "false"
      # CONDITIONAL_FORWARDING: "true"	# <<<<<< not sure if this and the below are reqd as I've disabled DHCP my the router
      # CONDITIONAL_FORWARDING_IP: "192.168.0.1"
      # CONDITIONAL_FORWARDING_DOMAIN: "home.local"
      # CONDITIONAL_FORWARDING_REVERSE: "0.168.192.in-addr.arpa"
    # Volumes store your data between container upgrades
    volumes:
      - /usr/local/etc/pihole/pihole-data:/etc/pihole/
      - /usr/local/etc/pihole/pihole-data/dnsmasq.d:/etc/dnsmasq.d/
    dns:
      - "127.0.0.1"
      #- "1.1.1.1" # fallback

  cloudflared:
    # Pinned to arm/v7 digest for crazymax/cloudflared:2020.2.0
    image: visibilityspots/cloudflared:latest # <<<<<< using this image instead of the one you've suggested
    deploy:
      replicas: 1
      placement: 
          constraints: [node.role == manager]
      update_config:
        delay: 30s    
    ports:
      - target: 5053
        published: 5053
        protocol: udp
      - target: 49312	# <<<<<< what is this for?
        published: 49312
        protocol: tcp
    environment:
      - "TZ=Asia/Kolkata"
      - "TUNNEL_DNS_UPSTREAM=https://1.1.1.1/dns-query,https://1.0.0.1/dns-query"

I"ve mentioned some comments in the above swarm-config as per my level of understanding / edits.
I was able to start the above via portainer, however, I ran into the multiple issues.

1. DHCP server was not doing it's job as I lost internet access
2. Pi was still able to access the internet, however, resolution was not happening via 1.1.1.1/DoH as per https://1.1.1.1/help

Do you happen to see any faulty configurations?

I was inspired to set up something similar to this blog wherein he talks about DNS server redundancy.

Thanks

[bluefangs]

I have an update.
I tried to work with the exact settings privided by you in the compose file. It works as intended EXCEPT for the fact that I'm unable to get the DHCP server of pi working.
I even triedto add

network_mode: 'host'
    cap_add:
      - NET_ADMIN

but still, to no avail. Any solutions on how to get the DHCP server working with this? Below is the yml file I'm using via portainer:

version: "3.7"

# https://github.com/pi-hole/docker-pi-hole/blob/master/README.md

services:
  pihole:
    
    image: pihole/pihole:latest
    hostname: "pihole"    
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        delay: 30s
      placement:
        constraints: [node.role == manager]
    # For DHCP it is recommended to remove these ports and instead add: network_mode: "host"
    ports:
     - "53:53/tcp"
     - "53:53/udp"
     - "67:67/udp"
     - "80:80/tcp"
     - "443:443/tcp"
    network_mode: 'host'
    cap_add:
      - NET_ADMIN

    environment:
      TZ: "Asia/Kolkata"
      WEBPASSWORD: "some_password"
      DNS1: "172.18.0.1#5053"
      DNS2: "172.18.0.1#5053"
      DNS_FQDN_REQUIRED: "true"
      DNS_BOGUS_PRIV: "true"
      DNSSEC: "false"
      CONDITIONAL_FORWARDING: "true"
      CONDITIONAL_FORWARDING_IP: "192.168.0.1"
      CONDITIONAL_FORWARDING_DOMAIN: "home.local"
      CONDITIONAL_FORWARDING_REVERSE: "0.168.192.in-addr.arpa"
    # Volumes store your data between container upgrades
    volumes:
      - etc-pihole:/etc/pihole/
      - etc-dnsmasq.d:/etc/dnsmasq.d/
      # run `touch ./var-log/pihole.log` first unless you like errors
      # - var-log:/var/log/
    dns:
      - "127.0.0.1"
      # - "1.1.1.1" # fallback

  cloudflared:
    # Pinned to arm/v7 digest for crazymax/cloudflared:2020.2.0
    image: crazymax/cloudflared:latest
    deploy:
      mode: replicated
      replicas: 1
      update_config:
        delay: 30s    
      placement:
        constraints: [node.role == manager]
    ports:
      - target: 5053
        published: 5053
        protocol: udp
      - target: 49312
        published: 49312
        protocol: tcp
    environment:
      - "TZ=America/New_York"
      - "TUNNEL_DNS_UPSTREAM=https://1.1.1.1/dns-query,https://1.0.0.1/dns-query"
volumes:
  etc-pihole: {}
  etc-dnsmasq.d: {}
  var-log: {}