Firewall and Network Ports

The confluent server binds a small, fixed set of ports. Most of what a deployment needs on the wire is served by other daemons on the same machine: the web server, a TFTP server, and optionally DNS, NTP and a log receiver. This page lists every port involved and gives worked firewalld and ufw configurations for them.

As described in the architecture overview, a confluent server normally reaches three separate networks, and which one a port belongs on matters as much as the port number:

• External, where administrators and the API live.
• Deployment, carrying the operating system NICs of the nodes. Network boot and OS installation happen here.
• BMC, a dedicated and fully isolated hardware management network carrying the BMCs, PDUs and the management ports of switches. Discovery and out-of-band control happen here.

Clusters usually have a fourth, high speed network as well, an InfiniBand, OmniPath or fast RoCE fabric used for traffic between compute nodes and to parallel storage. Confluent needs nothing open there unless you also deploy over it.

The deployment and BMC networks are normally completely separate from each other. The tables below name the network each port belongs to.

Ports bound by the confluent server¶

These are opened by confluentd itself.

The following are bound by confluent as well, but are not opened by the rules further down:

Ports served by other daemons¶

DNS and DHCP with dnsmasq (optional)¶

Confluent answers DHCP and PXE itself and needs no DNS server, so port 53 is only required if you choose to run one. confluent2dnsmasq and confluent2hosts can generate its configuration from the node attribute database. See Confluent and DHCP interaction.

dnsmasq then binds port 67 alongside confluent. confluent2dnsmasq emits bind-dynamic by default so the two can share it. Without that, confluent cannot bind the DHCP port and stops answering boot requests.

Node log forwarding (optional)¶

Nodes forward their logs to the server only when logging.servers is set. Leave it unset and none of 514, 6514 or 19532 is needed. The transport depends on logging.method and logging.tls, so only one of the three is normally in use. Certificates for the TLS transports are renewed by a daily timer on each node that calls back over HTTPS, so port 443 stays in use long after deployment finishes.

Collective (optional)¶

Collective members reach each other on port 13001, the same port used for the client API, authenticated with mutual TLS. No additional port is needed to form or run a collective. If members share state over NFS rather than a synchronized local filesystem, the NFS ports apply as well.

Native TLS API (optional)¶

Remote confluent clients connect directly to TCP port 13001. If those clients reach the server through its external address, allow that port on the external interface too. Restrict the allowance to the client addresses or subnet, as shown in the firewall examples below. HTTPS on port 443 does not replace this native TLS connection.

Outbound connections¶

Almost all of these go out over the BMC network. They rarely need rules on the server itself, since outgoing traffic is permitted by default, but they matter for any ACL between the server and the BMC network.

Configuring the firewall¶

The examples below cover firewalld (used on Enterprise Linux and SUSE) and ufw (used on Ubuntu and Debian). Each port is opened with a built-in service where one matches it exactly, and by number otherwise.

# external: administration only
firewall-cmd --permanent --zone=public --add-service=ssh              # 22/tcp, administrative access

# deployment: a new, empty zone for the ports confluent binds, plus the web server
firewall-cmd --permanent --new-zone=deployment
firewall-cmd --permanent --zone=deployment --add-port=13001/tcp       # confluent API and node credential handout
firewall-cmd --permanent --zone=deployment --add-service=dhcp         # 67/udp, DHCPv4, for network boot
firewall-cmd --permanent --zone=deployment --add-port=1900/udp        # SSDP, discovery and nodes finding the server
firewall-cmd --permanent --zone=deployment --add-service=https        # 443/tcp, deployment payload and node API

firewall-cmd --permanent --zone=deployment --add-service=ssh          # 22/tcp, node to server login

firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="10.1.0.50/32" port port="13001" protocol="tcp" accept'

firewall-cmd --permanent --zone=deployment --add-service=http         # 80/tcp, PXE boot, HTTP boot without TLS
firewall-cmd --permanent --zone=deployment --add-service=tftp         # 69/udp, PXE boot
firewall-cmd --permanent --zone=deployment --add-service=dhcpv6       # 547/udp, network boot over IPv6
firewall-cmd --permanent --zone=deployment --add-service=proxy-dhcp   # 4011/udp, ProxyDHCP, with a separate DHCP server

firewall-cmd --permanent --zone=deployment --add-service=dns          # 53, only if this server resolves for the cluster
firewall-cmd --permanent --zone=deployment --add-service=ntp          # 123/udp, only if this server is the time source

firewall-cmd --permanent --zone=deployment --add-port=514/tcp         # syslog, logging.method=rsyslog without TLS
firewall-cmd --permanent --zone=deployment --add-port=6514/tcp        # syslog over TLS, rsyslog with logging.tls
firewall-cmd --permanent --zone=deployment --add-port=19532/tcp       # logging.method=journal-remote

firewall-cmd --permanent --new-policy=deploy-out
firewall-cmd --permanent --policy=deploy-out --add-ingress-zone=deployment
firewall-cmd --permanent --policy=deploy-out --add-egress-zone=public
firewall-cmd --permanent --policy=deploy-out --set-target=ACCEPT

firewall-cmd --reload                                                 # load the rules before assigning interfaces
firewall-cmd --permanent --zone=public --change-interface=eno1        # external
firewall-cmd --permanent --zone=deployment --change-interface=eno2    # deployment
firewall-cmd --permanent --zone=trusted --change-interface=eno3       # BMC, trusted, see below
firewall-cmd --reload                                                 # apply where NetworkManager is not in use

for svc in https http dhcp dhcpv6 proxy-dhcp tftp; do   # leave out https if you use the web interface
    firewall-cmd --permanent --zone=public --remove-service=$svc
done
for port in 13001/tcp 1900/udp 427/udp 67/udp 69/udp 4011/udp 547/udp; do
    firewall-cmd --permanent --zone=public --remove-port=$port
done
firewall-cmd --reload

# external: administration only
ufw allow in on eno1 to any port 22 proto tcp     # SSH, administrative access

# deployment: the ports confluent binds, plus the web server
ufw allow in on eno2 to any port 13001 proto tcp  # confluent API and node credential handout
ufw allow in on eno2 to any port 67 proto udp     # DHCPv4, for network boot
ufw allow in on eno2 to any port 1900 proto udp   # SSDP, discovery and nodes finding the server
ufw allow in on eno2 to any port 443 proto tcp    # HTTPS, deployment payload and node API

# BMC: trusted, see below
ufw allow in on eno3

ufw enable

ufw allow in on eno2 to any port 22 proto tcp     # SSH, node to server login

ufw allow in on eno1 from 10.1.0.50/32 to any port 13001 proto tcp

ufw allow in on eno2 to any port 80 proto tcp     # HTTP, PXE boot, HTTP boot without TLS
ufw allow in on eno2 to any port 69 proto udp     # TFTP, PXE boot
ufw allow in on eno2 to any port 547 proto udp    # DHCPv6, network boot over IPv6
ufw allow in on eno2 to any port 4011 proto udp   # ProxyDHCP, with a separate DHCP server

ufw allow in on eno1 to any port 443 proto tcp    # HTTPS, web interface

ufw allow in on eno2 to any port 53               # DNS, only if this server resolves for the cluster
ufw allow in on eno2 to any port 123 proto udp    # NTP, only if this server is the cluster time source

ufw allow in on eno2 to any port 514 proto tcp    # syslog, logging.method=rsyslog without TLS
ufw allow in on eno2 to any port 6514 proto tcp   # syslog over TLS, rsyslog with logging.tls
ufw allow in on eno2 to any port 19532 proto tcp  # logging.method=journal-remote

ufw route allow in on eno2 out on eno1

for rule in 22/tcp 443/tcp 80/tcp 13001/tcp 67/udp 1900/udp 69/udp 547/udp 4011/udp OpenSSH 'Apache Secure' Apache; do
    ufw delete allow "$rule"
done

Discovery needs more than open ports¶

Opening 427 and 1900 is enough to receive discovery traffic that nodes and BMCs send to the server. It is not enough for confluent's own active scans. Those go out to multicast and broadcast addresses, and the answers come back as unicast packets from a different address than the request was sent to, which connection tracking does not associate with the outgoing request. The replies are therefore dropped as unsolicited.

Opening a fixed port does not help either: each scan sends from a freshly allocated source port, so there is no stable port to allow.

This affects the BMC network above all, because that is where the devices confluent scans for actually live, and it is also the one network where trusting an interface is genuinely safe.

Which device sits on which network¶

A confluent server typically has three interfaces, and what decides the firewall treatment is not how private the network looks but who can put a packet on the wire:

Putting the BMC interface in the trusted zone, as Configuring the firewall already does, is what makes active discovery work. No additional rule is needed. If you cannot trust it, see If you cannot trust the BMC interface below.

firewall-cmd --permanent --zone=trusted --add-source=172.30.1.192/26   # BMC addresses on the deployment network
firewall-cmd --reload

ufw allow in on eno2 from 172.30.1.192/26   # BMC addresses on the deployment network

If you cannot trust the BMC interface¶

Open only the two discovery ports on it instead. Devices announcing themselves are still heard, which BMCs do when they power on, but replies to confluent's active scans are dropped:

firewall-cmd --permanent --new-zone=bmc
firewall-cmd --permanent --zone=bmc --add-port=427/udp     # SLP, BMC discovery
firewall-cmd --permanent --zone=bmc --add-port=1900/udp    # SSDP, BMC discovery
firewall-cmd --reload                                      # load the zone and rules before assigning it
firewall-cmd --permanent --zone=bmc --change-interface=eno3
firewall-cmd --reload                                      # apply where NetworkManager is not in use

ufw allow in on eno3 to any port 427 proto udp    # SLP, BMC discovery
ufw allow in on eno3 to any port 1900 proto udp   # SSDP, BMC discovery
ufw delete allow in on eno3                       # then remove the wholesale rule

With firewalld, you can receive the scan replies as well by matching them with an ipset, as described next. ufw has no equivalent, so there the replies stay dropped.

Matching scan replies with an ipset¶

firewalld can single out the replies to confluent's active scans. The rules below record the source address and port of each outgoing scan in an ipset for three seconds, and accept the packets that come back to it. They only add the replies, so open the discovery ports as described above as well.

firewall-cmd --permanent --new-ipset=confluentv4 --type=hash:ip,port --option timeout=3
firewall-cmd --permanent --new-ipset=confluentv6 --type=hash:ip,port --option timeout=3 --family inet6
firewall-cmd --reload
firewall-cmd --permanent --direct --add-rule ipv6 filter OUTPUT 1 -p udp -m udp --dport 427 -j SET --add-set confluentv6 src,src --exist
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 1 -p udp -m udp --dport 427 -j SET --add-set confluentv4 src,src --exist
firewall-cmd --permanent --direct --add-rule ipv6 filter OUTPUT 1 -p udp -m udp --dport 1900 -j SET --add-set confluentv6 src,src --exist
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 1 -p udp -m udp --dport 1900 -j SET --add-set confluentv4 src,src --exist
firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 1 -p udp -m set --match-set confluentv4 dst,dst -j ACCEPT
firewall-cmd --permanent --direct --add-rule ipv6 filter INPUT 1 -p udp -m set --match-set confluentv6 dst,dst -j ACCEPT
firewall-cmd --reload

Web UI forwarding (optional)¶

The web interface can proxy a node's BMC web interface and remote console through the confluent server, so browsers do not need direct access to the BMC network. Port 3900 relays KVM video, and one further port starting at 3901 is allocated for each concurrent BMC session. There is no fixed upper bound, so size a range to the number of sessions you expect to have open at once.

These ports face the browser, so they belong on the external interface rather than the deployment one:

firewall-cmd --permanent --zone=public --add-port=3900-3999/tcp   # KVM video plus one port per BMC session
firewall-cmd --reload

ufw allow in on eno1 to any port 3900:3999 proto tcp   # KVM video plus one port per BMC session

SELinux¶

On distributions with SELinux enforcing, the web server needs permission to connect to confluent over the network:

setsebool -P httpd_can_network_connect=on

Without this, the reverse proxy on port 443 cannot reach the confluent API, and the web interface and node deployment both fail even though the firewall is correct.