Docker compose pokes holes in Firewalls

DateReadtime 4 minutes Tags

TLDR: Docker port forwarding uses iptables which can override your firewall rules

The problem

The other day, I noticed unusual activity on a VPS server, it was suddenly using 100% of the CPU. It appeared that one of my docker containers had been compromised and was running a piece of malware called "kdevtmpfsi"

Someone had run some sort of rootkit on a postgres container that I had running, and was now mining crypto! So after the machine was rebuilt and the threat removed, I was left with a question: how did someone get in?

Cautionary tale

Ok the first issue of access was easy to solve, it appeared that I had left the default user and password on this container, which obviously you shouldn't do. And they probably found the container by port scanning for open ports on my public ip. But I had firewalled the port!

First a little background, so a couple days before I decided that I wanted to share a database from this machine to another machine over via point-to-point wireguard tunnel. I have ufw setup on this machine so I researched how to limit access to a particular host IP.

Something like:

ufw allow from to port 5432 proto tcp

Which looks correct when you check the firewall:

$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp (OpenSSH)           LIMIT IN    Anywhere
80                         ALLOW IN    Anywhere
443                        ALLOW IN    Anywhere 5432/tcp        ALLOW IN
22/tcp (OpenSSH (v6))      LIMIT IN    Anywhere (v6)
80 (v6)                    ALLOW IN    Anywhere (v6)
443 (v6)                   ALLOW IN    Anywhere (v6)

However, when I was testing later after cleaning everything up, I noticed I could still connect to the postgres instance from the public ip.

Now this postgres instance is launched by a simpler docker-compose.yml like this:

    image: postgres:14.1
    restart: always
    env_file: .env
    command: postgres -c shared_preload_libraries=pg_stat_statements -c pg_stat_statements.track=all -c max_connections=20
      - pg_data:/var/lib/postgresql/data

And I recently added a posts section to the service like this:

      - "5432:5432"

After some playing around, I noticed that I could still connect to the postgres container via the public ip even without my fancy ufw rule. And I double checked, the default firewall rule:

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp (OpenSSH)           LIMIT IN    Anywhere
80                         ALLOW IN    Anywhere
443                        ALLOW IN    Anywhere
22/tcp (OpenSSH (v6))      LIMIT IN    Anywhere (v6)
80 (v6)                    ALLOW IN    Anywhere (v6)
443 (v6)                   ALLOW IN    Anywhere (v6)

In case you didn't know, ufw is just a wrapper around iptables that is more user friendly. So I decided to look at the actual iptables rules to see what I was setting up incorrectly using iptables -S.

As expected we can see a line generated by ufw:

-A ufw-user-input -s -d -p tcp -m tcp --dport 5432 -j ACCEPT

However in the first ten lines, I see my first hint. There is mention of "docker.":

$ sudo iptables -S | head
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward

Additionally, if we grep for port 5432, we'll see another rule slightly above our expected rule:

$ sudo iptables -S | grep -n 5432
68:-A DOCKER -d ! -i br-ac9ec4f755f9 -o br-ac9ec4f755f9 -p tcp -m tcp --dport 5432 -j ACCEPT
130:-A ufw-user-input -s -d -p tcp -m tcp --dport 5432 -j ACCEPT

So it appears on line 68 of my firewall rules I was allowing all host ips to connect to the docker container, this rule was allowing traffic before my rule to only allow traffic from a certain ip could be checked.

The solution

You can specify a host ip to bind the port to:

    - ""

This means that, the local ip will connect:

$ psql -h
Password for user paul:

While the public ip will not:

$ psql -h 350.350.32.350
psql: error: could not connect to server: Connection refused
    Is the server running on host "350.350.32.350" and accepting
    TCP/IP connections on port 5432?


  1. Don't use default passwords
  2. Don't make assumptions about your work, verify it
  3. Take backups and practice restoring from those backups
  4. The internet is very quick to find and take advantage of your mistakes