Hi folks, hope your weekend is going well.

So I have put myself into a situation. I have a home server with docker installed running fine so far. In my home network I have multiple networks for different purposes. The whole network stack looks like this OPNSense — Switch — Ubuntu Server

The server is connected to a switch port with pvid 100, and runs on vlan0.100 Now my goal is to move some docker containers to other vlans. To accomplish that I have set vlan0.101 and vlan0.102 on my server as interfaces with their own IP and default gateway on that subnet (e.g. 192.168.101.10) Next step I set up macvlans for my docker containers Then I set the port to also allow tagged traffic, but kept it on pvid 100. Now on my OPNSense I changed the host ip of my server from 192.168.100.10 to include all 3 IPs so homeserver 192.168.100.10, 192.168.101.10, 192.168.102.10

This setup seems to work fine for internal network, however no services are reachable from the outside (internet) anymore.

My first question is: Am I thinking correctly about this? Or is this over-engineered bs at this point and there is a better way to put docker containers on different subnets.

Second question is: Any ideas what’s breaking the internet access?

Thanks for the help in advance :D

EDIT: i have not changed the vlan of any container yet

  • zo0@programming.devOPEnglish
    2·
    16 hours ago

    Thanks, that’s a great write up. One thing I didn’t ubderstand however is in your Docker macvlan, you set the parent to br0.10 and br0.20, where are those parents defined?

    Maybe I misunderstood the macvlan documents but what I did was defining a vlan in server netplan vlan0.100 and set the macvlan parent to that vlan0.100. Is that not how it’s supposed to be?

    • buedi@feddit.orgEnglish
      1·
      16 hours ago

      The .10 or .20 just advises Docker to create that specific Subinterface automatically. In my example ip link will show new interfaces called br0.10 and br0.20 after creating the macvlan networks for VLAN IDs 10 and 20. You do not need to adjust your Netplan config when doing it like that. I would even assume that you are not allowed to define VLAN ID 10 and 20 in that particular case also in Netplan. I would expect that this will cause issues. Also see https://docs.docker.com/engine/network/drivers/macvlan/ in the 802.1Q trunk bridge mode section.

      There are probably multiple ways to do all of this, but this is how I did it and it works for me since a few years without touching it again. All VLANs are separated from each other and no VLAN has access to the LAN side. Everything is forced to go through tagged VLANs via the switch to the Firewall, where I then create rules to allow / deny traffic from / to all my networks and the Internet.

      For me, this setup is very simple to re-implement should my Host go down. No special configuration in Netplan is needed. Only create the Docker Networks and start up my stacks again.