I’m running an ESXi host with some virtual machines on it. Some of those machines are administered by me but some are rented out to clients. In my hosting agreement with my provider I have a few public Internet ip’s assigned to the physical port of the physical switch in which my physical machine is located. This means that any guest machine attached to the first vSwitch can grab any ip in my allotted range. This is fine for the machines I administer, but I don’t want my clients to be able to snatch any ip they see fit, only the ones they’ve been assigned. So I did some research and found that what I needed was probably some sort of virtual switch with bridged interfaces and - on top of that - some sort of ACL. VyOS was suggested for this task. What I’ve done so far is create a vSwitch for every client vps and a separate interface in VyOS for every vSwitch, with the only Internet interface being the one connected through VyOS. Please see the attachment below for a better idea of the current network topology.
So far I’ve been able to get traffic flowing to and from the VyOS-machine only. No matter how I bridge the interfaces there isn’t any traffic going through to the guests. I am probably doing something wrong. What I’m wondering is if what I’m trying to do is at all possible with VyOS, and if so how one would go about it? So far I’ve spent several long nights wrestling with this issue to no avail. Out of the different solutions I’ve tried, VyOS is by far the most easy to work with so I would really like this to work.
I presume you can reach vps0{a,b,c} thru vswitch0a, but cannot reach vps1{a,b,c} machines. Are the ports on the ESXi virtual switches in promiscuous mode?
Yes, that is correct. I did previously read the post you referred to above but I never did try to enable promiscious mode. Setting them as such allowed vps1{a,b,c} to reach the Internet (finally!). However, they are still able to snatch any ip from the alloted range as if they were connected directly to vSwitch0a (ie. a proper bridge). Perhaps this is when that ACL should come into play?
I’ve tried a few things, such as putting a /32 address on the bridge interface itself but that didn’t help. Any further assistance is very much appreciated.
Can you post your vyos config? It looks like you have three ethernet interfaces on the vyos, but are those all assigned to a bridge group? I think (I might be wrong) that you cannot add a firewall to the underlying components of a bridge group. If you really want to isolate those three clients, you could use rfc1918 addresses on the vswitch1{a,b,c} connections, and route their public /32 address toward them.
Suppose the vyos has 192.168.99.99 on the loopback interface. The three clients will need an interface route to that address via their respective interfaces, and a default route via that address. On the vyos, you have three static interface routes driving their packets out the proper interface. I think that might work.
So if I understand you correctly we need to put up our own network between the machines and vyos with their internal ip’s? Or did I misunderstand that completely?
I’m attaching my current configuration in this post. Please note that I’ve replaced the actual ip’s with X.Y.Z.n but I currently have access to the range of X.Y.Z.10 - X.Y.Z.15 in a /25 network. However, this may change as I can order additional ip’s at any time.
Please also note that currently my setup is not like what I described in the network topology the other night. The draft in there is what I imagine the finished product to be like. I’m still in the test lab with only one connected computer. If needed I can connected more for the sake of testing.
If you can help me solve this you’ve really saved my Christmas and I owe you a beer (or two). This is the one thing that’s blocking me from getting the ball rolling.
Okay, I’ve now read the article you referred to and did the following.
I set up a private network between vyos (10.0.0.1/24) and one of the guests (10.0.0.2/24). I then set up a 1-1 NAT between X.Y.Z.13 and 10.0.0.2 (using the official documentation here: http://vyos.net/wiki/User_Guide#1-to-1_NAT
So far it seems to work like I want. I have only tried with one client so far though, but in theory I think this will work with more clients provided they’re on different internal networks of course. Do you think I am correct in that regard?
Now this solution was a bit difficult from what I originally wanted since I wanted the guest machine to see the actual ip from the public domain, but if I understand you correctly this is not possible? In any case, the current solution is acceptable, but if you (or anyone else) know how to get the public domain ip properly forwarded please let me know.
