We tend to go a bit overkill on everything. Building a desktop? Yeah, 64GB of RAM should suffice, you know, just in case Windows 11 comes out next week. Testing a new service in Amazon EC2? Well t3.xlarge has
more than enough just enough in case of technical things I cannot yet comprehend.
In the event that I should need to test VPN configurations, network encapsulation techniques, VLANs and routing tables, maybe some seemingly large network emulations, there are several ways to go about testing them. I could go instantiate a handful of EC2 instances, or maybe grab a handful of Raspberry Pi and get going that way — virtualization seems beneficial in many ways, but I’m sorry, I’m not running 10 kubuntu (KDE > Gnome IMHO) images simultaneously so I can emulate Tor. Not even a GUI-less install could save my SSD from becoming a heaping ball of flames from the sweltering depths of swap space. No — way too complex to do a basic proof of concept.
Enter OpenWRT, recently(ish) reacquainted with it’s old spin-off LEDE Project (“Linux Embedded Development Environment”). In Creating a VMware machine, I end up with a 25M vmdk file and use an absolute maximum of 128M of ram per machine (see 4/32 warning here). I have used this in the past to create an easily distributable image of an IPSec tunnel gateway for safe(er) malware analysis on a protected network (packet encapsulation and even quick/weak encryption is a major sigh of relief). I am using it today to go over some basic GRE tunneling, just to rehash the little networking knowledge I picked up over the years and maybe learn a couple new tricks.
OpenWRT already has a pretty good walkthrough for creating a VMware VM from the latest image (yes, other virtualization environments have their own walkthroughs), I have a couple notes to add…
Once the VM boots, get in there and edit /etc/config/network (with vi) and comment out the bridge configuration or outright delete it. Start simple and just use eth0 by adding the following lines to the config:
Just make sure the ethernet device is set to share the connection with the host computer and either restart the VM or run /etc/init.d/networking restart
At this point your OpenWRT VM should be able to ping to the outside world.
The package manager is called opkg and has a pretty thorough repository sitting behind it. Start with a opkg update (skip this if you want to get angry) and start installing your favorite tools (or learn vi and accept it as a standard :D)
Preparing the lab
Let’s do a little GRE tunnel experiment, no encryption, easy peasy connect two networks. I’ll be using VMware fusion, but you can accomplish the same with any virtual environment.
Begin by creating two custom VM networks, and 3 fresh OpenWRT VMs. We’re going to have two machines on different networks connect to the outside world and each other via a machine acting as a router.
First the networks:
Uncheck all the boxes, we don’t need fancy auto-networking-enhancements. Create a second network the same way, and click the automatically assigned ‘vmnet#’ name to name them accordingly.
Now the three VMs:
Let’s create a template machine. The important thing to ensure here is to “Make a separate copy of the virtual disk” after choosing to use an existing virtual disk. This will allow you to spin up these vanilla VMs whenever you need to.
Click finish (you can also customize it here if you would like – technically you only need 128M of memory maximum for this image to work). And name it (we’re going to have a OWRT1, OWRT2, and OWRTGW) — clicking save will launch the virtual machine.
Start the new VM and change the root password as soon as it boots. Then configure the interface (you should only have eth0 at this point) by editing /etc/config/network with vi. A note about openwrt: in some cases, including with the firewall, the config is loaded from the disk image every time it reboots. You can add a couple rules in the existing chains in the scripts, edit the image and make sure it loads a clean firewall upon boot, OR… Flush the rules and clear the chains via /etc/rc.local, which I recommend. Edit the file to match above using vi.
Now shutdown the vm with the command poweroff.
Cloning the machines:
Clone this machine via the VMware Fusion Virtual Machine Library with a right click. Name it accordingly and repeat with ‘OWRT GW’. We should have 3 VMs when we are done.
Now we have to add two additional interfaces to the Gateway, we are going to triple NIC this. Add Device… -> Network Adaptor -> Add…
And select the network. I put network adaptor 2 on ‘Network 1’, network adaptor 3 on ‘Network 2’, and keep network adaptor 1 on ‘Share with my Mac’ (NAT)
Use the same settings screen to put ‘OWRT 1’ on ‘Network 1’ and ‘OWRT 2’ on ‘Network 2’
Finalizing the network
Power on all the VMs. I like to arrange them like so:
Important note: If
You can also type this by hand if you’d like, but this tell VMware to use the intel NIC emulation and everything should work. Do not forget to do this for ethernet2 and ethernet3 on your triple NIC’d gateway machine. (This is mentioned in the OpenWRT on VMware how-to)
Currently, OWRT1 has the only correct interface setting — We’ll need to edit /etc/config/network on each machine to reconfigure them like the following
The gateway needs to be able to pass traffic between networks. As such, it needs one firewall rule to handle NAT (Network Address Translation)
This line can be run after ever power cycle to the OWRT GW machine, or it can be added to /etc/rc.local after all the iptables rules have been cleared (recommended)
Now they all need a network restart via /etc/init.d/network restart or just a power off and power back on to get their new configurations.
Assuming everything is configured correctly, the following should hold true:
OWRT GW machine should be able to resolve and ping google, 10.1.0.2 and 10.2.0.2
OWRT 1 should be able to resolve and ping google, 10.1.0.1 and 10.2.0.2
OWRT 2 should be able to resolve and ping google, 10.2.0.1 and 10.1.0.2
Any issues with this, should identify where in our network diagram the fault lies and should be fixed accordingly. Any issues will make the next part impossible to complete.
A very simple GRE tunnel example
GRE (Generic Routing Encapsulation) is the most basic tunnel that can be established between two networks. There is no encryption to troubleshoot, nothing super complicated, it either works (and you can graduate to something more complicated) or it does not, and there may be a deeper problem going on.
In a more complete and mainstream linux distribution (Ubuntu, Fedora, etc.) the ability to configure a GRE tunnel tends to be precompiled into the kernel and the helper tools are preinstalled. OpenWRT is NOT a full distribution. It requires that anything more than the chosen fundamentals be installed by the user. OpenWRT is designed to be lightweight so that only the needed packages are installed and it can fit in the memory on a wireless router — so a few packages need to be installed to set up a GRE tunnel (and they are only necessary on OWRT 1 and OWRT 2)
opkg update; opkg install gre; opkg install ip-full
(this provides the required kernel module as well)
(ip-tiny package works for this as well if space is an issue on an embedded device – by default OpenWRT uses busybox to provide a lightweight toolset)
You can now create a GRE tunnel between OWRT 1 and OWRT 2
Three commands on each of the two tunnel endpoints is all it takes to set up a GRE tunnel:
OWRT 2 should be able to ping OWRT 1 at its tunnel address of 192.168.0.100
OWRT 1 should be able to ping OWRT 2 at its tunnel address of 192.168.0.200
Installing tcpdump on OWRT GW and listening on eth1 or eth2 should show all the tunneled traffic between the machines. (opkg update; opkg install tcpdump)
You should be able to move on to bigger and better tunnels, or more distant endpoints. You can add routes to default all traffic through the tunnel, make it persistent, or add encryption via IPSec.