First look at ZeroTier
Table of Contents
ZeroTier is:
…an imaginary Ethernet switch or a WiFi network with unlimited range. Actual traffic is carried by your local LAN or the Internet, but the network virtualization engine takes care of encrypting, authenticating, and routing it.
It’s (I think anyways) an ad-hoc virtual private network, and apparently it uses its own VXLAN-like encapsulation and includes encryption. The two use cases they espouse on their website are 1) hybrid cloud connections and 2) peer to peer VPNs.
I decided to take a look into what this is, or at least try it out, because I have recently been doing some work with OpenVPN and other VPN solutions, of which there are few, and none of them make me very happy. This zerotier essentially sounds like a simple to use VPN with a “joiner” concept.
Install
I tested this out on two Ubuntu Trusty boxes. I downloaded the deb from zerotier, installed it and started the service.
zclient1:~$ wget https://download.zerotier.com/dist/zerotier-one_1.0.5_amd64.deb
--2015-10-03 14:47:15-- https://download.zerotier.com/dist/zerotier-one_1.0.5_amd64.deb
Resolving download.zerotier.com (download.zerotier.com)... 104.207.155.202, 2001:19f0:6000:91c7:5400:ff:fe10:2421
Connecting to download.zerotier.com (download.zerotier.com)|104.207.155.202|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 326330 (319K) [application/octet-stream]
Saving to: ‘zerotier-one_1.0.5_amd64.deb’
100%[=====================================================================================================================>] 326,330 1.96MB/s in 0.2s
2015-10-03 14:47:15 (1.96 MB/s) - ‘zerotier-one_1.0.5_amd64.deb’ saved [326330/326330]
zclient1:~$ sudo dpkg -i zerotier-one_1.0.5_amd64.deb
sudo: unable to resolve host openvpn1
Selecting previously unselected package zerotier-one.
(Reading database ... 51354 files and directories currently installed.)
Preparing to unpack zerotier-one_1.0.5_amd64.deb ...
Unpacking zerotier-one (1.0.5) ...
Setting up zerotier-one (1.0.5) ...
*** ZeroTier One install/update ***
Getting version of existing install... NONE
Extracting files...
tmp/
tmp/systemd_zerotier-one.service
tmp/init.d_zerotier-one
var/
var/lib/
var/lib/zerotier-one/
var/lib/zerotier-one/zerotier-one
var/lib/zerotier-one/ui/
var/lib/zerotier-one/ui/main.js
var/lib/zerotier-one/ui/ZeroTierNetwork.jsx
var/lib/zerotier-one/ui/ztui.min.js
var/lib/zerotier-one/ui/index.html
var/lib/zerotier-one/ui/ZeroTierNode.jsx
var/lib/zerotier-one/ui/react.min.js
var/lib/zerotier-one/ui/simpleajax.min.js
var/lib/zerotier-one/ui/zerotier.css
var/lib/zerotier-one/uninstall.sh
Getting version of new install... 1.0.5
Creating symlinks...
Installing zerotier-one service...
Done! Installed and service configured to start at system boot.
To start now or restart the service if it's already running:
sudo service zerotier-one restart
Now to start it up…
zclient1# service zerotier-one start Starting ZeroTier One... </code>
Nice, that was super easy.
Then I joined the “earth” network, a test network provided by zerotier.
zclient2# zerotier-cli join e5cd7a9e1ce71cff
200 join OK
zclient2# ip ad sh
SNIP!
6: zt0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2800 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/ether 02:63:6c:ab:02:48 brd ff:ff:ff:ff:ff:ff
inet 28.183.225.45/7 brd 29.255.255.255 scope global zt0
valid_lft forever preferred_lft forever
inet6 fd80:56c2:e21c:0:199:9363:6cb7:e08a/88 scope global
valid_lft forever preferred_lft forever
inet6 fe80::63:6cff:feab:248/64 scope link
valid_lft forever preferred_lft forever
SNIP!
After doing so I have a zt0 network interface with an IP from the 28.0.0.0/7 network.
zclient2# zerotier-cli listnetworks
200 listnetworks
200 listnetworks 8056c2e21c000001 earth.zerotier.net 02:63:6c:ab:02:48 OK PUBLIC zt0 28.183.225.45/7,fd80:56c2:e21c:0000:0199:9363:6cb7:e08a/88
</code>
</pre>
## Creating your own private network
Unfortunately it seems that you have to create an account, login to the zerotier web interface (ie. it's a saas service), and create private networks from there. There are a few options, and the interface is a litle unpolished but it's straight forward enough. Once you've created a private network (which is default, and can only have a max of 10 devices or you are required to pay $4 a month which is fine) you can have systems/devices join that private network. One of the choices you'll have to make is what network IP space to give it, or to manage it yourself. By default zerotier will manage who gets what IP.
zclient1# zerotier-cli join
200 join OK
</code>
</pre>
And then in the interface set the device to be allowed and it'll get an IP.
zclient1# zerotier-cli listnetworks
200 listnetworks
200 listnetworks testnet fe:42:1f:5d:f2:7d OK PRIVATE zt0 10.147.17.23/24
</code>
</pre>
## Quick performance test
I ran a quick iperf test against the two nodes. So first I should note that I'm running these tests on virtual machines inside an OpenStack system and the network between them is also "virtualized", ie. is based on a VXLAN overlay, which is then running on VLANS underneath. So there is a lot of encapsulation going on here. :)
Performance over the zerotier VPN:
zclient2# iperf -c 10.147.17.23
------------------------------------------------------------
Client connecting to 10.147.17.23, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.147.17.61 port 59802 connected with 10.147.17.23 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 509 MBytes 426 Mbits/sec
Performance over the regular network:
zclient2# iperf -c 192.168.44.35
------------------------------------------------------------
Client connecting to 192.168.44.35, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.44.2 port 39575 connected with 192.168.44.35 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.06 GBytes 911 Mbits/sec
So you can see we get almost wirespeed without zerotier and then about half that with zerotier. So there is a performance hit. And, as I mentioned, the "normal network" is based on VXLAN, so it's already encapsulated. Perhaps the performance hit comes from the encryption. That is most likely. I haven't looked into the encryption provided by zerotier at all yet.
## Conclusion
I'm a big fan of software defined networking, whatever it actually means. I am not a big fan of the current state of virtual private networking. It seems OpenVPN is the best we have right now, and it's quite complicated and unwieldy. I think it's fairly obvious that one of the best things about SDN is going to be easier creation of secure virtual private networks, and I think they will either look something like zerotier, or even easier.
While I like how easy zerotier is, I'm not sure I like that it is essentially software as a service (saas), which means centralized. They say their system hasn't gone down, but it will. Then again they mention that existing devices will be able to communicate if their central system is down, but new devices can't be added and new networks can't be created. However, I'm a proponent of the public cloud (at least IaaS) so it'd be hypocritical of me to say that a centralized "cloud" network controller is not a good idea when I think using compute and storage in a public cloud is. That said, I'm not sure what performance issues there might be in having a cloud based controller. I would imagine that zerotier will eventually provide a controller that can be installed on premise. That'd be interesting, and if they made encryption optional then maybe performance would come up. Then again, it would be interesting to encrypt all network communication on premise too.
I quite like the idea of servers joining networks. Recently I have been installing a lot of Linux boxes, sure, using an automated process based on Cobbler, but I find the networking component to be extremely time consuming and frustrating, even just in terms of setting up templates. I'd really like to have the networks on servers setup by pulling from some kind of data center level source of truth (CMDB?) and just automatically being allowed to join specific networks based on their function. Can I hack that kind of process up, yes, but I don't want to do that work myself all the time. I want to just "happen" based on some tags pulled from the SST.