Both the Bluemix VPN service and the Bluemix Gateway service allow you to connect a Bluemix environment to an existing infrastructure in a very simple, secure, and straightforward manner. While the documentation on how to setup both of these services is well-organized, there isn’t very much information available on the performance of these services. So I setup a little experiment to attempt to get a rough idea of how well each service performs, and I got some very promising results in the process.
First, I wanted to test out the Bluemix VPN service, which is used to connect your Bluemix containers (and only the containers at this time) to any other IPSEC compatible endpoint. For this experiment, I am trying two different endpoints: a dedicated server and a Vyatta gateway. On the Bluemix side, I will have two containers running iperf3, which is used to push out as much traffic as possible.
The Bluemix VPN setup guides are a useful and great starting point, and are required reading for this experiment.
Like any good experiment, it is very important to be able to reproduce the results. Here is everything you will need to know to try this on your own. (A SoftLayer account and a Bluemix account will be required, of course.)
To test the SoftLayer side of things, we will use a single bare metal server with a 1Gbps public and private network. To reduce any latency or travel time issues, the server should be located in one of the Dallas data centers (which is where the Bluemix U.S. South region is hosted as well). Any operating system should be fine, but I choose the latest Ubuntu, since that is my personal favorite to work with. Any amount of RAM/CPU/disk space will be fine as well, since we are not really testing those.
Once the server is provisioned, make note of the private network subnet; mine was 10.37.82.128/26 and will be referenced in this experiment.
Iperf3 is the best tool that I have found for easily testing network speeds. Unfortunately, Ubuntu only ships with Iperf2, which is still pretty good, but I want to use the newest version (because newer is always better), so that will require a little bit of extra work to get installed.
apt-get install build-essential git
git clone https://github.com/esnet/iperf.git
After iperf3 is installed, we just need to setup a daemon to connect to. I used the following options:
iperf3 -f m -s -D -J --logfile /var/log/iperf3.1.log -p 9001
Luckily, there is already an iperf3 docker container available to use, so all that is required here is to publish it to our Bluemix repository and then spin one up.
If you haven’t already, setup the Bluemix Docker CLI.
Since Bluemix uses its own repositories for Docker images, we need to build the iperf3 container first, which is easy enough. Copy the contents of the iperf3 Dockerfile into a local file called “Dockerfile” then run:
cf ic build -t iperftest3 .
Assuming that build without errors, try running the test against the public network of your new SoftLayer server.
cf ic run -m 64 --name iperf01 registry.ng.bluemix.net/<NAMESPACE>/iperftest3 -c <SOFTLAYER PUBLIC IP> -t 600 -i 10 -R -p 9001
Bluemix has some great documentation on Setting up VPN endpoints (which you should read), but I will highlight the steps I took to get this all working.
The RIGHT side is Bluemix information. The LEFT side is SoftLayer information.
Just copy all the config files from that tutorial, and you should only need to edit the LEFT/RIGHT networks. Make sure your “rightid” field is set to the IP of your Bluemix VPN tunnel and not “%any” from stackoverflow. You will likely want to set your own secret key as well.
To setup a VPN connection with a Vyatta Endpoint instead, follow this guide on getting Bluemix + Vyatta Connected:
Now that everything is setup, we need to get some meaningful data. This is done by spinning up a Docker instance and pointing it at the private IP of our SoftLayer server.
cf ic run -m 64 --name iperf01 registry.ng.bluemix.net/cgallo/iperftest3 -c 10.37.82.159 -t 600 -i 10 -p 9001
Wait for that to finish (should be exactly 600 seconds), and then you can test the reverse speed (-R flag does this). A single iperf3 instance can only run one test at a time, so if you wanted to run two Docker instances and test both upstream and downstream speeds at the same time, you would need two iperf3 instances running on two different ports—which I will leave as an exercise for the reader.
cf ic run -m 64 --name iperf02 registry.ng.bluemix.net/cgallo/iperftest3 -c 10.37.82.159 -t 600 -i 10 -R -p 9001
Hopefully your results match mine, which are explained below.
To give the VPN speeds some perspective, I also wanted to test SoftLayer-to-SoftLayer and Bluemix-to-Bluemix.
Predictably, the connection between two bare metal servers in the same data center across the back-end network is really, really fast.
The connection between two containers was a lot lower than I expected, but they were still able to communicate at over 100Mbit/s, which is fairly respectable.
Now the part we really care about: transit over the VPN link.
I did two different tests: one through an instance of StrongSwan that I personally setup and one through a Vyatta Gateway. The speeds to Bluemix were about the same for both tests, but interestingly, the speeds to SoftLayer through the Vyatta were much higher. I suspect the difference in speeds here is likely due to a change in traffic congestion on the Bluemix side of things, but I’m sure having a dedicated gateway appliance played a small part as well.
Overall, the VPN connection should be viable for any workloads that do not exceed 100Mbit/s. It is very easy to setup, and once running, requires almost no maintenance.
Latency over this connection was also excellent.
--- 10.176.18.4 ping statistics ---
600 packets transmitted, 600 received, 0% packet loss, time 599734ms
rtt min/avg/max/mdev = 2.020/2.344/19.238/0.728 ms
The Bluemix Gateway service allows Bluemix applications to connect into private networks by exposing a URL and port combination on the Bluemix side for your application to connect to, and a process running inside your private network to actually move the traffic. (Here’s some required reading.)
For this section, I have created a small Bluemix application that contains the binary and library we need to launch iperf3, and a simple Python web server to launch the command.
My iperf3 project is available on hub.jazz.net and is very rough and simple, but it gets the job done. Download the code and upload it to your Bluemix account.
git clone https://hub.jazz.net/git/allmightyspiff/iperf3Test
Once the code is uploaded and the application is running, we need to get the secure gateway connected.
The securegateway_clientd application will run on the server that is running iperf3. Once it is installed and running, it will register itself (thanks to the provided authentication tokens) with the secure gateway. When you send data to the secure gateway URL, the secure gateway will send that data to the client running on the SoftLayer server, which is then sent to the address listed as the destination + port.
Traffic originating from the application will go to an “On Premises” destination, while traffic originating from a SoftLayer server going to your application will be a “Cloud Destination.”
The URL needed to launch the test is formatted as follows:
A normal test
For a reverse test
That will launch iperf3 in the background. You will need to check on it manually since I didn’t create anything fancy to acknowledge the test finishing. The files can be retrieved with the cf cli tool:
cf files iperf3Test app
cf files iperf3Test app/iperf3.secureR-1465496323.43.log
I ran two sets of tests: one over the public internet and one over the secure gateway.
SoftLayer-to-Bluemix had a decent average speed, averaging 89 Mbits/s. Bluemix-to-Softlayer, however, had significantly higher results, with a pretty impressive average speed of 514 Mbits/s. While the traffic spikes and dips a lot, even the minimum speed of 105 Mbits/s is really good.
The speeds of Bluemix-to-SoftLayer were rather poor, at an average of 20 Mbits/s, which is not really ideal for sending any decent amounts of data into a private network. However, SoftLayer-to-Bluemix was really great, at an average speed of 100 Mbits/s. Any data that lived in your private network that needed to be sent to Bluemix for processing would get there just fine.
Overall, both the VPN and secure gateways provide great ways of connecting your cloud and private infrastructures together so that data can be shared quickly and securely between them.