05 Sep Hosting Challenger 1
Using our new mindpack.io hosting platform, we decided to start running benchmarks against other hosting platforms to compare differences in network and hosting performance. This was the first challenger, and for descriptive purposes we’ll call them “Challenger 1”.
Challenger 1 was a relatively light weight WordPress site being hosted apparently on an IIS+PHP server. We aren’t interested in editing the site files directly; our job was just to copy the site over, update wp-config.php, and monitor the results.
The following tests were performed both before and after migration:
- a network “uptime” test
- a page speed test to see the real world results of page request response time
- a heavy concurrency test to see how the site would respond under load
Test 1 – Network Uptime
The uptime test makes a connection to the web server once every minute. It waits for a response from the hosting server and then logs the amount of time it takes to complete the request/response.
Network performance wasn’t bad, but the outage count looks pretty severe. The client didn’t seem aware that their site pulled this level of downtime, so I’m thinking that without us monitoring the site, they wouldn’t have known the outages existed.
Mindpack Studios Website Hosting
Look carefully – the two above graphs are using different scales. We’re not entirely certain what happened from the 12th to the 15th. It’s possible the client was removing any last hooks they had in there while still using the old hosting provider, or possibly the client just optimized their site a bit to grab a few more bits of performance from the site.
Test 3 – Concurrency
This test used the ApacheBench tool. The data may be a bit wordy, but if you give it a minute of your time, you’ll see it’s pretty simple to understand the numbers.
ApacheBench makes a bunch of concurrent connections (in this case 50) until it reaches a limit (in this case 200) and then times the results. This may not be a regular use case of website experience, but it does bring to light a few cases that will come up under heavily advertised Adwords or e-mail marketing campaigns. Usually, the most costly times your website is live is when you are directly paying for hits – and that is the worst time for your site to be delayed.
*Note: we’d have liked to run this test using only 20 concurrent connections over 2500 total, but that wasn’t possible as the remote site kept completely failing with our connection attempts. Eventually, we managed to get 3 tests completed with this provider using only 200 total connections and posted the median result here.
Concurrency Level: 50 Time taken for tests: 84.247 seconds Complete requests: 200 Failed requests: 1 (Connect: 0, Receive: 0, Length: 1, Exceptions: 0) Non-2xx responses: 1 Total transferred: 17205745 bytes HTML transferred: 17139486 bytes Requests per second: 2.37 [#/sec] (mean) Time per request: 21061.816 [ms] (mean) Time per request: 421.236 [ms] (mean, across all concurrent requests) Transfer rate: 199.44 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 145 425 399.6 241 1402 Processing: 1950 19667 11895.4 20578 38159 Waiting: 1884 19565 11897.1 20405 38033 Total: 2234 20092 11915.8 21624 38333
Those results show 20 second response times, likely the result of heavy rate limiting employed on the server.
Mindpack Studios Website Hosting
Concurrency Level: 50 Time taken for tests: 7.069 seconds Complete requests: 200 Failed requests: 0 Total transferred: 8199000 bytes HTML transferred: 8096200 bytes Requests per second: 28.29 [#/sec] (mean) Time per request: 1767.230 [ms] (mean) Time per request: 35.345 [ms] (mean, across all concurrent requests) Transfer rate: 1132.68 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 67 152 132.2 80 404 Processing: 512 1479 380.8 1520 2327 Waiting: 444 1408 382.4 1458 2259 Total: 652 1631 376.3 1636 2716
As expected, low latency responses from a site hosted with performance being one of our primary jobs.
The network and page speed were quickly in our favor, but the neat part of this test for us was the results that ApacheBench produced for concurrency. Considering the measurements were significantly slower than the standard page speed results, the assumption here is the last web hosting provider was performing some serious rate limiting on the host when it was accessed by the same IP address.
Global rate limiting is something that most commodity providers do without the customer’s knowledge, and only after an expensive marketing campaign goes south and the site throws some serious errors does the customer realize how bad they’ve had it all along.
This client was specifically noticing some serious delays to their site after they installed some high definition images. That, combined with the stats above, suggest that rate limiting was the culprit to the site’s delays. That doesn’t explain the downtime in the network test over that week, but it does at least give some indication as to why their site felt slow to them while making page edits. The other option is that the remote servers are just really overloaded, and sporadic delays were present depending on who else was hitting the server and how hard. Both scenarios not best for the client.
This was a pretty clean example showing that all hosting isn’t created equal. This client was apparently paying somewhere close to $14/monthly for website hosting. Our prices are a few bucks more a month, but deliver a product that won’t ever cost you a customer due to page-load frustration and losing that first impression.