Hosting Challenger 1

Hosting Challenger 1

Using our new mindpack.io hosting platform, we decided to start running benchmarks against other hosting platforms to compare differences in network and hosting performance.  This was the first challenger, and for descriptive purposes we’ll call them “Challenger 1”.

Challenger 1 was a relatively light weight WordPress site being hosted apparently on an IIS+PHP server.  We aren’t interested in editing the site files directly; our job was just to copy the site over, update wp-config.php, and monitor the results.

The following tests were performed both before and after migration:

  1. a network “uptime” test
  2. a page speed test to see the real world results of page request response time
  3. a heavy concurrency test to see how the site would respond under load

Test 1 – Network Uptime

The uptime test makes a connection to the web server once every minute.  It waits for a response from the hosting server and then logs the amount of time it takes to complete the request/response.

Challenger Hosting

Network performance wasn’t bad, but the outage count looks pretty severe.  The client didn’t seem aware that their site pulled this level of downtime, so I’m thinking that without us monitoring the site, they wouldn’t have known the outages existed.

Mindpack Studios Website Hosting

Look carefully – the two above graphs are using different scales. We’re not entirely certain what happened from the 12th to the 15th.  It’s possible the client was removing any last hooks they had in there while still using the old hosting provider, or possibly the client just optimized their site a bit to grab a few more bits of performance from the site.

Test 2 – Page Speed

This test used Pingdom Page Speed.  August 11th was the date the customer moved their website over to Mindpack Studios.  The graph speaks for itself.

Test 3 – Concurrency

This test used the ApacheBench tool.  The data may be a bit wordy, but if you give it a minute of your time, you’ll see it’s pretty simple to understand the numbers.

ApacheBench makes a bunch of concurrent connections (in this case 50) until it reaches a limit (in this case 200) and then times the results.  This may not be a regular use case of website experience, but it does bring to light a few cases that will come up under heavily advertised Adwords or e-mail marketing campaigns.  Usually, the most costly times your website is live is when you are directly paying for hits – and that is the worst time for your site to be delayed.

*Note: we’d have liked to run this test using only 20 concurrent connections over 2500 total, but that wasn’t possible as the remote site kept completely failing with our connection attempts.  Eventually, we managed to get 3 tests completed with this provider using only 200 total connections and posted the median result here.

Challenger Concurrency

Concurrency Level:      50
Time taken for tests:   84.247 seconds
Complete requests:      200
Failed requests:        1
   (Connect: 0, Receive: 0, Length: 1, Exceptions: 0)
Non-2xx responses:      1
Total transferred:      17205745 bytes
HTML transferred:       17139486 bytes
Requests per second:    2.37 [#/sec] (mean)
Time per request:       21061.816 [ms] (mean)
Time per request:       421.236 [ms] (mean, across all concurrent requests)
Transfer rate:          199.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:      145  425 399.6    241    1402
Processing:  1950 19667 11895.4  20578   38159
Waiting:     1884 19565 11897.1  20405   38033
Total:       2234 20092 11915.8  21624   38333

Those results show 20 second response times, likely the result of heavy rate limiting employed on the server.

Mindpack Studios Website Hosting

Concurrency Level: 50
Time taken for tests: 7.069 seconds
Complete requests: 200
Failed requests: 0
Total transferred: 8199000 bytes
HTML transferred: 8096200 bytes
Requests per second: 28.29 [#/sec] (mean)
Time per request: 1767.230 [ms] (mean)
Time per request: 35.345 [ms] (mean, across all concurrent requests)
Transfer rate: 1132.68 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 67 152 132.2 80 404
Processing: 512 1479 380.8 1520 2327
Waiting: 444 1408 382.4 1458 2259
Total: 652 1631 376.3 1636 2716

As expected, low latency responses from a site hosted with performance being one of our primary jobs.

Conclusion

The network and page speed were quickly in our favor, but the neat part of this test for us was the results that ApacheBench produced for concurrency. Considering the measurements were significantly slower than the standard page speed results, the assumption here is the last web hosting provider was performing some serious rate limiting on the host when it was accessed by the same IP address.

Global rate limiting is something that most commodity providers do without the customer’s knowledge, and only after an expensive marketing campaign goes south and the site throws some serious errors does the customer realize how bad they’ve had it all along.

This client was specifically noticing some serious delays to their site after they installed some high definition images.  That, combined with the stats above, suggest that rate limiting was the culprit to the site’s delays.  That doesn’t explain the downtime in the network test over that week, but it does at least give some indication as to why their site felt slow to them while making page edits.  The other option is that the remote servers are just really overloaded, and sporadic delays were present depending on who else was hitting the server and how hard.  Both scenarios not best for the client.

This was a pretty clean example showing that all hosting isn’t created equal.  This client was apparently paying somewhere close to $14/monthly for website hosting.  Our prices are a few bucks more a month, but deliver a product that won’t ever cost you a customer due to page-load frustration and losing that first impression.