LAB AND RUM

Imagine a pirate—with an eye patch, hat, and a parrot—sitting in a lab, drinking rum. Well, this article isn’t about the drinking habits of pirates. Instead, we are going to look at the speed measurement habits of Shopify store owners.

In the world of Shopify Speed optimization, Lab and RUM are different ways to test your site. Most store owners use the former and completely ignore the latter. But RUM is as essential to you, as it is to the well-being of pirates.

To understand why, let’s first look at what Lab and RUM stand for:

Lab speed testing—tests performed in a “sterile” environment with as little change as possible between measurements. Examples are the speed measurement tools Pingdom, GTMetrix and Page Speed Insights.

Why lab? Because the test runs on a device with predefined settings on predictable connections. These tests try to simulate 3-4G networks, but the results are not like what your visitors would experience in real life. At least not most of them.

Lab tests give consistent results most of the time, as the environment they are running in isn’t changing.

They are good at unpacking what makes a site slow. You can’t determine that by watching over the shoulders of your visitors or timing them with a stopwatch.

The Lab testing tools spit out a list of improvements that you can follow.

Let’s contrast that with RUM.

RUM stands for Real User Measurement. That’s testing with real users in their particular environment with their tools. You can get Real User Measurements of your Shopify’s store speed using:

• Google Analytics (with no extra setup)
• Pingdom Code Snippet (part of their paid performance monitoring service)
• “Think With Google” website.

These tools collect the speed reports sent from visitors to your website. Reviewing the results you can determine if a certain visitor segment is being underserved.

Results here may vary, as the devices and connectivity of your visitors are changing. There won’t be two similar measurements. And that’s OK because that’s real life— as people are different the details influencing their browsing behavior differ, too.

The effect of the speed of your site is not determined by what the lab tests say, but by what your visitors perceive.

Let’s say that according to Pingdom, you have a fast website. (Results from a Lab test) If some of your visitors are using an old device on a poor connection, they won’t think of your store as “lightning fast”. They won’t experience it like that. (Actual data by a RUM test)

RUM help you determine if it makes sense to optimize for speed or not

Speed is all about perception. If your target audience is browsing on a desktop with a broadband connection they’ll hardly notice any speed improvements to your site. Your store may already be loading fast enough for them.

The faster visitors are loading the website now, the harder it would be for them to pinpoint any “before and after” difference. After all, only improvements over 20% are truly perceivable. You’ll see less ROI from hiring a Speed Optimization expert in such case.

Yes, mobile traffic is booming. Some websites see up to 80% of their traffic coming from smartphones. Still, that varies from website to website. Your audience may be different.

Say you have a quilting website that caters to people aged 60+. You may get more desktop than mobile traffic, as that demographic still finds desktops and laptops a more comfortable way to browse the web.

But, if your visitors are using high-end flagship phones and living in metro areas the speed of loading will be higher. If they use old devices that struggle to process the pages, this will slow down the whole loading process, too.

When to use each kind of test

You need to gather Real User Measurements first, to know if you need to optimize or not.

Not all clients that come to me wanting speed optimization need one. In one such case, after examining the data we determined speed is not their biggest issue at the moment.

The size of their business didn’t justify the cost of my service. Their target demographic had good internet access and new devices (high speed) but the store hasn’t broken the $10k/mo barrier yet. That’s a setup for low project ROI for them.

But, if you are getting tens or hundreds of thousands of visitors per month, speed optimization is beneficial. You’ll also have the capital to invest in such a project. As a result, you may get 10x more in revenue than you paid for the optimization.

Recap

Both Lab and RUM tests have their place in the speed analysis of your Shopify store. You need RUM first to determine if speed optimization makes sense in the context of your visitors and your business. Then, after you’ve determined the viability of a speed optimization project, you need Lab tests to show you what you can improve.

If you’ve read this far, I believe you’ll benefit from my “No Code Shopify Speed Optimization” guide! It is a quick read that’ll help you get the right mindset towards your Shopify store customizations.