GTmetrix Performance Blog

News, updates and guides on GTmetrix and general web performance

Why is my GTmetrix Performance Score different from PageSpeed Insights, WebPageTest, and web.dev?

We explain why your GTmetrix Performance Score may be different from other Lighthouse-based tools.


Overview

The GTmetrix Performance Score is generated using Google Lighthouse and should be broadly similar to the Performance scores reported by other web performance tools.

However, there can sometimes be considerable differences in the GTmetrix Performance Score compared to what you may see in the other tools. These differences are mainly due to various factors including Lighthouse implementation, testing methodology, testing location, etc.

While this article focuses on three major Lighthouse tools, the overall rationale extends to any tool that uses Lighthouse to produce performance data.

Here are some of the key differences between GTmetrix, PageSpeed Insights, web.dev, and WebPageTest.

 


 

Why does my Performance score vary from tool to tool?

In a nutshell, Performance scores vary because different tools use different methodologies and configurations.

Even though Lighthouse is the common component among the tools, there are considerable differences in how Lighthouse itself is implemented.

For example, a notable difference is the Google tools’ use of something called Lantern, which simulates your page’s loading behaviour under certain network conditions, and can cause significant differences in Performance scores. More on this below.

Based on our observations, here’s how each tool generates a Lighthouse report:

GTmetrix

Loads page with real browser (Chrome)
Desktop view – 1366 x 768 (default)
Uses Lighthouse with custom audits
Applies Analysis Options (if selected)
Unthrottled Connection (default)
Vancouver, Canada (default)
Uses consistent hardware across all locations
Does not use Lantern

WebPagetest

Loads page with real browser (Chrome)
Desktop view – 1900 x 1200 (default)
Uses Lighthouse¹
Applies Analysis Options (if selected)
Throttled connection – Cable (default)
Dulles, USA (default)
Hardware may vary across different locations
Does not use Lantern

web.dev

Uses headless/emulated browsers
Mobile view – 1080 x 1920 (Emulated Moto G4)
Uses default Lighthouse²
Throttled CPU for mobile tests
Throttled Connection (likely Slow 4G connection)
Unknown test location (Possibly geolocated)
Uses mid to high-tier mobile CPU/Memory power
(e.g., Moto G4)
Uses Lantern

Google PageSpeed Insights

Uses headless/emulated browsers
Mobile – 360 x 640 / Desktop – 1350 x 940
Uses default Lighthouse²
Throttled CPU for mobile tests
Throttled connection (likely Slow 4G for mobile)
Unknown test location (Possibly geolocated)
Likely similar CPU/Memory resources as web.dev
(mobile tests only)
Uses Lantern

 

Key Takeaway

In general, all tools have differences in hardware, connection speed, locations, screen resolutions, and test methodology.

GTmetrix uses a real browser to load your page with an Unthrottled Connection using our specific hardware and test options. Some of these options (location, connection speed, screen resolution, etc.) can be changed based on your requirements.

WebPageTest also uses a real browser to load your page, albeit with different default test options and configurations. Some of these options can also be changed depending on your requirements.

PageSpeed Insights and web.dev launch performance tests using headless/emulated browsers and use network throttling to simulate your page load under different network conditions (i.e., desktop and mobile). You cannot change the test options like location, connection speed, screen resolution, etc.

Moreover, the Google tools (PageSpeed Insights and web.dev) use Lantern, which results in significantly different page loading behaviours, yielding different Performance scores and test results.

¹ Note that WebPageTest offers two types of tests. Their default/standard test launches a real browser using the specified configuration, but uses its own calculation and measurements to capture the different Lighthouse metrics (i.e., Web Vitals) from the Chrome trace data. You can also run a dedicated Lighthouse test from within WebPageTest, whose results will different from their standard test.

² Note that we’re assuming Google uses default Lighthouse in their tools. However, it is possible that modifications were made to suit their needs.

 


 

Implementation Differences

There could be several implementation differences among the different tools, including browser version differences, Lighthouse version differences, time to stop the test, etc.

As some of the performance metrics are very new, the scoring algorithms are continuously being tweaked, which often get rolled out with newer browser versions. This could yield different results on each tool.

Each tool may also stop the test at different times, which could result in things like missed requests, different metrics, and overall, different results.

While both of the above mentioned factors can cause discrepancies in your Performance scores, there is another chief implementation difference that could impact your Performance scores with each tool.

Some Lighthouse-powered tools use Lantern to speed up test execution and page analysis. This is a different approach to loading pages as it simulates load behaviour as opposed to measuring real-world performance with actual hardware specifications.
 

What is Lantern?

Lantern is an algorithm that initially loads your page as fast as possible (i.e., Unthrottled CPU/Network) and then applies something known as simulated throttling.

This essentially simulates what the actual page load would have been under given CPU/network conditions.

It’s an effective model for Google and, for the most part, provides representative data while speeding up test execution. However, in some cases, it may not accurately represent real-world conditions.
 


 

How does Lantern affect test results?

In the case of the Google tools, your page is initially loaded as fast as possible likely using powerful server-grade hardware.

The tool then uses Lantern to work backwards and applies simulated throttling to modify the results as if your page was being loaded on an “average” device.

This is significantly different from our implementation; GTmetrix loads pages in real time using specified device/connection options and generates results based on the measured performance at the time of the test.

In other words, we load the page using device/connection options from the start and do not work backwards to modify the results.

Lantern simulates network throttling to test your page, which may significantly affect your test results.

Lantern greatly shortens the time taken to generate a Lighthouse report as it gathers the data while the page loads.

However, this simulation of the browser execution may not be entirely accurate (compared to a real page load) and could result in significant differences in the various metrics on the Google tools compared to GTmetrix.

Learn more about Lantern here.

 


 

Do not compare them directly!

Keep in mind that there is no equal comparison as there are many differences among the tools, in both, implementation, as well as, scoring.

We do recommend using the other tools to do a spot check and for an overall idea of how your page is performing, but remember that Lighthouse alone is the common factor among all the tools, and each tool implements Lighthouse in its own way.

The goal should be to understand what the differences are with each tool and adjust your expectations accordingly to gain an accurate picture of your web performance.

Our recommendation is to always monitor your page frequently so that you can keep track of your performance and take appropriate action when performance issues crop up.

 


 

Detailed Differences

While we’ve summarized the key differences above, you may be interested in a more detailed explanation for the Performance score differences. Continue reading for more specific details on each tool.

 


 

Hardware Differences

Different tools use different hardware specifications, which results in different loading behaviours of your page.

Here are the key hardware differences among the different tools:

GTmetrix

Has particular hardware specifications
No CPU throttling for results
Uses consistent hardware across all locations
Does not use Lantern

WebPagetest

Has minimum hardware specifications
Applies CPU throttling for Lighthouse-only results
Hardware may vary across different locations
Does not use Lantern

web.dev

Unknown hardware specifications
Applies CPU throttling for mobile results
Uses mid to high-tier mobile CPU/Memory power
(e.g., Moto G4)
Uses Lantern

Google PageSpeed Insights

Unknown hardware specifications
Applies CPU throttling for mobile/desktop results
Likely similar CPU/Memory resources as web.dev
(mobile tests only)
Uses Lantern

 

Key Takeaway

Hardware specifications will play a major role in your Lighthouse results. More powerful hardware means more resources to efficiently load your page. This helps your browser execute tasks faster (i.e., CSS/ JavaScript parsing and execution, rendering, etc.).

GTmetrix is very selective with respect to server requirements so that we can ensure a consistent performance benchmark across all locations.

WebPageTest, however, may have significantly different hardware configurations across different locations as many of their servers are set up through members of the web community.

The Google tools (PageSpeed Insights and web.dev) likely use powerful server hardware and then apply appropriate CPU throttling for their page analysis results.

They also use Lantern, which can have a significant impact on Performance scores as the tools are set up to load your page as fast as possible and then simulate loading behaviour under certain network conditions.

Differences in CPU and Memory resources can particularly affect some Performance Score metrics like Time to Interactive (TTI) and Total Blocking Time (TBT) as they are CPU-focused (i.e., they rely on JavaScript execution).

 


 

Test Locations

The location used to conduct the tests can differ among the tools, which affects latency and other network connectivity related factors.

Here are the location differences with respect to each tool:

GTmetrix

Vancouver, Canada (default)
Test Location can be changed
Offers 22 locations in total

WebPagetest

Dulles, USA (default)
Test Location can be changed
Offers around 48 locations in total

web.dev

Test Location Unknown (possibly geolocated)
Test Location cannot be changed

Google PageSpeed Insights

Test Location Unknown (possibly geolocated)
Test Location cannot be changed

 

Key Takeaway

Test Locations can make a big difference to the Performance scores due to factors like distance from server (i.e., latency), network connection quality, etc.

GTmetrix and WebPageTest allow you to pick your test locations so that you can analyze web performance based on where your visitors are located. Both tools offer a number of different locations for you to choose from.

PageSpeed Insights and web.dev do not allow you to choose the location, and it is unknown where the test servers are located. It is possible that the test servers change based on geolocation, but there is no indication of this on the resulting reports.

 


 

Connection Speeds

All tools are likely to have connection speed differences, which impact the time taken to load your page. There may also be further implementation differences that affect each tool’s connection speed and results.

Here’s how each tool differs with respect to Connection Speeds:

GTmetrix

Uses an Unthrottled Connection (default)
Connection Speed can be changed
Does not use Lantern
Custom Connection Speeds can be used

WebPagetest

Uses a Cable connection (default)
Connection Speed can be changed
Does not use Lantern
Custom Connection Speeds can be used

web.dev

Uses Simulated throttling
Emulates a Slow 4G connection
Uses Lantern to simulate loading behaviour
Connection Speed cannot be changed

Google PageSpeed Insights

Uses Simulated throttling
Emulates a Slow 4G connection (mobile test only)
Uses Lantern to simulate loading behaviour
Connection Speed cannot be changed

 

Key Takeaway

Basically, faster connection speeds are likely to load your page faster, producing higher Performance scores.

GTmetrix, by default, uses an Unthrottled Connection but allows you to change the connection speed, based on your requirements.

WebPageTest, by default, uses a Cable connection and also allows you to change the connection speed, based on your requirements.

Web.dev runs emulated mobile-only tests using (what Google calls) a slow 4G connection.

In the case of PageSpeed Insights, the desktop and mobile tests are separated using their respective connection speeds.

Additionally, both Google tools use Lantern to simulate your page’s loading behaviour as if it were loaded on a mid-tier device/network.

Note that you cannot change the connection speed on either Google tool.

 


 

Real browser vs Headless/Emulated browser

Different tools can have fundamental differences with respect to the browser used to load your page. Consequently, loading behaviours may vary, resulting in different Performance scores.

Here’s what each tool uses, with respect to the browser option:

GTmetrix

Loads page with real browser (Chrome)
Default Screen Resolution – 1366 x 768
Screen Resolution can be changed
No browser caching

WebPagetest

Loads page with real browser (Chrome)
Default Screen Resolution – 1900 x 1200
Screen Resolution cannot be changed¹
No browser caching (for first view)

web.dev

Uses headless/emulated browser
Screen Resolution – 1080 x 1920
Screen Resolution cannot be changed
Unknown caching behaviour

Google PageSpeed Insights

Likely uses headless/emulated browser²
Mobile – 360 x 640 / Desktop – 1350 x 940
Screen Resolution cannot be changed
Unknown caching behaviour

 

Key Takeaway

Browser differences impact your page load in many ways, which can result in different Performance scores.

GTmetrix and WebPageTest use real browsers to load your page. This would be equal to the Chrome browser you’ve installed on your desktop.

PageSpeed Insights² and web.dev use headless/emulated browsers, which are essentially browsers without a user interface.

A real browser is the closest representation to a real-world user visiting your page. A headless/emulated browser, however, uses a script to load your page and capture the resulting loading data.

This could result in different loading behaviour and results for your pages.

¹ Note that you can only change screen resolution on a private instance of WebPageTest.

² Note that we’re assuming PageSpeed Insights uses an emulated browser but it’s not clear if this is the case.

 


 

You can read more about the new GTmetrix platform and other associated changes in the following articles:

 


 

Keep your site’s performance at the top of your mind

PRO Monitoring Splash

Get more On-Demand Tests, Monitored Slots and Premium Test Locations along with Hourly testing with a GTmetrix PRO plan.

Other features include: Priority queue access, Mobile Testing, Remote Location Monitoring (including Premium Locations), and more!


Upgrade to PRO today!