Digital performance has become increasingly important for successful e-commerce. MeasureWorks has been publishing the Twinkle100, a list of the best performing e-commerce websites, for the last ten years. What is the secret to ranking high on digital performance? In this series of blogs, Measurework’s own performance engineers talk with leading online retailers. What is the impact of website performance on their business? How does the company integrate performance in their day-to-day work? We are proud to start with number 1 in the Twinkle100: bol.com! For years now, they have been in the lead for website performance.
We talk with Berend Marttin (Engineering Manager), Lars de Bruijn (Product Tech Lead Shop) and Edwin van der Graaf (Software Engineer). “Bol.com has been inspired by the Velocity conference In the area of digital performance, as well as by Steve Souders and Patrick Meenan. At this point, website performance optimization is in our DNA.” says Marttin
“We know the ropes!” adds De Bruijn, jokingly. All three of them love to share their experience and do so on the blog of bol.com: Techlab. Their knowledge and experience go a long way: bol.com has been leading for years with outstanding website performance.
Its daily challenge: to guide clients through the shop as quickly as possible.
Bol.com is the number 1 shop for many Dutch and Belgian customers, with a lot of products in their own shop. It is also a platform for thousands of entrepreneurs that sell their products online. With such a huge scale and multiple daily releases, you have to be able to trust your IT-landscape. The complete landscape exists of many microsystems; the landscape itself is as strong as its weakest link. “If one system is slow, it creates a ripple effect in the entire landscape. At the expense of the customer experience.” says Van der Graaf.
“The customer experience has to be consistent and predictable.”
Edwin van der Graaf, Software Engineer at bol.com
“From our own tests we know that improving performance really impacts the conversion rates.” says Van der Graaf. “Our customers greatly appreciate it when we pay attention to good website performance. This makes shopping easy and more fun. You have to provide a good performance, front-end as well as back-end, to make the customer experience consistent and predictable. Just like in a supermarket, people want to go through your shop as quickly as possible.
Performance in practice: site reliability engineering
To get a grip on the site’s performance and stability, bol.com has embraced the Site Reliability Engineering (SRE) method. The principles of SRE are based on Service Level Objectives that put the customer experience central. “How good and how fast do we want to be? Business and IT have to decide on this together, and they should both be held accountable.” say De Bruijn. “It gives space to pay attention to site performance.”
One of the objectives of bol.com: visitors should be able to browse a list of products with a performance of less than a second (90th percentile). Next to this, there are additional objectives for filtering or paging products. “This way, we have a concrete definition of ‘what is a good customer journey’ and thus a great definition of our website’s performance.” says De Bruijn. “With those objectives, we can make an explicit tradeoff whether we should release updates or functionalities.”
“With objectives, we can make an explicit tradeoff whether we should release updates or functionalities.”
Lars de Bruijn, Product Tech Lead Shop at bol.com
Improving from good performance to perfect performance is not an easy task. “We are looking to improve on every millisecond. Ten minor changes, make a good improvement together.” says Van der Graaf. “To achieve this we work with Real User Monitoring, open-source tools like WebPageTest, DevTools and Lighthouse and supplement this with our own monitoring on the back-ends.”
To get the website performance fast and consistent, the team tests in advance which impact a release has. De Bruijn says determined: “Our performance must not decrease.” The team also experiments, because it wants to know what the effect is of referrals. De Bruijn: “We test both A/A (same page, different implementation) and A/B (adjusted page). We test these experiments based on two sets of data sources: Real User Monitoring and data from that web server.” With the use of SRE, bol.com makes the difference in online shopping.
With the use of SRE, bol.com makes the difference in online shopping
Ambition: #1 retailer with a perfect score on Web Vitals
Work concerning performance is never really finished: new technologies come with chances or threats. “We are on top of it. We have weekly meetings to exchange knowledge, hypothesis and test results in the field of performance with each other. We also mirror our performance to that of public benchmarks, like the Chrome User Experience data (CRUX). This is based on the Core Web Vitals. These vitals are the metrics for a perfect customer journey.” says Van der Graaf.
“Of course it is our ambition to remain #1 retailer, but also to conquer the #1 position in the Twinkle100 performance list on other benchmarks. Our ambitions for 2021: a perfect score for all web vitals. This would make our IT-management very happy, but more importantly: it will make the customer happy!”
“Our clients greatly appreciate it if we pay attention to fast website performance.”
Edwin van der Graaf, Software Engineer at bol.com
Work concerning performance is never really finished: new technologies come with chances or threats. “We are on top of it. We have weekly meetings to exchange knowledge, hypothesis and test results in the field of performance with each other. We also mirror our performance to that of public benchmarks, like the Chrome User Experience data (CRUX). This is based on the Core Web Vitals. These vitals are the metrics for a perfect customer journey.” says Van der Graaf.
“Of course it is our ambition to remain #1 retailer, but also to conquer the #1 position in the Twinkle100 performance list on other benchmarks. Our ambitions for 2021: a perfect score for all web vitals. This would make our IT-management very happy, but more importantly: it will make the customer happy!”
“Our clients greatly appreciate it if we pay attention to fast website performance.”
Edwin van der Graaf, Software Engineer at bol.com