Testing cadence & objectives — Will Critchlow // SearchPilot

Will Critchlow, CEO of SearchPilot, discusses a full series on SEO testing. SEO is an ever-evolving field as Google’s algorithm and your competitors will always be implementing changes. As a result, there’s been a shift in mindset toward the continual evolution of websites and that is also being applied to the cadence of SEO testing. Today, Will talks about testing cadence and objectives.

Show Notes

  • 02:34
    The cadence for SEO testing
    Cadence is one of the main ways to set goals in terms of SEO testing. Research from Bing indicates that its best to take an approach with less pre-filtering, where ideas are tested as quickly as possible and the losing ones are turned off immediately.
  • 07:18
    Prioritizing velocity over waiting for high levels of statistical confidence in SEO testing
    In SEO, things are continuously changing with your competitors and Google, so you have to constantly change things on your website. This has led to a mindset of continual evolution, process, and shifting.
  • 08:04
    Factors that may limit SEO testing cadence
    Constraints include how quickly you can come up with ideas, website size, and how many testable site sections you have. Bigger websites are able to run tests faster because they can come up with many test ideas.
  • 08:58
    Things that can be controlled in SEO testing
    Getting sign-off from different stakeholders proactively will enable you to always have a testing backlog before the previous test is finished. That way, youll always have a test running in every testable section at all times.
  • 09:34
    How to approach cheap vs expensive changes
    Always take into account the comfort level of the business with investments and uncertainty, then optimize for speed within those constraints. For cheap changes, rollout may occur with weak evidence, but expensive changes require a high degree of statistical confidence.


  • "In SEO testing, cadence is one of the key ways to set goals around a testing program. And this is actually backed up by academic research that came out of Bing." -Will Critchlow, CEO, SearchPilot

  • "Google is changing, your competitors are changing, and you're not going to reach some platonic ideal of a website that is the right answer forevermore. You're going to keep changing it." -Will Critchlow, CEO, SearchPilot

  • "The teams that can churn out hundreds and hundreds of test ideas also tend to have the bigger websites where they can run tests faster." -Will Critchlow, CEO, SearchPilot

  • "You might take 60-40 and say, It's cheap, let's go for it. For an expensive change, you'll want to run to a high degree of statistical confidence so that you don't end up wasting money." -Will Critchlow, CEO, SearchPilot

Up Next: