Screaming Frog is an SEO spider that crawls and analyses websites through an app that you can download from here for all main PC operation systems such as Windows, macOS and Ubuntu. It is an essential SEO tool in your arsenal and is recommended by hundreds of top SEOs in the world. 

Alongside Xenu’s Link Sleuth – Screaming Frog is the most used tool by our agency specialists. There are both free and premium versions of this software, unless you want it for specific reasons that only the premium version supports or you have a particularly large website, the free version will be good enough. If you would like to purchase the premium version – a licence costs just £149 per year. It is money worth investing in your digital marketing and including it in your yearly spending budget. As we already mentioned, unless you are looking for advanced options or have a particularly large website, the free version will still be a great tool to use.

The biggest difference between the paid and free versions is the limitation of 500 pages crawled per session. But if you are stubborn enough you can use Xenu’s Link Sleuth, obtain a list of URLs and crawl them in list mode of Screaming Frog with batches of 500 pages a time. Then you can export these in CSV files and merge them into a big file.

How to use Screaming Frog?

Screaming Frog has 3 modes that, depending on your intended operation, you can switch from top menu navigation of the application:

  • Spider Mode – when you submit the URL of the website and the spider crawls through your website and gathers all the data.
  • List Mode – when you can either manually copy and paste the list of URLs or upload it from a file.
  • SERP Mode – when you can upload page titles and meta descriptions directly into the SEO Spider to calculate pixel widths and character lengths.

Screaming Frog 1

How to use Screaming Frog in spider mode?

Using Screaming Frog in spider mode is pretty straightforward. All you will need is the URL of the website you want to crawl and confidence that the number of pages to crawl doesn’t exceed 500 if using the free version.

Screaming Frog 2

Once Screaming Frog has finished crawling your website, you will have a complete list of URLs and will be able to filter them by type as well as filter separate columns of the report for a better view. In the screenshot below we have crawled our website, filtered only HTML URLs and chose to see only Page Titles column:

Screaming Frog 3

And now to our favourite function of this tool – you can download/export the reports into an Excel workbook and work on it as you prefer. All you need is to click the “Export” button on the top left-hand side corner.

How to use Screaming Frog in list mode?

In list mode, you have the option to manually upload the list of URLs you would like to be crawled or copy and paste it to the spider. This gives you more flexibility in terms of being able to choose URLs you want to be crawled. The reports in both spider mode and list mode are the same and you can export them to an Excel file too.

In future, we will go into more details of how to use Screaming Frog, as it is difficult to cover such a versatile SEO tool in one blog post as Screaming Frog is.

Keep following our blog for more information on new cool ways to improve your website performance and push your SEO efforts to new heights.

Get in touch with us if you have any questions about this or any other digital marketing topics we discuss in our blog, or if you would like to make an enquiry about our digital marketing services.

Like this post? Read more from our blog here