Google URL Inspection API is now available in Screaming Frog

Screaming Frog wasted no time in incorporating Google’s new URL inspection API, which provides access to current indexing data. The new API enables Screaming Frog to include seven brand new filters that detect website issues that require attention.

In addition to expanding Screaming Frog’s utility, the new update codenamed Romeo, fixes a log4j vulnerability, adds new keyboard shortcuts, and fixes some bugs to make it more stable.

7 More Ways to Repair Websites

Google’s new URL Inspection API enables software tools such as Screaming Frog to gain access to Google Search Console and import data.

API is an abbreviation for Application Programming Interface. It functions as a bridge between two software programs, allowing them to connect in order to share data and achieve a certain level of integration.

The URL Inspection API integration in Screaming Frog enables the popular site auditing program to provide the following brand new troubleshooting filters.

Seven New Auditing Filters:

“URL Is Not on Google
The URL is not indexed by Google and won’t appear in the search results. This filter can include non-indexable URLs (such as those that are ‘noindex’) as well as Indexable URLs that are able to be indexed. It’s a catch all filter for anything not on Google according to the API.
Indexable URL Not Indexed
Indexable URLs found in the crawl that are not indexed by Google and won’t appear in the search results. This can include URLs that are unknown to Google, or those that have been discovered but not indexed, and more.
URL is on Google, But Has Issues
The URL has been indexed and can appear in Google Search results, but there are some problems with mobile usability, AMP or Rich results that might mean it doesn’t appear in an optimal way.
User-Declared Canonical Not Selected
Google has chosen to index a different URL to the one declared by the user in the HTML.
Page Is Not Mobile Friendly
The page has issues on mobile devices.
AMP URL Is Invalid
The AMP has an error that will prevent it from being indexed.
Rich Result Invalid
The URL has an error with one or more rich result enhancements that will prevent the rich result from showing in the Google search results.”

The URL Inspection Tool API integration adds value by allowing you to overlay Search Console indexing data onto Screaming Frog crawl data, allowing you to see all on-page data points extracted by Screaming Frog juxtaposed with the indexing status.

Screaming Frog now has a Rich Results inspection tool, which allows for bulk data export from the Bulk Export tab in the top menu.

Additional Information

Other updates, in addition to the log4j vulnerability patch, include the addition of keyboard shortcuts (ctrl + W to close crawling projects), making the Word Cloud Visualization tool language-aware to improve word-filtering, fixing a Page Speed Insights bug, an issue with JavaScript crawls that would cause a stall, fixing CSS Path extraction to support leading and trailing slashes, and various other fixes that make Screaming Frog run smoother.

Part 1 of Screaming Frog URL Inspection API Integration?

The update announcement included a statement stating that there is still some integration work to be done to improve it even more.

Due to practical constraints in how data can be fit into the Screaming Frog interface, not all URL inspection data could be fit into it. However, they promised to listen to user feedback and include additional data if users felt it was necessary.

The Screaming Frog announcement noted:

“With use and user feedback, the way we integrate the data to gain additional insights will be improved as well.
If you spot any issues, or would like to see additional data or reporting from the URL Inspection API – just let us know via support.
We already have a few items planned, but wanted to get the basic feature including the most important items out there for users to enjoy quickly.”

So it appears that Screaming Frog will be bringing us even more good news.