Google Indexing API tests with normal URLs, which have NEITHER job posting NOR livestream structured data

Setup

This is useful to start with nodejs and Google Indexing API:

URL_UPDATED

As described here https://developers.google.com/search/apis/indexing-api/v3/using-api?hl=en you can trigger a URL update e. g. like this

const urlUpdate = async (crawlurl,type) => {
const tokens = await jwtClient.authorize()
const options = {
url: 'https://indexing.googleapis.com/v3/urlNotifications:publish',
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
auth: { bearer: tokens.access_token },
json: {
url: crawlurl,
type: 'URL_UPDATED',
}
};
return await rp(options)
}

Next day:

In my setup first the Metadata is requested with https://developers.google.com/search/apis/indexing-api/v3/reference/indexing/rest/v3/UrlNotificationMetadata. Afterwards URL_UPDATED or URL_DELETED is called. Feedback of the my Lambda API looks like this:

URL_DELETED

When I push URL_DELETED for URLs, which were pushed for URL_UPDATED just a view minutes before Googlebot was not visiting. No Bot within 15 Minutes after pushing delete. For none of the test URLs. That’s an example with pushing URL_DELETED seconds after URL_UPDATED:

First learnings:

  • Looks like you can trigger Googlebot with the Indexing API, even for not job posting or livestream structured data URLs.
  • Googlebot always visited the URLs within the first 10 minutes after using the API
  • The first time I triggered the API with a new URL Googlebot was sent almost instantly. With repeated API calls for the same URL it was not willing to crawl again within the first hour. After some hours (don’t know exactly / maybe 8 hours) to push the same URL is possible again.
  • Parameter URLs seem to be handled like new URLs, even if canonicalized.
  • If you send URL_UPDATED and shortly after URL_DELETED Googlebot is not visiting. Seems to be the same “blocked time-frame” no matter what type of API call you do

Indexation

Until now I cannot see any change in Google SERPs

Test 2: Set meta=noindex first and trigger API afterwards

Set to meta=noindex 19:00

Test Results:

NO impact on index or deindex, if you don’t have a jobs or live video markup

Used Googlebots I have seen:

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P)

Used tool for log files:

https://www.screamingfrog.co.uk/log-file-analyser/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store