Handling Google Search Console API 403 Errors When Automating URL Inspections in Next.js

The Problem

When setting up an automated cron job in Next.js to inspect sitemap URLs via the Google Search Console URL Inspection API, the script immediately throws a 403 PERMISSION_DENIED error stating that the service account does not own the site, even though the service account email was added as a full owner in Search Console.

{
  "error": {
    "code": 403,
    "message": "You do not own this site, or the inspected URL is not part of this property.",
    "status": "PERMISSION_DENIED"
  }
}

This completely blocks automated indexing checks for internal platforms like PulseROI, where we track thousands of dynamic URLs across marketing pages and creator intelligence reports.

What I Tried First (and Why It Failed)

My initial assumption was that granting the service account email owner permissions inside the Google Search Console web UI was sufficient. I wrote a standard OAuth2 client loader using google-auth-library with a JSON key file stored in an environment variable.

import { google } from 'googleapis';

const auth = new google.auth.GoogleAuth({
  credentials: {
    client_email: process.env.GCP_CLIENT_EMAIL,
    private_key: process.env.GCP_PRIVATE_KEY?.replace(/\\n/g, '\n'),
  },
  scopes: ['https://www.googleapis.com/auth/webmasters.readonly'],
});

const searchconsole = google.searchconsole({ version: 'v1', auth });

// This failed with 403 even after permissions were verified
const res = await searchconsole.urlInspection.index.inspect({
  requestBody: {
    inspectionUrl: 'https://pulseroi.in/blog',
    siteUrl: 'https://pulseroi.in/',
  },
});

Even with valid credentials and correct scopes, the API continued throwing 403 errors. The documentation implied that site ownership was purely a property-level setting, but it missed a critical distinction regarding Domain Properties vs. URL-prefix properties in Search Console.

The Actual Fix

The root cause was twofold: first, the siteUrl parameter in the request body must match the exact property format registered in GSC; second, service accounts cannot access Domain-level properties (sc-domain:pulseroi.in) unless the property is explicitly set up as a URL-prefix property (https://pulseroi.in/).

Here is the corrected TypeScript implementation used in our background worker to handle authentication and format validation safely:

import { google } from 'googleapis';

interface InspectParams {
  url: string;
  siteUrl: string;
}

export async function inspectUrl({ url, siteUrl }: InspectParams) {
  const auth = new google.auth.GoogleAuth({
    credentials: {
      client_email: process.env.GCP_CLIENT_EMAIL,
      private_key: process.env.GCP_PRIVATE_KEY?.replace(/\\n/g, '\n'),
    },
    scopes: ['https://www.googleapis.com/auth/webmasters.readonly'],
  });

  const searchconsole = google.searchconsole({
    version: 'v1',
    auth,
  });

  try {
    const response = await searchconsole.urlInspection.index.inspect({
      requestBody:
      {
        inspectionUrl: url,
        siteUrl: siteUrl,
      },
    });

    return {
      success: true,
      data: response.data.inspectionResult,
    };
  } catch (error: any) {
    console.error(`[gsc-client] URL Inspection error 403:`, error.message);
    return {
      success: false,
      error: error.message,
    };
  }
}

Ensure your Google Search Console property is added as a URL-prefix (e.g., https://pulseroi.in/) rather than a Domain property. Service accounts are not granted access tokens that successfully resolve against DNS-verified domain properties in the URL Inspection API.

What Surprised Me

Google's error message for this specific failure mode is completely misleading. Instead of indicating that service accounts are unsupported on domain-level properties, it returns a generic authorization failure (You do not own this site). This led to hours of debugging IAM roles and service account JSON keys when the actual fix required recreating the property as a URL-prefix in the Search Console dashboard.

The Result

After switching our GSC property configuration to URL-prefix and updating our Next.js API route helper, our automated inspection cron jobs completed successfully. Error rates dropped from 100% (hard failures across all inspection calls) to 0%, allowing us to monitor indexation status alongside our creator intelligence pipelines.

Where to Go From Here

If you are automating SEO workflows in Next.js, make sure to implement robust error boundaries around your external API calls so a transient GSC or Neon DB timeout doesn't block your entire build or cron cycle. To see how we tie automated monitoring together with our core creator data pipelines, explore PulseROI.