Google no longer provides direct URLs in search results

The ability to scrape destination URLs directly from Google Search results requires an API call to fetch the redirect location.

Google has begun systematically rewriting organic search result links to a google.com/goto URL, obscuring the final destination URL directly within the search results page's HTML. Instead of displaying the target URL, users are now redirected through Google's own domain, where the actual destination URL is embedded in a custom-encoded parameter. This change, observed consistently in logged-out and private browsing sessions since late August 2026, differs from Google's previous redirect formats like google.com/url. The new mechanism requires an additional request to Google's server to retrieve the target URL from the `Location` header, making it harder for automated scrapers to extract vast numbers of links without repeated interaction with Google's systems. This move appears to be a strategic effort by Google to counter large-scale SERP harvesting by AI crawlers and SEO tools that previously could parse links directly from static HTML.

This shift represents a significant hurdle for automated SERP scraping services, forcing them to adapt their infrastructure to a two-step lookup process. By preventing direct URL extraction from HTML, Google increases the computational overhead and monitoring capabilities for any entity attempting to build an external index of search results. Services like Autom.dev have already updated their pipelines to handle this change by reading the `Location` header, ensuring their clients continue to receive destination URLs without modifying their integrations, though this adds latency and complexity.

The move underscores Google's ongoing commitment to protecting its search index and controlling access to its data, especially in the face of rising AI-driven content generation and analysis tools. This tactic, combined with previous measures, signals a clear intent to make unauthorized and large-scale data extraction from search results increasingly costly and difficult. Consequently, companies relying on scraped SERP data for competitive analysis, AI model training, or directory building will need to invest in more sophisticated scraping solutions or face reduced data acquisition efficiency.