If you’ve ever clicked on a link hoping to buy that amazing product that was number two in a “best of” list only to have your hopes crushed by landing on a 404 page, you’ve experienced HTTP status codes firsthand. When everything is going well, you won’t see these codes displayed, but they’re generated every time your browser interacts with a server. Understanding HTTP status codes can help you diagnose problems with your website that could be frustrating your visitors as well as negatively impacting your SEO ranking.
While it may sound intimidating and technical, understanding HTTP status codes is fairly simple, and it’s an important tool in understanding your SEO results. It’s important to monitor and resolve any error status codes on your website because Google uses them as an indicator in evaluating your site. Once you understand what they mean, you can use Google Search Console to find and repair any error status codes on your website.
How HTTP response status codes are classified and what they mean
HTTP status codes are three-digit numbers that are classified into five groups based on the type of response the server returns. Some common ones include:
- 100-199: These are informational responses that indicate that the request was received and is continuing.
- 200-299: These are successful response codes that let you know your request was successfully processed.
- 300-399: These codes indicate that additional action is needed to complete the request, usually a redirection.
- 400-499: These are error codes that indicate the problem is caused by the client.
- 500-599: These are error codes that indicate the problem is caused by the server.
The most common HTTP status codes
There are many different status codes that all mean something different within the category represented by the first number. There’s even a code, 418, that was created as an April Fool’s joke. It means, “I’m a teapot,” and it’s the server response when you ask a teapot to brew coffee. Code 451 is a reference to the dystopian novel Fahrenheit 451 and means the request is unavailable for legal reasons, such as being banned by a government.
However, those are unusual codes you’re not likely to see very often. The most common HTTP status codes you’ll have to deal with include:
- 200: This is not an error code. It means everything went OK, and the request was successfully processed.
- 301: This code indicates that the data you’re requesting has been permanently moved. This happens when the data you’re requesting has been permanently replaced with something else. You won’t see an error message with this request because the URL will automatically be changed.
- 302: This lets you know that what you’re looking for has been temporarily moved. The original URL is still valid, but it’s being temporarily redirected somewhere else. As with the 301 code, you won’t see an error message with this one. The redirect will happen automatically.
- 403: This is a “forbidden” error code that occurs when you don’t have access to whatever you’re requesting. This can happen when you try to access a password-protected site without logging in.
- 404: This is the dreaded “not found” message that pops up whenever the page you’re looking for isn’t there. It may have been taken off the website or moved without being redirected. It could also mean that you typed in the URL incorrectly.
- 500: You’ll get this code if there’s an internal server error.
- 503: This is another server error that occurs when the server is overloaded.
How do HTTP status codes affect your SEO ranking?
Googlebot is the software that Google uses to rank websites, and it will affect your SEO. It’s called a spider because it “crawls” through your website. It follows the links, gathers data, and indexes it. This index is then used to generate search engine results. The easier it is for the spider to crawl through your site, the higher your search engine ranking will be, all other things being equal. Ranking as high as possible on Google is an important part of search engine optimization.
Links are an important part of how the Googlebot gathers data because it works by going from link to link through your website. For every link it follows, it checks the status code. When it runs into an error status code, that code can interfere with the crawling. If your website can’t be crawled, it can’t be indexed, and it can’t show up in search engine results.
Using Google Search Console to monitor your website’s HTTP status codes
While you can check the status code of any page by typing the URL into a tool like Web Sniffer, Google Search Console is a faster, easier way to monitor problematic status codes on your entire website. Google Search Console provides tools that help you optimize your website for Google search results. Best of all, it’s completely free and works regardless of what platform you use.
Before you can get started with Google Search Console, you have to verify that you own your site. Google Search Console gives you a lot of sensitive information about your site that you don’t want just anyone to be able to access. There are several different ways you can verify your site, and the one that’s easiest for you will depend on what platform you’re using. Choose one of the methods listed in Search Console Help to get access.
Connecting your website to Google Search Console may be a little fiddly and time-consuming when you first attempt it. However, it’s such a valuable tool for monitoring so many aspects of your website’s SEO that you’ll find it’s worth the effort.
Once you have your site verified, you can use the Index Coverage report to find out which pages on your site are indexed by Google. It also shows you which pages have errors that are preventing your pages from being indexed. When you click on the report, you’ll see four tabs at the top of the page:
- Error: These pages couldn’t be indexed because of an error. Scroll down to see what errors were found and how many pages were affected.
- Valid with warning: These pages may have been indexed but have some problems you might need to fix.
- Valid: These pages have been indexed.
- Excluded: These pages don’t belong in the index for some reason, such as duplicate URLs, or it may have blocked robots.
Scroll down to see exactly what errors were found and how many pages are affected. Status codes that are causing problems with Google indexing will show up under the Error, Valid with warning, or Excluded sections. These codes will be in the 300s, 400s, or 500s, but the exact code may not be given.
To find the specific issue that’s causing problems, put the URL of the page with an error into the URL Inspection tool. You’ll be able to read about the error that’s causing problems and fix it.
Repairing HTTP status code errors found in Google Search Console
After Google Search Console shows you the errors that need addressing, you’ll want to understand what type they are to know how to go about fixing them. The errors that show up in Google Search Console will be grouped in the following categories:
Submitted URL not found (404)
This means the page couldn’t be found by Google. Double-check it by putting the URL in your browser. If the page doesn’t exist, you have two options. If you don’t want it to be indexed, just leave it and it will eventually be removed. This frequently happens when content is removed from websites.
If you want the URL to go to another page, you can redirect it. Your website platform may have built-in redirect commands. Just make sure that you redirect it with a 301 status code if it’s a permanent redirect. This will give your links equal weight as the original URL and is better for SEO.
Server error (5xx)
This indicates Google couldn’t access the site because the server was down or overloaded. This may be a temporary issue, but you should investigate it further if it doesn’t resolve. Server errors can be caused by user errors or problems with your server.
Redirect Error
If you see this error, it’s letting you know that the original URL has been redirected, but the redirected link isn’t working. This may be because the redirect chain was too long, there was a redirect loop, or the URL was bad. You’ll have to find the redirect on your platform and fix the error.
Submitted URL seems to be a soft 404
A soft 404 occurs when a webpage tells the user it doesn’t exist but returns a 200 HTTP status code, indicating it was successfully found. This can happen for several reasons. If the page no longer exists, it needs a 404 code or a 301 redirect code, or you can simply remove it from your sitemap.
There are times when a soft 404 is expected, and you won’t need to do anything to fix it. If you have a page on your site that can only be accessed after a visitor completes an action, it will return a soft 404 when you try to access it directly.
Submitted URL marked ‘noindex’
The ‘noindex’ tag tells search engines not to index your page. This error happens if you have a page that your sitemap submits for indexing, but it has a ‘noindex’ command. Sometimes there are pages you don’t want indexed, which is fine. Just verify that pages you’ve tagged ‘noindex’ are pages you truly don’t want indexed and remove them from your sitemap.
If this error occurs on a page you do want Google to index, simply remove the ‘noindex’ tag.
Submitted URL blocked by robots.txt
A robots.txt file blocks bots from accessing the page. This includes spiders like Googlebot. This is usually done to manage crawler traffic on your site, perhaps to keep crawlers from accessing unimportant pages. It’s not a reliable method for keeping a page off of Google search results, though, so use ‘noindex’ if you want to do that.
As with the ‘noindex’ errors, just verify that you want the pages listed to block bots. If not, remove them from the robots.txt file.
Submitted URL returns unauthorized request (401)
You’ll get this error if you have a password-protected page but it’s been submitted for indexing through your sitemap. If you want this page to remain password-protected, add a ‘noindex’ directive, and remove it from your sitemap. If you want it to be indexed, remove the password protection.
Submitted URL returned 403
This is similar to a 401 error but generally occurs because the URL requires authorized access, such as admin privileges. The fix is the same as for a 401 error.
Submitted URL blocked due to other 4xx issue
This error code means that the URL returned a 400-level HTTP status code other than 401, 403, or 404. You can further investigate the cause using the URL Inspection tool, or you can remove the page from your sitemap.
How to fix warnings in Google Search Console
Though they’re not as serious as an error, you should still check your warnings and resolve them to optimize your site. Possible warnings include:
Indexed, though blocked by robots.txt
Pages with this warning were indexed even though they were blocked by robots.txt. If you don’t want this page indexed by Google, use the ‘noindex’ attribute. If you do want it indexed, remove it from robots.txt.
Page indexed without content
This page was indexed, but Google couldn’t recognize the content. This could be because it’s in a format Google doesn’t recognize or it’s hidden. Use the URL Inspection tool to get more details about the cause of this warning.
You can use HTTP status codes to optimize your site
Now that you understand what HTTP status codes are, how they’re used, and how to find them on your website, make monitoring and repairing your status code errors part of your website maintenance. Create an account in Google Search Console and set up a routine for checking your Index Coverage report. Fix any unintentional status codes errors to increase the health and value of your website in the eyes of Google’s search engine ranking.
By understanding how SEO works and what factors affect it, you’ll be more in control of your organic traffic. Along with creating great content, optimizing your keyword research and maximizing backlinks, monitoring your status code errors can help you get the most out of your SEO marketing strategy.
