Understanding 404 errors: How they impact SEO and how to fix them

We’ve all seen them – a 404 error is one of the most commonly encountered issues when browsing. It occurs when a user tries to visit a webpage that doesn’t exist or has been removed from the server.

The term “404” refers to the HTTP status code that the server returns to indicate that it cannot find the requested page.

This error is part of the HTTP (Hypertext Transfer Protocol), a communication protocol used by the internet to transfer and display web pages.

The 404 Not Found error is displayed in the browser when the requested URL cannot be found on the server. While this might seem like a minor issue, it can have serious implications for your website’s performance, user experience, and rankings.

Why do 404 errors occur?

There are several reasons why 404 errors occur, including:

1.  Broken links: External websites or internal links that point to a page that no longer exists or has been moved.
2.  Deleted or moved content: If a page or product is deleted, and no redirects have been set up, users will encounter a 404 error.
3.  Misspelled URLs: When users manually type in an incorrect URL or follow a link with a typo in the address.
4.  Site restructuring: When you reorganise your website’s structure, some URLs might no longer point to valid content without proper redirects.

Why can 404 errors be bad for SEO?

404 errors can have a significant negative impact on your website’s SEO, affecting both search engine rankings and user experience. Here’s an expanded explanation of why 404s are bad for SEO:

Decreased crawl efficiency

Search engines like Google rely on web crawlers to explore and index pages across the internet. When a crawler encounters a 404 error, it essentially hits a dead-end, wasting crawl budget.

A crawl budget refers to the number of pages a search engine is willing to crawl on your site within a specific timeframe. If crawlers frequently come across 404 errors, they may not have the opportunity to index other important pages on your site, which can affect the overall indexing of your content.

Negative user experience

404 pages are frustrating for users, and Google takes user experience into account when ranking pages.

If visitors land on a 404 page, they’re likely to leave quickly, increasing your bounce rate. A high bounce rate signals to search engines that users are not finding the content they are looking for, which can hurt your rankings.

If users repeatedly encounter 404 errors, they may lose trust in your site, further damaging your reputation.

Lost link equity

When you have 404 errors on pages that once had backlinks (either from external sites or internal links), you’re losing valuable link equity.

Link equity, or “link juice,” is a metric that helps determine how much authority a page passes on to others via links. If a page with backlinks returns a 404 error, those backlinks no longer provide any value, which can negatively impact your overall SEO performance.

Losing these links means you’re also losing potential referral traffic and ranking power.

Broken internal linking structure

Internal links are crucial for spreading link equity throughout your site and helping search engines understand the structure of your website.

If your internal links lead to 404 pages, it disrupts the overall link flow and could create a poor user experience.

A well-structured internal linking system is vital for SEO because it helps both search engines and users navigate your site more efficiently.

Impact on indexation and site authority

Search engines might eventually start ignoring your pages if they consistently encounter 404 errors on your site.

If Google regularly encounters dead ends, it could reduce the number of pages it crawls or indexes, ultimately lowering your site’s authority and visibility.

This can cause a drop in organic rankings, especially if the 404 pages were previously indexed and had decent traffic.

Potential for duplicate content issues

In some cases, you may encounter “soft 404s,” where a page might return a “404” status code while still displaying content that appears to be an active page (e.g., a custom 404 page with content).

These can confuse search engines, causing them to mistakenly index a page that isn’t truly a valid one. This can lead to duplicate content issues or cause search engines to index incorrect versions of pages, further complicating your SEO efforts.

Hindering new content discovery

Search engines use links to discover new content. If there are too many 404 errors within your website’s structure, crawlers may miss important new pages that you’ve published, leading to delayed or failed indexing of your content.

This means your fresh content could remain invisible to search engines, stalling your efforts to gain organic traffic and rank for new keywords.

Impact on site reputation and trustworthiness

Google, as well as users, consider the overall health of your website when determining rankings. A site littered with 404 errors can appear untrustworthy, outdated, or neglected.

Trust is a crucial ranking factor, and if Google detects that your site has significant technical issues, it may negatively impact how your content is ranked, even if your content itself is high-quality.

Hard vs soft 404s

Understanding the difference between hard 404s and soft 404s is crucial for managing your website’s health and maintaining good SEO practices.

What is a hard 404?

A hard 404 is a straightforward error where a page is requested but does not exist on the server, and the server responds with a 404 status code.

This tells both users and search engines that the page cannot be found. A hard 404 error is the most direct and widely recognised form of a 404 error.

What is a soft 404?

A soft 404 occurs when a page appears to be available but returns a 200 status code (successful) with content that indicates the page is missing or not found.

Google may treat these pages as if they are 404 errors because they do not provide valuable content.

Soft 404s are tricky because they can still be indexed by search engines, potentially consuming crawl budget and lowering your site’s overall SEO performance.

For example, if you have a product page that no longer exists but instead of returning a 404 error, it shows a message saying “This product is no longer available,” Google might still consider it a valid page.

However, this page could result in lower rankings, as search engines are not able to index content that doesn’t add value.

How to fix soft 404s:

1.  Ensure proper status codes: Make sure that pages that no longer exist return a 404 or 410 Gone status code, rather than a 200 status code.
2.  Redirect the page: If the page has been moved or replaced with similar content, set up a 301 redirect to the relevant page.
3.  Remove low-quality pages: If the page has little or no content, consider removing it entirely or providing a proper redirect.

SEO best practices for 404s

It’s essential to handle 404 errors properly to ensure that they don’t harm your website’s SEO. Below are some best practices for managing 404 errors:

1. Create a custom 404 page

Rather than relying on a default error page, take the time to design a custom 404 page that is consistent with your branding.

This not only helps users understand that they’ve reached an error page but also provides them with useful options to continue browsing. Elements of a great 404 page include:

• A clear error message such as “Oops! This page doesn’t exist” or “Sorry, we couldn’t find that page.”
• A search bar that enables users to search for relevant content on your site.
• Links to popular pages such as your homepage, blog, or most visited categories to redirect users to valuable content.
• A call-to-action encouraging visitors to browse other parts of your site or sign up for a newsletter.

A good example of this can be seen on Royal Mail’s 404 page:

2. Implement 301 redirects

When a page is deleted or moved, a 301 redirect is a useful tool for redirecting users (and search engines) to a new, relevant page.

A 301 redirect tells search engines that the content has permanently moved, passing along the SEO value and link equity from the old page to the new one.

This ensures that the deleted or moved page doesn’t result in a 404 error and also helps retain the organic traffic associated with the original URL.

3. Regularly check for broken links

Regularly checking for broken links on your site is a vital aspect of maintaining a healthy website. Broken links can result in poor user experiences and negatively impact SEO. Use tools like Google Search Console, Screaming Frog, or Ahrefs to monitor for broken links and 404 errors. When you find them, either fix the broken link or set up a redirect to an appropriate page.

4. Avoid redirect chains

While redirects are useful, overusing them can cause performance issues. A redirect chain occurs when a URL is redirected multiple times before reaching its final destination.

This can slow down your site, increase load times, and cause search engines to devalue the content. To avoid redirect chains, ensure that your redirects go directly to the relevant page and not through multiple intermediary pages.

5. Monitor crawl errors

Google Search Console is a valuable tool for monitoring how Googlebot crawls your site. It provides reports on crawl errors, including 404 errors, which can help you identify issues with your URLs.

Regularly check these reports to ensure that there are no broken links or pages that are returning 404 errors.

6. Don’t overuse 404 pages for soft 404s

While a hard 404 error indicates a missing page, a soft 404 occurs when a webpage returns a successful HTTP status code (200 OK) but displays an empty or “Page Not Found” message.

Soft 404s can be problematic because they waste crawl budget and may confuse search engines. Ensure that any missing pages return the proper HTTP status code (404 or 410) to indicate that the page is truly gone.

Best practices for 301 redirects

301 redirects are a crucial aspect of maintaining SEO performance when content or URLs are moved, deleted, or changed on a website.

Proper implementation of 301 redirects ensures that you don’t lose search engine rankings or frustrate users. Below are essential best practices to follow when using 301 redirects.

1.  Prioritise relevance in redirects
   When implementing 301 redirects, ensure the destination URL matches the content of the original page as closely as possible. Redirecting to an unrelated page can confuse both search engines and users, resulting in a poor experience. The more relevant the redirect, the better the chances that users and search engines will view your website as valuable and well-structured.
2.  Ensure a one-step redirect
   A direct 301 redirect from the old URL to the new one is ideal. Avoid creating redirect chains, where one page redirects to another, and so on. Redirect chains not only slow down page loading times but can also impact the link equity passed to the final destination. To optimise SEO, ensure that each page is only redirected once, preferably to the most relevant and updated content.
3.  Use 301 redirects during site migrations
   When migrating to a new website or restructuring your URLs, a 301 redirect is essential to let search engines know that the pages have moved permanently. This will prevent your old pages from being indexed and losing traffic, allowing the new URLs to inherit the rankings and link equity of the original pages.
4.  Limit redirects to improve site speed
   While redirects are necessary for smooth user navigation, excessive use of redirects can slow down your site. Multiple redirects in a sequence create latency, which can lead to a slower page load time. This negatively impacts user experience and can also affect SEO rankings, as page speed is a ranking factor. Minimise the number of redirects and aim for direct and efficient redirections whenever possible.
5.  Update internal links to reflect new URLs
   After implementing 301 redirects, ensure that internal links within your website are updated to reflect the new URLs. Relying on redirects for internal linking can slow down page navigation and introduce unnecessary redirects. Updating internal links helps speed up the user experience and ensures that search engines crawl the new pages more efficiently.
6.  Use the correct HTTP status code
   A 301 redirect indicates a permanent move, which is crucial for SEO, as it allows link equity to be passed to the new page. If the change is temporary, you should use a 302 redirect instead. Avoid using a 301 redirect for temporary changes, as this could confuse search engines and affect the accuracy of your page indexing.
7.  Document your redirect strategy
   When implementing multiple redirects, it’s a good idea to document the strategy. This includes noting the original URLs, the corresponding destination pages, and why each redirect is being used. Proper documentation helps maintain consistency and ensures that all redirects are intentional, especially when scaling your website.
8.  Avoid redirecting low-value pages
   If you have pages with little to no SEO value or outdated content, it’s better to remove them entirely rather than redirect them to other pages. Redirecting low-value pages can clutter your site structure and pass irrelevant link equity to pages that don’t need it. In some cases, it might be better to return a 410 status code (content deleted) rather than use a 301 redirect.

Conclusion

404 errors are an unavoidable part of managing any website, but with careful attention to detail, you can ensure that they do not negatively impact your SEO. While 404 pages are common, a well-designed 404 page and proactive measures such as implementing 301 redirects, regular link checks, and correcting soft 404s can prevent these errors from harming your search engine rankings.

Remember, user experience is key. A 404 page that guides users to helpful content or allows them to search for what they need can turn a frustrating experience into a positive one. Meanwhile, addressing hard and soft 404 errors promptly helps to ensure your website’s health, maintaining good SEO practices and preserving your search rankings.

With regular monitoring and the proper tools in place, handling 404 errors can become a manageable task, helping you keep your website in top shape and providing an optimal experience for users.