Technical SEO
Duplicate Content in SEO: What It Is and What to Do
Duplicate content is text that is identical or very similar to text at another URL, on your own site or elsewhere. It is rarely a penalty. The harm is that Google must pick one URL to show, so the wrong page can rank and links get split. Fix it by choosing one preferred URL.
This guide covers what duplicate content is, whether Google penalises it, the common causes, how to find and fix it, and a real case from our own site. Sources are checked as of 12 November 2026.
What is duplicate content?
Duplicate content is substantial blocks of content that match, or are very similar to, content at another web address, so the same content is reachable at multiple URLs. Moz puts it simply: if the same content appears at more than one URL, you have duplicate content. Google's long-standing definition covers copies within one domain and across domains, and notes that most duplication is not deceptive in origin.
How does duplicate content differ from similar and thin content?
Screaming Frog separates three problems. Duplicate content is an exact copy, similar content is mostly the same with small differences, and thin content offers little value of its own. They need different fixes: duplicates need one preferred URL, similar pages need consolidating or differentiating, and thin pages need real substance.
Is duplicate content the same as keyword cannibalisation?
No. Duplicate content is about matching text. Keyword cannibalisation is about two pages competing for the same search, even when the text is different. They often occur together, and our guide to keyword cannibalisation covers the second problem.
Is there a duplicate content penalty?
Ordinary duplication does not trigger a penalty. In a 2008 post, Google said there is no such thing as a duplicate content penalty. Duplicate content on a site is not grounds for action unless the intent is to be deceptive and manipulate results. Scraping other sites' content and thin affiliate pages are separate problems that can be penalised. Screaming Frog reaches the same conclusion: in the vast majority of cases there is no penalty.
What does Google do with duplicate URLs?
Google groups duplicate URLs into a cluster and chooses one to show, which the 2008 post describes as the "best" URL to represent the cluster. It also consolidates signals such as link popularity onto that URL. Your problem arises when Google's choice differs from yours, so tell Google which version you prefer.
How does duplicate content hurt your site?
It hurts by confusing which URL deserves to rank, not by triggering a punishment. Three problems recur across Moz, Yoast and Screaming Frog:
- The wrong URL ranks. Google consolidates duplicates and may show a version you did not intend, such as a parameter URL or an old page.
- Link signals are split. If other sites link to different versions of the same page, the value is spread across several URLs instead of one.
- Crawling is wasted. Screaming Frog notes that Googlebot crawls a site for a limited time, so repeated copies use up visits that new or updated pages could have had.
What causes duplicate content?
Most duplicates are technical accidents rather than decisions, which is why duplicate content sits within technical SEO. Yoast says there are dozens of causes, and most of them are technical.
| Cause | Example | Fix |
|---|---|---|
| URL parameters | Tracking codes, session IDs, sorting and filters on the same page | Canonical tag or a clean URL structure |
| Parameter order | /?id=1&cat=2 and /?cat=2&id=1 showing the same page | Keep one order, canonical the rest |
| WWW and non-WWW, HTTP and HTTPS | The same page at four addresses | 301 redirect to one version |
| Trailing slash differences | /guide and /guide/ both load | Pick one, redirect the other |
| Printer-friendly pages | A second URL for the same article | Block or canonical |
| Comment pagination | An article repeated at /comment-page-1/ and other paginated URLs | Turn off or canonical |
| Tag and category archives | WordPress archive pages that repeat post text | Noindex the archive pages |
| Staging or test sites | A development copy that Google can index | Password-protect or noindex it |
| Separate mobile or AMP URLs | Desktop and mobile versions on different addresses | Canonical and consistent annotations |
| Scraped or syndicated copies | Another site republishes your article | Link back to the original; ask for a canonical |
| Copied content or templated text | Headings or FAQs copied from a competitor, or location pages that change only the town | Rewrite from your own research |
| Repeated titles and descriptions | The same title or meta description on many pages | Write unique ones for each page |
On the last row, Google says identical or similar descriptions across a site are not helpful when pages appear in results. Our guide to title tags and meta descriptions shows how to fix them.
How do you find duplicate content?
Look at what Google has indexed, search for your own text and crawl the site to find the same content at different URLs. Follow these steps:
- Check [Search Console](/blog/gsc-opportunities/). In the Page indexing report, look for the duplicate statuses: "Duplicate without user-selected canonical", "Duplicate, Google chose different canonical than user" and "Duplicate, submitted URL not selected as canonical". Each means Google found another version it preferred.
- Compare the indexed count. Search
site:yourdomain.comor read the indexed total in Search Console. It should roughly match the number of pages you created. A much larger number means pages are being generated automatically. - Search for your own wording. Paste a distinctive sentence or heading in quotation marks into Google and see whether the same text appears on another URL.
- Crawl the site. A crawler such as Screaming Frog finds exact duplicates by default and near duplicates when you enable that option, with a 90% similarity threshold you can adjust. Siteliner scans a site for pages with a lot of duplicate content.
- Check titles and descriptions. Export them and look for repeats.
- Test the site versions. Load
http://,https://,wwwand non-wwwversions of your home page. All of them should end up at one address.
How do you fix duplicate content?
Decide which URL is the right one, then tell Google with a redirect, a canonical tag or a consolidation. Google says redirects are a strong signal that the target should become canonical, rel="canonical" is also a strong signal and including a URL in a sitemap is a weak one. None is required, and Google may choose a different URL if it judges it better.
| Situation | Fix | Why |
|---|---|---|
| An old or duplicate URL should not exist | 301 redirect to the preferred page | Sends visitors and signals to one URL |
| Both URLs must stay live | rel="canonical" on the duplicate | Names the preferred version without removing the page |
| A page should exist but not appear in search | noindex | Removes it from results, but passes no signals |
| Several similar pages split the topic | Merge into one stronger page, then redirect | Combines the value of all of them |
| Text copied from elsewhere | Rewrite it | No technical tag fixes copied words |
| Your content copied by another site | Ask for a link back or a canonical | You cannot edit their pages |
Use internal links consistently, too. If your preferred address is https://www.example.com/page/, every internal link should point to that exact version, as Moz advises.
What changed since older guides?
Some advice in older guides no longer applies. Moz and Screaming Frog both describe Search Console's URL Parameters tool, but Google announced on 28 March 2022 that it was deprecating the tool, saying only about 1% of the parameter configurations in it were useful for crawling. Google now handles parameters automatically, and site owners who need more control can use robots.txt rules or hreflang. Check the date of any guide before following its Search Console steps.
What does a real duplicate-content fix look like?
We fixed one on our own site. In September 2026 we launched a German-language site listing a coffee chain's menu and prices on a new domain. Search Console reported the homepage as "Crawled - currently not indexed": Google had visited it and chosen not to index it, even though the robots rules and meta tags were correct.
We compared the page text with competing sites and found 26 section headings and a 30-question FAQ copied word for word from a competitor. They sat inside the script that builds the menu, so a proofread of the visible copy had missed them. We rewrote all 26 headings and replaced the FAQ with 24 new questions drawn from real "People also ask" results and a keyword sheet. We also rewrote the homepage introduction, tripled the length of three thin sub-pages and rewrote all ten meta descriptions. The homepage was indexed within hours of the rewrite going live. Clicks rose from 84 (1 to 28 September) to 908 (1 to 28 October 2026), and impressions from 10.5K to 41.9K. This is one site, and the rewrite was only part of the work, so treat it as an example, not a promise. The full case study has the Search Console figures and the check we recommend.
How do you prevent duplicate content?
Prevent it by building one clear version of every page and checking templates before you publish. Use this checklist:
- Redirect every alternative of your domain to one version.
- Give each page a self-referencing canonical tag.
- Link internally to the preferred URL only.
- Noindex low-value archive and tag pages.
- Write a unique title, description and body for each page.
- Check templates and scripts, not only visible copy, for copied text.
- Protect staging sites with a password.
- If you syndicate an article, make sure the copy links back to your original.
A worked example from this website
Our solicitor pages showed a real intent overlap. The industry page at /industries/lawyers/ and the guide at /blog/seo-for-solicitors/ both answered the same commercial search. We carried unique practice-area prioritisation material into the industry page, removed the guide from the sitemap and set a permanent redirect. This is consolidation of intent, not a claim that every repeated sentence needs a redirect.
For a similar decision, compare Search Console queries and page purpose first. If two URLs answer different stages of the journey, keep both and make the distinction clear. If they answer the same decision, choose the stronger owner, preserve useful sections and point body links to that URL. Test the old URL with a single-hop redirect and check the canonical on the destination.
See the live SEO for solicitors page, our SEO services and the free SEO audit for the broader process.
01What is an example of duplicate content?+
The same product description appearing at two web addresses is the classic example. Others include a page reachable with and without www, a printer-friendly copy of an article and two location pages that differ only by the town name.
02Is duplicate content bad for SEO?+
It can be, but not through a penalty. Google clusters duplicates and picks one to show, so the risk is that the wrong URL ranks or that links are split. Fixing it means choosing one preferred URL.
03How can I check whether my content has been copied?+
Paste a distinctive sentence or heading in quotation marks into Google and see which other URLs show it. Tools such as Copyscape and Siteliner scan for copies, and a crawler flags duplicates inside your own site. If you find a copy, ask for a link back or a canonical tag.
04How much duplicate content is acceptable?+
Google publishes no percentage. Tools use their own thresholds, such as the 90% similarity default for near duplicates in Screaming Frog, but these are settings for finding problems, not Google rules. Aim for each page to have a distinct purpose and text.
05Does Google penalise duplicate content?+
Not for ordinary duplication. Google has said duplicate content on a site is not grounds for action unless the intent is to be deceptive and manipulate results. Scraping other sites' content is treated differently from accidental duplicates such as parameters.
06Should I use a canonical tag or a 301 redirect?+
Use a 301 redirect when the duplicate URL should no longer exist, and a canonical tag when both must stay live. Google treats both as strong signals, but redirects remove the duplicate entirely, which is simpler to maintain.
07Do canonical tags always work?+
No. Google treats a canonical tag as a signal, not an instruction, and it can choose a different URL if it judges that one better. Make sure your internal links, sitemap and redirects all point to the same preferred URL so the signals agree.
08Do duplicate meta descriptions count as duplicate content?+
They are a related problem, not a ranking penalty. Google says identical or similar descriptions on every page are not helpful when individual pages appear in results, so write a specific one for each page.
09What if another site copies my content?+
Google clusters the copies and chooses a version to show, which is not always the original. Ask the other site to link back to your article or to use a canonical tag pointing to it. A link to the original is one of the few fixes available when you cannot edit their site.
10Does www versus non-www matter?+
Yes, if both load without redirecting. Moz and Search Engine Land both warn that the same page at both addresses creates duplicates. Pick one version and redirect the other, then make sure your internal links match.
11Does duplicate content matter on multi-country sites?+
It can. Screaming Frog warns that targeting several countries in the same language, such as Spain, Mexico and Argentina, can create very similar content. Make each version genuinely local rather than a copy with a new flag.
12How often should I check for duplicate content?+
Check after every migration or template change, when you add many pages at once and on a regular schedule. A crawl and a look at the Page indexing report in Search Console will surface most problems.
Free SEO audit
Get a straight answer on what to fix first.
Request a free SEO audit and we will come back with the largest constraint on your organic growth and the first practical fix. No sales sequence.
Prefer to hire through Upwork? Hire me on Upwork