Duplicate Content Checker
Enter a domain to crawl the site and compare every page with every other. Find pages that are copies or near-copies of each other, the same page at several URLs, and repeated titles and descriptions, with what to keep and how to fix the rest.
What the duplicate content checker finds
The checker crawls your site from the homepage and adds the URLs in your XML sitemap, then compares the text of every page with every other page. It reports:
| Finding | What it means |
|---|---|
| Exact duplicates | Two or more URLs serve the same text. Often one page reachable at several addresses: with and without a trailing slash, in different letter case, as /index.html, or with tracking and sorting parameters. |
| Near duplicates | Pages whose text is at least 90% the same (you can choose 80 to 95%), such as location pages that only swap the town name, or product variants with copied descriptions. |
| Mostly shared text | Pages where almost everything is text that also appears across much of the site, so there is little that is unique to the page. |
| Duplicate titles, descriptions and H1s | Indexable pages that share a title tag, meta description or main heading, which makes them look interchangeable in search results. |
Each group lists every URL with its similarity to the URL worth keeping, and shows whether it is already handled by a canonical tag, noindex or hreflang, and how to fix the rest.
How pages are compared
Menus, footers, cookie notices and other text that repeats on at least half of the site's pages are set aside first, so pages are compared on their own content, not on the template they share. The checker then splits each page's text into overlapping runs of five words. Similarity is the share of those runs two pages have in common: 100% means the same text, and 90% means only small differences, such as a changed name, price or sentence.
Pages are read as the server sends them, before JavaScript runs, which is the version search engines see first.
How to fix duplicate content
- Same page, several URLs: pick one address and 301-redirect the others to it. Where a variant has to stay reachable, such as a filtered or tracked URL, add a canonical tag on it pointing at the main URL.
- Two pages doing the same job: merge them into the stronger one, usually the page with more links and traffic, and 301-redirect the other to it.
- Pages that each need to exist: rewrite them so each has substantial content of its own: local details, specifications, reviews, photos and answers to the questions specific to that page.
- Duplicate titles and descriptions: write a unique title and description for each page that says what makes it different.
Questions
Does Google penalise duplicate content?
Not as a penalty, unless the copying is deceptive or spammy. What Google does is pick one version of duplicate pages to show and filter out the rest, and it may pick a different URL from the one you want. Links and signals are split across the copies, and crawl time is spent on pages that add nothing. Fixing duplicates puts you back in control of which URL ranks.
What percentage of duplicate content is acceptable?
There is no official figure. Shared navigation, footers and short boilerplate are normal on every site, which is why the checker sets them aside before comparing. Pages whose own content is 90% or more the same are the ones search engines tend to treat as duplicates, so that is the default threshold.
Does it check for content copied from other websites?
No. It compares the pages of one site with each other, which is where most duplicate content comes from. To find copies of your text elsewhere, search Google for an exact sentence from the page in quotes.
Should I use a canonical tag or a 301 redirect?
Use a 301 redirect when visitors don't need the duplicate URL at all; it consolidates the pages for everyone. Use a canonical tag when the duplicate has to stay reachable, for example a URL with sorting or tracking parameters, or a printable version. A canonical is a strong hint to Google rather than a rule, so keep internal links pointing at the canonical URL too.
Why are some groups marked as handled?
A group is handled when search engines are already told which URL to index: every other page in it has a canonical tag pointing at one URL, is set to noindex, or the pages are hreflang alternates of each other for different regions. Those need no action, and are shown so you can confirm they point where you expect.
Is the duplicate content checker free?
Yes. Up to 1,000 pages per check with no sign-up, and the URLs you check are not stored. For a full SEO check of the same crawl, broken links, redirects, indexing and on-page tags included, run the Website SEO Audit.