How to Remove Duplicate Lines in Any Text
Duplicate lines are everywhere — pasted email lists, exported CSV rows, log files, keyword lists. Here's how to clean them fast, plus a free instant tool.
The fastest way to remove duplicate lines is to use a free online duplicate line remover that runs in your browser. Paste your list, and unique lines appear instantly — nothing is uploaded.
1. Using a free online tool (instant)
Copy your list, paste it into the Duplicate Line Remover, and the duplicates vanish in a second. Optional controls let you ignore case, trim whitespace, remove empty lines, sort, or keep the first or last occurrence of each duplicate.
2. In Excel or Google Sheets
In Excel: select the column → Data → Remove Duplicates. In Google Sheets: Data → Data cleanup → Remove duplicates.
3. In the terminal (macOS / Linux)
To remove all duplicate lines while preserving original order: awk '!seen[$0]++' file.txt > clean.txt
To sort alphabetically and deduplicate: sort -u file.txt
What counts as a duplicate?
By default, two lines are duplicates only if they match exactly, including case and spacing. Turn on ignore case so Apple and apple collapse to one, and trim whitespace so "apple" and " apple " are treated as the same.