What counts as a duplicate
Two lines are the same only if they match exactly, which means Alice and alice are different, and so are alice and alice with a trailing space. That trailing space is the usual reason a de-duplicated list still looks like it has repeats — the options for ignoring case and trimming whitespace exist for exactly that.
Order is preserved: the first occurrence stays where it was and later ones are dropped. Sorting first would also group duplicates together, but it destroys an order that often matters.
Lists that came from somewhere else
Email lists exported from a CRM, keyword lists from a research tool, log lines pasted from a terminal — all of them tend to carry invisible baggage: a byte-order mark on the first line, carriage returns from a Windows export, non-breaking spaces from a web page.
Trimming whitespace catches most of it. If a list still de-duplicates oddly, the whitespace cleaner normalises the exotic space characters that a plain trim leaves alone.
Frequently asked questions
Why are duplicates still showing after I run it?
Almost always trailing spaces or mixed case, which make two lines genuinely different strings. Turn on trimming and case-insensitive matching.
Does it change the order?
No. The first occurrence stays in place and later copies are removed, so a list that was in a meaningful order stays in it.
Can it show me only the duplicates?
Not here — this keeps the unique lines. To find what repeated, the word frequency counter reports counts.
Something wrong with this tool, or an idea for it? Tell us