Remove duplicate rows from a CSV by one column or the whole row

How to use the CSV duplicate remover

1. Press Choose a file to load a CSV, TSV or Excel file, or paste rows into CSV data.
2. Check Delimiter (Detect automatically works for most files) and First row is a header.
3. In Match on columns (blank = the whole row), type the column names or numbers that identify a record. Leave it blank to remove only rows that repeat completely.
4. Choose whether to Ignore upper/lower case and Ignore spaces around values, and whether to Keep the First occurrence or the Last occurrence.
5. Copy the Deduplicated CSV or press Download to save <name>-deduplicated.csv.

What it does and when to use it

Duplicates creep in whenever lists are combined: two sign-up forms exported into one sheet, a contact list merged with last year’s, an order export run twice. Sending two emails to the same person or counting the same order twice causes real problems, so remove repeats before you import, mail or report.

Matching on a key column is usually better than matching the whole row. Two rows for “asha@example.com” may differ in the name (“Asha Verma” and “Asha V.”), yet they are the same customer. If you are joining several files first, use the CSV merger and then dedupe the merged result. Stray spaces in other columns can be fixed with the CSV cleaner. For a plain list with one value per line, the remove duplicate lines tool is simpler.

How it works

For each data row the tool builds a key from the chosen columns (or every column). With Ignore spaces around values, each value is trimmed; with Ignore upper/lower case, it is lower-cased. The first row with a key is kept, or, with Last occurrence, the last one. All other rows with that key go to the removed list. The header row is never compared and is always written first. The output uses the same delimiter as the input, and fields are quoted only when they need it.

Worked examples

The sample has five rows, matched on email, with case and spaces ignored:

email,name,plan
asha@example.com,Asha Verma,Pro
ben@example.com,Ben Carter,Free
ASHA@example.com ,Asha V.,Pro
mei@example.com,Mei Lin,Team
ben@example.com,Ben Carter,Free

Default settings. “ASHA@example.com “ (capitals and a trailing space) matches “asha@example.com”, and Ben appears twice. Result: unique rows kept 3, duplicates removed 2, values with duplicates 2.

Whole row, case-sensitive. Clear the match box and untick Ignore upper/lower case. Only Ben’s exact repeat goes: 1 duplicate removed, matched on whole row. Asha’s two rows differ, so both stay.

Keep the last occurrence. The output becomes ASHA@example.com ,Asha V.,Pro, then Mei, then Ben’s second row, each in its original position.

Limits and tips

• Matching is exact after the optional trim and lower-casing. “Ben Carter” and “Benjamin Carter” are different.
• A column name that does not exist gives a message listing the real column names.
• To count how often each value appears before removing anything, try the frequency counter.