DeDuplicate is a utility to identify and remove duplicate items from a list of text. Type or paste the list of text items you want de-duplicated into the top box. Click the OK button to remove all duplicate occurrences from the list of items.
DeDuplication types include:
Remove Duplicates — sorts the list then removes the excess occurrences. Unique items will always be included, but only one occurrence of each duplicate will be included.
Show only Duplicates — sorts the list then shows only those items that occur two or more times.
Show only Uniques — Sorts the list then shows only those items that occur once.
The DeDuplication takes into account accented and foreign characters.
If you want to DeDuplicate by words (rather than paragraphs) you will need to check the DeDuplicate by Words checkbox.
Note that Case is not taken into account so ‘ABC’ and ‘Abc’ are considered duplicates.
[New] Added new Wikipedia entries for all conversion types.
[New] Added the ability to copy the Converted text to the Source text window.
[New] Added the ability to get a frequency count from each duplicate paragraph.
[New] Font size can now be up to 48 points.
[New] Improved the layout of the Wikipedia windows.
[New] Reconfigured screen layout to be horizontal
[New] Windows can now display full screen.
[New] Added the ability to deduplicate as case-sensitive — ‘ABC’ and ‘AbC’ are not considered duplicates.
[New] Added the ability to remove blank lines from the duplication.