Python Remove Duplicates From List

Removing duplicates in from a list in Python is a simple operation and the results can be achieved using different methods and techniques. First very common way is to use Python set. Set is a collection of un-ordered unique object. Which implies when using set to remove duplicate original order of the items in the list will not be preserved.

>>> sampleList=[1, 5, 3, 6, 3, 5, 6, 1]
>>> sampleList=list(set(sampleList))
>>> sampleList
[1, 3, 5, 6]
>>>

if order of the items are important and Python version is <= 3.6 use OrderedDict from collections

>>> from collections import OrderedDict
>>> sampleList=[1, 5, 3, 6, 3, 5, 6, 1]
>>> sampleList=list(OrderedDict.fromkeys(sampleList))
>>> sampleList
[1, 5, 3, 6]
>>>

if your Python version is 3.7 and above. It comes out of box with dict function that guarantees uniqueness whiles preserving order.

>>> sampleList=[1, 5, 3, 6, 3, 5, 6, 1]
>>> list(dict.fromkeys(sampleList))
>>> sampleList
>>> [1, 5, 3, 6]