Trending September 2023 # How To Remove Duplicates From A List In Python # Suggested October 2023 # Top 17 Popular

You are reading the article How To Remove Duplicates From A List In Python updated in September 2023 on the website Uyenanhthammy.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested October 2023 How To Remove Duplicates From A List In Python

Python remove Duplicates from a List

A list is a container that contains different Python objects, which could be integers, words, values, etc. It is the equivalent of an array in other programming languages.

So here will go through different ways in which we can remove duplicates from a given list in Python.

Method 1) Remove duplicates from list using Set

To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of set() method is that it returns distinct elements.

Example

my_list = [1,1,2,3,2,2,4,5,6,2,1] my_final_list = set(my_list) print(list(my_final_list))

The output that we get is distinct elements where all the duplicates elements are eliminated.

Output:

[1, 2, 3, 4, 5, 6] Method 2) Using the Temporary List

To remove duplicates from a given list, you can make use of an empty temporary list. For that first, you will have to loop through the list having duplicates and add the unique items to the temporary list. Later the temporary list is assigned to the main list.

Here is a working example using temporary list.

my_list = [1, 2, 3, 1, 2, 4, 5, 4 ,6, 2] print("List Before ", my_list) temp_list = [] for i in my_list: if i not in temp_list: temp_list.append(i) my_list = temp_list print("List After removing duplicates ", my_list)

Output:

List Before [1, 2, 3, 1, 2, 4, 5, 4, 6, 2] List After removing duplicates [1, 2, 3, 4, 5, 6]

Method 3) Using Dict

We can remove duplicates from the given list by importing OrderedDict from collections. It is available from python2.7 onwards. OrderedDict takes care of returning you the distinct elements in an order in which the key is present.

Let us make use of a list and use fromkeys() method available in OrderedDict to get the unique elements from the list.

To make use of OrderedDict.fromkey() method, you have to import OrderedDict from collections, as shown below:

from collections import OrderedDict

Here is an example to remove duplicates using OrderedDict.fromkeys() method.

Example

from collections import OrderedDict my_list = ['a','x','a','y','a','b','b','c'] my_final_list = OrderedDict.fromkeys(my_list) print(list(my_final_list))

Output:

['a', 'x', 'y', 'b', 'c']

From Python 3.5+ onwards, we can make use of the regular dict.fromkeys() to get the distinct elements from the list. The dict.fromkeys() methods return keys that are unique and helps to get rid of the duplicate values.

An example that shows the working of dict.fromkeys() on a list to give the unique items is as follows:

Example

my_list = ['a','x','a','y','a','b','b','c'] my_final_list = dict.fromkeys(my_list) print(list(my_final_list))

Output:

['a', 'x', 'y', 'b', 'c'] Method 4) Using for-loop

Using for-loop, we will traverse the list of items to remove duplicates.

First initialize array to empty i.e myFinallist = [].Inside the for-loop, add check if the items in the list exist in the array myFinallist. If the items do not exist, add the item to the array myFinallist using the append() method.

So whenever the duplicate item is encountered it will be already present in the array myFinallist and will not be inserted. Let us now check the same in the example below:

Example

my_list = [1,2,2,3,1,4,5,1,2,6] myFinallist = [] for i in my_list: if i not in myFinallist: myFinallist.append(i) print(list(myFinallist))

Output:

[1, 2, 3, 4, 5, 6]

Method 5) Using list comprehension

List comprehensions are Python functions that are used for creating new sequences (such as lists, dictionaries, etc.) using sequences that have already been created. This helps you to reduce longer loops and make your code easier to read and maintain.

Let us make use of list comprehension to remove duplicates from the list given.

Example

my_list = [1,2,2,3,1,4,5,1,2,6] my_finallist = [] [my_finallist.append(n) for n in my_list if n not in my_finallist] print(my_finallist)

Output:

[1, 2, 3, 4, 5, 6] Method 6) Using Numpy unique() method.

The method unique() from Numpy module can help us remove duplicate from the list given.

To work with Numpy first import numpy module, you need to follow these steps:

Step 1) Import Numpy module

import numpy as np

Step 2) Use your list with duplicates inside unique method as shown below. The output is converted back to a list format using tolist() method.

myFinalList = np.unique(my_list).tolist()

Step 3) Finally print the list as shown below:

print(myFinalList)

The final code with output is as follows:

import numpy as np my_list = [1,2,2,3,1,4,5,1,2,6] myFinalList = np.unique(my_list).tolist() print(myFinalList)

Output:

[1, 2, 3, 4, 5, 6] Method 7) Using Pandas methods

The Pandas module has a unique() method that will give us the unique elements from the list given.

To work with Pandas module, you need to follow these steps:

Step 1) Import Pandas module

import pandas as pd

Step 2) Use your list with duplicates inside unique() method as shown below:

myFinalList = pd.unique(my_list).tolist()

Step 3) Print the list as shown below:

print(myFinalList)

The final code with output is as follows:

import pandas as pd my_list = [1,2,2,3,1,4,5,1,2,6] myFinalList = pd.unique(my_list).tolist() print(myFinalList)

Output:

[1, 2, 3, 4, 5, 6] Method 8) Using enumerate() and list comprehension

Here the combination of list comprehension and enumerate to remove the duplicate elements. Enumerate returns an object with a counter to each element in the list. For example (0,1), (1,2) etc. Here the first value is the index, and the second value is the list item. W

Each element is checked if it exists in the list, and if it does, it is removed from the list.

Example

my_list = [1,2,2,3,1,4,5,1,2,6] my_finallist = [i for j, i in enumerate(my_list) if i not in my_list[:j]] print(list(my_finallist))

Output:

[1, 2, 3, 4, 5, 6] Summary

To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of the set() method is that it returns distinct elements.

You can remove duplicates from the given list by importing OrderedDictfrom collections. It is available from python2.7 onwards. OrderedDictdict takes care of returning you the distinct elements in an order in which the key is present.

You can make use of a for-loop that we will traverse the list of items to remove duplicates.

The method unique() from Numpy module can help us remove duplicate from the list given.

The Pandas module has a unique() method that will give us the unique elements from the list given.

The combination of list comprehension and enumerate is used to remove the duplicate elements from the list. Enumerate returns an object with a counter to each element in the list.

You're reading How To Remove Duplicates From A List In Python

Update the detailed information about How To Remove Duplicates From A List In Python on the Uyenanhthammy.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!