How to get a list of all the duplicate items using Pandas in Python?

Given a Pandas DataFrame, we have to get a list of all the duplicate items.
Submitted by Pranit Sharma, on June 04, 2022

Duplicity in a column of pandas DataFrame occurs when there is more than 1 occurrence of similar elements.

Here, we are going to learn how to get a list of duplicates in pandas DataFrame? For this purpose, we will use the pandas.DataFrame.duplicated() method having the following syntax:

DataFrame.duplicated(subset=None, keep='first')

The pandas.DataFrame.duplicated() method will iterate over entire DataFrame and check that if any element occurs more than 1 or not, if the occurrence is more than 1 for any element it will return that element, by passing 'keep=False' it will return the 1st occurrence of the elements.

To work with pandas, we need to import pandas package first, below is the syntax:

import pandas as pd

Let us understand with the help of an example,

# Importing pandas package
import pandas as pd

# Defining two DataFrames
df = pd.DataFrame(
    data = {
        'Products':['Frooti','Krack-jack','Hide&seek','Frooti','Chawanprash','Honey','Hair oil','Honey','Maggie','Kitkat','EveryDay','Crunch']

# Display DataFrame
print("Original DataFrame:\n",df,"\n")


Example 1: Get a list of all the duplicate items

Now, get all the duplicate items in the DataFrame,

# Getting the duplicates from parle column
result = df[df.duplicated(['Products'], keep='first')]

# Display result


Example 2: Get a list of all the duplicate items

Python Pandas Programs »


Comments and Discussions!


Languages: » C » C++ » C++ STL » Java » Data Structure » C#.Net » Android » Kotlin » SQL
Web Technologies: » PHP » Python » JavaScript » CSS » Ajax » Node.js » Web programming/HTML
Solved programs: » C » C++ » DS » Java » C#
Aptitude que. & ans.: » C » C++ » Java » DBMS
Interview que. & ans.: » C » Embedded C » Java » SEO » HR
CS Subjects: » CS Basics » O.S. » Networks » DBMS » Embedded Systems » Cloud Computing
» Machine learning » CS Organizations » Linux » DOS
More: » Articles » Puzzles » News/Updates

© some rights reserved.