Make pandas DataFrame to a dict and dropna

Learn, how to make pandas DataFrame to a dict and drop nan values?
Submitted by Pranit Sharma, on November 11, 2022

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

While creating a DataFrame or importing a CSV file, there could be some NaN values in the cells. NaN values mean "Not a Number" which generally means that there are some missing values in the cell.

Problem statement

Suppose, we are given a DataFrame with multiple columns. These columns contain integer values as well as some nan values. We need to convert this DataFrame into a dictionary and at the same time, we need to drop all the nan values as well. As a result, we need a dictionary with no nan values in it.

Pandas DataFrame to a dict and dropna

To convert dataframe into a dictionary, we will use pandas.DataFrame.to_dict() method but since we need to drop the nan values simultaneously, we will use this method in a comprehension statement.

Let us understand with the help of an example,

Python program to make pandas DataFrame to a dict and dropna

# Importing pandas package
import pandas as pd

# Importing numpy package
import numpy as np

# Creating a dictionary
d = {
    'A':{'a':'b','c':'d','e':np.nan},
    'B':{'a':np.nan,'b':'c','d':'e'}
}

# Creating DataFrame
df = pd.DataFrame(d)

# Display dataframe
print('Original DataFrame:\n',df,'\n')

# Dropping nan and converting to dict
res = [v.dropna().to_dict() for k,v in df.iterrows()]

# Display result
print('Result:\n',res,'\n')