Dynamically filtering a pandas dataframe

Given a pandas dataframe, we have to dynamically filter it.
Submitted by Pranit Sharma, on November 15, 2022

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

Dynamically filter a pandas DataFrame

So, we are creating a DataFrame with multiple columns and then we need to filter the df using thresholds for three columns.

We can do this by simply applying all the conditions and if the data satisfies the condition, it will be the final result. However, we need to do this inside a function where the name of the thresholds and the columns are given in the form of a dictionary.

For this purpose, we will use a quick and easy way of doing this. We will build a dynamic query using list comprehension and the join method of string.

Finally, we have a method called pandas.DataFrame.query() inside which we can pass any comprehended query and it will dynamically calculate all the results.

Let us understand with the help of an example,

Python code to dynamically filter a pandas DataFrame

# Importing pandas package
import pandas as pd

# Importing numpy package
import numpy as np

# Creating a dictionary
d ={
    "A" : [6, 2, 10, -5, 3],
    "B" : [2, 5, 3, 2, 6],
    "C" : [-5, 2, 1, 8, 2]
}

# Creating DataFrame
df = pd.DataFrame(d)

# Display dataframe
print('Original DataFrame:\n',df,'\n')

# Giving columns and thresholds
dic = {"A" : 0, "B" : 2, "C" : -1}

# Making a query
q = ' & '.join(['{}>{}'.format(k, v) for k, v in dic.items()])

# Running query
res = df.query(q)

# Display new DataFrame
print("Result:\n",res)

Output:

Example: Dynamically filtering a pandas dataframe

Python Pandas Programs »



ADVERTISEMENT
ADVERTISEMENT




Comments and Discussions!




Languages: » C » C++ » C++ STL » Java » Data Structure » C#.Net » Android » Kotlin » SQL
Web Technologies: » PHP » Python » JavaScript » CSS » Ajax » Node.js » Web programming/HTML
Solved programs: » C » C++ » DS » Java » C#
Aptitude que. & ans.: » C » C++ » Java » DBMS
Interview que. & ans.: » C » Embedded C » Java » SEO » HR
CS Subjects: » CS Basics » O.S. » Networks » DBMS » Embedded Systems » Cloud Computing
» Machine learning » CS Organizations » Linux » DOS
More: » Articles » Puzzles » News/Updates

© https://www.includehelp.com some rights reserved.