Convert Pandas dataframe to Sparse Numpy Matrix Directly

Learn, how to convert pandas dataframe to sparse numpy matrix directly in Python? By Pranit Sharma Last updated : September 17, 2023

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

Problem statement

Suppose that we are given a pandas dataframe and we need to convert it into a sparse matrix directly. By directly, we mean that we do not want to convert it into a matrix first.

Converting dataframe to sparse numpy matrix

The scipy.sparse.csr_matrix() is used to create a sparse matrix, with a dataframe, we need to pass all of its values directly, hence, we need to pass df.values as an argument which will work as a numpy array.

Let us understand with the help of an example,

Python program to convert pandas dataframe to sparse numpy matrix directly

# Importing pandas package
import pandas as pd

# Import numpy
import numpy as np

# Importing scipy
import scipy

# Creating a dataframe
df = pd.DataFrame(data={'X': [1,2,3], 'Y': [4,5,6], 'Z': [7,8,9]})

# Display the DataFrame
print("Original DataFrame:\n",df,"\n\n")

# Creating a sparse matrix
res = scipy.sparse.csr_matrix(df.values)

# Display Result
print("Result:\n",res)