Interpolation is a powerful technique in Pandas that is used for handling the missing values in a dataset. This technique estimates the missing values based on other data points of the dataset. Pandas provides the interpolate() method for both DataFrame and Series objects to fill in missing values using various interpolation methods.
In this tutorial, we will learn about the interpolate() methods in Pandas for filling the missing values in a time series data, numeric data, and more using the different interpolation methods.
The Pandas interpolate() method of the both DataFrame and Series objects is used to fills the missing values using different Interpolation strategies. By default, Pandas automatically uses linear interpolation as the default method.
Here is a basic example of calling the interpolate() method for filling the missing values.
import numpy as np
import pandas as pd
df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})
print("Original DataFrame:")
print(df)
# Using the interpolate() method
result = df.interpolate()
print("\nResultant DataFrame after applying the interpolation:")
print(result)Original DataFrame:
A B
0 1.1 0.25
1 NaN NaN
2 3.5 NaN
3 NaN 4.70
4 NaN 10.00
5 NaN 14.70
6 6.2 1.30
7 7.9 9.20
Resultant DataFrame after applying the interpolation:
A B
0 1.100 0.250000
1 2.300 1.733333
2 3.500 3.216667
3 4.175 4.700000
4 4.850 10.000000
5 5.525 14.700000
6 6.200 1.300000
7 7.900 9.200000Pandas supports several interpolation methods, including linear, polynomial, pchip, akima, spline, and more. These methods provide flexibility for filling the missing values depending on the nature of your data.
The following example demonstrates using the interpolate() method with the barycentric interpolation technique.
import numpy as np
import pandas as pd
df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})
print("Original DataFrame:")
print(df)
# Applying the interpolate() with Barycentric method
result = df.interpolate(method='barycentric')
print("\nResultant DataFrame after applying the interpolation:")
print(result)Original DataFrame:
i A B
0 1.1 0.25
1 NaN NaN
2 3.5 NaN
3 NaN 4.70
4 NaN 10.00
5 NaN 14.70
6 6.2 1.30
7 7.9 9.20
Resultant DataFrame after applying the interpolation:
A B
0 1.100000 0.250000
1 2.596429 57.242857
2 3.500000 24.940476
3 4.061429 4.700000
4 4.531429 10.000000
5 5.160714 14.700000
6 6.200000 1.300000
7 7.900000 9.200000By default, Pandas interpolation fills all the missing values, but you can limit how many consecutive NaN values are filled using the limit parameter of the interpolate() method.
The following example demonstrates filling the missing values of a Pandas DataFrame by limiting the consecutive fills using the limit parameter of the interpolate() method.
import numpy as np
import pandas as pd
df = pd.DataFrame({"A": [1.1, np.nan, 3.5, np.nan, np.nan, np.nan, 6.2, 7.9],
"B": [0.25, np.nan, np.nan, 4.7, 10, 14.7, 1.3, 9.2],
})
print("Original DataFrame:")
print(df)
# Applying the interpolate() with limit
result = df.interpolate(method='spline', order=2, limit=1)
print("\nResultant DataFrame after applying the interpolation:")
print(result)Original DataFrame:
i A B
0 1.1 0.25
1 NaN NaN
2 3.5 NaN
3 NaN 4.70
4 NaN 10.00
5 NaN 14.70
6 6.2 1.30
7 7.9 9.20
Resultant DataFrame after applying the interpolation:
i A B
0 1.100000 0.250000
1 2.231383 -1.202052
2 3.500000 NaN
3 4.111529 4.700000
4 NaN 10.000000
5 NaN 14.700000
6 6.200000 1.300000
7 7.900000 9.200000Interpolation can be applied to the Pandas time series data as well. It is useful when filling gaps in missing data points over time.
Example statement −
import numpy as np
import pandas as pd
indx = pd.date_range("2024-01-01", periods=10, freq="D")
data = np.random.default_rng(2).integers(0, 10, 10).astype(np.float64)
s = pd.Series(data, index=indx)
s.iloc[[1, 2, 5, 6, 9]] = np.nan
print("Original Series:")
print(s)
result = s.interpolate(method="time")
print("\nResultant Time Series after applying the interpolation:")
print(result)Original Series:
Date Value
2024-01-01 8.0
2024-01-02 NaN
2024-01-03 NaN
2024-01-04 2.0
2024-01-05 4.0
2024-01-06 NaN
2024-01-07 NaN
2024-01-08 0.0
2024-01-09 3.0
2024-01-10 NaN
Resultant Time Series after applying the interpolation:
Date Value
2024-01-01 8.000000
2024-01-02 6.000000
2024-01-03 4.000000
2024-01-04 2.000000
2024-01-05 4.000000
2024-01-06 2.666667
2024-01-07 1.333333
2024-01-08 0.000000
2024-01-09 3.000000
2024-01-10 3.000000