ENH: allow pct change to generate ghost dates to avoid missing values

[DanielGoncalves47264]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

I'm taking advantage of the DateOffset to calculate % changes. However, this approach generates some NaN values due to irregularities in months. I would like to be able to avoid this NaN by filling with the values of the adjacent days. eg. monthly_change of 30 of March be calculated as 30 March / 28 or 29 February.

  returns_calculation = {
      'daily_change': {'days': 1},
      'weekly_change': {'weeks': 1},
      'monthly_change': {'months': 1},
      'quarterly_change': {'months': 3},
      'semester_change': {'months': 6},
      'yearly_change': {'years': 1},
      'five_year_change': {'years': 5},
  }
   for period, freq in returns_calculation.items():
            df[period] = df['col1'].pct_change(
                freq=pd.tseries.offsets.DateOffset(**freq))

Feature Description

Add a new parameter to pct_change method that would temporarily reindex the series by introducing this shadow dates, and back fill if the closest date is available.

def pct_change(...., ghost_dates: bool = True):
    if ghost_dates:
         # Do something
´´´

### Alternative Solutions

I believe one could simply map every occurrence of such irregularities and use loc to fill those dates although it seems hardcoding and hard to scale.

### Additional Context

_No response_

[phofl]

Hi, thanks for your report. Could you add a complete example?

[DanielGoncalves47264]

import yfinance as yf
import pandas as pd
data=yf.download('AAPL')
returns_calculation = {
      'daily_change': {'days': 1},
      'weekly_change': {'weeks': 1},
      'monthly_change': {'months': 1},
      'quarterly_change': {'months': 3},
      'semester_change': {'months': 6},
      'yearly_change': {'years': 1},
      'five_year_change': {'years': 5},
  }
df=pd.DataFrame()
for period, freq in returns_calculation.items():
            df[period] = data['Close'].pct_change(
                freq=pd.tseries.offsets.DateOffset(**freq))
df.loc[['2022-03-31','2022-02-28']]

In this case, on 2022-03-31 we have a nan on the monthly_change i.e because there is no "2022-02-31". On this cases I would like to see it filled with the value from the 28th (same for the 31st day of months).

[MarcoGorelli]

sorry could you make the example without requiring us to download data? it should be possible to write

data = pd.DataFrame(...

and construct your example like that, so it's easily copy-and-pasteable

[Pvlb1998]

Take

[MarcoGorelli]

I think this might be a bug: subtracting a month from 2022-03-31 already goes back to 2022-02-28:

In [31]: pd.Timestamp('2022-03-31') - pd.offsets.DateOffset(months=1)
Out[31]: Timestamp('2022-02-28 00:00:00')

[DanielGoncalves47264]

@MarcoGorelli
Haven't noticed it before. However it somehow is not applying that rules with pct_change:

import pandas as pd
data = pd.Series([1.1,1.2,0.9,0.5],index=['2022-02-28','2022-03-31','2022-07-30','2022-08-31'])
data.index=pd.to_datetime(data.index)
data.pct_change(freq=pd.offsets.DateOffset(months=1))

>>> Result
2022-02-28   NaN
2022-03-31   NaN
2022-07-30   NaN
2022-08-31   NaN

>>> Expected
2022-02-28   NaN
2022-03-31   0.0909
2022-07-30   NaN
2022-08-31   -0.444