Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Update Values from MultiIndex Dataframe after using loc is not applied correctly #47867

Closed
2 of 3 tasks
bbizarro opened this issue Jul 26, 2022 · 4 comments · Fixed by #47949
Closed
2 of 3 tasks

BUG: Update Values from MultiIndex Dataframe after using loc is not applied correctly #47867

bbizarro opened this issue Jul 26, 2022 · 4 comments · Fixed by #47949
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex Regression Functionality that used to work in a prior pandas version
Milestone
1.4.4

Comments

@bbizarro
Copy link

bbizarro commented Jul 26, 2022 •
edited
Loading

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

df = pd.DataFrame()
df['idx1'] = [0,1,2,3,4]
df['idx2'] = [0,1,2,3,4]
df['abool'] = True
df['a'] = np.arange(5, dtype='int64')
df['b'] = np.arange(5, dtype='float64')
df['c'] = np.arange(5, dtype='int64')
df = df.set_index(['idx1','idx2'])

idx = [(1,1),(3,3)]
df.loc[idx, 'c'] = 0
df.loc[idx, 'c']

df.loc[idx, ['a', 'b']] # This line is the problem

df.loc[idx, 'c'] = 15
df.loc[idx, 'c']

Issue Description

In Pandas 1.4.x the last line is not showing the change in the column c and the values still being 0.

>>> idx1  idx2
1     1       0
3     3       0
Name: c, dtype: int64

When I call the column c as DataFrame I get the correct values

df.loc[idx, ['c']]

>>>         c
idx1 idx2    
1    1     15
3    3     15

Expected Behavior

On Pandas 1.3.5 I get

>>> idx1  idx2
1     1       15
3     3       15
Name: c, dtype: int64

Installed Versions

INSTALLED VERSIONS

commit : 66e3805
python : 3.8.12.final.0
python-bits : 64
OS : Windows
OS-release : 7
Version : 6.1.7601
machine : AMD64
processor : Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : Spanish_Chile.1252

pandas : 1.3.5
numpy : 1.23.1
pytz : 2022.1
dateutil : 2.8.2
pip : 22.2
setuptools : 59.8.0
Cython : None
pytest : None
hypothesis : None
...
xarray : None
xlrd : 2.0.1
xlwt : None
numba : None
@bbizarro bbizarro added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 26, 2022
@BeRT2me BeRT2me mentioned this issue Jul 26, 2022
Closed
1 task
@simonjayhawkins
Copy link
Member

@bbizarro Thanks for the report.

In Pandas 1.4.x the last line is not showing the change in the column c and the values still being 0.

Not able to reproduce. The code sample gives the expected result on 1.3.5, 1.4.3 and main for me. Can you double check the code sample.

@BeRT2me
Copy link

BeRT2me commented Jul 27, 2022

This is missing a line where you have to try to access df.loc[idx, 'c'] before df.loc[idx, ['a', 'b']], see my above mentioned report for a working example. @simonjayhawkins

@bbizarro
Copy link
Author

I updated the code sample with the missing line mentioned by @BeRT2me and I get the problem on 1.4.3

simonjayhawkins added a commit to simonjayhawkins/pandas that referenced this issue Jul 27, 2022
@simonjayhawkins
code sample for pandas-dev#47867
c0743a8
simonjayhawkins added a commit to simonjayhawkins/pandas that referenced this issue Jul 28, 2022
@simonjayhawkins
use incorrect result to find fix for pandas-dev#47867
9471705
@simonjayhawkins
Copy link
Member

In Pandas 1.4.x the last line is not showing the change in the column c and the values still being 0.

first bad commit: [573c063] BUG: frame.loc[2:, 'z'] not setting inplace when multi-block (#44345)

but looks like fixed in commit: [c80f656] BUG: frame[x].loc[y] inconsistent with frame.at[x, y] GH#22372 (#45287)

I'll milestone as 1.4.x for now as we maybe able to backport that fix.

@simonjayhawkins simonjayhawkins added this to the 1.4.4 milestone Jul 28, 2022
@simonjayhawkins simonjayhawkins added Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version MultiIndex and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 28, 2022
This was referenced Aug 2, 2022
Merged
Merged
@phofl phofl mentioned this issue Aug 3, 2022
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex Regression Functionality that used to work in a prior pandas version
Projects
None yet
Milestone
1.4.4
Development

Successfully merging a pull request may close this issue.

TST: Add test for loc not updating cache correctly phofl/pandas
3 participants
@bbizarro @simonjayhawkins @BeRT2me