Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ER: Bug accessing a series MultiIndex with a Timestamp #4294

Closed
l736x opened this issue Jul 19, 2013 · 5 comments · Fixed by #5088
Closed

ER: Bug accessing a series MultiIndex with a Timestamp #4294

l736x opened this issue Jul 19, 2013 · 5 comments · Fixed by #5088
Assignees
Labels
Bug Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Milestone

Comments

@l736x
Copy link
Contributor

l736x commented Jul 19, 2013

I have a series with a multiindex (datetime, int).
When I try to access a slice corresponding to a date using a Timestamp I have an exception printed to screen if the size of the series is >= 1e6.
No problem if I access the same slice with a string.
No problem with an equivalent dataframe.
Tested on 0.10.1 and 0.11, I'm sorry but I don't have access to 0.12rc1.

In [1]: import pandas as pd

In [2]: pd.__version__
Out[2]: '0.11.0'

In [3]: np.__version__
Out[3]: '1.7.1'

In [4]: df = pd.DataFrame(randn(1000, 1000), index=pd.date_range('2000-1-1', periods=1000))

In [5]: s = df.stack() # s has 1e6 lines

In [6]: s.head()
Out[6]:
2000-01-01  0   -0.555228
            1   -0.152952
            2    0.401490
            3   -1.057106
            4   -0.987895
dtype: float64

In [7]: s['2000-1-4'].head()
Out[7]:
0   -1.009647
1    0.094427
2    0.105180
3   -2.166810
4   -0.767165
dtype: float64

In [8]: s[pd.Timestamp('2000-1-4')].head()
Exception TypeError: TypeError('Cannot compare Timestamp with (<Timestamp: 2002-09-26 00:00:00>, 999)',) in 'pandas.index._bin_search' ignored
Out[8]:
0   -1.009647
1    0.094427
2    0.105180
3   -2.166810
4   -0.767165
dtype: float64

In [9]: s2 = s[:-1].copy() # s2 has 1e6 - 1 lines

In [10]: s2['2000-1-4'].head()
Out[10]:
0   -1.009647
1    0.094427
2    0.105180
3   -2.166810
4   -0.767165
dtype: float64

In [11]: s2[pd.Timestamp('2000-1-4')].head()
Out[11]:
0   -1.009647
1    0.094427
2    0.105180
3   -2.166810
4   -0.767165
dtype: float64

In [12]: df2 = pd.DataFrame(s)

In [13]: df2.ix['2000-1-4'].head()
Out[13]:
          0
0 -1.009647
1  0.094427
2  0.105180
3 -2.166810
4 -0.767165

In [14]: df2.ix[pd.Timestamp('2000-1-4')].head()
Out[14]:
          0
0 -1.009647
1  0.094427
2  0.105180
3 -2.166810
4 -0.767165
@cpcloud
Copy link
Member

cpcloud commented Jul 19, 2013

I get slightly less (more?) informative error message on master

In [11]: s[pd.Timestamp('2000-1-4')].head()
Exception TypeError: TypeError("Cannot compare Timestamp with 'tuple'",) in 'pandas.index._bin_search' ignored
Out[11]:
0   -0.834
1    0.374
2    0.755
3    1.747
4    0.366
dtype: float64

but it's still the same issue.

@jreback
Copy link
Contributor

jreback commented Jul 19, 2013

ok....so on the timestamp it is raising (but being caught) and continues...weird....marking as an error reporting bug

@cpcloud
Copy link
Member

cpcloud commented Jul 19, 2013

Indeed, why doesn't this raise "for real"?

@cpcloud
Copy link
Member

cpcloud commented Jul 19, 2013

_bin_search is not propagating errors it generates

here's why: http://docs.cython.org/src/userguide/language_basics.html#error-return-values

@jreback
Copy link
Contributor

jreback commented Oct 2, 2013

this is a bit tricky because it hits a different cython path (for really large indexes and returned a TypeError rather than a KeyError)...so takes a different path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants