Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

COMPAT: datetime.date comparisons with datetime64[ns] #17965

Closed
jreback opened this issue Oct 24, 2017 · 9 comments · Fixed by #18188
Closed

COMPAT: datetime.date comparisons with datetime64[ns] #17965

jreback opened this issue Oct 24, 2017 · 9 comments · Fixed by #18188
Labels
Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Milestone
0.21.1

Comments

@jreback
Copy link
Contributor

jreback commented Oct 24, 2017

This should probably work

In [16]: df = pd.DataFrame({'A':[pd.Timestamp('20130101'), np.nan]})

In [17]: df
Out[17]: 
           A
0 2013-01-01
1        NaT

In [18]: pd.Timestamp('20130101').to_pydatetime()>=df.A
Out[18]: 
0     True
1    False
Name: A, dtype: bool

In [20]: pd.Timestamp('20130101').to_pydatetime().date()>=df.A
ValueError: cannot set a Timestamp with a non-timestamp
@jreback jreback added Compat pandas objects compatability with Numpy or Python functions Difficulty Intermediate Dtype Conversions Unexpected or buggy dtype conversions Datetime Datetime data dtype labels Oct 24, 2017
@jreback jreback added this to the 0.21.1 milestone Oct 24, 2017
@jbrockmendel
Copy link
Member

What would date(2017, 10, 24) < Timestamp('2017-10-24 17:02') be?

@jreback
Copy link
Contributor Author

jreback commented Oct 26, 2017

These just need wrapping

In [7]: pd.Timestamp(datetime.date(2017, 10, 24)) < pd.Timestamp('2017-10-24 17:02')
Out[7]: True

@jbrockmendel
Copy link
Member

I think the internally consistent thing to do would be to wrap date(2017, 10, 24) with pd.Period(date(2017, 10, 24), 'D')

(the constructor should also be able to infer the freq should be "D", but that is a different issue)

@jreback
Copy link
Contributor Author

jreback commented Oct 26, 2017

no it’s a Timestamp (conceptually it’s just a shortened datetime)

@jbrockmendel
Copy link
Member

(conceptually it’s just a shortened datetime)

That part isn't obvious to me, but conditional on this, then I agree this issue is straightforward.

@TomAugspurger
Copy link
Contributor

FWIW, python raises here

In [4]: datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-c0cf43ef6360> in <module>()
----> 1 datetime.datetime(2017, 1, 1, 12) > datetime.date(2017, 1, 1)

TypeError: can't compare datetime.datetime to datetime.date

@jbrockmendel jbrockmendel mentioned this issue Oct 26, 2017
Closed
59 tasks
@jaradc jaradc mentioned this issue Oct 27, 2017
Closed
TheChymera added a commit to TheChymera/LabbookDB that referenced this issue Nov 3, 2017
@TheChymera
pandas-0.21.0 breaks labbookdb - isue #17965:
6aeb9cc 
pandas-dev/pandas#17965
@liuli1735
Copy link 

dates = data['TradingDay'].unique()
for date in dates:
print(date)
raw_data = data.loc[data['TradingDay'] == date].copy()

ValueError: cannot set a Timestamp with a non-timestamp```

I get the same error, i just upgrade my Anaconda yesterday.

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Nov 8, 2017 via email

@akrherz
Copy link

akrherz commented Nov 8, 2017 •
edited
Loading

I think a MCVE for @liuli1735 's post is this

import pandas as pd
df = pd.DataFrame({'dates': pd.date_range('2000-01-01', '2010-12-31')})
date = df['dates'].unique()[0]
print(df.loc[df['dates'] == date])

Result with 0.20.3-py27_1 on conda-forge (linux64 python2.7)

$ python mcve.py 
       dates
0 2000-01-01

Result with 0.21.0-py27_0 on conda-forge (linux64 python2.7)

$ python mcve.py 
Traceback (most recent call last):
  File "mcve.py", line 5, in <module>
    print(df.loc[df['dates'] == date])
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 879, in wrapper
    res = na_op(values, other)
  File "/opt/miniconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 808, in na_op
    y = libindex.convert_scalar(x, _values_from_object(y))
  File "pandas/_libs/index.pyx", line 494, in pandas._libs.index.convert_scalar
  File "pandas/_libs/index.pyx", line 509, in pandas._libs.index.convert_scalar
ValueError: cannot set a Timestamp with a non-timestamp

watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 9, 2017
@watercrossing
Fix pandas-dev#17965 datetime64 comparison
95140e6
@watercrossing watercrossing mentioned this issue Nov 9, 2017
Merged
4 tasks
watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 9, 2017
@watercrossing
Expand tests for pandas-dev#17965 to various datetimelikes
7943bcb
watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017
@watercrossing
Fix pandas-dev#17965 to allow full comparison of datetimelike objects
8b01fe3
watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017
@watercrossing
Fix pandas-dev#17965 to allow full comparison of datetimelike objects
ddfe7dc
watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 10, 2017
@watercrossing
Fix pandas-dev#17965 to allow full comparison of datetimelike objects
ebe805b
watercrossing added a commit to watercrossing/pandas that referenced this issue Nov 13, 2017
@watercrossing
Fix pandas-dev#17965 to allow full comparison of datetimelike objects
d9f12f4
jreback pushed a commit that referenced this issue Nov 14, 2017
@watercrossing @jreback
Fix #17965 to allow full comparison of datetimelike objects (#18188)
77f10f0
@discort discort mentioned this issue Dec 6, 2017
Closed
TomAugspurger pushed a commit to TomAugspurger/pandas that referenced this issue Dec 8, 2017
@watercrossing @TomAugspurger
Fix pandas-dev#17965 to allow full comparison of datetimelike objects (…
ea94667 
…pandas-dev#18188)

(cherry picked from commit 77f10f0)
TomAugspurger pushed a commit that referenced this issue Dec 11, 2017
@watercrossing @TomAugspurger
Fix #17965 to allow full comparison of datetimelike objects (#18188)
ee68bb3 
(cherry picked from commit 77f10f0)
@set92 set92 mentioned this issue Dec 4, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Milestone
0.21.1
Development

Successfully merging a pull request may close this issue.

Fix #17965 datetime64 comparison watercrossing/pandas
5 participants
@akrherz @jreback @TomAugspurger @liuli1735 @jbrockmendel