Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OutOfBoundsDatetime for big-endian np.datetime64 arrays #29684

Closed
lr4d opened this issue Nov 18, 2019 · 3 comments · Fixed by #30976
Closed

OutOfBoundsDatetime for big-endian np.datetime64 arrays #29684

lr4d opened this issue Nov 18, 2019 · 3 comments · Fixed by #30976
Assignees
@jbrockmendel
Labels
Datetime Datetime data dtype Regression Functionality that used to work in a prior pandas version
Milestone
1.1

Comments

@lr4d
Copy link

lr4d commented Nov 18, 2019

Code Sample, a copy-pastable example if possible

ipdb> value = np.array([np.datetime64(1, 'ms')], dtype="<M8[ms]")
ipdb> pd.Series(value)
0   1970-01-01 00:00:00.001
dtype: datetime64[ns]
ipdb> value = np.array([np.datetime64(1, 'ms')], dtype=">M8[ms]")
ipdb> pd.Series(value)
*** pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 2285384-04-02 23:52:07

Problem description

Seems like something weird is going on when converting the big-endian timestamp, which does not happen for the little-endian one.

Expected Output

The same as for the little-endian data type.

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

ipdb> pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 18.5.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.3
numpy : 1.17.3
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 41.6.0.post20191101
Cython : None
pytest : 5.0.1
hypothesis : 4.44.2
sphinx : 2.2.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.9.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 0.13.0
pytables : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
@lr4d
Copy link
Author

lr4d commented Nov 18, 2019

This doesn't affect pandas 0.24.2. With that version, it works as expected

@lr4d lr4d mentioned this issue Nov 18, 2019
Open
@jorisvandenbossche jorisvandenbossche added Regression Functionality that used to work in a prior pandas version Datetime Datetime data dtype labels Nov 19, 2019
@jorisvandenbossche
Copy link
Member

@lr4d Thanks for the report, I can confirm this is a regression compared to pandas 0.24.

@jbrockmendel do you know if we have / had special handling for this in the tslib code?

@jbrockmendel
Copy link
Member

I'm not aware of anything endian-specific in the tslibs code. I'll try to take a look at this

@jbrockmendel jbrockmendel self-assigned this Nov 19, 2019
@jbrockmendel jbrockmendel mentioned this issue Jan 13, 2020
Merged
5 tasks
@jreback jreback added this to the 1.1 milestone Jan 14, 2020
@spencerkclark spencerkclark mentioned this issue Sep 17, 2022
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jbrockmendel jbrockmendel

Labels
Datetime Datetime data dtype Regression Functionality that used to work in a prior pandas version
Projects
None yet
Milestone
1.1
Development

Successfully merging a pull request may close this issue.

BUG: ensure_datetime64ns with bigendian array jbrockmendel/pandas
4 participants
@jreback @jorisvandenbossche @jbrockmendel @lr4d