Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_csv - allow to create folders. #24306

Closed
OmerJog opened this issue Dec 16, 2018 · 7 comments · Fixed by #42250 or #43436
Closed

to_csv - allow to create folders. #24306

OmerJog opened this issue Dec 16, 2018 · 7 comments · Fixed by #42250 or #43436
Labels
Milestone

Comments

@OmerJog
Copy link

OmerJog commented Dec 16, 2018

currently to_csv will create a file if it does not exist. However it will not create a folder (nor sub-folders).
In this case it will raise
IOError: [Errno 2] No such file or directory:

The user must wrap his code in order for it to create a folder.

I'm not sure why the function behaves like this.. It would be great if it will also create folders if needed.

Reference from stack overflow:
https://stackoverflow.com/questions/47143836/pandas-dataframe-to-csv-raising-ioerror-no-such-file-or-directory

@jreback
Copy link
Contributor

jreback commented Dec 16, 2018

this is out of scope for pandas itself as it can easily be accomplished by the user

@jreback jreback closed this as completed Dec 16, 2018
@JivanRoquet
Copy link

JivanRoquet commented Aug 28, 2019

In this case, pd.to_csv() is also out of scope, since the user can "easily" open the file from Python and stream the dataframe content into it.

@manugarri
Copy link

@JivanRoquet i agree with you, pandas has been known for being beginner friendly, and a simple SO search can guarantee that making directories for output files is a very common issue for python users, particularly analysts that know how to use pandas, but dont really now about filesystems.

On the other side, the last pandas community survey indicated that feature creep is an important issue for the community. @jreback maybe makes sense to add a comment with instructions to the docstring of the to_csv function as a compromise?

@mchappychen
Copy link

Is possible to make folder drive from colab:
https://stackoverflow.com/questions/60706752/how-to-make-directory-on-the-google-colab
so you is can do it

@MarcoGorelli MarcoGorelli added this to the Contributions Welcome milestone Jun 10, 2021
@MarcoGorelli
Copy link
Member

maybe makes sense to add a comment with instructions to the docstring of the to_csv function

A note in the docs might be useful, pull requests welcome

@MarcoGorelli MarcoGorelli reopened this Jun 10, 2021
feefladder pushed a commit to feefladder/pandas that referenced this issue Jun 26, 2021
feefladder pushed a commit to feefladder/pandas that referenced this issue Jun 26, 2021
feefladder pushed a commit to feefladder/pandas that referenced this issue Jun 26, 2021
@leonarduschen
Copy link
Contributor

Quoting from #42255 (review)

would also accept a better error message for cases like this (not sure how hard this is to detect that if you create the parent folders). (separate PR)

Do we want this? I'd be happy to make a PR, should be straightforward with os.path.isdir or pathlib.Path.is_dir (don't have to catch IOError), would help a lot of new users

I don't think automatic creation of directories is a great idea to do so automatically, so after the 2 above could consider via a keyword (yes some work for this). but please propose before you do the PR.

After that, maybe we can consider this too?

@jreback
Copy link
Contributor

jreback commented Sep 6, 2021

Do we want this? I'd be happy to make a PR, should be straightforward with os.path.isdir or pathlib.Path.is_dir (don't have to catch IOError), would help a lot of new users

would take this as an enhancement PR.

After that, maybe we can consider this too?

we would need a new keyword to do this (to preserver backward compabitility), but -1 on adding new keywords so unlikely

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment