-
Notifications
You must be signed in to change notification settings - Fork 216
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
azcopy thinks file is directory when sas token has slashes #873
Comments
@adreed-msft Can you take a look, and add to backlog as appropriate? |
Sure thing, though I have my suspicions this is actually fixed after the text processing errors PR. I'll check both sides of that. |
Myself and others have been getting this error also, see MicrosoftDocs/azure-docs#41969 (comment) |
I've created an issue on the az-cli side to fix the encoding: Azure/azure-cli#12291 |
Im having the same issue when generating a sas token with the ip information as well, trying to copy from a blob to fileshare. Anyone has fixed that? |
Depending on the result of @adreed-msft 's check, above, it may be fixed in the upcoming release 10.4. (No date announced) |
As far as I tested, SAS token with '/' will work fine when I hit the URL in the browser. The SignedVersion from az-cli is |
Given that there's a workaround (to escape the slashes) we won't look at this right now. But we'll leave this issue open to remind us to look at it later. |
Not solved in AzcopyVersion 10.5.0 Trying to fetch a ZIP file from Azure storage on MacOS (tried zsh and pwsh). File URL copied from portal and do get failure: URL is treated as directory Note, the SAStoken has "/" escaped with %2F, does not help. GET request shows in URL still "restype=directory" |
Is there a workaround for this? I saw mention of escaping the slashes, but that doesn't seem to work. I just confirmed the issue happens in 10.5.1, causing an error on this Microsoft Learn module. When the tokens are retrieved in the portal there, the slashes are already escaped. |
@patridge To workaround it, I've been using the powershell module instead of az cli:
|
Version 10.6.0
|
AzCopy 10.8.0 ... it was working well for us ... Since last week we are seeing this error
We are trying to copy a blob from one storage account to another with SAS token. Is there any workaround? |
any update on this? |
AzCopy 10.8.0 not working for me either failed to perform copy command due to error: cannot use directory as source without --recursive or a trailing wildcard (/*) there are not slashes in my SAS token. Trying to copy files from one storage account's blobs to another storage account's file share. |
@jackfoxy This error usually means azcopy could not list the properties of a certain path on the source account - invalid path or incorrect/insufficient permissions. Can you verify from azcopy log file if it indeed was the case? |
@nakulkar-msft thanks for reaching out. I ran the test just now, which wrote the following to stout:
but did not write a log file with today's timestamp. (And none of my last tests from yesterday have a matching timestamp. I know the TS is in UTC.) Other odd things about the logs: it records delete statements for the dest file, even though there was no file to delete, all the messages are for the dest paths, no messages at all about the source path. |
and the log version is AzcopyVersion 10.7.0, even though I am running 10.8.0 |
The only messages about properties look like this:
|
Please take this issue into priority |
I'm still getting this issue .. any update on this? |
any update on this? I'm still seeing this issue .. this is blocking our weekly release. What is the resolution? Seems like this issue is there since a month and moving from one version to another version. We regenerated the token multiple times with keeping the start date in past, but no way we could make it work.
As you can see we are trying to copy a file and not a directory, but somehow it always detect this as directory |
Not sure if others are experiencing the same problem I encountered when they got this message, but my error may not have been related to the token containing the slash as much as I was trying to operate on several objects in a container, for which my SAS token was not configured to allow. I was able to resolve this issue by adding Object to the Allowed resource types when I generated a SAS (shared access signature). |
@patridge comment helped resolved this for me. The error message was not helpful. Perhaps the error should guide the user to validate their token. |
There's still a PR open related to this issue, but in 10.12.1, /s in SAS tokens work as expected. |
This fixed my problem. Had to select "object" when generating the SAS token. |
The above PR has been merged to improve error reporting when determining if a source is a directory. Closing this issue as a result. If you are still experiencing any issues, please reopen the issue or create a new one. |
Which version of the AzCopy was used?
AzCopy 10.3.4
Which platform are you using? (ex: Windows, Mac, Linux)
Linux
(Running in a powershell script on a Jenkins node)
What command did you run?
What problem was encountered?
Got the error:
failed to perform copy command due to error: cannot use directory as source without --recursive or a trailing wildcard (/*)
The files being copied are .vhd files and definitely not directories.
The primary problem occurred when changing from using the Az powershell module to the az cli to generate the SAS URIs. e.g.:
The issue seems to be that the az cli does not properly escape the
/
character. e.g.:https://(account).blob.core.windows.net/(container)/(filename).vhd?...&sig=.../.../...
https://(account).blob.core.windows.net/(container)/(filename).vhd?...&sig=...%2F...%2F...
This is probably also a bug for the az cli, but I think azcopy should update its parsing to better handle the SAS tokens. The current parsing should not mistake a file for a directory because of URI parameters.
How can we reproduce the problem in the simplest way?
Use the az cli to generate two file Uris+SAS and see if they have an unescaped
/
in the SAS token.Have you found a mitigation/solution?
Switching back to the AZ powershell module works.
Manually escaping the slashes prior to the azcopy call should also work.
The text was updated successfully, but these errors were encountered: