Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke-WebRequest/RestMethod: Rename TimeoutSec to ConnectTimeoutSec and alias to TimeoutSec and add NetworkTimeoutSec timeout #19519

Closed
stevenebutler opened this issue Apr 16, 2023 · 13 comments · Fixed by #19558
Labels
Resolution-Fixed The issue is fixed.

Comments

@stevenebutler
Copy link
Contributor

stevenebutler commented Apr 16, 2023

Rename the parameter with an alias based on this wg-powershell-cmdlets review comment

@PowerShell/wg-powershell-cmdlets reviewed this. Based on the documentation currently, -TimeoutSec is for the initial connection and the definition of this parameter should not change. We suggest renaming this parameter to -ConnectTimeoutSec and alias to -TimeoutSec. For this new capability, we propose adding a new -NetworkTimeoutSec (default to 0 meaning infinite) to handle the case of intermittent network issues.

Originally posted by @SteveL-MSFT in #18005 (comment)

@stevenebutler stevenebutler changed the title Rename TimeoutSec to ConnectTimeoutSec and alias to TimeoutSec and add NetworkTimeoutSec timeout Invoke-WebRequest/RestMethod: Rename TimeoutSec to ConnectTimeoutSec and alias to TimeoutSec and add NetworkTimeoutSec timeout Apr 16, 2023
@kasini3000
Copy link

kasini3000 commented Apr 16, 2023

how many timeout with http client?It's best to add it all at once.

connectTimeout
sendTimeout
readTimeout

https://learn.microsoft.com/en-us/dotnet/api/system.net.httplistenertimeoutmanager?view=net-7.0

The timeout manager to use for an HttpListener object.

I saw you adding a cancellation token to IWR. Do you have time to address this issue?
PowerShell/ThreadJob#26

@stevenebutler
Copy link
Contributor Author

stevenebutler commented Apr 16, 2023 via email

@iSazonov
Copy link
Collaborator

We have to proceed from realistic scenarios. There are actually two types (1) get/send small data with a request, (2) upload/download file.
From this it is easy to draw the right conclusion how Timeout should work.

@stevenebutler
Copy link
Contributor Author

You could also be uploading/downloading data you have no intention of storing in a file so I'm not sure it's as clear cut as having two options and an easy, automatic choice between what to do with the parameter.

Simplest implementation is a straight timeout on the receipt of all data but as has been discussed before this doesn't work well when we don't know ahead of time how long the download will take because of varying loads/network speeds etc. The main advantage of this sort of timeout is it could be implemented by using cts.CancelAfter(timeout) before reading the response stream.

Another option is a "no packet received" timeout which would reset every time some data is received. This is more challenging to implement because there's no direct support for this kind of timeout in the core libraries used.

Unfortunately, the WG didn't really give much guidance when writing that statement as to whether it was intending it to be a timeout for the overall download time or a "no packet received in x period" timeout or some hybrid of the two.

Both might be useful depending on the use case but it may be surprising if the same parameter has a different behaviour depending on whether we are downloading to a file or not.

@iSazonov could you be a bit more specific about what you expect since others may have a different view point and it's good to be explicit.

@iSazonov
Copy link
Collaborator

@iSazonov could you be a bit more specific about what you expect since others may have a different view point and it's good to be explicit.

#18005 (comment)

In short, the only meaning of this parameter is the timeout for any atomic operation.
(In any case cmdlet will return a generic error and it does not matter what the reason is - DNS, connection, network timeout or any other of an infinite number of reasons. Although actually this reason will be an InnerException.)

@kasini3000
Copy link

kasini3000 commented Apr 17, 2023

Rename TimeoutSec to ConnectTimeoutSec
add NoDataReceiveTimeoutSec
sounds good

@stevenebutler
Copy link
Contributor Author

@iSazonov - so what you're saying is the timeout should only ever mean "length of time between received data chunks/packets/whatever" has been exceeded, and should in no way apply to the entire length of the download, regardless of the use case involved.

There might be some benefit in having an additional parameter that caps the overall download time, but that is probably overkill and wasn't requested by the WG. Do you have any thoughts on that?

@iSazonov
Copy link
Collaborator

@stevenebutler The parameter you ask makes no sense because download time is unpredictable. Any component (client, server, network) can be either faster or slower. Moreover, the size of the file is in general also unpredictable. (For example, the size of a daily log file can be 1 Mb or 10Gb.)

You can say that "my file always uploads no more than a minute and I agree to wait even an hour". But a timeout of one hour makes no sense because the server will close the session if there is a delay of any operation for about a minute, and a high-loaded server will do it even earlier.

@CarloToso
Copy link
Contributor

I think the -NetworkTimeoutSec parameter is meant to handle this issue #16122 (we could also avoid the new parameter and set a timeout at 5 minutes with no data recieved)

@stevenebutler
Copy link
Contributor Author

From the WG notes they suggested defaulting to not have any timeout which is leaving the default situation where it will hang until someone CTRL-C the download if the OS doesn't report the loss of connection. Are you suggesting the default should be something other than no timeout?

If the WG have decided on a course of action, is that what should be implemented, or is it still up for further debate?

@CarloToso
Copy link
Contributor

I don't know if it's up for further debate, but in my opinion we should follow PS5 behaviour described in #12249. Would you be open to reconsider @SteveL-MSFT

@stevenebutler
Copy link
Contributor Author

PR above adds the feature as per the discussion here, and with the timeout defaulting to disabled. If WG wants to change the default that will be trivial to do.

@ghost ghost added In-PR Indicates that a PR is out for the issue and removed In-PR Indicates that a PR is out for the issue labels Apr 23, 2023
@ghost ghost added Resolution-Fixed The issue is fixed. and removed In-PR Indicates that a PR is out for the issue labels Jun 19, 2023
@ghost
Copy link

ghost commented Jun 29, 2023

🎉This issue was addressed in #19558, which has now been successfully released as v7.4.0-preview.4.:tada:

Handy links:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Resolution-Fixed The issue is fixed.
Projects
None yet
4 participants