Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--parallel flag, migrated from dotnet vstest, is missing in dotnet test documentation #35931

Open
1 of 3 tasks
uladz-zubrycki opened this issue Jun 23, 2023 · 2 comments
Open
1 of 3 tasks

--parallel flag, migrated from dotnet vstest, is missing in dotnet test documentation #35931

uladz-zubrycki opened this issue Jun 23, 2023 · 2 comments
Labels
Pri3 ⌚ Not Triaged Not triaged

Comments

@uladz-zubrycki
Copy link

uladz-zubrycki commented Jun 23, 2023 •
edited
Loading

Issue description

According to the dotnet vstest documentation page, the command is superseded by the dotnet test command, which can now be used to run assemblies.

Which, I believe, was done as a result of microsoft/vstest#1453 request in a scope of microsoft/vstest-docs#196.

dotnet vstest is able to run tests from the assemblies in parallel if --parallel flag is provided. Per the referenced RFC, there was an intention to support the same behavior for the dotnet test command as well.

Success criteria (as per user ask)

  1. The "dotnet test" command can be used in place of the "dotnet vstest" command.
  2. The "dotnet test" command can accept a list of dlls
  3. Using "dotnet test" and targeting a list of dlls, there is an argument for "--parallel" which will run the tests from the dlls in parallel.
  4. Using "dotnet test" and targeting a dll, the inclusion or exclusion of the "--no-build" argument will have no impact on the behavior of execution.

My local testing confirms that --parallel flag is respected and tests are run in parallel; test from the assemblies are run sequentially otherwise. Note, that I'm talking parallel execution of the assemblies themselves and not the internal assembly-level of tests parallelization, which is handled by NUnit and its adapter for me.

However, there is no documentation for that feature at the dotnet test page and in the output of the dotnet test --help command, which is confusing.

  1. Is parallel execution of tests from assemblies supported by the dotnet test command officially, and it's safe to rely on that, or it's an implementation detail exposed accidentally?
  2. Should the documentation for dotnet test be updated to reflect that?

Target framework

  • .NET Core
  • .NET Framework
  • .NET Standard
dotnet --info output 
.NET SDK:
 Version:   7.0.304
 Commit:    7e794e2806

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.304\

Host:
  Version:      7.0.7
  Architecture: x64
  Commit:       5b20af47d9

.NET SDKs installed:
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.426 [C:\Program Files\dotnet\sdk]
  6.0.100 [C:\Program Files\dotnet\sdk]
  6.0.118 [C:\Program Files\dotnet\sdk]
  7.0.302 [C:\Program Files\dotnet\sdk]
  7.0.304 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found
dotnet test --version output 17.6.3.22601
@dotnet-bot dotnet-bot added the ⌚ Not Triaged Not triaged label Jun 23, 2023
@dhrumil0902
Copy link

@uladz-zubrycki I am at a similar cross-road, have you made a descision or got more information regarding this?

@uladz-zubrycki
Copy link
Author

uladz-zubrycki commented Jul 15, 2024 •
edited
Loading

@dhrumil0902 as I remember, I've ended up using the flag and haven't ever rechecked it since then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Pri3 ⌚ Not Triaged Not triaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@uladz-zubrycki @dotnet-bot @dhrumil0902