-
-
Notifications
You must be signed in to change notification settings - Fork 516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration test framework #3121
Conversation
Also rename pytest example to integration_tests to not have name-collisions
…ion-test-framework
Co-authored-by: Falko Schindler <falko@zauberzeug.com>
True. The magic fixture loading from plugins occasionally leads to unpredictable name clashes. See pytest-dev/pytest#3834 and pytest-dev/pytest#3966 for a lengthly discussions. The example of Still, I would really like to keep the simple fixture names @pytest.mark.module_under_test(main)
async def test_basic_startup_appearance(user: User) -> None:
"""Test basic appearance of the chat app."""
await user.open('/')
await user.should_see('simple chat app')
await user.should_see('message')
... I have not found any other plugins which use that name. But they plugin space is vast.
I have experimented with explicit activation of our fixtures and have a working prototype where you need to pass @falkoschindler came up with an interesting pattern which could solve the dillema: We would provide the fixtures @pytest.mark.fixture
def screen(nicegui_screen):
return nicegui_screen
@pytest.mark.fixture
def user(nicegui_user):
return nicegui_user But that would make using nicegui tests a bit more complicated. All in all, I'm in favour of just keeping at as it is right now in this pull request:
|
@rodja I have an alternative suggestion based on @codingpaula's experiments from yesterday: She struggled to get our pytest plugin (provided via poetry entrypoint) to work in a separate test project. Even though it works in the NiceGUI example, it doesn't seem to be correctly registered when used outside of the NiceGUI workspace. But it does work when using the pytest_plugins = ['nicegui.testing.fixtures'] Strangely, this works even without defining the plugin via poetry in NiceGUI. We couldn't figure out why. But currently we have doubts that the magic registration of pytest plugins via poetry entrypoints actually works. Yes, RoSys also provides such a plugin. But do we have proof it actually works? We couldn't find any project that actually uses fixtures from the RoSys pytest plugin. Maybe we are completely on the wrong track, and it would certainly help to chat about it together on Monday. Nevertheless, the
The latter still allows writing |
Regarding the filter rules: I noticed that "within(instance=)" is a bit nonsensical when used with multiple elements. Since it now only matches when the element's ancestors include all given instances, you could as well specify the one closest to the element. All others don't contribute anything. Example: root > A > B > C > label
within(instance=[A, C])
within(instance=C) # equivalent But in favor of a very symmetric rule set, I think we can leave it as it is. |
About the pytest plugin
The Field Friend is using the
Oh wow. I did not get that in their documentation. And also ChatGPT 4o and Claude Sonnet did not tell me about it when I asked how I could "not autoload a plugin, but do it explicitly". That is great! A slightly modified strategy of what you are suggesting would be:
While it would make the documentation a bit more lengthly, small projects like our About the
I'm not so sure. Consider the scenario where there is another
|
About the pytest plugin Ok, that makes three ways to use About the Sure, there can be multiple elements with marker or content "C". But one |
* remove pytest-prefix from plugin specification * fix requirements and settings for pytests example * rename screen and user with prefix nicegui_ * add conftest with renaming to pytests example * add internal renaming of nicegui_screen to screen and nicegui_user to user * remove unused pylint ignore * fix rename in examples * remove entry point * remove renaming * remove renaming in examples * remove renaming fixtures and add example code * double quotes * reworked as described in docs (#3413) * add pytest plugin loading to examples * run the pytest examples as part of the test_startup.sh to avoid conflicts in setup * renamed the pytest plugin from fixtures.py to plugin.py * fix todo tests * rename folder --------- Co-authored-by: Rodja Trappe <rodja@zauberzeug.com> Co-authored-by: Falko Schindler <falko@zauberzeug.com>
* begin with documentation of fixtures and pytest setup * code review * also prepare simulation for screen fixture * replace hacky docs.pytest and Demo.raw with generic docs.part * better split of testing topics in the docs * improve user docs * add UserInteraction reference * review * simplify doc.part * fix storage test * web driver info * review --------- Co-authored-by: Falko Schindler <falko@zauberzeug.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this PR is finally ready to merge.
In 161c51f I removed the need for activate()
and deactivate()
by replacing ui.navigate
whenever an attribute of a user is accessed. Since the slot stack manipulation was obsolete anyway, we could get rid of quite some code.
@rodja Feel free to merge if you agree with the latest changes. 🙂
This pull request introduces a test framework which allows to write high-level integration tests which run as fast as unit tests.
Note: Partly based on PR #1931.
Open tasks:
__str__
: We need to clarify and test the general layout of the generated string.user.should_not_see
: We need to handleui.notify
?user.gather_elements
: Do we really need nestedlist
initializers? And should we work with sets to avoid duplicates?ElementFilter
: Maybe "Styling & Appearance" is a better place for it?pycheck nicegui
isn't happy yet (mainly unused fixure arguments,T
vs.Element
, missing_iterator
, missing docstrings)response.encoding = 'utf-8'
? --> We don't.Screen
?activate()
? It doesn't seem to make any difference.ui.number
,ui.editor
,ui.codemirror
etc.element_contents
. It allows a test like this to succeed:ElementFilter.within_kinds
has no test and does not seem to workElementFilter.exclude
may be anelement
and notkind
...ElementFilter
.screen
,user
,create_user
, link to examples) -> PR Add documentation of pytest fixtures and pytest configuration #3413user.activate
anduser.deactivate
?