chore(e2e): refactor "Toggle visibility" org-members test #12871

Merged

Gusted merged 1 commit from 0ko/e2e-org-members-toggl into forgejo 2026-06-02 14:52:16 +02:00 AGit

Conversation 0 Commits 1 Files changed 2 +11 -9

0ko commented 2026-06-01 18:01:09 +02:00

Owner

Copy link

This test fails very often in CI. This should hopefully improve it.

With a downclocked CPU I've got 2 errors in 10 tries. After adding

await page.waitForLoadState();

I got no errors in 20 tries. But my testing methodology is unreliable so there's no guarantee that this helps:

• Locator.click() documentation says that it already waits by default for caused navigation to finish
• I added some date time measurement and waitForLoadState was clocking in about 5-15 ms, which shouldn't be causing the issue for expect that times out in 3000ms
  ...but e2e issues were element(s) not found on timeout, not wrong state, so my assumption is that somehow it is asserting the wrong snapshot of a page that isn't fully loaded

Additionally, I've converted revertion logic to make consistent before testing, which is better at handling test retries in case that the revertion part of the code was never ran in previous failure. In such case test retry is guaranteed to fail as well, at least when testing locally, not sure about CI.

Additionally, a completely unrelated comment fix in Playwright config.

This test fails very often in CI. This should hopefully improve it. With a downclocked CPU I've got 2 errors in 10 tries. After adding ``` await page.waitForLoadState(); ``` I got no errors in 20 tries. But my testing methodology is unreliable so there's no guarantee that this helps: * `Locator.click()` documentation says that it already waits by default for caused navigation to finish * I added some `date` time measurement and `waitForLoadState` was clocking in about 5-15 ms, which shouldn't be causing the issue for `expect` that times out in 3000ms ...but e2e issues were `element(s) not found` on timeout, not wrong state, so my assumption is that somehow it is asserting the wrong snapshot of a page that isn't fully loaded Additionally, I've converted _revertion_ logic to _make consistent before testing_, which is better at handling test retries in case that the _revertion_ part of the code was never ran in previous failure. In such case test retry is guaranteed to fail as well, at least when testing locally, not sure about CI. Additionally, a completely unrelated comment fix in Playwright config.

0ko added 1 commit 2026-06-01 18:01:10 +02:00

chore(e2e): refactor "Toggle visibility" org-members test ... 508f3ee237

This should hopefully improve the reliability of this test in CI.

With a downcloced CPU I've got 2 errors in 10 tries. After adding

```
await page.waitForLoadState();
```

I got no errors in 20 tries. But my testing methodology is unreliable so there's no guarantee that this helps:
* `Locator.click()` documentation says that it already waits by default for caused navigation to finish
* I added some `date` time measurement and `waitForLoadState` was clicking in about 5-15 ms, which shouldn't be causing the issue for `expect` that times out in 3000ms
...but e2e issues were `element(s) not found` on timeout, not wrong state, so my assumption is that somehow it is asserting the wrong snapshot of a page that isn't fully loaded

Additionally, I've converted _revertion_ logic to _make consistent before testing_, which is better at handling test retries in case that the _revertion_ part of the code was never ran in previous failure.

Additionally, a completely unrelated comment fix in Playwright config.

0ko 2026-06-01 18:01:10 +02:00

• added the
  test
  not-needed
  label
• requested review from fnetX

Gusted approved these changes 2026-06-02 14:45:42 +02:00

Gusted left a comment

Copy link

Sure thanks!

Sure thanks!

Gusted merged commit eb76582261 into forgejo 2026-06-02 14:52:16 +02:00

viceice-bot referenced this pull request from a commit 2026-06-02 14:56:22 +02:00

chore(e2e): refactor "Toggle visibility" org-members test (#12871)

forgejo-release-notes-assistant added this to the Forgejo v16.0.0 milestone 2026-06-02 14:58:43 +02:00

Sign in to join this conversation.

Reviewers

No reviewers

fnetX

Gusted

Labels

Clear labels   

arch

riscv64

Archived

  

backport/v1.19

Scheduled for backport to Forgejo v1.19

Archived

  

backport/v1.20

Scheduled for backport to Forgejo v1.20

Archived

  

backport/v1.21/forgejo

Scheduled for backport to Forgejo v1.21

Archived

  

backport/v10.0/forgejo

Automated backport to v10.0

Archived

  

backport/v11.0/forgejo

Automated backport to v11.0

  

backport/v12.0/forgejo

Automated backport to v12.0

Archived

  

backport/v13.0/forgejo

Automated backport to v13.0

Archived

  

backport/v14.0/forgejo

Automated backport to v14.0

Archived

  

backport/v15.0/forgejo

Automated backport to v15.0

  

backport/v16.0/forgejo

Automated backport to v16.0

  

backport/v7.0/forgejo

Scheduled for backport to Forgejo v7.0

Archived

  

backport/v8.0/forgejo

Scheduled for backport to Forgejo v8.0

Archived

  

backport/v9.0/forgejo

Scheduled for backport to Forgejo v9.0

Archived

  

breaking

The release containing this change is not backward compatible

  

bug

Something is not working

Archived

  

bug

confirmed

it can be reproduced

  

bug

duplicate

bug has already been reported in the Forgejo tracker

  

bug

needs-more-info

the information provided does not contain enough details

  

bug

new-report

bug has just been reported and need triage (default label on issue creation)

  

bug

reported-upstream

bug cannot be fixed within Forgejo easily, it has been reported upstream

  

code/actions

Forgejo Actions feature

  

code/api

API

  

code/auth

Forgejo Authentication

  

code/auth/faidp

Forgejo as Identity Provider (in OAuth/OIDC flow)

  

code/auth/farp

Forgejo as Relying Party / Client (in OAuth/OIDC flow)

  

code/email

Everything related to email in Forgejo

  

code/federation

Federation

  

code/git

Related to the Git backend in Forgejo

  

code/migrations

Migration between Git forges (i.e. for GitHub, GitLab, Gitea, Forgejo, etc.). NOT for database migrations.

  

code/packages

Forgejo package and container registry

  

code/wiki

  

database

MySQL

https://www.mysql.com/

  

database

PostgreSQL

https://www.postgresql.org/

  

database

SQLite

https://sqlite.org/

  

dependency-upgrade

https://codeberg.org/forgejo/forgejo/issues/2779

  

dependency

Chi

https://github.com/go-chi/chi/

Archived

  

dependency

Chroma

https://github.com/alecthomas/chroma/

Archived

  

dependency

F3

https://f3.forgefriends.org/

  

dependency

ForgeFed

https://forgefed.org

  

dependency

garage

https://git.deuxfleurs.fr/Deuxfleurs/garage

  

dependency

Gitea

https://github.com/go-gitea/gitea

Archived

  

dependency

Golang

https://go.dev/

  

Discussion

  

duplicate

This issue or pull request already exists

  

enhancement/feature

New feature

  

forgejo/accessibility

Accessibility (a11y)

  

forgejo/branding

Branding (logo, name, tagline etc.)

  

forgejo/ci

Forgejo Actions CI configuration

  

forgejo/commit-graph

The commit graph feature and page.

  

forgejo/documentation

  

forgejo/furnace cleanup

Keeping Forgejo in sync with its dependencies and contributing back to them

Archived

  

forgejo/i18n

t9n/translation, l10n/localization, and i18n/internationalization of Forgejo

  

forgejo/interop

Interoperability with other services: Webhooks, bridges, integrations

  

forgejo/moderation

Moderation

  

forgejo/privacy

Privacy first

  

forgejo/release

Release management

  

forgejo/scaling

Performance and scaling

  

forgejo/security

Security (please disclose responsibly)

  

forgejo/ui

User interface

  

Gain

High

User research provides indicators that this would be good to have, interested contributors are encouraged to pick this.

  

Gain

Nice to have

This is likely worth having, but the assumption is not backed by user research data (it might benefit a small amount of users only.) Unlikely to receive much attention, but feel free to pick.

  

Gain

Undefined

Not enough information to assess the request's benefits. This issue may be closed if no gain is established: You can help by giving us more input.

  

Gain

Very High

User research indicates that this is an important improvement for Forgejo users. Contributions very welcome!

  

good first issue

Optimal for first-timers! Make sure to look for further explanations and ask for help if needed. If you want, you can consider the person who added this label as a point of contact.

  

i18n/backport-stable

This PR needs to be backported to stable branch of Forgejo safely and manually, using a migration script.

  

impact

large

Large impact: Potential data loss, many users affected, major degradation in UX.

  

impact

medium

Medium impact: Several users affected, degradation in UX, workarounds might be available but inconvenient.

  

impact

small

Small impact: No data loss, workarounds might be available, affects few users.

  

impact

unknown

Report was not yet triaged to assess impact.

  

Incompatible license

This pull request contains changes that are not (yet) compatible with the current Forgejo license

  

issue

closed

The issue was resolved in the repository of the dependency

  

issue

do-not-exist-yet

An issue should be created in the respository of the dependency

  

issue

open

An open issue exists in the upstream repository of the dependency

  

manual test

Pull requests that have been merged with a manual test

Archived

  

Manually tested during feature freeze

The manual test instructions were followed

  

OS

FreeBSD

Specific to the FreeBSD Operating System

  

OS

Linux

Specific to (GNU/)Linux Operating Systems

  

OS

macOS

Specific to the MacOS Operating System

  

OS

Windows

Specific to the Windows Operating System

  

problem

A user report about a problem. Needs to be triaged to find potential solutions.

  

QA

  

regression

found in the version of the milestone and not before

  

release blocker

Issues that must be fixed before the release can be published

  

Release Cycle

Feature Freeze

Only bug fixes with automated tests (except for CSS/JavaScript)

  

release-blocker

v7.0

Issues that must be fixed before Forgejo v7.0 can be released 17 April 2024

Archived

  

release-blocker

v7.0.1

Issues that must be fixed before Forgejo v7.0.1 can be released

Archived

  

release-blocker

v7.0.2

Issues that must be fixed before Forgejo v7.0.2 can be released

Archived

  

release-blocker

v7.0.3

Issues that must be fixed before Forgejo v7.0.3 can be released

Archived

  

release-blocker

v7.0.4

Issues that must be fixed before Forgejo v7.0.4 can be released

Archived

  

release-blocker

v8.0.0

Issues that must be fixed before Forgejo v8.0.0 can be released

Archived

  

release-blocker/v9.0.0

Issues that must be fixed before Forgejo v9.0.0 can be released

Archived

  

run-all-playwright-tests

Add this label to a PR to run all playwright tests manually.

  

run-end-to-end-tests

Trigger additional tests on the PR when it is ready to be merged

  

stage

2-research

Needs research to moe on.

  

stage

3-design

Needs design to move on

  

stage

4-implementation

This is actionable and waits for implementation

  

test

manual

manual testing has been documented

  

test

needed

test should be added

  

test

needs-help

help needed to add a test

  

test

not-needed

no additional test is needed

  

test

present

test has been added

  

untested

Pull requests that have been merged with no test and submitted as is to the dependency where they belong

Archived

  

User research - time-tracker

Time tracking feature for issues and the JS stopwatch.

  

valuable code

This PR was closed because the implementation is incomplete

  

worth a release-note

Add this PR to the release notes

  

User research - Accessibility

Requires input about accessibility features, likely involves user testing.

  

User research - Blocked

Do not pick as-is! We are happy if you can help, but please coordinate with ongoing redesign in this area.

  

User research - Community

Community features, such as discovering other people's work or otherwise feeling welcome on a Forgejo instance.

  

User research - Config (instance)

Instance-wide configuration, authentication and other admin-only needs.

  

User research - Errors

How to deal with errors in the application and write helpful error messages.

  

User research - Filters

How filter and search is being worked with.

  

User research - Future backlog

The issue might be inspiring for future design work.

  

User research - Git workflow

AGit, fork-based and new Git workflow, PR creation etc

  

User research - Labels

Active research about Labels

  

User research - Moderation

Moderation Featuers for Admins are undergoing active User Research

  

User research - Needs input

Use this label to let the User Research team know their input is requested.

  

User research - Notifications/Dashboard

Research on how users should know what to do next.

  

User research - Rendering

Text rendering, markup languages etc

  

User research - Repo creation

Active research about the New Repo dialog.

  

User research - Repo units

The repo sections, disabling them and the "Add more" button.

  

User research - Security

  

User research - Settings (in-app)

How to structure in-app settings in the future?

No labels

arch

riscv64

backport/v1.19

backport/v1.20

backport/v1.21/forgejo

backport/v10.0/forgejo

backport/v11.0/forgejo

backport/v12.0/forgejo

backport/v13.0/forgejo

backport/v14.0/forgejo

backport/v15.0/forgejo

backport/v16.0/forgejo

backport/v7.0/forgejo

backport/v8.0/forgejo

backport/v9.0/forgejo

breaking

bug

bug

confirmed

bug

duplicate

bug

needs-more-info

bug

new-report

bug

reported-upstream

code/actions

code/api

code/auth

code/auth/faidp

code/auth/farp

code/email

code/federation

code/git

code/migrations

code/packages

code/wiki

database

MySQL

database

PostgreSQL

database

SQLite

dependency-upgrade

dependency

Chi

dependency

Chroma

dependency

F3

dependency

ForgeFed

dependency

garage

dependency

Gitea

dependency

Golang

Discussion

duplicate

enhancement/feature

forgejo/accessibility

forgejo/branding

forgejo/ci

forgejo/commit-graph

forgejo/documentation

forgejo/furnace cleanup

forgejo/i18n

forgejo/interop

forgejo/moderation

forgejo/privacy

forgejo/release

forgejo/scaling

forgejo/security

forgejo/ui

Gain

High

Gain

Nice to have

Gain

Undefined

Gain

Very High

good first issue

i18n/backport-stable

impact

large

impact

medium

impact

small

impact

unknown

Incompatible license

issue

closed

issue

do-not-exist-yet

issue

open

manual test

Manually tested during feature freeze

OS

FreeBSD

OS

Linux

OS

macOS

OS

Windows

problem

QA

regression

release blocker

Release Cycle

Feature Freeze

release-blocker

v7.0

release-blocker

v7.0.1

release-blocker

v7.0.2

release-blocker

v7.0.3

release-blocker

v7.0.4

release-blocker

v8.0.0

release-blocker/v9.0.0

run-all-playwright-tests

run-end-to-end-tests

stage

2-research

stage

3-design

stage

4-implementation

test

manual

test

needed

test

needs-help

test

not-needed

test

present

untested

User research - time-tracker

valuable code

worth a release-note

User research - Accessibility

User research - Blocked

User research - Community

User research - Config (instance)

User research - Errors

User research - Filters

User research - Future backlog

User research - Git workflow

User research - Labels

User research - Moderation

User research - Needs input

User research - Notifications/Dashboard

User research - Rendering

User research - Repo creation

User research - Repo units

User research - Security

User research - Settings (in-app)

Milestone

Clear milestone

No items

No milestone

Forgejo v16.0.0

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

forgejo/forgejo!12871

No description provided.