Page MenuHomePhabricator

Test Kitchen (Experiment Platform Sprint 26)Milestone
ArchivedPublic

Members (8)

Watchers

  • This project does not have any watchers.
  • View All

Details

Description

Experiment Platform Sprint for June 25 - July 16 2026.

Recent Activity

Yesterday

brouberol moved T431015: GrowthBook staging sluggish and throws authentication errors from Blocked/Waiting to Done on the Data-Platform-SRE (2026-07-03 - 2026-07-31) board.
Fri, Jul 17, 9:33 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
brouberol updated Other Assignee for T431015: GrowthBook staging sluggish and throws authentication errors, added: brouberol.
Fri, Jul 17, 9:17 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)

Thu, Jul 16

KReid-WMF archived Test Kitchen (Experiment Platform Sprint 26).
Thu, Jul 16, 5:04 PM
KReid-WMF closed T428010: A/A test with logged-in users on Commons, WikiData, Mediawiki, MetaWiki to investigate SRM as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 26), Essential-Work
KReid-WMF closed T427958: [Spike] Determine best method to guide traffic limits for NCS experiments, a subtask of T429648: [EPIC] Traffic limit exception: request and grant mechanism, as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 27), Epic
KReid-WMF closed T427958: [Spike] Determine best method to guide traffic limits for NCS experiments as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
KReid-WMF closed T406576: Test Kitchen UI: Add link to Codesearch to action menu as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 26), RoadToWiki, good first task, Essential-Work, Test Kitchen UI
KReid-WMF closed T430842: Align validation rules of running and draft experiments as Resolved.
Thu, Jul 16, 4:04 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26)
KReid-WMF closed T428984: [GrowthBook Integration] Implement alerting to Phabricator for invalid experiment configuration as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 26), Patch-For-Review, OKR-Work
KReid-WMF closed T429420: [Customize GrowthBook] Auto-check Pre-Launch Checklist items via the backend poller as Resolved.
Thu, Jul 16, 4:04 PM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
KReid-WMF closed T431015: GrowthBook staging sluggish and throws authentication errors as Resolved.
Thu, Jul 16, 4:04 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
KReid-WMF moved T395891: Test Kitchen UI (FKA MPIC) NPM updates to recent major version from Done to In Progress on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 4:03 PM · Test Kitchen (Experiment Platform Sprint 27), Essential-Work, Test Kitchen UI
Maintenance_bot removed a project from T431015: GrowthBook staging sluggish and throws authentication errors: Patch-For-Review.
Thu, Jul 16, 2:31 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci added a comment to T429420: [Customize GrowthBook] Auto-check Pre-Launch Checklist items via the backend poller.

Because of the issue we have been dealing with via T431015: GrowthBook staging sluggish and throws authentication errors, the Checklist autocompletion feature will be fully disabled for now. We have found that the number of requests done by that feature (about 20 req/minute) was creating a high number of concurrent jobs that were competing for same resources (to access and modify some structures) and that has been causing a excessive high CPU load in the GrowthBook's database.

Thu, Jul 16, 2:13 PM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
Sfaci moved T428984: [GrowthBook Integration] Implement alerting to Phabricator for invalid experiment configuration from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Test Kitchen (Experiment Platform Sprint 26), Patch-For-Review, OKR-Work
Sfaci moved T406576: Test Kitchen UI: Add link to Codesearch to action menu from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Test Kitchen (Experiment Platform Sprint 26), RoadToWiki, good first task, Essential-Work, Test Kitchen UI
Sfaci moved T430842: Align validation rules of running and draft experiments from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26)
Sfaci moved T429420: [Customize GrowthBook] Auto-check Pre-Launch Checklist items via the backend poller from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
Sfaci moved T431015: GrowthBook staging sluggish and throws authentication errors from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci moved T395891: Test Kitchen UI (FKA MPIC) NPM updates to recent major version from To Deploy to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 2:08 PM · Test Kitchen (Experiment Platform Sprint 27), Essential-Work, Test Kitchen UI
gerritbot added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

Change #1311414 merged by jenkins-bot:

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.4.9 release to production

https://gerrit.wikimedia.org/r/1311414

Thu, Jul 16, 1:50 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
gerritbot added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

Change #1311413 merged by jenkins-bot:

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.4.9 release to staging

https://gerrit.wikimedia.org/r/1311413

Thu, Jul 16, 1:45 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
gerritbot added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

Change #1311414 had a related patch set uploaded (by Santiago Faci; author: Santiago Faci):

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.4.9 release to production

https://gerrit.wikimedia.org/r/1311414

Thu, Jul 16, 10:05 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
gerritbot added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

Change #1311413 had a related patch set uploaded (by Santiago Faci; author: Santiago Faci):

[operations/deployment-charts@master] Test Kitchen UI: Deploy v1.4.8 release to staging

https://gerrit.wikimedia.org/r/1311413

Thu, Jul 16, 10:03 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci moved T431015: GrowthBook staging sluggish and throws authentication errors from Needs Review to To Deploy on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 9:53 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
CodeReviewBot added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

Disable Checklist completion

Thu, Jul 16, 9:47 AM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
phuedx updated the task description for T432144: Update how start/end dates and phases are handled and validated.
Thu, Jul 16, 6:13 AM · Test Kitchen (Experiment Platform Sprint 27), Essential-Work
Sfaci moved T427958: [Spike] Determine best method to guide traffic limits for NCS experiments from Needs Review to Done on the Test Kitchen (Experiment Platform Sprint 26) board.
Thu, Jul 16, 1:25 AM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
Sfaci added a comment to T427958: [Spike] Determine best method to guide traffic limits for NCS experiments.

After reviewing both proposals based on the great comparison table here, some numbers there and some other considerations, a decision was made a we will implement the approach based on UI:

  • The numbers showed that the implementation effort was lower for the MR-based on
  • The numbers also showed that there would be a constant additional effort in the MR-based one because every exception would mean a deployment. And that could be a blocker in some cases. An unrelated deployment that is being delayed for any reason could delay an exception that needs to be applied
  • We have also considered that the additional effort in implementing the UI can be an investment for the future: No additional constant effort will be needed (no deployments) to register and activate a traffic limit exception
  • Also the new UI can fit with the new lading page we are considering for test-kitchen.wikimedia.org
Thu, Jul 16, 1:23 AM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
Sfaci added a subtask for T429648: [EPIC] Traffic limit exception: request and grant mechanism: T432313: [Traffic limit exception] Implement a validation layer for the traffic limit exception UI.
Thu, Jul 16, 12:33 AM · Test Kitchen (Experiment Platform Sprint 27), Epic
Sfaci added a subtask for T429648: [EPIC] Traffic limit exception: request and grant mechanism: T432310: [Traffic limit exception] Traffic limit exception UI is available only for growthbook-admin LDAP group.
Thu, Jul 16, 12:14 AM · Test Kitchen (Experiment Platform Sprint 27), Epic

Wed, Jul 15

Sfaci added a subtask for T429648: [EPIC] Traffic limit exception: request and grant mechanism: T432309: [Traffic limit exception] TK API responses include the traffic limit exception when registered and active.
Wed, Jul 15, 11:50 PM · Test Kitchen (Experiment Platform Sprint 27), Epic
Sfaci moved T431015: GrowthBook staging sluggish and throws authentication errors from In Progress to Needs Review on the Test Kitchen (Experiment Platform Sprint 26) board.
Wed, Jul 15, 11:26 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci added a comment to T431015: GrowthBook staging sluggish and throws authentication errors.

From our side, Experiment Platform team, we have already disabled the feature that caused this issue in both staging and production environments.

Wed, Jul 15, 11:26 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
CodeReviewBot added a project to T431015: GrowthBook staging sluggish and throws authentication errors: Patch-For-Review.

sfaci opened https://gitlab.wikimedia.org/repos/data-engineering/test-kitchen/-/merge_requests/371

Wed, Jul 15, 11:22 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci added a subtask for T429648: [EPIC] Traffic limit exception: request and grant mechanism: T432305: [Traffic limit exception] Implement both controller and service layer to manage traffic limit exceptions.
Wed, Jul 15, 11:03 PM · Test Kitchen (Experiment Platform Sprint 27), Epic
Sfaci added a subtask for T427958: [Spike] Determine best method to guide traffic limits for NCS experiments: T432304: [Traffic limit exceptions] Design and build an UI to list/register/update/remove traffic limit exceptions.
Wed, Jul 15, 10:53 PM · Test Kitchen (Experiment Platform Sprint 26), OKR-Work
mpopov added a comment to T428010: A/A test with logged-in users on Commons, WikiData, Mediawiki, MetaWiki to investigate SRM.

I have stopped the A/A test.

Wed, Jul 15, 7:13 PM · Test Kitchen (Experiment Platform Sprint 26), Essential-Work
mpopov moved T428010: A/A test with logged-in users on Commons, WikiData, Mediawiki, MetaWiki to investigate SRM from In Progress to Done on the Test Kitchen (Experiment Platform Sprint 26) board.

We ran this A/A test to check whether there's something special about certain wikis (Wikidata, Meta-Wiki, Wikimedia Commons, and MediaWiki) and Test Kitchen SDKs that might have contributed to the SRM issue we saw in PSI's Email Confirmation Banner A/B test. After running this A/A test for about 3 weeks we do not see any SRM on those wikis, which has improved our confidence in the platform.

Wed, Jul 15, 7:13 PM · Test Kitchen (Experiment Platform Sprint 26), Essential-Work
Sfaci added a project to T431015: GrowthBook staging sluggish and throws authentication errors: Essential-Work.
Wed, Jul 15, 4:12 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
brouberol reassigned T431015: GrowthBook staging sluggish and throws authentication errors from brouberol to Sfaci.
Wed, Jul 15, 3:30 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci moved T431015: GrowthBook staging sluggish and throws authentication errors from Ready for Development to In Progress on the Test Kitchen (Experiment Platform Sprint 26) board.
Wed, Jul 15, 3:07 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci edited projects for T431015: GrowthBook staging sluggish and throws authentication errors, added: Test Kitchen (Experiment Platform Sprint 26); removed Test Kitchen.
Wed, Jul 15, 3:06 PM · Essential-Work, Test Kitchen (Experiment Platform Sprint 26), Data-Platform-SRE (2026-07-03 - 2026-07-31)
Sfaci moved T413296: Improve session tick instrument's handling of lack of localStorage from Needs Review to To Deploy on the Test Kitchen (Experiment Platform Sprint 26) board.
Wed, Jul 15, 3:02 PM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), Essential-Work
ReleaseTaggerBot added a project to T427090: Capture experiment version in all experiment-related analytics events: MW-1.47-notes (1.47.0-wmf.12; 2026-07-21).
Wed, Jul 15, 2:00 PM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), RoadToWiki, good first task, Essential-Work
Maintenance_bot removed a project from T427090: Capture experiment version in all experiment-related analytics events: Patch-For-Review.
Wed, Jul 15, 1:31 PM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), RoadToWiki, good first task, Essential-Work
gerritbot added a comment to T427090: Capture experiment version in all experiment-related analytics events.

Change #1298112 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] TestKitchen: Capture experiment version in analytics events

https://gerrit.wikimedia.org/r/1298112

Wed, Jul 15, 1:16 PM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), RoadToWiki, good first task, Essential-Work
ReleaseTaggerBot added a project to T413296: Improve session tick instrument's handling of lack of localStorage: MW-1.47-notes (1.47.0-wmf.12; 2026-07-21).
Wed, Jul 15, 12:00 PM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), Essential-Work
Adarsh2406 moved T427382: Cleanup redundant mw.loader.using statements from Incoming to Completed on the RoadToWiki board.
Wed, Jul 15, 11:40 AM · MW-1.47-notes (1.47.0-wmf.9; 2026-06-30), Test Kitchen (Experiment Platform Sprint 26), Patch-For-Review, RoadToWiki, MediaWiki-extensions-WikimediaEvents, good first task, Technical-Debt
Maintenance_bot removed a project from T413296: Improve session tick instrument's handling of lack of localStorage: Patch-For-Review.
Wed, Jul 15, 11:31 AM · Test Kitchen (Experiment Platform Sprint 27), MW-1.47-notes (1.47.0-wmf.12; 2026-07-21), Essential-Work