The dataset and pipeline for generating it are at: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/tree/main/projects/vandalism

Next step is to find some negative samples from the unreverted abuse filter hits. Looking through them, many seem like they should have been reverted (and probably were, but not in a way we can detect via tags or mediawiki_history) but some seem like genuine false positives.

I'm started with a first pass made from abuse filter hits, using this process:

• Choose a few filters from Special:AbuseFilter that seem to target silly vandalism
• Look up the time range when these filters were set to an action other than 'disallow' (e.g. 'warn') - because no edit is saved when they disallow
• Query for revision IDs of hits in that time range
• Check whether each one was reverted or not (using mediawiki_history.revision_is_identity_reverted
• Positive samples are the reverted edits

Awaiting more information, to see if we can close this task - we may have captured everything essential in the topline metrics.

Date range: 27 March to 10 June 2026 (76 days)
Number of distinct users: 83
Number of wikis: 34

Updates are in a few merge requests, mostly:

• https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/merge_requests/12
• https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/merge_requests/14

Thanks @kevinbazira ! I think we will only need violation, p_violation and p_safe.

Pipeline is here: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/tree/main/projects/suppressed_revisions

Let's leave this open until the backfills are complete. Dashboard is done though.

I don't think we need this any more, since we're pivoting away from the metric, and have analyzed the stability of it historically in T426598: Analyse stability of time to revert bad faith edit metric. We will keep tracking it going forwards, but I don't believe we need the historical comparisons on the dashboard.

This is done, and we are moving forward with the same-day handling metric: T428419: Set up Superset dashboard for monitoring DE4 metric same day handling of bad-faith content

In T417056#11987896, @Tchanders wrote:

Hi, I need to update my key again (hard disk failed, lost the old keys). Can we re-use this task?

Hi, I need to update my key again (hard disk failed, lost the old keys). Can we re-use this task?

There is still discussion over whether this metric needs any more changes, so let's leave this task open until those are resolved.

We will make a task for each metric, so I'm moving this back to In Refinement until that happens.

In T425292#11931394, @OKryva-WMF wrote:

I am thinking of "% none automatically identified accounts that were mitigated", i think this would not add additional complexity? If it's too hard to measure at this point we can de-prio this for now too.

Is there a way to measure manually identified accounts that were not mitigated?

In T425292#11929615, @OKryva-WMF wrote:

Thank you Thalia! I agree that some of these should be removed (updated the general description to remove them)

max_revert_seconds will be 7 * 86400 (7 days)

The dashboard is available here: https://superset.wikimedia.org/superset/dashboard/788

Thanks for the detailed explanations! My comments concern the metric for time-to-revert by UWER (user with extended rights): P91457.

Weekly trend since temporary accounts were deployed to English Wikipedia (5 November, so the first full day of data is 6 November 2025):

Note: this concerns what mitigations, preventions etc already exist. Anything we build later would be incorporated into the metric, but not the baseline calculation.

Let's close this task.

The notebook for gathering the dataset is here: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/tree/main/projects/suppressed_revisions?ref_type=heads