Fresh stack trace from ruwiki on T430727:

In T430594#12068303, @Dzahn wrote:

@thcipriani hello, please take a look

Looks like this is resolved, thanks for the task and the patch @vaughnwalters

In T423917#12056375, @cscott wrote:

Risky Patch! 🚂🔥

• Change: https://gerrit.wikimedia.org/r/c/1305746
• Summary:
  • This configuration change depends on Parsoid 0.24.0-a12, which is included in 1.47.0-wmf.8 (but not 1.47.0-wmf.7)
  • It should be reverted if the train is rolled back to -wmf.7, or if the version of Parsoid included in -wmf.8 is rolled back to -a11.
• Revert plan: Revert https://gerrit.wikimedia.org/r/c/1305746 before rolling back group2 to -wmf.7
• Affected wikis: all (as the config is currently written)
• IRC contact: cscott, arlolra (timezone UTC-4)
• UBN Task Projects/tags: Content-Transform-Team
• Would you like to backport this change rather than ride the train?: Yes

In T429610#12039261, @MoritzMuehlenhoff wrote:

@thcipriani This needs your approval for the deployment group.

Adding some detail here after chatting with @Mhurd

In T429824#12042981, @cscott wrote:

@thcipriani how do you remove the cache for a job?

NPM cache data was corrupted. Removed the cache for the job and it looks like changes are flowing again.

After some initial exploration there may be pieces to break out. Assigning to @jnuche to do the initial exploration of what's going to break.

Sounds like @jeena is already working on this after talking to @jnuche reassigning to reflect that.

Egress is now set up in the helm chart. Resolving. @jeena if you spot any problems, feel free to reopen.

Tagging with Catalyst (PatchDemo) means we'll see it during triage, setting this as low for now. If maniphest.info is slated for removal soon we can up the priority. If not, it's in the backlog.

Thanks for the fix @vaughnwalters should be live now.

In T428903#12020551, @Dzahn wrote:

Oh, interesting. Thanks for the explanation @thcipriani. Yea, I was only looking at actual file system. So GitLab has not reclaimed the space.

In T428903#12020525, @Dzahn wrote:

Maybe let me also ask Tyler if 98.8 GB Thcipriani / Flaky Tests is expected, needed and growing, given how the second largest is less than half the size.