Page MenuHomePhabricator

Cloud-ServicesUmbrella
ActivePublic

Details

Description

Cloud-Services is the umbrella project for tasks related to Wikimedia Cloud Services infrastructure as well as the various projects hosted there. A general overview of services offered can be found at wikitech:Help:Cloud Services Introduction

Subprojects:

The teams working on the infrastructure have separate projects at tools-infrastructure-team and tools-platform-team that are used to track the backlogs and current priorities of the teams. Please do not tag those directly if you do not know what you are doing.

Recent Activity

Thu, Jul 16

taavi edited Description on Cloud-Services.
Thu, Jul 16, 5:29 PM

Tue, Jul 14

cmooney updated the task description for T430918: codfw: rack B5 maintenance.
Tue, Jul 14, 11:08 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF updated the task description for T430918: codfw: rack B5 maintenance.
Tue, Jul 14, 9:05 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF updated the task description for T430918: codfw: rack B5 maintenance.
Tue, Jul 14, 9:00 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF closed T430920: Switchover s7 master (db2159 -> db2220), a subtask of T430918: codfw: rack B5 maintenance, as Resolved.
Tue, Jul 14, 8:59 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Mon, Jul 13

BTullis added a comment to T430918: codfw: rack B5 maintenance.

I have cordoned dse-k8s-worker2003

root@deploy2003:~# kubectl cordon dse-k8s-worker2003.codfw.wmnet
node/dse-k8s-worker2003.codfw.wmnet cordoned

I can do a full drain tomorrow as well.

Mon, Jul 13, 6:58 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
LSobanski moved T430918: codfw: rack B5 maintenance from Incoming to Consultation on the collaboration-services board.
Mon, Jul 13, 3:49 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF updated the task description for T430918: codfw: rack B5 maintenance.
Mon, Jul 13, 3:45 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
cmooney updated the task description for T430918: codfw: rack B5 maintenance.
Mon, Jul 13, 2:40 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
hnowlan added a comment to T430918: codfw: rack B5 maintenance.

graphite2004 doesn't need depooling. centrallog will have some gaps in logs on-disk but that's unavoidable and low-impact.

Mon, Jul 13, 11:13 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Thu, Jul 9

cmooney updated the task description for T430918: codfw: rack B5 maintenance.
Thu, Jul 9, 5:18 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
ayounsi moved T430918: codfw: rack B5 maintenance from Backlog to This quarter on the netops board.
Thu, Jul 9, 9:56 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Mon, Jul 6

Eevans updated the task description for T430918: codfw: rack B5 maintenance.
Mon, Jul 6, 11:42 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Dzahn added a comment to T430918: codfw: rack B5 maintenance.

re: contint2002 - This is not the main CI server, that is currently contint1002, but it is still used to a certain extent.

Mon, Jul 6, 4:53 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Dzahn added a project to T430918: codfw: rack B5 maintenance: Release-Engineering-Team.
Mon, Jul 6, 4:52 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Fri, Jul 3

Gehel moved T430918: codfw: rack B5 maintenance from Backlog - project to Backlog - operations on the Data-Platform-SRE (2026-07-03 - 2026-07-31) board.
Fri, Jul 3, 9:49 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Gehel edited projects for T430918: codfw: rack B5 maintenance, added: Data-Platform-SRE (2026-07-03 - 2026-07-31); removed Data-Platform-SRE (2026-06-05 - 2026-06-26).
Fri, Jul 3, 9:49 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
MLechvien-WMF moved T430918: codfw: rack B5 maintenance from Inbox to Radar (Awareness) on the ServiceOps new board.
Fri, Jul 3, 9:27 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Thu, Jul 2

Gehel edited projects for T430918: codfw: rack B5 maintenance, added: Data-Platform-SRE (2026-06-05 - 2026-06-26); removed Data-Platform-SRE.
Thu, Jul 2, 2:57 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Gehel moved T430918: codfw: rack B5 maintenance from Incoming to Scratch on the Data-Platform-SRE board.
Thu, Jul 2, 2:56 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Maintenance_bot removed a project from T430918: codfw: rack B5 maintenance: Patch-For-Review.
Thu, Jul 2, 2:32 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
gerritbot added a comment to T430918: codfw: rack B5 maintenance.

Change #1307118 merged by Majavah:

[operations/puppet@production] hieradata: Indicate cloudweb2002-dev does not need depooling

https://gerrit.wikimedia.org/r/1307118

Thu, Jul 2, 1:42 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
cmooney updated the task description for T430918: codfw: rack B5 maintenance.
Thu, Jul 2, 12:27 PM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
gerritbot added a project to T430918: codfw: rack B5 maintenance: Patch-For-Review.
Thu, Jul 2, 11:55 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
gerritbot added a comment to T430918: codfw: rack B5 maintenance.

Change #1307118 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] hieradata: Indicate cloudweb2002-dev does not need depooling

https://gerrit.wikimedia.org/r/1307118

Thu, Jul 2, 11:54 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
cmooney updated the task description for T430918: codfw: rack B5 maintenance.
Thu, Jul 2, 10:06 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
cmooney claimed T430918: codfw: rack B5 maintenance.
Thu, Jul 2, 10:04 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
Marostegui added a comment to T430918: codfw: rack B5 maintenance.

db2160 needs just a downtime. dbproxy will fail during the maintenance (just irc notification) and they'll recover on their own once the host has connectivity again.

Thu, Jul 2, 9:42 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF added a subtask for T430918: codfw: rack B5 maintenance: T430920: Switchover s7 master (db2159 -> db2220).
Thu, Jul 2, 9:30 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
ayounsi added a parent task for T430918: codfw: rack B5 maintenance: T426197: codfw: pod AB switches upgrade (2026).
Thu, Jul 2, 9:28 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF updated the task description for T430918: codfw: rack B5 maintenance.
Thu, Jul 2, 9:25 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
CWilliams-WMF updated the task description for T430918: codfw: rack B5 maintenance.
Thu, Jul 2, 9:25 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
ayounsi added a comment to T430918: codfw: rack B5 maintenance.

@cmooney @Papaul can one of you take that upgrade ? And update the date/time to anything that suits you in that week.

Thu, Jul 2, 9:22 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops
ayounsi created T430918: codfw: rack B5 maintenance.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Thu, Jul 2, 9:12 AM · tools-infrastructure-team, Cloud-VPS, Release-Engineering-Team, Data-Platform-SRE (2026-07-03 - 2026-07-31), ServiceOps new, Infrastructure-Foundations, Machine-Learning-Team, Traffic, collaboration-services, observability, Data-Persistence, netops

Wed, Jul 1

dancy created T430875: PuppetStaleCertificates alert is too aggressive for deployment-prep.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Wed, Jul 1, 7:35 PM · cloud-services-team, Cloud-VPS

Mon, Jun 29

hashar created T430479: Zuul executor can not reach WMCS OpenStack API.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Mon, Jun 29, 6:33 AM · cloud-services-team, Data-Services, collaboration-services, Continuous-Integration-Infrastructure (Zuul upgrade)

Tue, Jun 23

poro26 added a comment to T429857: [striker] Unable to create phabricator project for tool odivoir – HTTP 500 Internal Server Error on toolsadmin.
Dans T429857#12043354, @Herald a écrit :

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Tue, Jun 23, 1:19 AM · tools-platform-team, cloud-services-team, Striker
poro26 created T429857: [striker] Unable to create phabricator project for tool odivoir – HTTP 500 Internal Server Error on toolsadmin.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Tue, Jun 23, 1:18 AM · tools-platform-team, cloud-services-team, Striker

Jun 17 2026

BLiviero-WMF created T429557: Implement magnum monitoring.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Jun 17 2026, 11:19 PM · Cloud-VPS, tools-infrastructure-team
dancy renamed T429542: debian-12.0-bookworm and debian-13.0-trixie image still reference mirrors.wikimedia.org from debian-13.0-trixie image still references mirrors.wikimedia.org to debian-12.0-bookworm and debian-13.0-trixie image still reference mirrors.wikimedia.org.
Jun 17 2026, 8:56 PM · cloud-services-team, Cloud-VPS
dancy added a parent task for T429542: debian-12.0-bookworm and debian-13.0-trixie image still reference mirrors.wikimedia.org: T416707: Sunsetting mirrors.wikimedia.org.
Jun 17 2026, 8:29 PM · cloud-services-team, Cloud-VPS
dancy created T429542: debian-12.0-bookworm and debian-13.0-trixie image still reference mirrors.wikimedia.org.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Jun 17 2026, 8:29 PM · cloud-services-team, Cloud-VPS
Samwalton9-WMF triaged T429442: Migrate Moderator Tools Cloud VPS tools from Debian Bullseye as High priority.
Jun 17 2026, 7:37 AM · Cloud-VPS (Debian Bullseye Deprecation), Epic, Moderator-Tools-Team
Samwalton9-WMF updated the task description for T429442: Migrate Moderator Tools Cloud VPS tools from Debian Bullseye.
Jun 17 2026, 7:37 AM · Cloud-VPS (Debian Bullseye Deprecation), Epic, Moderator-Tools-Team
Samwalton9-WMF moved T429442: Migrate Moderator Tools Cloud VPS tools from Debian Bullseye from Inbox to Epics on the Moderator-Tools-Team board.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Jun 17 2026, 7:35 AM · Cloud-VPS (Debian Bullseye Deprecation), Epic, Moderator-Tools-Team

Jun 12 2026

cmooney added a parent task for T429013: Upgrade cloudsw1-e4-eqiad: T390813: Upgrade End Of Support Junos.
Jun 12 2026, 11:12 AM · tools-infrastructure-team, Cloud-VPS, netops, Infrastructure-Foundations, SRE
cmooney created T429013: Upgrade cloudsw1-e4-eqiad.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Jun 12 2026, 11:11 AM · tools-infrastructure-team, Cloud-VPS, netops, Infrastructure-Foundations, SRE

Jun 11 2026

Green_Cardamom created T428830: Urgent: Backup restore request (Toolforge).

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Jun 11 2026, 12:17 AM · cloud-services-team, Data-Services, Toolforge

May 26 2026

BLiviero-WMF created T427338: Streamline cloud services reboots to minimize admin impact.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

May 26 2026, 10:52 PM · tools-infrastructure-team

May 21 2026

Dzahn edited projects for T422559: @wikimedia.org email addresses don't seem to be receiving emails sent by the test Phabricator instance, added: Cloud-Services; removed VPS-project-Phabricator.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

May 21 2026, 6:24 PM · VPS-project-Phabricator, Infrastructure-Foundations, Mail, collaboration-services