Getting Operations Assistance
- The majority of operations requests should be filed within the Wikimedia Phabricator installation using the #operations project tag.
- If you keep the default priority to 'Needs Triage' and it is in the
#Operations project, our Ops Clinic Duty assignee for the week will triage your request.
- This link will create a task in the operations project.
- If you further refine your request using the below instructions, it will usually result in faster triage.
Getting help via IRC
- Operations team members idle in
#wikimedia-operations on Freenode.
- This is generally useful for vague questions or project planning, but non-ideal for hardware requests, access requests, or ongoing work.
- If the request will result in work on the part of the operations team member, a Phabricator task will be requested to track the work.
- There is an Ops Clinic Duty assignee from the operations team for every week.
- Clinic duty person is listed in the topic for
#wikimedia-operations, as well as on Ops Clinic Duty this changes every Monday.
- The clinic duty person can be pinged, and is the first point of contact in IRC for operations issues.
- Please note that our operations team works in multiple time zones, and the clinic assignee for any given week will likely be working within their own local time zone.
Access Requests
- Requesting shell access - fully documented instructions on how to gain shell access; required for any user requesting access.
Blocked By Ops
- If your particular project or task is currently blocked by operations, please detail such on the task (or sub-task) and flag it with the project tag Blocked-on-Operations.
Domain Requests
- This project #domains is for all domain registration requests, nameserver updates, and anything involving a domain registrar.
- Example: Volunteer transferring domain to WMF control.
- Example: Incoming domains needing implementation/support on cluster.
- This link will create a task in the domain & operations projects.
- It is advised that you leave your priority as 'Needs Triage' and not assign it to a specific person. This will result in it showing in the top of the operations triage lists.
- If you are requesting that Wikimedia register a domain that is currently unregistered, you will want to select option Security: Other confidential issue.
- This allows you, plus the wmf-nda to view the task, but not the entire internet; requesting we register an unregistered domain in an open task is a nice way to let squatters know what to register.
Hardware Requests
- TL;DR Click Hardware requests to file a task for requesting hardware. But please read the following.
- Requesting a server for your service should only occur after the following:
- You can click the link above to pre-populate a hardware request ticket with the basic fields for entry. Please also include the operations and hardware-requests projects. These include:
- Labs Project Tested, Site/Location, Number of systems, Service, Networking Requirements like access to specific networks, Processor Requirements, Memory, Disks:, NIC(s), Partitioning Scheme, and any other relevant notes/info.
- Note that Operations might suggest using a VM instead if it deems it applicable.
- Note that you don't place server requests on our procurement project.
- A single #hardware-request can generate multiple #procurement sub-tasks, as each sub-task is pricing from a specific vendor.
Virtual Machine Requests (Production)
- TL;DR. Click VM Requests and fill in the form. But please do read the following.
- THIS IS NOT LABS.
- This is for requesting a virtual machine in the production cluster. (This is usually as an alternative to a bare metal server.)
- VMs are great for hardware sharing, increasing hardware usage. If your service does not have specific hardware requirements, a VM is an ideal candidate for it. But if it has critical performance requirements, it might very well not be.
- Requesting a server for your service should only occur after the following:
- You can click the link above to pre-populate a vm-request ticket with the basic fields for entry. These include:
- Labs Project Tested, Site/Location, Service, Networking Requirements, Processor Requirements, Memory, Disks, and any other relevant notes/info.
- Do note that Operations might suggest using bare metal hardware instead if it deems it necessary
- Networking wise, multiple NICs for throughput increase is not viable in a VM
- Disk performance is limited by the underlying technology and resource sharing.
Other Purchases: SSL Certificates, Support Contracts
- All other requests for operations purchasing of support contracts, ssl certificates, and other related items should be placed in the Procurement project
Mail Aliases
- Please note that operations only handles group mailing aliases that are routed before our google routing of mailing aliases. All individual user mail aliases under the wikimedia.org domain are handed by the Corporate OIT team here. Please note that if you are not staff, and require a mail alias, you should request it via your working group/team leads/technical mentor/staff.
- Otherwise this is just a typical operations request.
- If you keep the default priority to 'Needs Triage' and it is in the #operations project, our Ops Clinic Duty assignee for the week will triage your request.
- This link will create a task in the operations project.
Mailing Lists
Creation Requests
- The operations team doesn't create all mailing lists. Instead, you should file a general request until the Wikimedia-Mailing-lists project in Phabricator; please leave the priority as 'Needs Triage' for our our Ops Clinic Duty assignee to better notice it.
- This link will create a task in the Wikimedia-Mailing-lists project.
Administration
- General list administration is handled by an individual lists administrators; administrators can be viewed on the lists information page.
- Operations involvement is typically only required when a list administrator is not listed on the list information page, or if the administrator has become unavailable for the role.
- We will NOT simply change list owners; all attempts to handle the request via the usual means/admins must be exhausted. We will attempt to also contact the list administrator before we change anything.
- If you still want operations assistance, please file a task with both the #operations & #Wikimedia-Mailing-lists projects.
- This link will create a task in both operations & Wikimedia-Mailing-lists projects.
Patch Review
- Any patches that require an operations team member review should have a Phabricator task and have both the operations and Patch-For-Review project tags assigned to it.
- Please do not assign a specific team member for review unless they are the subject matter expert (though CCing them if you are uncertain is valid); otherwise our Ops Clinic Duty assignee will attempt to triage to the appropriate parties.
- This link will create a task in the Patch-For-Review project.
- PuppetSWAT takes place twice weekly. Any simple patches can be included during its SWAT window. Please see page for further details.
Network Configuration
- Network requests (router configuration, switch port descriptions, vlan assignments, etc) should have a Phabricator task and have both the operations and network project tags assigned to it.
- Please do not assign a specific team member for review unless they are the subject matter expert (though CCing them if you are uncertain is valid); otherwise our Ops Clinic Duty assignee will attempt to triage to the appropriate parties.
- This link will create a task with the operations and network projects associated with it.
- Subnets/VLANS are listed on the switches (not public accessible) and in our operations/dns git repo (public accessible).
Schema Change
- Schema changes on production databases have to be approved and applied by DBAs. Instructions on how to request its application are on the Schema change page.
- Please do not assign a specific team member for review (though CCing them if you are uncertain is valid); otherwise our Ops Clinic Duty assignee will attempt to triage to the appropriate parties.
- This link will create a task in the #Blocked-on-schema-change and #DBA projects
- Only use #Blocked-on-schema-change when the change is final, now while it is in progress/hasn't been reviewed
- Normal schema changes can take up to 2 weeks to take effect. Those involving key tables such as revision, page or image may take more.
