Add a wiki

This page will walk you through how to add a new wiki project. This includes new languages on sister projects, and wikis for committees, chapters etc.

Start

Tell in advance the Ops list, jcrespo (jynus) or Coren that this is happening so the storage layer can be prepared and checked (labs, backups, dumps). In particular, it needs to be made clear whether the wiki will be public - if so, ops will arrange for the wiki to be replicated to labs. If not, ops will need to add the wiki to $private_wikis in the puppet repository's manifests/realm.pp.

This can be fixed after the wiki creation, but it is more painful/involves custom queries.

DNS

First of all, the DNS entry for the new wiki has to be added. Make the following changes to your local checkout of operations/dns.git and submit to gerrit for review.
- If it is a language project, add the language code to /templates/helpers/langs.tmpl in operations/dns repo. If it is not a new language, then it has probably been done already as these are shared by all the sister projects.
- If it is a wiki in *.wikimedia.org domain (chapter wiki or special wiki), add it to /templates/wikimedia.org. Make sure to add a mobile entry if necessary.
Merge the change in gerrit and run authdns-update. Query the DNS servers to make sure it has been correctly deployed. See DNS#HOWTO for details.

Apache configuration

Secondly, add Apache configuration for the new wiki by submitting it to gerrit first. They are now located at /modules/mediawiki/files/apache/sites/ in operations/puppet.git.
Common configuration:
- For languages projects, this step is usually not needed as it has been configured already.
- For chapter wikis, add ServerAlias to wikimedia-chapter VirtualHost in /wikimedia.conf.
- For Wikimania wikis, add ServerAlias to /wikimania.conf.
After merging on Gerrit, do sync-apache and apache-graceful-all^{(needs verification?)}. See Apache for details.
If there are any other domains which should point to the same wiki, add it to /redirects.conf and /redirects/redirects.dat.

You will firstly have to login to our deployment host. As of sometime, this is the host tin, you will have to go via bast1001. Then gather all relevant information for the new project, including all configuration variables and entries. Each wiki will have different requirements (project name, namespaces, extensions to be enabled etc).

You now have to determine if this is a new language project (say something like Spanish Wikibooks) or a chapter wiki, or a special wiki (anything else, including private wikis). Then choose the correct part of this manual:

If it is a language project, follow steps in #Language Project)
Else go directly to #Finish

Language Project

Make sure the language has been approved by the Language committee at Requests for new languages on Meta-Wiki. Usually, the Maniphest task will contain a link to the approval page.
Make the following changes on your own checkout of operations/mediawiki-config.git:
- Add the language code (ISO 639 code: this will usually be provided in the task) to langlist. This will be used at Special:SiteMatrix and for interwiki linking etc.
Go to #Finish

Finish

IMPORTANT: For Private Wikis

Private wiki databases must not be replicated to the labsdb MySQL instances!
- Before creating a database for a new private wiki, make sure to add the db name to the puppet global array $private_wikis in manifests\realm.pp.
- Deploy this config change with puppet and manually restart the Prelabsdb-db (Sanitarium) MySQL instance that will house this wiki's db (most likely s3).
- If you need help with this, please ask a member of the Ops team for help. This is very important.

MediaWiki configuration

Make the following changes on your own checkout of operations/mediawiki-config.git and submit to Gerrit for review:
- If it is a new language, add the language code (ISO 639 code: usually provided in the task) to langlist. This will be used at Special:SiteMatrix and for interwiki linking etc.
- Update the configuration files (located at /wmf-config). Some may not be required but the following are the most common wiki-specific settings in InitialiseSettings.php:
  - Manual:$wgServer, Manual:$wgCanonicalServer, Manual:$wgLogo, Manual:$wgSitename, Manual:$wgExtraNamespaces, Manual:$wgLocaltimezone, groupOverrides.
  - Overrides for $wgServer and $wgCanonicalServer are usually not done for language projects as the default works for these.
  - Also set Manual:$wgCategoryCollation if it is needed for a language. If you are not sure, ask on the task.
  - If an extension needs to be enabled: usually done by setting a wmg variable to true and if custom configuration needs to be done for the extension, do them as well. Some extensions' configuration are located in a different file located at /wmf-config directory.
- For *wikimedia databases, add the subdomain to the list in MWMultiVersion::setSiteInfoForWiki
Add the wiki to the relevant dblists (under the dblists directory).

Database list	Purpose
s1.dblist s2.dblist s3.dblist s4.dblist s5.dblist s6.dblist s7.dblist	Database lists of wikis in each MySQL database cluster
all.dblist	All wikis should be listed in here
closed.dblist	Any closed (no write access, full read access) wikis
deleted.dblist	Wiki databases which MediaWiki is no longer configured to access
small.dblist medium.dblist large.dblist	Database lists of wikis arranged into their relevant size
flaggedrevs.dblist	All wikis running the FlaggedRevs extension
securepollglobal.dblist	$wgSecurePollCreateWikiGroups wikis: Board Election wikis
visualeditor-default.dblist	All wikis where VisualEditor is enabled by default
commonsuploads.dblist	All wikis which should have local uploading soft-disabled. Uploads go to Commons instead.
fishbowl.dblist	All fishbowl (restricted write access, full read access) wikis
private.dblist	All private (read and write restricted) wikis
special.dblist	All special wikis
wikidata.dblist	All wikis running the Wikidata repo
wikidataclient.dblist	All wikis running the Wikidata client
wikimania.dblist	All Wikimania wikis
wikimedia.dblist	All chapter wikis
wikibooks.dblist	All Wikibooks wikis
wikinews.dblist	All Wikinews wikis
wikipedia.dblist	All Wikipedia wikis
wikiquote.dblist	All Wikiquote wikis
wikisource.dblist	All Wikisource wikis
wikiversity.dblist	All Wikiversity wikis
wikivoyage.dblist	All Wikivoyage wikis
wiktionary.dblist	All Wiktionary wikis

Add an entry to wikiversions.json. See Deployments (one week) and mw:Deployment Train.

Now it is time to actually create the database (this script also performs other tasks, such as notifying the 'newprojects' mailing list). There are two different ways to do this, depending on if you did Part A, B, or C above:

mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain
Part A is for standard language additions to projects, like adding a Spanish Wikinews, which will be my example.

mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain

EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki es wikinews eswikinews es.wikinews.org to add a Spanish Wikinews.

Part B is for Chapters Wikis. My example will be adding a Finnish chapter wiki:

mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain

EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki fi wikimedia fiwikimedia fi.wikimedia.org

Part C is for non-standard special wikis such as committees, chapters, and the like. My example will be strategy.wikimedia.org.

mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain

EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki en wikimedia strategywiki strategy.wikimedia.org

(Note the 'aawiki' in these examples: aawiki is in s3, so this will create the database in s3, which is usually the right place for new small wikis.)

Merge the config change in Gerrit, and pull it onto tin
Check *.dblist files now contain the new wiki
Run sync-dir dblists to synchronize all database lists
Check wikiversions.json is sane
Run sync-wikiversions to synchronize the version number to use for this wiki
Run sync-file wmf-config/InitialiseSettings.php
Run sync-file w/static/images/project-logos/path-to-logo.png
Run sync-file langlist (this must be done before the interwiki cache below)
Unless it's a language project, update the meta:Interwiki map
updateinterwikicache

RESTBase

RESTBase is a service providing a RESTful API for the projects' wikis. To enable it to serve the new wiki as well, create a patch for ops/puppet adding its domain to RESTBase's configuration.

Parsoid

Parsoid has it's own copy of SiteMatrix, which needs updating using tools/fetch-sitematrix.js in mediawiki/services/parsoid.git.

Search

You shouldn't have to do much (maybe adjust shard sizes preemptively if it'll be big), the index should be automatically created by addWiki.php and all wikis now opt into Cirrus/Elastic by default. See Search/New#Adding_new_wikis for more information.

Swift

Public wikis:
- Create the container for thumbnails. mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite
Private wikis:
- Create the container for thumbnails. mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite --private

Wikidata

dblist

Add to dblists/wikidataclient.dblist and sync-dir dblists (might need to touch/sync InitialiseSettings/CommonSettings)

sites table

Add the new wiki to all site tables of current wikidataclient wikis and also wikidatawiki.
Populate the sites table on the new wiki, which can be done with the populateSitesTable script in Wikibase. It has to be run once for each site group (e.g. wikipedia, wikivoyage, ...), as well as for wikidata (wikidata.dblist, with --site-group wikipedia) and commons (--site-group wikipedia).

Both of these steps can be done with populateSitesTable:

foreachwikiindblist wikidataclient.dblist extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https

In addition to updating the sites table, the script clears the sites cache (memcached) which is needed for things to work correctly.

After you finished

If there's something to import (as is often the case in new Language wikis), someone will do so, following the process described at MetaWikipedia:incubator:Incubator:Importing from Incubator (logged at MetaWikipedia:incubator:Incubator:Site creation log).