peps/pep-0102.txt at main · python/peps

Cannot retrieve contributors at this time

512 lines (382 sloc) 19.6 KB

	PEP: 102
	Title: Doing Python Micro Releases
	Author: Anthony Baxter <anthony@interlink.com.au>,
	Barry Warsaw <barry@python.org>,
	Guido van Rossum <guido@python.org>
	Status: Superseded
	Type: Informational
	Content-Type: text/x-rst
	Created: 09-Jan-2002
	Post-History:
	Superseded-By: 101


	Replacement Note
	================

	Although the size of the to-do list in this PEP is much less scary
	than that in :pep:`101`, it turns out not to be enough justification
	for the duplication of information, and with it, the danger of one
	of the copies to become out of date. Therefore, this PEP is not
	maintained anymore, and micro releases are fully covered by :pep:`101`.


	Abstract
	========

	Making a Python release is an arduous process that takes a
	minimum of half a day's work even for an experienced releaser.
	Until recently, most -- if not all -- of that burden was borne by
	Guido himself. But several recent releases have been performed by
	other folks, so this PEP attempts to collect, in one place, all
	the steps needed to make a Python bugfix release.

	The major Python release process is covered in :pep:`101` - this PEP
	is just :pep:`101`, trimmed down to only include the bits that are
	relevant for micro releases, a.k.a. patch, or bug fix releases.

	It is organized as a recipe and you can actually print this out and
	check items off as you complete them.


	How to Make A Release
	=====================

	Here are the steps taken to make a Python release. Some steps are
	more fuzzy than others because there's little that can be
	automated (e.g. writing the NEWS entries). Where a step is
	usually performed by An Expert, the name of that expert is given.
	Otherwise, assume the step is done by the Release Manager (RM),
	the designated person performing the release. Almost every place
	the RM is mentioned below, this step can also be done by the BDFL
	of course!

	XXX: We should include a dependency graph to illustrate the steps
	that can be taken in parallel, or those that depend on other
	steps.

	We use the following conventions in the examples below. Where a
	release number is given, it is of the form X.Y.MaA, e.g. 2.1.2c1
	for Python 2.1.2 release candidate 1, where "a" == alpha, "b" ==
	beta, "c" == release candidate. Final releases are tagged with
	"releaseXYZ" in CVS. The micro releases are made from the
	maintenance branch of the major release, e.g. Python 2.1.2 is made
	from the release21-maint branch.

	1. Send an email to python-dev@python.org indicating the release is
	about to start.

	2. Put a freeze on check ins into the maintenance branch. At this
	point, nobody except the RM should make any commits to the branch
	(or his duly assigned agents, i.e. Guido the BDFL, Fred Drake for
	documentation, or Thomas Heller for Windows). If the RM screwed up
	and some desperate last minute change to the branch is
	necessary, it can mean extra work for Fred and Thomas. So try to
	avoid this!

	3. On the branch, change Include/patchlevel.h in two places, to
	reflect the new version number you've just created. You'll want
	to change the PY_VERSION macro, and one or several of the
	version subpart macros just above PY_VERSION, as appropriate.

	4. Change the "%define version" line of Misc/RPM/python-2.3.spec to the
	same string as ``PY_VERSION`` was changed to above. E.g::

	%define version 2.3.1

	You also probably want to reset the %define release line
	to '1pydotorg' if it's not already that.

	5. If you're changing the version number for Python (e.g. from
	Python 2.1.1 to Python 2.1.2), you also need to update the
	README file, which has a big banner at the top proclaiming its
	identity. Don't do this if you're just releasing a new alpha or
	beta release, but /do/ do this if you're release a new micro,
	minor or major release.

	6. The LICENSE file also needs to be changed, due to several
	references to the release number. As for the README file, changing
	these are necessary for a new micro, minor or major release.

	The LICENSE file contains a table that describes the legal
	heritage of Python; you should add an entry for the X.Y.Z
	release you are now making. You should update this table in the
	LICENSE file on the CVS trunk too.

	7. When the year changes, copyright legends need to be updated in
	many places, including the README and LICENSE files.

	8. For the Windows build, additional files have to be updated.

	PCbuild/BUILDno.txt contains the Windows build number, see the
	instructions in this file how to change it. Saving the project
	file PCbuild/pythoncore.dsp results in a change to
	PCbuild/pythoncore.dsp as well.

	PCbuild/python20.wse sets up the Windows installer version
	resource (displayed when you right-click on the installer .exe
	and select Properties), and also contains the Python version
	number.

	(Before version 2.3.2, it was required to manually edit
	PC/python_nt.rc, this step is now automated by the build
	process.)

	9. After starting the process, the most important thing to do next
	is to update the Misc/NEWS file. Thomas will need this in order to
	do the Windows release and he likes to stay up late. This step
	can be pretty tedious, so it's best to get to it immediately
	after making the branch, or even before you've made the branch.
	The sooner the better (but again, watch for new checkins up
	until the release is made!)

	Add high level items new to this release. E.g. if we're
	releasing 2.2a3, there must be a section at the top of the file
	explaining "What's new in Python 2.2a3". It will be followed by
	a section entitled "What's new in Python 2.2a2".

	Note that you /hope/ that as developers add new features to the
	trunk, they've updated the NEWS file accordingly. You can't be
	positive, so double check. If you're a Unix weenie, it helps to
	verify with Thomas about changes on Windows, and Jack Jansen
	about changes on the Mac.

	This command should help you (but substitute the correct -r tag!)::

	% cvs log -rr22a1: \| python Tools/scripts/logmerge.py > /tmp/news.txt

	IOW, you're printing out all the cvs log entries from the
	previous release until now. You can then troll through the
	news.txt file looking for interesting things to add to NEWS.

	10. Check your NEWS changes into the maintenance branch. It's easy
	to forget to update the release date in this file!

	11. Check in any changes to IDLE's NEWS.txt. Update the header in
	Lib/idlelib/NEWS.txt to reflect its release version and date.
	Update the IDLE version in Lib/idlelib/idlever.py to match.

	11. Once the release process has started, the documentation needs to
	be built and posted on python.org according to the instructions
	in :pep:`101`.

	Note that Fred is responsible both for merging doc changes from
	the trunk to the branch AND for merging any branch changes from
	the branch to the trunk during the cleaning up phase.
	Basically, if it's in Doc/ Fred will take care of it.

	12. Thomas compiles everything with MSVC 6.0 SP5, and moves the
	python23.chm file into the src/chm directory. The installer
	executable is then generated with Wise Installation System.

	The installer includes the MSVC 6.0 runtime in the files
	MSVCRT.DLL and MSVCIRT.DLL. It leads to disaster if these files
	are taken from the system directory of the machine where the
	installer is built, instead it must be absolutely made sure that
	these files come from the VCREDIST.EXE redistributable package
	contained in the MSVC SP5 CD. VCREDIST.EXE must be unpacked
	with winzip, and the Wise Installation System prompts for the
	directory.

	After building the installer, it should be opened with winzip,
	and the MS dlls extracted again and check for the same version
	number as those unpacked from VCREDIST.EXE.

	Thomas uploads this file to the starship. He then sends the RM
	a notice which includes the location and MD5 checksum of the
	Windows executable.

	Note that Thomas's creation of the Windows executable may generate
	a few more commits on the branch. Thomas will be responsible for
	merging Windows-specific changes from trunk to branch, and from
	branch to trunk.

	13. Sean performs his Red Hat magic, generating a set of RPMs. He
	uploads these files to python.org. He then sends the RM a notice
	which includes the location and MD5 checksum of the RPMs.

	14. It's Build Time!

	Now, you're ready to build the source tarball. First cd to your
	working directory for the branch. E.g.
	% cd .../python-22a3

	15. Do a "cvs update" in this directory. Do NOT include the -A flag!

	You should not see any "M" files, but you may see several "P"
	and/or "U" files. I.e. you better not have any uncommitted
	changes in your working directory, but you may pick up some of
	Fred's or Thomas's last minute changes.

	16. Now tag the branch using a symbolic name like "rXYMaZ",
	e.g. r212

	::

	% cvs tag r212

	Be sure to tag only the python/dist/src subdirectory of the
	Python CVS tree!

	17. Change to a neutral directory, i.e. one in which you can do a
	fresh, virgin, cvs export of the branch. You will be creating a
	new directory at this location, to be named "Python-X.Y.M". Do
	a CVS export of the tagged branch.

	::

	% cd ~
	% cvs -d cvs.sf.net:/cvsroot/python export -rr212 \
	-d Python-2.1.2 python/dist/src

	18. Generate the tarball. Note that we're not using the 'z' option
	on the tar command because 1) that's only supported by GNU tar
	as far as we know, and 2) we're going to max out the compression
	level, which isn't a supported option. We generate both tar.gz
	tar.bz2 formats, as the latter is about 1/6th smaller.

	::

	% tar -cf - Python-2.1.2 \| gzip -9 > Python-2.1.2.tgz
	% tar -cf - Python-2.1.2 \| bzip2 -9 > Python-2.1.2.tar.bz2

	19. Calculate the MD5 checksum of the tgz and tar.bz2 files you
	just created

	::

	% md5sum Python-2.1.2.tgz

	Note that if you don't have the md5sum program, there is a
	Python replacement in the Tools/scripts/md5sum.py file.

	20. Create GPG keys for each of the files.

	::

	% gpg -ba Python-2.1.2.tgz
	% gpg -ba Python-2.1.2.tar.bz2
	% gpg -ba Python-2.1.2.exe

	21. Now you want to perform the very important step of checking the
	tarball you just created, to make sure a completely clean,
	virgin build passes the regression test. Here are the best
	steps to take::

	% cd /tmp
	% tar zxvf ~/Python-2.1.2.tgz
	% cd Python-2.1.2
	% ls
	(Do things look reasonable?)
	% ./configure
	(Loads of configure output)
	% make test
	(Do all the expected tests pass?)

	If the tests pass, then you can feel good that the tarball is
	fine. If some of the tests fail, or anything else about the
	freshly unpacked directory looks weird, you better stop now and
	figure out what the problem is.

	22. You need to upload the tgz and the exe file to creosote.python.org.
	This step can take a long time depending on your network
	bandwidth. scp both files from your own machine to creosote.

	23. While you're waiting, you can start twiddling the web pages to
	include the announcement.

	1. In the top of the python.org web site CVS tree, create a
	subdirectory for the X.Y.Z release. You can actually copy an
	earlier patch release's subdirectory, but be sure to delete
	the X.Y.Z/CVS directory and "cvs add X.Y.Z", for example::

	% cd .../pydotorg
	% cp -r 2.2.2 2.2.3
	% rm -rf 2.2.3/CVS
	% cvs add 2.2.3
	% cd 2.2.3

	2. Edit the files for content: usually you can globally replace
	X.Ya(Z-1) with X.YaZ. However, you'll need to think about the
	"What's New?" section.

	3. Copy the Misc/NEWS file to NEWS.txt in the X.Y.Z directory for
	python.org; this contains the "full scoop" of changes to
	Python since the previous release for this version of Python.

	4. Copy the .asc GPG signatures you created earlier here as well.

	5. Also, update the MD5 checksums.

	6. Preview the web page by doing a "make" or "make install" (as
	long as you've created a new directory for this release!)

	7. Similarly, edit the ../index.ht file, i.e. the python.org home
	page. In the Big Blue Announcement Block, move the paragraph
	for the new version up to the top and boldify the phrase
	"Python X.YaZ is out". Edit for content, and preview locally,
	but do NOT do a "make install" yet!

	24. Now we're waiting for the scp to creosote to finish. Da de da,
	da de dum, hmm, hmm, dum de dum.

	25. Once that's done you need to go to creosote.python.org and move
	all the files in place over there. Our policy is that every
	Python version gets its own directory, but each directory may
	contain several releases. We keep all old releases, moving them
	into a "prev" subdirectory when we have a new release.

	So, there's a directory called "2.2" which contains
	Python-2.2a2.exe and Python-2.2a2.tgz, along with a "prev"
	subdirectory containing Python-2.2a1.exe and Python-2.2a1.tgz.

	So...

	1. On creosote, cd to ~ftp/pub/python/X.Y creating it if
	necessary.

	2. Move the previous release files to a directory called "prev"
	creating the directory if necessary (make sure the directory
	has g+ws bits on). If this is the first alpha release of a
	new Python version, skip this step.

	3. Move the .tgz file and the .exe file to this directory. Make
	sure they are world readable. They should also be group
	writable, and group-owned by webmaster.

	4. md5sum the files and make sure they got uploaded intact.


	26. the X.Y/bugs.ht file if necessary. It is best to get
	BDFL input for this step.

	27. Go up to the parent directory (i.e. the root of the web page
	hierarchy) and do a "make install" there. You're release is now
	live!

	28. Now it's time to write the announcement for the mailing lists.
	This is the fuzzy bit because not much can be automated. You
	can use one of Guido's earlier announcements as a template, but
	please edit it for content!

	Once the announcement is ready, send it to the following
	addresses::

	python-list@python.org
	python-announce@python.org
	python-dev@python.org

	29. Send a SourceForge News Item about the release. From the
	project's "menu bar", select the "News" link; once in News,
	select the "Submit" link. Type a suitable subject (e.g. "Python
	2.2c1 released" :-) in the Subject box, add some text to the
	Details box (at the very least including the release URL at
	www.python.org and the fact that you're happy with the release)
	and click the SUBMIT button.

	Feel free to remove any old news items.

	Now it's time to do some cleanup. These steps are very important!

	1. Edit the file Include/patchlevel.h so that the PY_VERSION
	string says something like "X.YaZ+". Note the trailing '+'
	indicating that the trunk is going to be moving forward with
	development. E.g. the line should look like::

	#define PY_VERSION "2.1.2+"

	Make sure that the other ``PY_`` version macros contain the
	correct values. Commit this change.

	2. For the extra paranoid, do a completely clean test of the
	release. This includes downloading the tarball from
	www.python.org.

	3. Make sure the md5 checksums match. Then unpack the tarball,
	and do a clean make test.

	::

	% make distclean
	% ./configure
	% make test

	To ensure that the regression test suite passes. If not, you
	screwed up somewhere!

	Step 5 ...

	Verify! This can be interleaved with Step 4. Pretend you're a
	user: download the files from python.org, and make Python from
	it. This step is too easy to overlook, and on several occasions
	we've had useless release files. Once a general server problem
	caused mysterious corruption of all files; once the source tarball
	got built incorrectly; more than once the file upload process on
	SF truncated files; and so on.


	What Next?
	==========

	Rejoice. Drink. Be Merry. Write a PEP like this one. Or be
	like unto Guido and take A Vacation.

	You've just made a Python release!

	Actually, there is one more step. You should turn over ownership
	of the branch to Jack Jansen. All this means is that now he will
	be responsible for making commits to the branch. He's going to
	use this to build the MacOS versions. He may send you information
	about the Mac release that should be merged into the informational
	pages on www.python.org. When he's done, he'll tag the branch
	something like "rX.YaZ-mac". He'll also be responsible for
	merging any Mac-related changes back into the trunk.


	Final Release Notes
	===================

	The Final release of any major release, e.g. Python 2.2 final, has
	special requirements, specifically because it will be one of the
	longest lived releases (i.e. betas don't last more than a couple
	of weeks, but final releases can last for years!).

	For this reason we want to have a higher coordination between the
	three major releases: Windows, Mac, and source. The Windows and
	source releases benefit from the close proximity of the respective
	release-bots. But the Mac-bot, Jack Jansen, is 6 hours away. So
	we add this extra step to the release process for a final
	release:

	1. Hold up the final release until Jack approves, or until we
	lose patience <wink>.

	The python.org site also needs some tweaking when a new bugfix release
	is issued.

	2. The documentation should be installed at doc/<version>/.

	3. Add a link from doc/<previous-minor-release>/index.ht to the
	documentation for the new version.

	4. All older doc/<old-release>/index.ht files should be updated to
	point to the documentation for the new version.

	5. /robots.txt should be modified to prevent the old version's
	documentation from being crawled by search engines.


	Windows Notes
	=============

	Windows has a GUI installer, various flavors of Windows have
	"special limitations", and the Windows installer also packs
	precompiled "foreign" binaries (Tcl/Tk, expat, etc). So Windows
	testing is tiresome but very necessary.

	Concurrent with uploading the installer, Thomas installs Python
	from it twice: once into the default directory suggested by the
	installer, and later into a directory with embedded spaces in its
	name. For each installation, he runs the full regression suite
	from a DOS box, and both with and without -0.

	He also tries every shortcut created under Start -> Menu -> the
	Python group. When trying IDLE this way, you need to verify that
	Help -> Python Documentation works. When trying pydoc this way
	(the "Module Docs" Start menu entry), make sure the "Start
	Browser" button works, and make sure you can search for a random
	module (Thomas uses "random" <wink>) and then that the "go to
	selected" button works.

	It's amazing how much can go wrong here -- and even more amazing
	how often last-second checkins break one of these things. If
	you're "the Windows geek", keep in mind that you're likely the
	only person routinely testing on Windows, and that Windows is
	simply a mess.

	Repeat all of the above on at least one flavor of Win9x, and one
	of NT/2000/XP. On NT/2000/XP, try both an Admin and a plain User
	(not Power User) account.

	WRT Step 5 above (verify the release media), since by the time
	release files are ready to download Thomas has generally run many
	Windows tests on the installer he uploaded, he usually doesn't do
	anything for Step 5 except a full byte-comparison ("fc /b" if
	using a Windows shell) of the downloaded file against the file he
	uploaded.


	Copyright
	=========

	This document has been placed in the public domain.