,
so I invested resolving RC bugs. A couple of NMUs later the graph now looks better -
but of course, there is much more that needs to be done.
How to read text in somebody's text which is not there, and not meant to be there - or: dunc-tank didn't slow down Etch
That sometimes journalists have obvious reading problems,
well, isn't a too new thing. One has to deal with that, e.g. by appending more text to the cited blog entry.
If someone inside the Debian community fails for the same is way more disappointing for me. I have never stated that "[I] couldn't compensate this" - that is just totally wrong. I think that dunc-tank helped us with release of Etch, but the help could have been greater if some people wouldn't behave as childish as they do. And I also try to look both ways, and not be singled-minded about it.
There is another thing I learned as well during the last 12 hours - people don't appreciate if one tries to summarise openly what happened, and look both ways. Is it so wrong to look in both directions, and not only say "that and that were the improvements"? I thought at least within Debian we don't want to fall the usual management mistake of only speaking about how great everything is, but be honest to ourselfs. People starting to rip only half of the words out, citing them out of the context, and inventing new quotes don't help with that.
attr and libpng
Monday morning started with seeing lots of brand new release bugs about attr being broken so bad
that "ls -la" failed, and also buildds were affected negatively.
As a consequence, I few-hour-NMUed this package back to a sane (i.e. working) state.
As any library maintainer, please remember:
Whenever you drop a symbol - whichever this symbol is this means:
You need to bump SONAME. Or just use library versioning from the beginning.
Failing to do so can lead break any program linking to your code, which is especially bad if it is ls and cp.
Rest of the day was mostly spent on further libpng-analysis. There is now a nice table on http://ries.debian.org/~aba/png.abis which symbols were exported in which version. We are progressing into resolving the libpng problems now.
midterm report
This is the first of the two "larger" reports during the release management funding experiment.
I suppose I don't need to be more specific about what I did in detail - my blog postings in http://blogs.turmzimmer.net/en/debian/etch-rm/ cover that already. The thing that ate most of my time are the release critical bugs. While it is possible for me on normal days to fix one bug every few days, it was now possible to fix a couple of bugs per day. This sounds rather good, but at the same time took away most of my time for other things I should have also worked on - that even involved blogging about my work. (Also in looking back that was still a good decision to work so much an release critical bugs - it would have been just more convenient if that wasn't necessary so much.)
Looking back, I started with improving tools. I think that this was a good decision, good tools make work easier - and also, everyone can use the better tools, so they multiply their benefit. And e.g. the structural changes I did to bts.turmzimmer.net to allow "show also related bugs" were so deep that I probably wouldn't have been able to do them, unless I reserve a couple of hours in a row. In other words, being able to explicitly set some time aside for Debian which is not eaten up by the daily routine tasks (which involves reading quite much mail, BTW), is a good thing.
So, looking at the status changes during the time I spent full-time on release issues I think it worked well. Of course, not everything is perfect, but there is a clear improvement. On the other hand, there was a large disadvantage of the whole experiment: Some people who used to do good work reduced their involvement drastically. There was nothing I could do about, and that happened way before I started full-time on release, but on the global picture that still counts. So, as a first summary, I am happy with my own involvement, but that doesn't necessarily apply to the full experiment.
Update: There are media rumours floating around that "[Etch has] been delayed because some developers have deliberately slowed down their work". This doesn't reflect what I said.
I just noted that the dunc-tank experiment has positive and negative effects, and we shouldn't watch only one side - whichever that side is. The reasons for the release being delayed from the original planned date has other reasons, please read the mails on debian-devel-announce for details (also all linked on http://release.debian.org/). And, I'm quite happy with the involvement of most Developers in the release. (And this paragraph isn't part of what this blog posting should be about really, but as it is cited, it is still here ...)
Release update and more
(This blog entry is a bit delayed because I have been on the QA meeting.)
On Monday and Tuesday last week we did one important step:
We sent out the release update that etch is fully frozen now.
Preparing that, and then handling the incoming freeze requests took some of the time.
Of course, some of the regular RC bug squashing happened as well - but that is rather a routine task now.
Preparing freeze
Blogging about recent Friday is a bit late now, but there you are:
On Friday, I was busy preparing the full freeze for Etch - this included
e.g. polishing the announcement mail as well as letting linux 2.6.18 finally
into Etch (and breaking some packages we cannot remove now), and of course
more RC bug squashing.
Thankfully, most of that worked out well, so we were able to freeze Etch today.
More bugs need to go away
On Monday, I sent out the next release update (which was under preparation for a few days).
Basically, Debian is doing quite well - we just have a few items left open before we can have the full
freeze.
Of course, I worked also on one of the preconditions, namely on RC bugs - the usual trying to reproduce, uploading fixes, encouraging other people to upload fixes, easying testing transition, ...
Letting non-free packages catch up and more NMUs
On Wednesday, I send out the announcement about how (and which) non-free packages can be autobuild.
Of course, that was not the only action - some more time was invested in fixing RC-bugs, and,
unfortunatly, in finding an RC-bug within apt-get source.
I also transfered the "which programs are uninstallable in etch"-program to the new ftp-master -
and the good news is we only have left two programs uninstallable on arm (ignoring arch=all except on i386).
More RC bug fixing and PHP got into testing
One of the good news is that PHP 5.2 finally got into testing.
A bit of handholding PHP was part of my Tuesdays tasks
(together with a removal of php-imagick, which I also NMUed afterwards,
and which I need to handhold to go back into testing now).
Tuesday allowed me to also upload a few more RC bug fixes. Besides of that, lots of the usual gruntwork and small items were done - nothing special to talk about, just answering couple of mails, reviewing packages, and all the rest of daily routine job (which becomes quite much shortly prior to release).
Bug fixing
I have been away for 5 days without any net access
(and had a few hours of mail reading on Sunday evening).
On Monday morning, the trend on the RC-Graph didn't looked too good
,
so I invested resolving RC bugs. A couple of NMUs later the graph now looks better -
but of course, there is much more that needs to be done.
Beside fixing packages, I did the "usual" small jobs, including reviewing cyrus-sasl2, answering lots of mails, and so on.
Small items
Tuesday was mainly busy with lots of small items - nothing really to speak about.
Answering mails, looking into small testing transition issues,
cleanup of stuff that has been done before, ...
RC bug p0rn, bugsquashing
Monday morning started with setting up two more graphical stats about the number of Release Critical bugs.
They contain how many RC bugs affect etch currently, how many affect only etch (i.e. usually untransitioned fixes),
how many affect both suites, and how many affect both suites after 20 days.
Both graphs are available for the last 3 weeks and 3 months, respectively at
http://bts.turmzimmer.net/graph-small.png and
http://bts.turmzimmer.net/graph-large.png.
As you can see with the large graph, we did some important progress. As the small graphs show, we should try to get better again. :) So, let's work together on that, and let all three curves go down again.
After that was done, some normal RC bug squashing started - nothing special about, just "another few bugs gone".
Release update sent, and upgrade testing
On Thursday, the release update we have been busy preparing was finally sent out by Steve.
This has clarified some of the questions people have been asking for some time.
Also, more comments on the release notes have been merged. We are now at an state where I'm happy to let users work with the release notes - though one large section still needs work: What is the appropriate order for upgrades of kernel and packages.
After that was done, I started myself with some upgrade tests to see how the packages really behave. More on that on one of the next days ...
More Documentation
Another day spent mostly on documentation.
It seems incredible how much time it takes to get things right,
but still - it is an very important task, as our users rely on good documentation.
The release notes are now mostly ready, except for some more information on the kernel/udev/...
(and thanks for Dann for so much input on the kernel situation).
Thursday will be spent on merging some comments on the release notes which I got in the late evening,
and trying out some upgrade scenarios.
Documentation and other long standing tasks
After a day off yesterday I spent some more day on release today.
Some was to work on our proposed mail to debian-devel-announce (hopefully, we can finally send it out today).
More time was spent on updating the release notes, and work on the submitted requests what should be included.
Now the only requests still open are connected with the kernel/udev (and I asked the kernel list for some more input),
security situation with php and mozilla, related to sarge (which probably translates into a "wontfix"),
and a few minor issues.
That's good news because we want to have release notes available now as we have an Release Candidate of the installer published.
And as always, some time is spent on the thousands minor little things that are so time-expansive. But still need to be done.
The first apache day
On Friday, I spent most of my time on apache2.
I originally didn't intend to spend so much time there, but as apache2 is an important package,
and we still have lots of bug reports flowing in, I decided that spending a bit more time
than the minimum effort is useful. I commited an NMU to fix a first row of 8 bug reports
(which boiled down to 3 changes), but I expect to work more on apache during the weekend and
on Monday.
Spreading the news
Today started with drafting a release update (did anybody tell so far how
much time it costs to write such mails? - of course it is time well spent,
but still lots of time). Two review rounds, and a couple of smaller issues
made most of my morning.
In the afternoon, I worked on my mail backlog, and answered e.g. a longish mail from an journalist about release management in Debian. Not something that directly improves the release, but I believe it is also time well spent - if we get our point of view distributed better. Amazingly again how much time one can spend with writing stuff.
Second day: Release planing
Today started with finishing the list of open issues for Etch,
and trying to add time expectations to the list.
This sounds like an easy task at first glance,
but actually it involves asking many people,
and getting time expectations updated etc.
As result of that list, and some further discussions,
I started drafting the next release update mail.
One also is hinted to many issue which escaped attention so far, so it involved also a round of bug severity changing.
Also, I finished working on "how does apt behave with more than one signature" for now: There is http://testmirror.turmzimmer.net/debian which has an additional signature on the sarge Release-file (thanks to Netcologne for hosting) - please feel free to just use it for any sarge system, and report errors back. I'll revisit the experiences in two weeks time.
The rest of day involved smaller issues, e.g. fixing the sort-by-bugnumber-mode on bts.turmzimmer.net (which I broke with yesterdays changes for "related" bugs), and a couple of other things. I also started to update the etch-release-page, but that needs to be continued tomorrow it seems. After I saw some more people not too happy with apache2.2 configuration changes (or rather, some modules are no longer there automatically), I decided to dedicate this Thursday to Apache.
First day of experiment
Today, I started with the well-known experiment by making sure I'm not dropped out from social security
for taking part there.
After that was started, I invested some time on a long-standing wishlist bug on bts.turmzimmer.net, so that one can see even the RC-bugs related to a (source) package which would be ignored otherwise if at least one bug would be shown. This required some more code cleanup first. Now everything should work.
I started the afternoon with creating a list of open tasks that are not marked as RC bugs, and dependencies between those. During that, I learned that the security teams thinks that we lost the security sparc autobuilder - not something that makes me happy, and something that needs to be fixed independent of Etch. Actually, it seems the autobuilder is just recovering, but I'll continue looking at it.
One item open on the list of open tasks is the final usage of apt-keys (which involves me in more than one way), so I started a testbed for having the Release-file signed by more than one key, and check whether apt is still happy. Adding another signature turned out to be quite easy with just adding it to the Release.gpg-file.
If there is more than one signature on the release-file, but not all keys are know to apt, apt warns, but doesn't stop:
W: There are no public key available for the following key IDs: 010908312D230C5F
If however there is a bad signature, apt fails, even if there is also a good signature. (And please note, if the key is unknown, the signature is not bad but unknown - this gives the interesting behaviour that removing a trusted key by apt-key my remove the need to explicitly confirm to go on with the upgrade.)
Current behaviour is ok, we just need to sign one file extra with each stable (point) release, and this can be done with: gpg --no-default-keyring --secret-keyring srm-secring.gpg -b --armor < etch/Release >> etch/Release.gpg (or rather, sign off-line, and append on ftp-master).
The behaviour with multiple keys might look a bit strange, but IMHO we can live with it.
I'm currently preparing to use real keys, and publish the result on some place where anybody could test it (for the latest sarge point release), and I hope we can put it on for the next point release on ftp-master.
