During current QA meeting, Amaya did a nice picture of us all.
Lenny: done
It is strange to watch for the first time a debian release to happen,
without being part of a though time schedule, and with the possibilities to
sleep uninterrupted etc. And it is a good feeling to see how well it went.
I want to congratulate the people who made it happen, especially the people who worked for it for many months or even years. To avoid the pain of not mentioning someone who did lot of work, I restrict myself to congratulating my successors in the release team - a better list of people is part of a mail Marc just sent out. Thanks, and well done!
That time again
It seems it is that time again.
People seem to believe that Debian is actually able to ship a new stable very soon, and
so they do actions that delay it.
Good news is that this means: We're really quite near the release. Only some RC bugs to crack (anyone can help here, seriously), and we should be done. Let's work together and make this happen again!
bts.turmzimmer.net with delayed queue again
Thanks to Thomas Viehmann for the patch on both ftp-master side (aka
http://ftp-master.debian.org/deferred/
and on "client side" (and for nagging me enough to activate it),
http://bts.turmzimmer.net/details.php
shows now again when bugs are fixed in delayed. Thanks!
stale vs incomplete: xen vs kvm
Currently, I'm together with Marc Brockschmidt evaluating which
virtualization to use on our new server. We want that our virtual systems
feel like real systems, and we want an open source solution. So, vserver
and the like is out of the game, as well as VMware.
The two remaining solutions we looked at are Xen and kvm.
Xen has of course the advantage of the matured. Also, we have experience with running xen servers - and with the issues that can happen, like the chances to disconnect dom0 from xend, and then reboot the server the hard way. However, the most serious disadvantage is that development has practically stalled with the 2.6.18-kernel. Of course, even of today one could install a new server based on Etch, but that doesn't really feel right. There is some development ongoing to run domUs with newer kernels (like in Lenny), but there isn't currently any new kernel available for dom0.
kvm is a more recent addition to the virtualization camp, and is basically "qemu on steroids". All looks rather promising, development happens with the recent kernels. However, kvm lacks a few features of e.g. Xen.
This includes the ability to reboot dom0 (and the hardware) and just let the domUs survive. Or to have a nice management script where one could just say "xm shutdown $domU", and have basically the power button be pressed on the virtual machine. Or to just attach and detach to the virtual console whenever one wants. Nothing of that is impossible with kvm, one could attach the command-terminal to some pipe, and the linux console to some other, and attach and distach via own scripts. But - all of that should be expected to be available from some solution that calls itself enterprise ready. (And - writing own scripts has always the possibility to make own mistakes.)
However, among all the worst possible issue is that kvm is underdocumented (or rather: There are lots of different places where some parts of the documentation is hidden - including the great remark in the man page "The other options are similar to those of qemu.").
So, what to do? Invest more time into a solution that seems like a dead end. Or put up with the incompletness of another solution?
ldap and webauth / kerberos
I'm currently looking for some software that user can login on the webserver,
but for the scripts, it doesn't look too different from basic auth.
One example of that kind is webauth, however that requires that all user accounts are in kerberos. I however want to continue to keep accounts in ldap, because that works well for most issues.
So, my next step is either to set up kerberos in a way that allows the accounts to be synced with ldap, or find another software with the same effect.
Why can't apache just bind against an ldap-dn?
The current apache ldap documentation explains that apache first looks up in the directory before performing an bind.
Sometimes it would be much easier if one can just tell apache "take this dn, add the supplied user name there, and there you go". Any ideas how to do that?
Binding with ldap to attributes
I wanted to be able to not only bind with the normal dn, but also to attributes.
This means I e.g. have an attribute mail, and want the people to be able to login with their
mailaddress as username.
Stephen Gran gave me some valuable hints to using the rwm-rewriting engine.
After some time, I ended up with this setup:
overlay rwm
rwm-rewriteEngine on
rwm-rewriteMap ldap attr2dn "ldap://127.0.0.1/ou=myorg?dn?sub"
rwm-rewriteContext bindDN
rwm-rewriteRule "^anyid=([^,]*@[^,]*)" "${attr2dn(mail=$1)}" ":"
rwm-rewriteRule "^anyid=([^,@]*)" "${attr2dn(uid=$1)}" ":"
rwm-rewriteRule "^(uid=[^,]*)" "${attr2dn($1)}" ":"
rwm-rewriteRule "^(mail=[^,]*)" "${attr2dn($1)}" ":"
The only thing that doesn't work is to make rwm using ldap version 3 to log into itself, so I had to allow read-only access to the relevant attributes from peername.ip=127.0.0.1 - but well, I can live with that.
Update: Added anyid for not thinking in client code, and made sure only the start of entries is used.
apache, ldap and using different attributes as user names
I'm currently considering how to allow users to log on with different attributes as user names,
e.g. with their "real" user name or their mail adress. Unfortunatly as described on
http://httpd.apache.org/docs/2.2/mod/mod_authnz_ldap.html#authldapurl,
though RFC 2255 allows a comma-separated list, only the first attribute is used.
Now, of course an idea would be to specify all different attributes as a new "loginas"-type one. Another solution would be to use ldap overlay modules, and just convert them "on the fly". Better ideas would be welcome.
Update: Thanks to Faidon Liambotis (again!) one can probably use mod_authn_alias to combine authentication with user name, mail adress etc.
aba@ries:~$ grep final update_out/update.OUTPUT_py | cut -f 2 -d ' '| tr , '\n'|wc -l 901
This means 901 packages went from unstable to testing with the most recent britney run (though counting each binNMUed source package as ~12 packages here). Only about 200 packages which are otherwise ready for migration tried to go to testing, the vast majority reached testing.
Major reason for this move is that now libselinux was ready, and we were able to migrate libselinux, pango, glib2.0 and ocaml which dependend on all of them (the last transition being responsible for about 700 packages).
Good news for armel as well:
start: 61+442: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-11:p-0:s-0:s-12:a-442 end: 60+367: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-10:p-0:s-0:s-12:a-367So some progress as well - it's a steady move towards a reasonable uninstallability count (the last number being armel).
semantics in ldap - anyone out there?
Ldap seems like a good database to store accounts in:
Lots of tools can use it, it's easily integrated into pam, and lots more.
But one sometimes just hits borders in the data model of openldap:
There are no semantics available.
So, let me give you an example. Take e.g. the dnsZoneEntry attribute in the debian.org-ldap. Actually, one wants to allow anyone to write into that attribute in his own entry. But - and that's a big but - one wants to have two additional checks before writing: One is that one cannot claim any dns entry already used by someone else. And the other is that the format needs to comply to specific standards.
Now the question is, is there some way to use ldap with more semantics? Or does one need to write ones own backend to do that (and in all the non-plain-db*-backends, it is specified that one needs to write his own authorization code as well). Or some other recommended way to do that? (Or is the only existing incarnation of that ActiveDirectory?)
Update: Thanks to Faidon Liambotis I know now about the overlays in openldap. The standard overlays unique and constraint will probably solve the case above - of course, I have a few more complex cases, but that are at least good starters.
trying: mysql-dfsg-5.0 accepted: mysql-dfsg-5.0 ori: 71+890: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-890 pre: 71+887: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-887 now: 71+773: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-773
In other words, the transition of mysql-dfsg-5.0 to testing has reduced the uninstallability on armel from about 900 packages to less than 800. Still some way to go, but good progress.
Any good web user selfmanagemenet out there?
I'm looking for some time for a good user self management.
"Good" means: User can add their own accounts, gets approved (or not),
can recover the password, ...
Of course, there are plenty implementations of that available in the market -
most wikis have that, as well as lots of other applications.
But all of them are embedded-only, i.e. one has to use that web application
and gets "vendor-locked". Isn't there anything open source and standalone
available, following the basic unix principles of "do one thing and do it well"?
Top testing migration blockers
During the recent weeks, we started to hunt and hint large
batches of packages together to testing (and this blog is just
to show a bit of what is part of the tasks of release team members).
This started with a large jasper hint, continued with a very large
poppler / texlive / jack / abiword-hint (which moved about 75% of the
packages that were ready for testing migration in at the same time),
then moved a more recent linux-2.6 version in, and now finally ghc6 /
haskell.
However, now we reached a point where a few more release teamisch hints don't help anymore: The next large blocker is gcj-4.1 / gccdefaults / ecj. This is now blocked by different bugs on different architecture:
Update: Aurelien Jarno informed me that gcj-4.1 had to be bootstrapped on all architectures, but it hasn't happened yet on arm due to bugs in gcj.
test svn-buildpackage
A new version of svn-buildpackage has appeared in unstable now. If you use this
package, please test it, and if something bad happens, please file appropriate
bugs and alert the release team. Thanks.
Neglected RC bugs
I dived through the list of open RC issues, and found some
without any (real) activity for at least 20 days. Please remember: If we want to release Etch, we need to get rid of any such
bug prior to release.
Who is lying?
One library is lying on ia64 - it tells libtool to have a link to /usr/lib/libasound.la, but there is no dependency to that on the package.
Please compare
http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=ia64&stamp=1166917613&file=log with
http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=s390&stamp=1166227920&file=log.
Now, one needs to locate the lying package and fix it.
How to read text in somebody's text which is not there, and not meant to be there - or: dunc-tank didn't slow down Etch
That sometimes journalists have obvious reading problems,
well, isn't a too new thing. One has to deal with that, e.g. by appending more text to the cited blog entry.
If someone inside the Debian community fails for the same is way more disappointing for me. I have never stated that "[I] couldn't compensate this" - that is just totally wrong. I think that dunc-tank helped us with release of Etch, but the help could have been greater if some people wouldn't behave as childish as they do. And I also try to look both ways, and not be singled-minded about it.
There is another thing I learned as well during the last 12 hours - people don't appreciate if one tries to summarise openly what happened, and look both ways. Is it so wrong to look in both directions, and not only say "that and that were the improvements"? I thought at least within Debian we don't want to fall the usual management mistake of only speaking about how great everything is, but be honest to ourselfs. People starting to rip only half of the words out, citing them out of the context, and inventing new quotes don't help with that.
attr and libpng
Monday morning started with seeing lots of brand new release bugs about attr being broken so bad
that "ls -la" failed, and also buildds were affected negatively.
As a consequence, I few-hour-NMUed this package back to a sane (i.e. working) state.
As any library maintainer, please remember:
Whenever you drop a symbol - whichever this symbol is this means:
You need to bump SONAME. Or just use library versioning from the beginning.
Failing to do so can lead break any program linking to your code, which is especially bad if it is ls and cp.
Rest of the day was mostly spent on further libpng-analysis. There is now a nice table on http://ries.debian.org/~aba/png.abis which symbols were exported in which version. We are progressing into resolving the libpng problems now.
midterm report
This is the first of the two "larger" reports during the release management funding experiment.
I suppose I don't need to be more specific about what I did in detail - my blog postings in http://blogs.turmzimmer.net/en/debian/etch-rm/ cover that already. The thing that ate most of my time are the release critical bugs. While it is possible for me on normal days to fix one bug every few days, it was now possible to fix a couple of bugs per day. This sounds rather good, but at the same time took away most of my time for other things I should have also worked on - that even involved blogging about my work. (Also in looking back that was still a good decision to work so much an release critical bugs - it would have been just more convenient if that wasn't necessary so much.)
Looking back, I started with improving tools. I think that this was a good decision, good tools make work easier - and also, everyone can use the better tools, so they multiply their benefit. And e.g. the structural changes I did to bts.turmzimmer.net to allow "show also related bugs" were so deep that I probably wouldn't have been able to do them, unless I reserve a couple of hours in a row. In other words, being able to explicitly set some time aside for Debian which is not eaten up by the daily routine tasks (which involves reading quite much mail, BTW), is a good thing.
So, looking at the status changes during the time I spent full-time on release issues I think it worked well. Of course, not everything is perfect, but there is a clear improvement. On the other hand, there was a large disadvantage of the whole experiment: Some people who used to do good work reduced their involvement drastically. There was nothing I could do about, and that happened way before I started full-time on release, but on the global picture that still counts. So, as a first summary, I am happy with my own involvement, but that doesn't necessarily apply to the full experiment.
Update: There are media rumours floating around that "[Etch has] been delayed because some developers have deliberately slowed down their work". This doesn't reflect what I said.
I just noted that the dunc-tank experiment has positive and negative effects, and we shouldn't watch only one side - whichever that side is. The reasons for the release being delayed from the original planned date has other reasons, please read the mails on debian-devel-announce for details (also all linked on http://release.debian.org/). And, I'm quite happy with the involvement of most Developers in the release. (And this paragraph isn't part of what this blog posting should be about really, but as it is cited, it is still here ...)
Release update and more
(This blog entry is a bit delayed because I have been on the QA meeting.)
On Monday and Tuesday last week we did one important step:
We sent out the release update that etch is fully frozen now.
Preparing that, and then handling the incoming freeze requests took some of the time.
Of course, some of the regular RC bug squashing happened as well - but that is rather a routine task now.
QA Meeting-picture
During current QA meeting, Amaya did a nice picture of us all.
Reading is difficult - or why the current number of RC bugs is ok for the freeze
People who have following our release update mails have seen that we are tracking different numbers this time.
There is the total number of RC bugs relevant for etch, which is currently 122.
And there is the number of RC bugs relevant for both etch and sid (i.e. "not handled yet"-bugs), which is currently
78 - and that number was basically the only number we could track for release of sarge.
(Then there are also the numbers of bugs only in etch, and the number of aged bugs relevant for etch and sid - please
see http://bts.turmzimmer.net/graph-small.png for the full stats.)
So, if you look back we have written "RC bugs below 80" as precondition in our timeline. We copied that from the sarge release, were we had "~85 RC bugs" at time of freeze. During detailed planing, it became obvious that if we want to basically "go by the same numbers", that is (and can only be) the number of RC bugs applying to both etch and sid - as that was the only stats we really had during sarge release planing (please remember, we didn't had version tracking at that time, at the moment where a bug fix was uploaded to sid, the bug disappeared at all, unless someone reopened the bug report and tagged it sarge).
So, we agreed that we can and should freeze Etch as soon as the combined number is below 80, and the number of fixes waiting for transition is not out of control (and a few other issues are resolved, please see the previous mail to debian-devel-announce for that).
Please allow me also a personal remark: It is really sad when somone tries to hurt active Developers working on the release, while not contributing anything except spite. I don't think that is adequate behaviour.
Preparing freeze
Blogging about recent Friday is a bit late now, but there you are:
On Friday, I was busy preparing the full freeze for Etch - this included
e.g. polishing the announcement mail as well as letting linux 2.6.18 finally
into Etch (and breaking some packages we cannot remove now), and of course
more RC bug squashing.
Thankfully, most of that worked out well, so we were able to freeze Etch today.
More bugs need to go away
On Monday, I sent out the next release update (which was under preparation for a few days).
Basically, Debian is doing quite well - we just have a few items left open before we can have the full
freeze.
Of course, I worked also on one of the preconditions, namely on RC bugs - the usual trying to reproduce, uploading fixes, encouraging other people to upload fixes, easying testing transition, ...
Bug needs help
There is a long-standing RC bug
libglide2: Reproducible segmentation fault with Voodoo2
where it is easy to reproduce this bug if you have the hardware (warning: it will crash your computer),
but - well, it just needs to be done.
What maintaining means
During my cruise through release critical bugs, one sometimes wonder
how often "team" should be translated with "toll, ein anderer macht's" (great, someone else will do it).
Looking at bugs like e.g. #399589 - asterisk-oh323: Asterisk OH-323 crashes latest asterisk package, it doesn't seem to demanding to expect also in a team-maintained environment that any of the maintainers recheck (or at least send mail) to the person reporting the bug within a week. However, the bug was reported on November 20th (that's a Monday), and nothing happend up to now from their side - means: Nothing within 11 days. Unfortunatly, there are more so buggy packages from the VoIP-Team - most can just be moved out from Etch, but it is always unsatisfactory to remove packages.
Letting non-free packages catch up and more NMUs
On Wednesday, I send out the announcement about how (and which) non-free packages can be autobuild.
Of course, that was not the only action - some more time was invested in fixing RC-bugs, and,
unfortunatly, in finding an RC-bug within apt-get source.
I also transfered the "which programs are uninstallable in etch"-program to the new ftp-master -
and the good news is we only have left two programs uninstallable on arm (ignoring arch=all except on i386).
More RC bug fixing and PHP got into testing
One of the good news is that PHP 5.2 finally got into testing.
A bit of handholding PHP was part of my Tuesdays tasks
(together with a removal of php-imagick, which I also NMUed afterwards,
and which I need to handhold to go back into testing now).
Tuesday allowed me to also upload a few more RC bug fixes. Besides of that, lots of the usual gruntwork and small items were done - nothing special to talk about, just answering couple of mails, reviewing packages, and all the rest of daily routine job (which becomes quite much shortly prior to release).
Please upload RC bug fixes
In case you are maintainer and have any RC bug sitting with your package, and you have
already commited a resolution to your source repository, please upload your package in a timely
manner. It is really annoying if such bugs sit there, and are not resolved (and have their negative
impact about our users!), and at the same time, it wouldn't be too difficult for the maintainers
to upload the fix.
Bug fixing
I have been away for 5 days without any net access
(and had a few hours of mail reading on Sunday evening).
On Monday morning, the trend on the RC-Graph didn't looked too good
,
so I invested resolving RC bugs. A couple of NMUs later the graph now looks better -
but of course, there is much more that needs to be done.
Beside fixing packages, I did the "usual" small jobs, including reviewing cyrus-sasl2, answering lots of mails, and so on.
Small items
Tuesday was mainly busy with lots of small items - nothing really to speak about.
Answering mails, looking into small testing transition issues,
cleanup of stuff that has been done before, ...
RC bug p0rn, bugsquashing
Monday morning started with setting up two more graphical stats about the number of Release Critical bugs.
They contain how many RC bugs affect etch currently, how many affect only etch (i.e. usually untransitioned fixes),
how many affect both suites, and how many affect both suites after 20 days.
Both graphs are available for the last 3 weeks and 3 months, respectively at
http://bts.turmzimmer.net/graph-small.png and
http://bts.turmzimmer.net/graph-large.png.
As you can see with the large graph, we did some important progress. As the small graphs show, we should try to get better again. :) So, let's work together on that, and let all three curves go down again.
After that was done, some normal RC bug squashing started - nothing special about, just "another few bugs gone".
Bugs to sell
There are a couple of bugs around where it would be a shame to remove the package
(or where it is next to impossible), which are hard to fix for outsiders but should be
way easier for people inside:
There are a couple of more such bugs available, please feel free to visit our RC bugs list yourself.
Test KPilot
KPilot has a long-standing RC bug report
about problems with syncing events and addressbook.
There are now some test packages available at
http://mirror.pusling.com/kdepim/kdepim_4:3.5.5.dfsg.1-2~sune1/
that should work, so please give it a try. (Writing about this because pusling doesn't blog.)
Release update sent, and upgrade testing
On Thursday, the release update we have been busy preparing was finally sent out by Steve.
This has clarified some of the questions people have been asking for some time.
Also, more comments on the release notes have been merged. We are now at an state where I'm happy to let users work with the release notes - though one large section still needs work: What is the appropriate order for upgrades of kernel and packages.
After that was done, I started myself with some upgrade tests to see how the packages really behave. More on that on one of the next days ...
More Documentation
Another day spent mostly on documentation.
It seems incredible how much time it takes to get things right,
but still - it is an very important task, as our users rely on good documentation.
The release notes are now mostly ready, except for some more information on the kernel/udev/...
(and thanks for Dann for so much input on the kernel situation).
Thursday will be spent on merging some comments on the release notes which I got in the late evening,
and trying out some upgrade scenarios.
Documentation and other long standing tasks
After a day off yesterday I spent some more day on release today.
Some was to work on our proposed mail to debian-devel-announce (hopefully, we can finally send it out today).
More time was spent on updating the release notes, and work on the submitted requests what should be included.
Now the only requests still open are connected with the kernel/udev (and I asked the kernel list for some more input),
security situation with php and mozilla, related to sarge (which probably translates into a "wontfix"),
and a few minor issues.
That's good news because we want to have release notes available now as we have an Release Candidate of the installer published.
And as always, some time is spent on the thousands minor little things that are so time-expansive. But still need to be done.
Experimental and autobuilding
Experimental is autobuild, but on an best-effort basis
(which is a different thing than what happens with unstable).
In other words, it can happen that a package is not build on all architectures,
and nobody does anything to fix that.
If that happens to one of your packages, please contact us (Martin Zobel or me),
and we'll try to fix that.
I just have rescheduled glibc in experimental on two architectures.
More apache chasing
Today, I continued chasing easy apache2 bugs. The following bugs seem to be easy to fix (and
should be fixed in my opinion prior to Etch).
Some more bugs were obviously outdated, so I closed them. I also started collecting information for the release notes about apache2.
The first apache day
On Friday, I spent most of my time on apache2.
I originally didn't intend to spend so much time there, but as apache2 is an important package,
and we still have lots of bug reports flowing in, I decided that spending a bit more time
than the minimum effort is useful. I commited an NMU to fix a first row of 8 bug reports
(which boiled down to 3 changes), but I expect to work more on apache during the weekend and
on Monday.
