During current QA meeting, Amaya did a nice picture of us all.
apache, ldap and using different attributes as user names
I'm currently considering how to allow users to log on with different attributes as user names,
e.g. with their "real" user name or their mail adress. Unfortunatly as described on
http://httpd.apache.org/docs/2.2/mod/mod_authnz_ldap.html#authldapurl,
though RFC 2255 allows a comma-separated list, only the first attribute is used.
Now, of course an idea would be to specify all different attributes as a new "loginas"-type one. Another solution would be to use ldap overlay modules, and just convert them "on the fly". Better ideas would be welcome.
Update: Thanks to Faidon Liambotis (again!) one can probably use mod_authn_alias to combine authentication with user name, mail adress etc.
aba@ries:~$ grep final update_out/update.OUTPUT_py | cut -f 2 -d ' '| tr , '\n'|wc -l 901
This means 901 packages went from unstable to testing with the most recent britney run (though counting each binNMUed source package as ~12 packages here). Only about 200 packages which are otherwise ready for migration tried to go to testing, the vast majority reached testing.
Major reason for this move is that now libselinux was ready, and we were able to migrate libselinux, pango, glib2.0 and ocaml which dependend on all of them (the last transition being responsible for about 700 packages).
Good news for armel as well:
start: 61+442: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-11:p-0:s-0:s-12:a-442 end: 60+367: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-10:p-0:s-0:s-12:a-367So some progress as well - it's a steady move towards a reasonable uninstallability count (the last number being armel).
semantics in ldap - anyone out there?
Ldap seems like a good database to store accounts in:
Lots of tools can use it, it's easily integrated into pam, and lots more.
But one sometimes just hits borders in the data model of openldap:
There are no semantics available.
So, let me give you an example. Take e.g. the dnsZoneEntry attribute in the debian.org-ldap. Actually, one wants to allow anyone to write into that attribute in his own entry. But - and that's a big but - one wants to have two additional checks before writing: One is that one cannot claim any dns entry already used by someone else. And the other is that the format needs to comply to specific standards.
Now the question is, is there some way to use ldap with more semantics? Or does one need to write ones own backend to do that (and in all the non-plain-db*-backends, it is specified that one needs to write his own authorization code as well). Or some other recommended way to do that? (Or is the only existing incarnation of that ActiveDirectory?)
Update: Thanks to Faidon Liambotis I know now about the overlays in openldap. The standard overlays unique and constraint will probably solve the case above - of course, I have a few more complex cases, but that are at least good starters.
trying: mysql-dfsg-5.0 accepted: mysql-dfsg-5.0 ori: 71+890: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-890 pre: 71+887: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-887 now: 71+773: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-773
In other words, the transition of mysql-dfsg-5.0 to testing has reduced the uninstallability on armel from about 900 packages to less than 800. Still some way to go, but good progress.
Any good web user selfmanagemenet out there?
I'm looking for some time for a good user self management.
"Good" means: User can add their own accounts, gets approved (or not),
can recover the password, ...
Of course, there are plenty implementations of that available in the market -
most wikis have that, as well as lots of other applications.
But all of them are embedded-only, i.e. one has to use that web application
and gets "vendor-locked". Isn't there anything open source and standalone
available, following the basic unix principles of "do one thing and do it well"?
Top testing migration blockers
During the recent weeks, we started to hunt and hint large
batches of packages together to testing (and this blog is just
to show a bit of what is part of the tasks of release team members).
This started with a large jasper hint, continued with a very large
poppler / texlive / jack / abiword-hint (which moved about 75% of the
packages that were ready for testing migration in at the same time),
then moved a more recent linux-2.6 version in, and now finally ghc6 /
haskell.
However, now we reached a point where a few more release teamisch hints don't help anymore: The next large blocker is gcj-4.1 / gccdefaults / ecj. This is now blocked by different bugs on different architecture:
Update: Aurelien Jarno informed me that gcj-4.1 had to be bootstrapped on all architectures, but it hasn't happened yet on arm due to bugs in gcj.
test svn-buildpackage
A new version of svn-buildpackage has appeared in unstable now. If you use this
package, please test it, and if something bad happens, please file appropriate
bugs and alert the release team. Thanks.
Neglected RC bugs
I dived through the list of open RC issues, and found some
without any (real) activity for at least 20 days. Please remember: If we want to release Etch, we need to get rid of any such
bug prior to release.
Who is lying?
One library is lying on ia64 - it tells libtool to have a link to /usr/lib/libasound.la, but there is no dependency to that on the package.
Please compare
http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=ia64&stamp=1166917613&file=log with
http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=s390&stamp=1166227920&file=log.
Now, one needs to locate the lying package and fix it.
How to read text in somebody's text which is not there, and not meant to be there - or: dunc-tank didn't slow down Etch
That sometimes journalists have obvious reading problems,
well, isn't a too new thing. One has to deal with that, e.g. by appending more text to the cited blog entry.
If someone inside the Debian community fails for the same is way more disappointing for me. I have never stated that "[I] couldn't compensate this" - that is just totally wrong. I think that dunc-tank helped us with release of Etch, but the help could have been greater if some people wouldn't behave as childish as they do. And I also try to look both ways, and not be singled-minded about it.
There is another thing I learned as well during the last 12 hours - people don't appreciate if one tries to summarise openly what happened, and look both ways. Is it so wrong to look in both directions, and not only say "that and that were the improvements"? I thought at least within Debian we don't want to fall the usual management mistake of only speaking about how great everything is, but be honest to ourselfs. People starting to rip only half of the words out, citing them out of the context, and inventing new quotes don't help with that.
attr and libpng
Monday morning started with seeing lots of brand new release bugs about attr being broken so bad
that "ls -la" failed, and also buildds were affected negatively.
As a consequence, I few-hour-NMUed this package back to a sane (i.e. working) state.
As any library maintainer, please remember:
Whenever you drop a symbol - whichever this symbol is this means:
You need to bump SONAME. Or just use library versioning from the beginning.
Failing to do so can lead break any program linking to your code, which is especially bad if it is ls and cp.
Rest of the day was mostly spent on further libpng-analysis. There is now a nice table on http://ries.debian.org/~aba/png.abis which symbols were exported in which version. We are progressing into resolving the libpng problems now.
midterm report
This is the first of the two "larger" reports during the release management funding experiment.
I suppose I don't need to be more specific about what I did in detail - my blog postings in http://blogs.turmzimmer.net/en/debian/etch-rm/ cover that already. The thing that ate most of my time are the release critical bugs. While it is possible for me on normal days to fix one bug every few days, it was now possible to fix a couple of bugs per day. This sounds rather good, but at the same time took away most of my time for other things I should have also worked on - that even involved blogging about my work. (Also in looking back that was still a good decision to work so much an release critical bugs - it would have been just more convenient if that wasn't necessary so much.)
Looking back, I started with improving tools. I think that this was a good decision, good tools make work easier - and also, everyone can use the better tools, so they multiply their benefit. And e.g. the structural changes I did to bts.turmzimmer.net to allow "show also related bugs" were so deep that I probably wouldn't have been able to do them, unless I reserve a couple of hours in a row. In other words, being able to explicitly set some time aside for Debian which is not eaten up by the daily routine tasks (which involves reading quite much mail, BTW), is a good thing.
So, looking at the status changes during the time I spent full-time on release issues I think it worked well. Of course, not everything is perfect, but there is a clear improvement. On the other hand, there was a large disadvantage of the whole experiment: Some people who used to do good work reduced their involvement drastically. There was nothing I could do about, and that happened way before I started full-time on release, but on the global picture that still counts. So, as a first summary, I am happy with my own involvement, but that doesn't necessarily apply to the full experiment.
Update: There are media rumours floating around that "[Etch has] been delayed because some developers have deliberately slowed down their work". This doesn't reflect what I said.
I just noted that the dunc-tank experiment has positive and negative effects, and we shouldn't watch only one side - whichever that side is. The reasons for the release being delayed from the original planned date has other reasons, please read the mails on debian-devel-announce for details (also all linked on http://release.debian.org/). And, I'm quite happy with the involvement of most Developers in the release. (And this paragraph isn't part of what this blog posting should be about really, but as it is cited, it is still here ...)
Release update and more
(This blog entry is a bit delayed because I have been on the QA meeting.)
On Monday and Tuesday last week we did one important step:
We sent out the release update that etch is fully frozen now.
Preparing that, and then handling the incoming freeze requests took some of the time.
Of course, some of the regular RC bug squashing happened as well - but that is rather a routine task now.
QA Meeting-picture
During current QA meeting, Amaya did a nice picture of us all.
Reading is difficult - or why the current number of RC bugs is ok for the freeze
People who have following our release update mails have seen that we are tracking different numbers this time.
There is the total number of RC bugs relevant for etch, which is currently 122.
And there is the number of RC bugs relevant for both etch and sid (i.e. "not handled yet"-bugs), which is currently
78 - and that number was basically the only number we could track for release of sarge.
(Then there are also the numbers of bugs only in etch, and the number of aged bugs relevant for etch and sid - please
see http://bts.turmzimmer.net/graph-small.png for the full stats.)
So, if you look back we have written "RC bugs below 80" as precondition in our timeline. We copied that from the sarge release, were we had "~85 RC bugs" at time of freeze. During detailed planing, it became obvious that if we want to basically "go by the same numbers", that is (and can only be) the number of RC bugs applying to both etch and sid - as that was the only stats we really had during sarge release planing (please remember, we didn't had version tracking at that time, at the moment where a bug fix was uploaded to sid, the bug disappeared at all, unless someone reopened the bug report and tagged it sarge).
So, we agreed that we can and should freeze Etch as soon as the combined number is below 80, and the number of fixes waiting for transition is not out of control (and a few other issues are resolved, please see the previous mail to debian-devel-announce for that).
Please allow me also a personal remark: It is really sad when somone tries to hurt active Developers working on the release, while not contributing anything except spite. I don't think that is adequate behaviour.
Preparing freeze
Blogging about recent Friday is a bit late now, but there you are:
On Friday, I was busy preparing the full freeze for Etch - this included
e.g. polishing the announcement mail as well as letting linux 2.6.18 finally
into Etch (and breaking some packages we cannot remove now), and of course
more RC bug squashing.
Thankfully, most of that worked out well, so we were able to freeze Etch today.
More bugs need to go away
On Monday, I sent out the next release update (which was under preparation for a few days).
Basically, Debian is doing quite well - we just have a few items left open before we can have the full
freeze.
Of course, I worked also on one of the preconditions, namely on RC bugs - the usual trying to reproduce, uploading fixes, encouraging other people to upload fixes, easying testing transition, ...
Bug needs help
There is a long-standing RC bug
libglide2: Reproducible segmentation fault with Voodoo2
where it is easy to reproduce this bug if you have the hardware (warning: it will crash your computer),
but - well, it just needs to be done.
What maintaining means
During my cruise through release critical bugs, one sometimes wonder
how often "team" should be translated with "toll, ein anderer macht's" (great, someone else will do it).
Looking at bugs like e.g. #399589 - asterisk-oh323: Asterisk OH-323 crashes latest asterisk package, it doesn't seem to demanding to expect also in a team-maintained environment that any of the maintainers recheck (or at least send mail) to the person reporting the bug within a week. However, the bug was reported on November 20th (that's a Monday), and nothing happend up to now from their side - means: Nothing within 11 days. Unfortunatly, there are more so buggy packages from the VoIP-Team - most can just be moved out from Etch, but it is always unsatisfactory to remove packages.
Letting non-free packages catch up and more NMUs
On Wednesday, I send out the announcement about how (and which) non-free packages can be autobuild.
Of course, that was not the only action - some more time was invested in fixing RC-bugs, and,
unfortunatly, in finding an RC-bug within apt-get source.
I also transfered the "which programs are uninstallable in etch"-program to the new ftp-master -
and the good news is we only have left two programs uninstallable on arm (ignoring arch=all except on i386).
More RC bug fixing and PHP got into testing
One of the good news is that PHP 5.2 finally got into testing.
A bit of handholding PHP was part of my Tuesdays tasks
(together with a removal of php-imagick, which I also NMUed afterwards,
and which I need to handhold to go back into testing now).
Tuesday allowed me to also upload a few more RC bug fixes. Besides of that, lots of the usual gruntwork and small items were done - nothing special to talk about, just answering couple of mails, reviewing packages, and all the rest of daily routine job (which becomes quite much shortly prior to release).
Please upload RC bug fixes
In case you are maintainer and have any RC bug sitting with your package, and you have
already commited a resolution to your source repository, please upload your package in a timely
manner. It is really annoying if such bugs sit there, and are not resolved (and have their negative
impact about our users!), and at the same time, it wouldn't be too difficult for the maintainers
to upload the fix.
Bug fixing
I have been away for 5 days without any net access
(and had a few hours of mail reading on Sunday evening).
On Monday morning, the trend on the RC-Graph didn't looked too good
,
so I invested resolving RC bugs. A couple of NMUs later the graph now looks better -
but of course, there is much more that needs to be done.
Beside fixing packages, I did the "usual" small jobs, including reviewing cyrus-sasl2, answering lots of mails, and so on.
Small items
Tuesday was mainly busy with lots of small items - nothing really to speak about.
Answering mails, looking into small testing transition issues,
cleanup of stuff that has been done before, ...
RC bug p0rn, bugsquashing
Monday morning started with setting up two more graphical stats about the number of Release Critical bugs.
They contain how many RC bugs affect etch currently, how many affect only etch (i.e. usually untransitioned fixes),
how many affect both suites, and how many affect both suites after 20 days.
Both graphs are available for the last 3 weeks and 3 months, respectively at
http://bts.turmzimmer.net/graph-small.png and
http://bts.turmzimmer.net/graph-large.png.
As you can see with the large graph, we did some important progress. As the small graphs show, we should try to get better again. :) So, let's work together on that, and let all three curves go down again.
After that was done, some normal RC bug squashing started - nothing special about, just "another few bugs gone".
Bugs to sell
There are a couple of bugs around where it would be a shame to remove the package
(or where it is next to impossible), which are hard to fix for outsiders but should be
way easier for people inside:
There are a couple of more such bugs available, please feel free to visit our RC bugs list yourself.
Test KPilot
KPilot has a long-standing RC bug report
about problems with syncing events and addressbook.
There are now some test packages available at
http://mirror.pusling.com/kdepim/kdepim_4:3.5.5.dfsg.1-2~sune1/
that should work, so please give it a try. (Writing about this because pusling doesn't blog.)
Release update sent, and upgrade testing
On Thursday, the release update we have been busy preparing was finally sent out by Steve.
This has clarified some of the questions people have been asking for some time.
Also, more comments on the release notes have been merged. We are now at an state where I'm happy to let users work with the release notes - though one large section still needs work: What is the appropriate order for upgrades of kernel and packages.
After that was done, I started myself with some upgrade tests to see how the packages really behave. More on that on one of the next days ...
More Documentation
Another day spent mostly on documentation.
It seems incredible how much time it takes to get things right,
but still - it is an very important task, as our users rely on good documentation.
The release notes are now mostly ready, except for some more information on the kernel/udev/...
(and thanks for Dann for so much input on the kernel situation).
Thursday will be spent on merging some comments on the release notes which I got in the late evening,
and trying out some upgrade scenarios.
Documentation and other long standing tasks
After a day off yesterday I spent some more day on release today.
Some was to work on our proposed mail to debian-devel-announce (hopefully, we can finally send it out today).
More time was spent on updating the release notes, and work on the submitted requests what should be included.
Now the only requests still open are connected with the kernel/udev (and I asked the kernel list for some more input),
security situation with php and mozilla, related to sarge (which probably translates into a "wontfix"),
and a few minor issues.
That's good news because we want to have release notes available now as we have an Release Candidate of the installer published.
And as always, some time is spent on the thousands minor little things that are so time-expansive. But still need to be done.
Experimental and autobuilding
Experimental is autobuild, but on an best-effort basis
(which is a different thing than what happens with unstable).
In other words, it can happen that a package is not build on all architectures,
and nobody does anything to fix that.
If that happens to one of your packages, please contact us (Martin Zobel or me),
and we'll try to fix that.
I just have rescheduled glibc in experimental on two architectures.
More apache chasing
Today, I continued chasing easy apache2 bugs. The following bugs seem to be easy to fix (and
should be fixed in my opinion prior to Etch).
Some more bugs were obviously outdated, so I closed them. I also started collecting information for the release notes about apache2.
The first apache day
On Friday, I spent most of my time on apache2.
I originally didn't intend to spend so much time there, but as apache2 is an important package,
and we still have lots of bug reports flowing in, I decided that spending a bit more time
than the minimum effort is useful. I commited an NMU to fix a first row of 8 bug reports
(which boiled down to 3 changes), but I expect to work more on apache during the weekend and
on Monday.
One day off
On Thursday, I did lots of "small" items that needed be done done for quite some time:
I checked the new hosting location for volatile (thanks to the physics department of the
Ludwig-Maximilians-Universität München for sponsoring),
(hopefully) fixed the wanna-build setup for debian-edu (and are now hung at uploading),
and so on.
Also, I had a short look at the apache2-bugs count, and noticed this package needs serious
QA activities.
All "good" activities, but obviously not really "working on the release", so I mark Thursday as day off, and will add another day at the end of my schedule.
Spreading the news
Today started with drafting a release update (did anybody tell so far how
much time it costs to write such mails? - of course it is time well spent,
but still lots of time). Two review rounds, and a couple of smaller issues
made most of my morning.
In the afternoon, I worked on my mail backlog, and answered e.g. a longish mail from an journalist about release management in Debian. Not something that directly improves the release, but I believe it is also time well spent - if we get our point of view distributed better. Amazingly again how much time one can spend with writing stuff.
Second day: Release planing
Today started with finishing the list of open issues for Etch,
and trying to add time expectations to the list.
This sounds like an easy task at first glance,
but actually it involves asking many people,
and getting time expectations updated etc.
As result of that list, and some further discussions,
I started drafting the next release update mail.
One also is hinted to many issue which escaped attention so far, so it involved also a round of bug severity changing.
Also, I finished working on "how does apt behave with more than one signature" for now: There is http://testmirror.turmzimmer.net/debian which has an additional signature on the sarge Release-file (thanks to Netcologne for hosting) - please feel free to just use it for any sarge system, and report errors back. I'll revisit the experiences in two weeks time.
The rest of day involved smaller issues, e.g. fixing the sort-by-bugnumber-mode on bts.turmzimmer.net (which I broke with yesterdays changes for "related" bugs), and a couple of other things. I also started to update the etch-release-page, but that needs to be continued tomorrow it seems. After I saw some more people not too happy with apache2.2 configuration changes (or rather, some modules are no longer there automatically), I decided to dedicate this Thursday to Apache.
testbed for secure apt
I was allowed to add an additional vhost to debian.netcologne.de
to try out secure apt signature issues. There is now available
deb http://testmirror.turmzimmer.net/debian sarge main -
the only difference to the real host is an additional signature on
http://testmirror.turmzimmer.net/debian/dists/sarge/Release.gpg.
Please feel free to try out that host as mirror for your sarge-based systems, and report any breakages that happen. The second key I used is the volatile archive key (as that was easy available) - I'll switch one day to the real Stable Release Managers key.
First day of experiment
Today, I started with the well-known experiment by making sure I'm not dropped out from social security
for taking part there.
After that was started, I invested some time on a long-standing wishlist bug on bts.turmzimmer.net, so that one can see even the RC-bugs related to a (source) package which would be ignored otherwise if at least one bug would be shown. This required some more code cleanup first. Now everything should work.
I started the afternoon with creating a list of open tasks that are not marked as RC bugs, and dependencies between those. During that, I learned that the security teams thinks that we lost the security sparc autobuilder - not something that makes me happy, and something that needs to be fixed independent of Etch. Actually, it seems the autobuilder is just recovering, but I'll continue looking at it.
One item open on the list of open tasks is the final usage of apt-keys (which involves me in more than one way), so I started a testbed for having the Release-file signed by more than one key, and check whether apt is still happy. Adding another signature turned out to be quite easy with just adding it to the Release.gpg-file.
If there is more than one signature on the release-file, but not all keys are know to apt, apt warns, but doesn't stop:
W: There are no public key available for the following key IDs: 010908312D230C5F
If however there is a bad signature, apt fails, even if there is also a good signature. (And please note, if the key is unknown, the signature is not bad but unknown - this gives the interesting behaviour that removing a trusted key by apt-key my remove the need to explicitly confirm to go on with the upgrade.)
Current behaviour is ok, we just need to sign one file extra with each stable (point) release, and this can be done with: gpg --no-default-keyring --secret-keyring srm-secring.gpg -b --armor < etch/Release >> etch/Release.gpg (or rather, sign off-line, and append on ftp-master).
The behaviour with multiple keys might look a bit strange, but IMHO we can live with it.
I'm currently preparing to use real keys, and publish the result on some place where anybody could test it (for the latest sarge point release), and I hope we can put it on for the next point release on ftp-master.
Migrating from SuSE to Debian?
On the most recent trade show Systems in Munich, lots of people asked about migrating from SuSE to Debian,
and the differences. I'm quite sure there is somewhere a reference, but does someone know where?
GR: Handling source-less firmware in the Linux kernel
The plain proposal allows us to ship with most of the firmware drivers (as they just require the *license*
complies with the DFSG, which is e.g. ok with BSD-style licenses). Two drivers one might still include are
still doubtfull with this decision: Typhoon and tg3.
The Amendment has a similar effect - it will definitly allow us to ship with tg3, typhoon is unclear. It also supports a decision by the release team - but well, we can also live without supporting that decision (as long as nobody overwrites it).
Summary: Any of the proposals allows us releaseing Etch soon. There is a "better" proposal upcoming, but that is not a reason to refuse the current one.
