LVM bug affecting badling our buildd network
In wheezy (and all other current linux distros) there is a well-hidden bug
that might freeze LVM if deleting a snapshot fails. Doesn't sound too bad
one would think, but actually, creating and deleting snapshots is what
buildds do all the time, and so they run into that issue quite often. It's
worse on some architectures than on others, and so some packages are
currently not built as fast as we're used to.
Details about the bug are in #659762. Anyone fixing that would (not only) do our buildd network a big help (and you don't need to know anything about autobuilders for fixing that).
Cleaning up wanna-build
After adding the new triggers to auto-build wheezy-backports and jessie
yesterday, today I cleaned up the remaining bits in wanna-build from lenny:
wanna-build=> delete from packages where distribution ~ 'lenny' ;
DELETE 8250
wanna-build=> delete from distribution_architectures where distribution ~ 'lenny';
DELETE 63
wanna-build=> delete from locks where distribution ~ 'lenny';
DELETE 63
wanna-build=> delete from pkg_history where distribution ~ 'lenny';
DELETE 16504
wanna-build=> delete from distributions where distribution ~ 'lenny';
DELETE 6
wanna-build=> delete from architectures where architecture in ('alpha', 'arm', 'hppa');
DELETE 3
Traveling to DebConf 11 - back to Zagreb and München
Back from Banja Luka was the DebConf bus trip to Zagreb. Even though the train
connection should allow to catch the night train to München, the time in
Zagreb was too short to be sure (as with the other Bosnian track, two
trains per day - taking a earlier train was not possible if I wanted to
get at least a bit of sleep). So, I went on the DebConf-bus to Zagreb,
and arrived there on time. Border checks into Hrvatska were a bit more time
consuming then in the other direction - too much traffic but nothing else.
In Zagreb, I dropped off my luggage at the train station again, and then went by tram to the parts of the city I hadn't been before (and which were more normal parts, and not the tourist-areas). Incidently I also experienced tram track works, and so had to switch to the bus; however, information was so bad that not only I didn't notice it (which wasn't too bad and unexpected - I plan with enough time as tourist) but also locals were taken by surprise.
(Many parts of) Zagreb appears to have many too wide roads, with pedestrians pushed away. Not too uncommon for some cities here as well. But sad to see if public space is not created with humans in mind. This used to be modern in the 60ies, whereas it should have now advanced to center around humans again. If one compares the situation how one feels while standing in a too far road (where the wind blows easily cold) with a decent road in any city center, one could see the difference. But as said, many german cities make the same mistake.
After arriving back at the train station, this was in time for taking the night train. Obviously there were many DDs on board. Passport checks in Hravatska went smooth but with many boarder guards. The train had an extra stop not in the official timetable so that they didn't leave their territory armed. On arriving in Slovenija the train had to stop some time. Border checks were quite strict (as this was entering the european union) and time consuming, e.g. partly cover sheets of the train were removed.
After entering Slovenija the reminder of the trip was uneventful (or at least: ignored while sleeping), so the trip ended in München as planned.
Summarising it was an interesting and nice trip. I had no problem using public transport in spite of the warnings before. Of course, as always while traveling in foreign countries one should expect the country to be more different to home than just temperature and language - i.e. one should expect a bit of the unexpected, and be able to cope with. But that's true for any place one is going to. And these areas are worth another visit another time.
I also learned more about "local" history (whereas local covers everything within 1000 kilometers around München). However, the really bad thing is when comparing Hrvatska and Bosna i Hercegovina to see how much more Bosna i Hercegovina could have advanced within the last 16 years, but didn't due to incompetent management. Thinking that the same ways of obstructing decision-making happens in this country (and the european union as a whole) as well (but isn't as visible - we hadn't had a war, but also not much advances in our infrastructure) makes me more sad. Having said this, I still enjoyed the tour quite much - it was a good decision to have done it.
Traveling to DebConf 11 - in Sarajevo
Sarajevo gave a rainy look the next morning. That wasn't too bad, as I was
able to walk around a bit without it being too crowded. Incidently I also
saw the corner where Franz Ferdinand was shot in 1914, which started the
first world war - Europe has many links beyond todays state borders, in
good and bad times. It's not like transeuropean politics (or communications)
are something too new.
Bazar in Sarajevo
Franz Ferdinand was shot here in 1914
Trolley buses
River, tram and city
nice building hidden by cars - should our cities be dominated that way?
more of the old city
After having spent the morning exploring the old part of Sarajevo and along the river, I used the time after hotel checkout to use the tram link and take a few more pictures. I arrived at the train station as planned, but the train I wanted to take didn't go that day (in spite of checking it within the hotel in the morning). So, I ended with another bus trip (this time unplanned) to Banja Luca. Again, bus trips are worse than train trips - one cannot move in the vehicle, and there are delays for every stop. The events in Banja Luca are discussed elsewhere, so nothing about that here (except that DebConf was great - I enjoyed it very much).
Traveling to DebConf 11 - Mostar to Sarajevo
During daytime, I finally bought a train ticket from Mostar to
Sarajevo. The ticket for the 100 kilometers trip was 10 KM, which are 5
Euro.
Mostar train station
Right on time the train entered the station. However, we had a 30 minutes delay there without any obvious cause or communications. This train was (as well as the previous in Hrvatska) between full and overcrowded, in spite of the bad connections. I however learned soon why these carriages shouldn't go to other parts of europe: The window in our compartment was replaced, so it couldn't be opened any more (and we had no means of fresh air). That wasn't as bad as it sounded, as the train doors weren't locked, so someone opened the door while the train was moving and "locked" it with paper so we could get fresh air (and this configuration stayed all the way to Sarajevo at least, so it was that way for about 4 hours).
After leaving Mostar, there were soon signs between the train and the river that the area was mined, so one shouldn't go there. The train drives through beautiful landscape. After it got dark, the electric light within the train was not working, so we had (at best) our mobil phones to provide us with light (strange modern times).
Mine warning between the train tracks and the River Neretva
In spite (or: because?) of these technical issues, I meet a few local people from Sarajevo and from all parts of Europe. So the train trip proved to be nice and entertaining, and I learned a bit about government issues there.
Along the tracks, the signals were non-functional. The trains were directed only by flags (and hand-lights) from personal. On entering any station the train had to slow down till I got shown the relevant flag or hand-light that allowed it to leave the station again. Another heritage from the last war.
With due delay, the train arrived in Sarajevo. Due to missing information at the tram stop, I didn't know that the trams to the train station didn't run that late (but I had the information that there would be trams if the train was on time). As it was late, I shortcut that by taking a taxi to the hotel (if I were to arrive in Sarajevo again, I would walk one tram station to the main line - but well, travelling is also about experiences).
Unplanned that evening was also the Sarajevo film festival, with one stage opposite of the hotel window. As this was almost closing down when I arrived in Sarajevo, nothing to worry but to enjoy.
Traveling to DebConf 11 - in Mostar
The next day was reserved for seeing Mostar.
I could leave my luggage at the hotel, so I first went to the train station.
The train station was rather large and impressive, in spite of only two
trains per day and direction. However, the train station was closed at that time, so no
ticket purchase yet.
Into the historic city of Mostar: Visiting the Ottoman House was interesting, as well as climbing on the tower of one mosque. In the city, many houses still had effects of shelters and gun shots from the war. (At least) One of the trees in the pedestrian area had many shots as well.
river Neretva within the city of Mostar
Seeing Stari Most, the old (or rather: rebuilt) bridge was nice, as well as the masses of people looking there. However, the area with many tourists was quite small. Whereas Stari Most was shoot down during the war, there was a small sister bridge which survived the war damaged but still existed. However, that bridge collapsed during one high water afterwards but had also been rebuilt.
Stari Most with tourists, sister bridge of Stari Most without tourists and houses along the river
Below Stari Most, there was an area that looked like a nice picknick-place to look at the bridge and old city, but it was obviously unmaintained since some time. Same at other places: Really great, but many of them unmaintained (as in "too less money", not as in "vandalized"). Soon in the afternoon, most tourists disappeared, so I had the chance to look at the city with only few tourists arround before I had to leave for the train.
Walking around Mostar, I meet a few people who were in Germany before, for work, studying or school. The city was quite fascinating - in some way reminded me of 20 years ago. I saw a few more nice vegetables, like kiwis.
Mostar wasn't as tourismn-oriented as I would have expected from the monuments available to see there. All people were very nice and friendly, and in the cafes around there was free wifi. A nice place, but also depressing when thinking how much better it could be.
Traveling to DebConf 11 - Split to Mostar
Second half of the second day saw me entering the bus to Mostar in Split.
Starting in Split, the bus was rather empty. As the bus went along the
coast in Hrvatska to Ploce, it filled with more and more people (and was in
many traffic jams). In Ploce,
it met the train station which is in an industrial area. On this trip, one
could see the disadvantages of busses: Not only one couldn't get up and
meet new people, but also the bus needs to leave the main road for every
stop, so a stop has a drastic effect on the speed of the connection.
(However, as this bus was after the last train of the day, I had little
choice; going from Zagreb to Mostar via Split in one day with minimum bus is
basically impossible unless the train to Split is strictly on time.)
From Ploce, the bus went parallel to the train tracks. The boarder checks were quite easy to pass - except for one passenger who needed the passport stamped, but the stamp had to be fetched from the office first. That took a few minutes, but nothing too bad either. On entering Bosna i Hercegovina, I had the feeling of a rather dark country, at least compared to Hrvatska (but might be influenced as well that it was just getting dark, and I was in Zagreb on the previous evening). The journey for this day ended in Mostar, where I found a warm welcome in the hotel (which was some 10 minutes away from the bus and train station). An additional difficulty was that the hotel booking system I used to book hotels uses Google maps, but different to openstreetmaps, Google maps doesn't really know much about Bosna i Hercegovina.
Traveling to DebConf 11 - Zagreb to Split
On the second day, I entered the train to Split early in the morning. The
train was a bit crowded, but having a seat reserved helped.
Unfortunatly I knew that kind of train already from home (here known
as 612), and changing colours didn't make it a better train.
Train from Zagreb to Split in Gračac
After leaving Zagreb, the train went through nice countryside. After some time hills started - with given-up houses, walls, areas. As I learned in the meantime, this track used to be not the main line to Split but a backland-line. However, since the last war (the one from 1991-1995), the shorter and faster direct line is still cut. In Knin one could still see the reminders of the direct line which was electrified.
Landscape from within the train
The railway was a mixture of historic operations (with many people, changing switches by going there and by hand and using flags instead of signals) and directly into the 21nd century with electronic signal boxes just being built.
train junction in Perković
In spite of "the train should be an alternative to cars and busses"-speeches, as there was only one (and overcrowded) train suiteable for going to Split, priorities seem to be elsewhere (not too uncommon for politics on public transport in Europe). Also, there was obviously much more money put into the streets than into the trains.
The journey to Split should have taken about 6 hours, but the train in the opposite direction was a bit late, so I arrived late about 30 minutes. Directly on the platform there was a crowd of people trying to let appartments to tourists etc. Ignoring that, I first got rid of my luggage in the train station, bought the bus ticket for the next leg, and as having planned for late trains still had ample of time to see Split. This was my first "southern" target on this journey, so I could say Hi to fig trees (and their smell as they were blooming at that time).
Split itself has an historic center (being dated back to the Romans). One could also go up a hill and have an overview over both Split and the Adria.
All in all I found Hrvatska being compareable to other nice parts of Europe: Many tourists, overcrowded, and prices similar to Germany. Usage of public transport by tourists seems strange, and mostly only done by interrailers.
Traveling to DebConf 11 - München to Zagreb
These articles cover my way to DebConf 11 through Hrvatska and Bosna i Hercegovina. After telling my initial plans to go by train (mostly) and bus (where it couldn't be avoided) on IRC, I was warned that public transport is quite bad and unreliable. Also, as I live in München, of course this part of Europe was always known as "near" and "could be visited any time" (which means "one never gets to it"; and I can still remember the time when it was Yugoslavia - there are and always were many people living here from that part of Europe; in fact, it's nearer than some parts of Germany). I plan to publish more parts of my way within the next days.
Already at home I also learned that most trains in Hrvatska (and all in Bosna i Hercegovina) are not part of the usual train information system, so it was a bit more advanced to find out the appropriate connections. And in both countries there are only a few trains running, so one shouldn't miss a train ("few" means e.g. two per day and direction - but one at most unappropriate times, so really only one suiteable).
The first day saw me boarding the Suburban train (S-Bahn) at my usual station, changing platforms and trains at München Ost station and then I was sitting in a train to Zagreb. Nothing strange there, except that the coach I had an seat reserved in was missing. I learned later on that it happens more often that the coach from Serbia is not there because it's technically too unfit to send it to Germany -- and usually, there will be more coaches added in Ljubljana. Border checks were uneventfull but at the slovenijan border my passport was stamped (not sure why, didn't happen on the way back).
Train in Österreich
After arriving in Zagreb and dropping off my luggage in the hotel, I first got some Kunas and then bought my ticket to Split for the next day (as a direct ticket from München to Split was not available). After having done that, I took a brief tour through Zagreb until it became dark, with visiting some of the tourist places.
Zagreb
Schäfflertanz
Every 7 years with the Schäfflertanz the end of the Black Death in 1517 is celebrated.
After lots of people died, everyone was too scared to go out in the streets again,
even after the Black Death was gone.
The Schäffler (cooper = people who traditionally build barrels) started to
cheer the people in Munich up with their dance, and made them go on the
streets and start their normal lifes again.
This is one of the few local traditions that even survived the modern times.
The barrel is signed with "Schäfflertanz 2012" and "Zur Erinnerung an das Pestjahr 1517" = "To remind of the Black Death-year 1517", and should remind us that even in the darkest times, there is still hope and life goes on.
professional opensource email backup?
I'm looking for an opensource mail backup which doesn't treat mails as "unix files", but instead knows a bit more. So that upon recovery mode I could recover mails by title or sender, and not only just "all files as of that date". Did I miss that, or can this only be bought by money currently? (Speaking of mails, I really mean "Maildir".)
Building a mipsel porter box and buildd
As grub is now working on the loongson 2e boxes as well (thanks to phcoder
and Colin Watson), it is time to move the buildds running at my home to
a data center (previously we couldn't remote manage the kernels / boot
flags without VGA console, which means no data center usage).
Also one of the boxes could be converted to a porter machine, so that we
could get an mipsel porter box again.
The machines are delivered only with 256M of ram, which is a bit too less
for usage. Thanks to Zugschlus (Marc Haber) I got 1g ram for both machines
going to Vienna (one buildd, one porter box), and thanks to Robert Grimm
a 160g harddisk to replace the build-in 40g in the porter box (the buildd
can cope with 40g fine). The additional ram modules and hard disk are
visible on the following picture.
eysler.debian.org is now DSAed, online and building packages (including autosigning). I will shutdown monteverdi.ayous.org which is still at my place in the next days, reinstall the system and send it to Darmstadt as another buildd for data center hosting there. After that happens all mipsel buildds are DSAed as it should be, and are running in a data center and not via some DSL line.
(In case you're looking for hardware at your place, there are a couple of loongson 2f-systems available to buy. 2f is the successor cpu of 2e. Some 2f systems get delivered with Debian installed on it, see e.g. http://www.aliexpress.com/product-gs/290632002-Yeeloong-8101-Laptop-10-inch-Debian-EU-Adaptor-version--wholesalers.html. However, for buildd usage, the 2e are fine as well, and we got them sponsored some time ago.)
RFH: buildd and multiple builds in parallel
On some machines, we have enough cpus to run more than one build in parallel.
However, as of now, this isn't supported by buildd.
What needs to be done is to change the following code to have up to n builds happening in parallel (including of getting the next package if there is a free slot).
while( 1 ) {
$self->check_restart();
$self->read_config();
my ( $dist_config, $pkg_ver) = get_next_REDO($self);
$self->do_build( $dist_config, $pkg_ver) if $pkg_ver;
next if $pkg_ver;
( $dist_config, $pkg_ver) = get_next_WANNABUILD($self);
$self->do_build( $dist_config, $pkg_ver) if $pkg_ver;
next if $pkg_ver;
# sleep a little bit if there was nothing to do this time
$self->log("Nothing to do -- sleeping " .
$self->get_conf('IDLE_SLEEP_TIME') . " seconds\n");
my $idle_start_time = time;
sleep( $self->get_conf('IDLE_SLEEP_TIME') );
my $idle_end_time = time;
$self->write_stats("idle-time", $idle_end_time - $idle_start_time);
}
Any takers? The full code is available from git://git.debian.org/buildd-tools/sbuild.git in the file lib/Buildd/Daemon.pm (and I'm happy to try out appropriate patches).
Update The code above lives in the git branch buildd, and not master. Also, I already have one proposal in my mail.
How to autorotate images?
I'm still receiving messages by fax. Sometimes pages get inserted wrong
(head-down). To make my life easier, I'd like to execute code which
automatically rotates the page if it was inserted with bottom first - any
idea if there is free software suiteable to recognize if it's wrong (I
won't mind if hand-written pages appear wrong - if it works in 90% of the
cases, I'm entirely happy).
RFH: Release Team
The next Debian release needs help. Releasing Debian is a community effort,
only we together can make it happen.
What needs to be done is mostly one of:
Whatever you do: Write about what you achived. Coordinate with others in time. Have fun!
Changes in wanna-build
During recent weeks, not only sbuild and buildd were changed, but also
wanna-build. Many changes were small and don't have direct impact, but will
ease our life in future. This includes a bunch of code cleanups. Most
changes were done by Kurt Roeckx and me, but as usual Marc Brockschmidt
and Philipp Kern were also involved.
This round of changes was started with redoing our priority calculation. Up to now, any package had a fixed place in the list, and our list was built from top to bottom as far as buildd power was available (putting aside manual intervention by setting build-priority by admins). That meant of course that some packages could be stalled if buildd time isn't enough anymore, like currently on mipsen. (The queue order was determined by the following sort options: build-priority, (>= standard?), already built in the past, priority, section, name, and the first difference decided list order.) Now, of course >= standard packages are still built first, but waiting days increase priority so that old extra packages could be built before young optional package (in other words, they shouldn't stall. The new formula is about: {required: 50, important: 40, standard: 30, optional: 5}[priority] + {libs: 4, devel: 2}[section] + {contrib: -20, non-free: -40}[component] + {out-of-date: 20}[notes] + max(6, waitingdays) * 2 + manual priorities, and packages are ordered by this number, then by waitingdays, then by name.)
While adding code to add bonus for long-waiting packages, we stumbled across the fact that there were non-C dates in the database stored, which in turn means that export of the database stopped to work. For fixing that we replaced the last change field in the database by an postgres now() on insert, and converted that field to an date field (instead of freetext). Which in turn broke mkstats and a few more things, which are fixed as of now.
While doing that, we also introduced the format option, which allows to do queries like:
wanna-build --format='%t %u: %p/%v%{+b}B%B' -A mipsel --list=building
which gives output like:
2010-03-03 15:24:38.642988 buildd_mipsel-mayer: cracklib2/2.8.16-1 2010-03-03 15:30:00.341313 buildd_mipsel-rem: liblouisxml/2.1.0-1+b1
Of course, there are even better possibilities what one could do with that. :) More changes are pending, like the injector for log files was changed so that we record building times in the database. This will allow us to include build time on at least a few buildds, so that large packages cannot so easily stall all buildds completely anymore. So, more to come ...
How to (not) protect privacy
Protecting our privacy is important. For example, I wouldn't like if my
mailing list subscriptions get known to anybody else (except the relevant
listmasters of course as part of their job), for the simple reason that
this is just my own decisions which lists I get mail from (and read, but
that's not necessarily the same). This is an classical example
of "personal information are handed out on a need-to-know-basis" (like
to listmasters if necessary), and is also in line with european laws (I
don't know the legal situation in other parts of the world enough).
Now, publishing mailing list subscriptions hased sounds like an good idea to protect privacy. But if thinking twice, this just doesn't work. Most peoples subscription addresses could be known by other means or are even the addresses they use for sending mails (some systems even enforce this). So this is a huge privacy fail.
Getting back to debian, and speaking the obvious out: The new interface to udd is just broken and wrong. Please remove all my adresses from being displayed, either in direct or hashed form, even in restricted mode. Thanks.
Transit network during olympic games Vancouver?
As Munich is currently trying to get the olympic games in 2018, I looked at what Vancouver did.
I could see that there is a
new streetcar operating, however the tracks aren't brand new but used to be there while I was in Vancouver
(just with not so new platforms and rails).
To Whistler there are Busses operating. Inside the city there seem to be Olympic Lanes (which of course makes
public transport faster). I'm wondering if there are new built lines at all for the games?
Backup non-NIH?
I'm currently looking for some backup tool, but each tool lacks some features.
I'd like to
I took a deeper look at duplicity, but the encryption to gnupg means I need to copy files around. Also the webdav-uploader has some issues if backups get too large (i.e. uploads take too long).
Any other hints? Or is it worth to start to enhance duplicity?
Update: I'd like to generate the backup on the backuped system, but to manage them (i.e. purge old versions) on the backup system. And of course restore to any host within my control. Without exposing any affected gpg key, but just the one-time key(s) for the affected backup(s).
cups and samba
Currently samba cannot transition to testing, as cups
doesn't build on
mips* anymore. We first thought that the issue is that cups is using
PIE as build option, but even
after changing cups to not do that, and making binutils to give a specific
error message for using PIE instead of building corrupted binaries (thanks,
Matthias for the fast change), it still continues to FTBFS. No answer from
the porters yet though. And no idea what to do, at least none that I know of.
Feed of RC bug changes
In response to Andrews request for a feed of RC buggy packages
there is currently a 3 times per day changelist on http://bts.turmzimmer.net/.
Converting that list to an RSS feed shouldn't be an issue if it's helpful to people. (And if you want it, just sent me an example
how the two current top entries should look like as rss feed.)
more on the mysql-transition
One of the great things within Debian is how fast things happen if they are
important. The php5-FTBFS-bug
was fixed already yesterday. I'm now looking into more details of that
transition, as my usual strategy is first to look at packages not updated
at all architectures. Yesterday I took a deeper look into amd64-specific
packages not updated to the new library yet. A few packages are just not
uploaded yet (that's normal).
Three packages had RC bugs without patch for "doesn't build anymore" since
ages
(asterisk-addons as #534037
since June 21st,
mysql-gui-tools as #527652
since May 8th and
ulogd as #527534 since May
7th). The first two packages are as of this writing already pending
removal, the third might follow any minute.
This is one of the bad things of unhandled RC bugs: They make us way more
work, and they are really pending for removal from testing. Another few
packages got fresh RC bugs for not building. Also e.g. pike7.6
does need an upload, but with
obvious patch.
So the conclusion is: However (un)important an RC bug looks to you, it is
necessary that it is fixed as soon as possible. Please fix it.
Fix it now. There is no use in waiting, things will only get worse. And if
there are reasons for not uploading now, please document them, and make
sure there is at least a patch in your bug report.
Bugs in need of love
Currently a few larger transitions hang for different reasons: octave
together with hdf5 is blocked by bug
#542333 that the current
swig1.3-version makes gdal FTBFS, for this reason we cannot get the
necessary binNMUs for testing transition. The migration of the current
mysql5.1 is blocked by 5 RC bugs, where two are trivial (just replacement
of build dependencies), two are easily removable, but php5 now started to
FTBFS with the autoconf in testing and unstable, but still builds with the
version in stable (#542906).
And thanks to libgdal-ruby1.8 these two transitions want to glue together
to one. Fixing any of these two bugs would be very welcome.
improving kfreebsd* installability
trying: libxi/kfreebsd-amd64_tpu accepted: libxi/kfreebsd-amd64_tpu ori: 127+4202: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-2129:k-2073 pre: 127+4202: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-2129:k-2073 now: 127+3326: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-1253:k-2073 all: -sibyl-installer audiere/hppa linphone/hppa sdl-mixer1.2/hppa unicap/hppa libxi/kfreebsd-amd64_tpu trying: libxi/kfreebsd-i386_tpu accepted: libxi/kfreebsd-i386_tpu ori: 127+4202: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-2129:k-2073 pre: 127+3326: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-1253:k-2073 now: 127+2463: i-26:a-8:a-22:a-10:h-17:i-12:m-7:m-7:p-3:s-8:s-7:k-1253:k-1210 all: -sibyl-installer audiere/hppa linphone/hppa sdl-mixer1.2/hppa unicap/hppa libxi/kfreebsd-amd64_tpu libxi/kfreebsd-i386_tpuThis quote from current britneys output mean that thanks to the binNMU of libxi to testing-proposed-updates (wasn't in testing yet for these two architectures at all) the uninstallability count of the two bsd-variants dropped down again by a large value (kfreebsd-amd64 from 2129 to 1253, kfreebsd-i386 from 2073 to 1210). Still much to do, but also much progress (and yes, one can now build testing chroots on the newcomers). (And please thank the porters for that, they do the hard work. I just pull the strings a bit here and there to optimize testing migration.)
"IPv6 as release goal"
A few days ago, some people wondered why ipv6 is still on the list of our
release goals. The answer is easy: It still doesn't fully work.
Most of the issues are not debian specific. But did you every try to use nrpe (from nagios) to check an ipv6-only host? Or use the httplib python library to an ipv6-only host with an https-connect? Just to name two examples of the interessting journey with ipv6-only machines. We are just not there. "We" as the linux world at large, not only "we" as in Debian.
forcing eglibc and gcc, breaking *glx*ia32 and lib32ffi on amd64
finish: [eglibc,gcc-4.4,alsa-lib,bzip2,gcj-4.4,gnat-4.4,readline5,ia32-libs,
ecj,zlib,ncurses,libshout/alpha,openafs/alpha,acpitool/armel,aespipe/armel,
calcurse/armel,consolekit/armel,dwm/armel,freej/armel,gadmin-rsync/armel,
galleta/armel,geeqie/armel,gpicview/armel,halevt/armel,iec16022/armel,
jinja2/armel,keytouch-editor/armel,libjavascript-minifier-xs-perl/armel,
libsys-virt-perl/armel,libtest-leaktrace-perl/armel,libwant-perl/armel,
libwww-curl-perl/armel,logapp/armel,luckybackup/armel,matplotlib/armel,
missidentify/armel,moreutils/armel,myrescue/armel,pasco/armel,pipebench/armel,
pycairo/armel,pymssql/armel,python-enable/armel,pywebkitgtk/armel,
reglookup/armel,rifiuti/armel,rpy/armel,safecopy/armel,scponly/armel,
scrounge-ntfs/armel,shed/armel,sleuthkit/armel,ssdeep/armel,swapd/armel,
tct/armel,tmux/armel,trousers/armel,virt-manager/armel,xcftools/armel,
flasm/ia64,fontforge-extras/ia64,frama-c/ia64,freecell-solver/ia64,gcom/ia64,
geeqie/ia64,gpicview/ia64,gupnp-tools/ia64,heimdal/ia64,hex/ia64,
janest-core/ia64,jinja2/ia64,joystick/ia64,keytouch-editor/ia64,ldm/ia64,
libhtml-template-pro-perl/ia64,libjavascript-minifier-xs-perl/ia64,
librep/ia64,libshout/ia64,libsys-virt-perl/ia64,libtest-leaktrace-perl/ia64,
libtioga-ruby/ia64,libwant-perl/ia64,libwww-curl-perl/ia64,lua-bitop/ia64,
lua-lpeg/ia64,luckybackup/ia64,lxrandr/ia64,matplotlib/ia64,mklibs/ia64,
mlt/ia64,mpdscribble/ia64,mtasc/ia64,ncmpcpp/ia64,newsx/ia64,
ocaml-libvirt/ia64,open-cobol/ia64,openafs/ia64,oprofile/ia64,
pangomm/ia64,pari/ia64,pidgin-facebookchat/ia64,pipebench/ia64,
postgresql-8.4/ia64,prelude-lml/ia64,printfilters-ppd/ia64,ptlib/ia64,
pymssql/ia64,python-enable/ia64,qd/ia64,quodlibet/ia64,rasmol/ia64,
reglookup/ia64,reprepro/ia64,rifiuti/ia64,rpm2html/ia64,rpy/ia64,
safecopy/ia64,scponly/ia64,scrounge-ntfs/ia64,serf/ia64,shed/ia64,
sleuthkit/ia64,ssdeep/ia64,structure-synth/ia64,tct/ia64,
telepathy-gabble/ia64,tex-guy/ia64,timidity/ia64,tmux/ia64,transmission/ia64,
trousers/ia64,util-vserver/ia64,varconf/ia64,yasm/ia64,gmp,nss-mdns]
endloop: 30+15272: i-5:a-2:a-2:a-2:h-5:i-3:m-2:m-2:p-2:s-3:s-2:k-7623:k-7649
now: 34+5455: i-5:a-2:a-6:a-2:h-5:i-3:m-2:m-2:p-2:s-3:s-2:k-2725:k-2730
* amd64: fglrx-glx-ia32, lib32ffi-dev, lib32ffi5, nvidia-glx-ia32
* kfreebsd-amd64: gcj-4.4-jdk, gcj-4.4-jre, libgcj10-awt, libgcj10-dev
* kfreebsd-i386: gcj-4.4-jdk, gcj-4.4-jre, libgcj10-awt, libgcj10-dev
This means that we pushed eglibc and gcc-4.4 through to testing. This
reduces the uninstallability count on *bsd by more than half (rather 64%
gone). On the other hand, we broke 4 packages additionally on amd64, which
are for the i386-compatibility stuff on amd64. Two of them are non-free
and linked to the xorg-transition (and we would like to avoid waiting for
that transition before we can update eglibc in testing). The other two
packages are discussed in bug
#533009, and not uploaded yet.
After the package is fixed, it can move to testing anytime. As there
weren't any reverse dependencies broken, we decided that this decision is
the best for our users.
I realized also that it's quite tempting to just fix testing a bit here and there. However, I'm not intending to be back in the "need to fix testing every day"-camp. It's quite a bit of fun to do that after a long pause (and definitly very tempting), but don't expect me to do that every day again.
About being a press officer
Some of you know that besides Debian I'm also volunteering in the germans
national association of public transport users. A few know that I'm a press
officer for more than 15 years on the local level, and more than 10 years
on the national level. As that, I've had my fair deal (or even more) of
issues and experiences (and also seen how other organisations deal with
that). Some recent discussions have convinced me to write up my experiences,
and try to clarify some common misunderstandings - press communications is
very different from normal open source stuff.
One of the most important things to respect as an press officer is the proper division of tasks: As press officer I'm not making the decisions nor do I communicate them to the inside. I'm "only" communicating them to the outside, and trying to get the focus of the media set right. That doesn't mean I'm not discussing afairs with the responsible persons for the decisions, and giving advise (and sometimes I'm also voicing an opinion as delegate to the national council - but that's non-public then). But in the end, they need to decide which decision is the right one. Disagreeing with a decision only allows me two ways to handle them: Either ignore my disagreement and still distribute the decision (and that means also publically welcome). Or to step back.
To avoid misunderstandings, one usually considers at the beginning how critical and how much potential for trouble a position statement has. Depending on that one decides how many people need to review a position statement before handing it out - next to no position statement goes out unreviewed. It is always recommended to let any position statement be signed off by the people responsible for the decision. (But in constrast to signed-off patches, never tell in public who did review and sign-off a position statement - either the organisation has decided it according to their internal governance process, then the decision is proper and signed off by the organisation as whole. Or it isn't, then there is no position statement.)
Asking the press officers to make their own decisions and contradicting the decisions as taken by the governancy rules is counter-productive: This will at best only lead to confusion to the outside, and unnecessary conflicts on the inside. Usually it will get way worse though.
Looking at Debians press team, I'm quite happy to say that they work in a very good way. Please continue to keep up your good work.
Finishing off transitions
On Sunday, Phil (mostly) and I (a bit) were finishing off some transitions:
ocaml and suitesparse went to testing (over 1000 packages involved), as well as
imagemagick (which included an upload of meta-gnome2 to
testing-proposed-updates to change the gnome-package). Both can be qualified as
quite major transitions, and I didn't expect that we could finish two at almost
the same time.
Some transitions didn't work though: The transition of eglibc to testing failed as ia32-libs were changed quite a bit on amd64 (where is multiarch?).
All that together was more than six hours of running our testing migration scripts, analyzing issues, changeing hints, ...
Why I'm not commenting on the release plans ... and why the publicity members can only welcome them
I have been approached by some people why I don't comment on the decision
to commit to time-based releases.
Actually, that's quite simple, and there are basically two aspects of the same reason involved. First of all, I'm no longer one of the release managers. I stepped down after a having a very good time, and agreed to hand over the power to the current release managers. A transition of powers that mostly are social engineering powers can only work if one really stops using the powers. If I am only quiet in public when I disagree with the current release managers, it would be obvious when I agree and disagree. So I must be quiet in public about release decisions all the time, at least currently (this will hopefully change some time).
That doesn't mean I'm not voicing my opinions inside the release team - there I voice them openly, and will continue to do so. But of course I respect the authority of the release managers, granted to them by the constitution, and I will base my own actions on the decisions by the persons or bodies empowered through the constitution.
Now, some people told me I could just voice my opinion as normal developer. Though this seems be true, it isn't. Anything I say will always be read as the opinion of the former release manager. I might perhaps return to that state in a few more years time, but as of now, I just cannot.
Now, my position is quite easy. I could just shut up, and not write this blog, and ignore the few people who are directly asking me. For some people inside of Debian, this is not possible. For example, the press team has the task to transport the decisions and happenings inside of Debian to the outside. So they have, no matter what there own opinions are, to send out a press announcement which is positive of the decision. Their only option for not sending out that mail would be to step back. And frankly speaking, I'm really happy with what the publicity team has done. They are doing a very good job. Thank you for that.
Please remember: It's of course ok to not like the message. But please do not shoot the messenger. And remember, the publicity team (and for that decision also the DPL) are only messenger.
Lenny: done
It is strange to watch for the first time a debian release to happen,
without being part of a though time schedule, and with the possibilities to
sleep uninterrupted etc. And it is a good feeling to see how well it went.
I want to congratulate the people who made it happen, especially the people who worked for it for many months or even years. To avoid the pain of not mentioning someone who did lot of work, I restrict myself to congratulating my successors in the release team - a better list of people is part of a mail Marc just sent out. Thanks, and well done!
That time again
It seems it is that time again.
People seem to believe that Debian is actually able to ship a new stable very soon, and
so they do actions that delay it.
Good news is that this means: We're really quite near the release. Only some RC bugs to crack (anyone can help here, seriously), and we should be done. Let's work together and make this happen again!
bts.turmzimmer.net with delayed queue again
Thanks to Thomas Viehmann for the patch on both ftp-master side (aka
http://ftp-master.debian.org/deferred/
and on "client side" (and for nagging me enough to activate it),
http://bts.turmzimmer.net/details.php
shows now again when bugs are fixed in delayed. Thanks!
stale vs incomplete: xen vs kvm
Currently, I'm together with Marc Brockschmidt evaluating which
virtualization to use on our new server. We want that our virtual systems
feel like real systems, and we want an open source solution. So, vserver
and the like is out of the game, as well as VMware.
The two remaining solutions we looked at are Xen and kvm.
Xen has of course the advantage of the matured. Also, we have experience with running xen servers - and with the issues that can happen, like the chances to disconnect dom0 from xend, and then reboot the server the hard way. However, the most serious disadvantage is that development has practically stalled with the 2.6.18-kernel. Of course, even of today one could install a new server based on Etch, but that doesn't really feel right. There is some development ongoing to run domUs with newer kernels (like in Lenny), but there isn't currently any new kernel available for dom0.
kvm is a more recent addition to the virtualization camp, and is basically "qemu on steroids". All looks rather promising, development happens with the recent kernels. However, kvm lacks a few features of e.g. Xen.
This includes the ability to reboot dom0 (and the hardware) and just let the domUs survive. Or to have a nice management script where one could just say "xm shutdown $domU", and have basically the power button be pressed on the virtual machine. Or to just attach and detach to the virtual console whenever one wants. Nothing of that is impossible with kvm, one could attach the command-terminal to some pipe, and the linux console to some other, and attach and distach via own scripts. But - all of that should be expected to be available from some solution that calls itself enterprise ready. (And - writing own scripts has always the possibility to make own mistakes.)
However, among all the worst possible issue is that kvm is underdocumented (or rather: There are lots of different places where some parts of the documentation is hidden - including the great remark in the man page "The other options are similar to those of qemu.").
So, what to do? Invest more time into a solution that seems like a dead end. Or put up with the incompletness of another solution?
ldap and webauth / kerberos
I'm currently looking for some software that user can login on the webserver,
but for the scripts, it doesn't look too different from basic auth.
One example of that kind is webauth, however that requires that all user accounts are in kerberos. I however want to continue to keep accounts in ldap, because that works well for most issues.
So, my next step is either to set up kerberos in a way that allows the accounts to be synced with ldap, or find another software with the same effect.
Why can't apache just bind against an ldap-dn?
The current apache ldap documentation explains that apache first looks up in the directory before performing an bind.
Sometimes it would be much easier if one can just tell apache "take this dn, add the supplied user name there, and there you go". Any ideas how to do that?
Binding with ldap to attributes
I wanted to be able to not only bind with the normal dn, but also to attributes.
This means I e.g. have an attribute mail, and want the people to be able to login with their
mailaddress as username.
Stephen Gran gave me some valuable hints to using the rwm-rewriting engine.
After some time, I ended up with this setup:
overlay rwm
rwm-rewriteEngine on
rwm-rewriteMap ldap attr2dn "ldap://127.0.0.1/ou=myorg?dn?sub"
rwm-rewriteContext bindDN
rwm-rewriteRule "^anyid=([^,]*@[^,]*)" "${attr2dn(mail=$1)}" ":"
rwm-rewriteRule "^anyid=([^,@]*)" "${attr2dn(uid=$1)}" ":"
rwm-rewriteRule "^(uid=[^,]*)" "${attr2dn($1)}" ":"
rwm-rewriteRule "^(mail=[^,]*)" "${attr2dn($1)}" ":"
The only thing that doesn't work is to make rwm using ldap version 3 to log into itself, so I had to allow read-only access to the relevant attributes from peername.ip=127.0.0.1 - but well, I can live with that.
Update: Added anyid for not thinking in client code, and made sure only the start of entries is used.
apache, ldap and using different attributes as user names
I'm currently considering how to allow users to log on with different attributes as user names,
e.g. with their "real" user name or their mail adress. Unfortunatly as described on
http://httpd.apache.org/docs/2.2/mod/mod_authnz_ldap.html#authldapurl,
though RFC 2255 allows a comma-separated list, only the first attribute is used.
Now, of course an idea would be to specify all different attributes as a new "loginas"-type one. Another solution would be to use ldap overlay modules, and just convert them "on the fly". Better ideas would be welcome.
Update: Thanks to Faidon Liambotis (again!) one can probably use mod_authn_alias to combine authentication with user name, mail adress etc.
aba@ries:~$ grep final update_out/update.OUTPUT_py | cut -f 2 -d ' '| tr , '\n'|wc -l 901
This means 901 packages went from unstable to testing with the most recent britney run (though counting each binNMUed source package as ~12 packages here). Only about 200 packages which are otherwise ready for migration tried to go to testing, the vast majority reached testing.
Major reason for this move is that now libselinux was ready, and we were able to migrate libselinux, pango, glib2.0 and ocaml which dependend on all of them (the last transition being responsible for about 700 packages).
Good news for armel as well:
start: 61+442: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-11:p-0:s-0:s-12:a-442 end: 60+367: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-10:p-0:s-0:s-12:a-367So some progress as well - it's a steady move towards a reasonable uninstallability count (the last number being armel).
semantics in ldap - anyone out there?
Ldap seems like a good database to store accounts in:
Lots of tools can use it, it's easily integrated into pam, and lots more.
But one sometimes just hits borders in the data model of openldap:
There are no semantics available.
So, let me give you an example. Take e.g. the dnsZoneEntry attribute in the debian.org-ldap. Actually, one wants to allow anyone to write into that attribute in his own entry. But - and that's a big but - one wants to have two additional checks before writing: One is that one cannot claim any dns entry already used by someone else. And the other is that the format needs to comply to specific standards.
Now the question is, is there some way to use ldap with more semantics? Or does one need to write ones own backend to do that (and in all the non-plain-db*-backends, it is specified that one needs to write his own authorization code as well). Or some other recommended way to do that? (Or is the only existing incarnation of that ActiveDirectory?)
Update: Thanks to Faidon Liambotis I know now about the overlays in openldap. The standard overlays unique and constraint will probably solve the case above - of course, I have a few more complex cases, but that are at least good starters.
trying: mysql-dfsg-5.0 accepted: mysql-dfsg-5.0 ori: 71+890: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-890 pre: 71+887: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-887 now: 71+773: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-773
In other words, the transition of mysql-dfsg-5.0 to testing has reduced the uninstallability on armel from about 900 packages to less than 800. Still some way to go, but good progress.