Oerwoud Apestaart

Beste Ecover,

Toen ik deze middag thuiskwam stond er mij 10cm schuim op te wachten in de gang.

Zou u de verpakkingen van uw “Wasverzachter” en uw “Wol en Fijnwasmiddel” meer kunnen differentiëren? Die flessen zien er nu hetzelfde uit (op de kleur van het dopje na) .

Daardoor had ik fijn wasmiddel in het bakje van de wasverzachter gedaan, wat klaarblijkelijk niet de bedoeling kan zijn.

Ondergeschuimde Groeten,
Fons

ps:
Ok, ik overdrijf, dat is niet echt gebeurd. Gelukkig. Maar ik had wel schrik dat het zou gebeuren! Toen de was 10min inzat en ik mij plots mijn fout realiseerde, heb ik als een gek het etiket nagelezen maar daar stond NIET vermeld “In het geval u dit product per ongeluk in het bakje van de wasverzachter giet, hoeft u niet te vrezen voor een ontploffing van schuim. De gevolgen zullen zich beperken tot een wasresultaat dat niet aan de gemiddelde zachtheidseisen voldoet”. Had DAT op het etiket gestaan, ik had de wasbeurt niet moeten onderbreken en herbeginnen. Want dat is dus niet groen he, de was moeten herbeginnen. Dat is dubbel waterverbruik! Als Ecover echt als een groen merk bestempeld wil worden, verwacht ik dat jullie zoveel mogelijk wasonderbrekingen proberen te vermijden door een simpele vermelding op het etiket.

May 09, 2012 08:00 AM

Het Belgacom gebouw in Brussel (waar ik werk) bestaat uit drie torens: twee grote en één kleintje. Die drie torens zijn verbonden door een gelijkvloerse verdieping die het hele perceel beslaat en waar ook de restaurants, shops en het auditorium in zijn ondergebracht.

Omdat die gelijkvloerse verdieping veel open ruimtes en toegangen tot torens (en de buitenwereld) bevat, is het daar altijd een beetje tochtig en fris. Niet koud, maar toch frisser dan op mijn bureau. Wanneer ik uit de lift stap om te gaan eten, voel ik altijd een korte rilling over mijn rug gaan.

Ik kan kiezen uit drie opties om te gaan eten. Er is een Foodmaker, een broodjeszaak en een typisch self-service restaurant gelijk de Carrestel. De Foodmaker is met voorsprong de hipste, lekkerste en gezondste oplossing. Maar toch kies ik meestal voor het self-service restaurant.

Want daar hebben ze verwarmde dienbladen! De dienbladen die je aan de ingang moet nemen komen net uit de vaatwas een hebben lekker warm. Om de friste van de benedenverdieping te counteren, houd ik mijn dienblad strak tegen mijn lichaam en dan voel ik mezelf opwarmen.

Ik geniet daar altijd een beetje van.

May 08, 2012 01:07 PM

Op den bureau zijn ze de toiletten (grondig) aan het renoveren. In de eerste fase worden alle oneven verdiepingen aangepakt en aangezien ik op het 17de werk, moet ik nu noodgedwongen uitwijken naar de wc’s op het 16de of het 18de.

Ik ga altijd naar het 16de en het heeft mij drie dagen gekost om mij te realiseren dat ik evengoed naar het 18de zou kunnen gaan! De reden is puur psychologisch: ik ben een uitsteller. Ik doe eerst de leuke dingen en stel het moeilijke werk uit tot later. In dit geval: ik neem eerst de trap naar beneden en pas als het echt niet anders kan, doe ik de moeite om de trap naar boven te nemen. Dat was een confronterende vaststelling! De eerste drie dagen heb ik het 18de op geen enkel moment zelfs maar overwogen. Ik zag alleen het 16de.

Sinds ik deze nieuwe psychologische profileringstechniek (patent pending) ontdekt heb, pas ik hem uiteraard toe op al mijn collega’s. Mijn hypothese blijkt te kloppen, want de harde werkers gaan effectief naar het 18de. Straf.

Uit mijn kleine onderzoekje zijn ook een paar andere profielen naar voor gekomen:

• Neuroten: dat zijn mensen die (zelfs volgens mijn normen) onnatuurlijk hard nagedacht hebben over hun keuze van WC. Drie mensen vertelden mij dat ze eerst naar beneden gaan omdat dat efficiënter is. Bij het naar boven gaan zullen ze namelijk lichter zijn (minus pipi of kaka) en zal het hen minder moeite kosten om de trappen te bestijgen bij de terugkeer. Het waren allemaal ingenieurs, trouwens.
• Sociopaten: deze mensen kiezen de verdieping op basis van wie ze kunnen tegenkomen onderweg. Ze willen niet gaan plassen op een verdieping waar collega’s werken die hen niet liggen.
• Hedonisten: op sommige verdiepingen is de make-over al klaar. De nieuwe toiletten zijn prachtige en er zijn collega’s die speciaal een omweg maken om te genieten van de luxe van de nieuwe toiletten :-)

April 24, 2012 09:05 AM

April 18, 2012 07:46 AM

I pretty much never drink and hack, and last Friday’s evening is a good reason why. I was having a rare beer and managed to spill part of it on my keyboard and desk. So I turned the keyboard around, started cleaning it as fast as I could, forgetting to actually unplug it. I called it a night because nothing good was going to come from that night anymore.

And on Saturday morning I noticed that my INBOX was gone. Hm, is it really gone? Yep, gone from my laptop too. Crap, must have deleted it on the server by accident while cleaning my keyboard…

And because my NAS is a little full lately, I haven’t been as diligent with backups as I normally have been. Hm, and the modest cache on my N900 isn’t very useful either…

Luckily, evolution on my work machine was shut down for some reason, so yay, it has a reasonably fresh cache of my INBOX!

Except that it’s not all that straightforward to actually get this cache back into Evolution. Just copying its contents to an existing or new folder doesn’t do anything. The files themselves are split up versions of the actual email, assumingly because the evo guys thought it would be faster to search header and body by splitting them off from the attachments and saving them separately, inventing their own caching format. Which is fine, but makes it impossible to actually restore a backup with…

After lots of Googling, I stumbled upon this tool that did the trick for me. A lot of hours wasted over a bunch of emails… But what would happen if I really lost my IMAP server mail ? Run this script by hand on all the folders ? Shudder…

April 01, 2012 02:36 PM

I’ve been having fun recently on a new project where I put myself through all sorts of pain by nesting git submodules into team submodules into platform submodules and so on. The goal here is to be able to tag a root repository and thus identify exact commit hashes of all the submodules to any level. This was an idea Andoni had when he was working on livetranscoding in response to a request of mine where I want to be able to use a single ‘tag’ to identify a complete deployment.

That’s been working better than I expected, and I even hacked git-submodule-tools so that I can do git rlog and get a recursive git log between two root version tags, and get a list of every commit between the master and all submodules. That’s pretty neat for writing out release notes.

However, the way I embedded submodules causes a bit of pain when going back and forth. One of my hackers once gave me a PS1 bash prompt that includes info of which git branch you’re on in your shell prompt. So today I decided to extend that a little, and I now have this:

(b:release-0.2.x d:deploy-pro-2012-03-29) [thomas@otto platform]$ ls
Makefile platform puppet RELEASE-0.2.1
(b:release-0.2.x d:deploy-pro-2012-03-29) [thomas@otto platform]$ cd puppet/pro/
(s:puppet/pro b:release-0.2.x d:v0.2.1) [thomas@otto pro]$

This is showing me submodule name, branch, and description of the current commit.

If you want this for your prompting fun too, here’s the github repo

In the near future, simple portknocking for fun and profit with bash!

March 29, 2012 07:39 PM

The jury is still out on puppet as far as I’m concerned.

On the one hand, of course I relish that feeling of ultimate power you are promised over all those machines… I appreciate the incremental improvements it lets you make, and have it give you the feeling that anything will be possible.

But sometimes, it is just so painful to deal with. Agent runs are incredibly slow. It really shouldn’t take over a minute for a simple configuration with four machines. Also, does it really need to be eating 400 MB of RAM while it does so ? And when running with the default included web server (is that webrick ?), I have to restart my puppetmaster for every single run because there is this one multiple definition that I can’t figure out that simply goes away when you restart, but comes back after an agent run:
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate definition: Class[Firewall::Drop] is already defined; cannot redefine at /etc/puppet/environments/testing/modules/manifests/firewall/drop.pp:19 on node esp

And sometimes it’s just painfully silly. I just spent two hours trying to figure out why my production machine couldn’t complete its puppet run.

All it was telling me was
Could not evaluate: 'test' is not executable

After a lot of googling, I stumbled on this ticket. And indeed, I had a file called ‘test’ in my /root directory.

I couldn’t agree with the reporter more:

I find it incredibly un-pragmatic to have policies fail to run whenever someone creates a file in root which matches the name of an executable I am running.

March 27, 2012 01:53 PM

March 24, 2012 10:37 AM

March 19, 2012 08:22 PM

On the bad side of life, I was planning to go to an awesome Calcotada in Lleida today, but I spent last night awake until 4:30 with an upset stomach, so I had to cancel and stay home feeling like shit.

On the good side of life, I really had no excuse left to not do a little long overdue hacking on Digital Audio Database.

I still use it regularly to listen to music, but the GNonLin-based player is just really not very stable. I should really just rewrite it using simply adder just like roughly ten years ago, but my brain won’t be able to do that. So instead I decided to clean up the web-based WebSockets using player I prototyped at OVC last year.

I started with some refactoring, clearly defining model/view/controller base classes and adapting the player and playerview classes to them.

WebSocket code seems to need an update every few months – I pulled the latest revision of txWebsocket on my fork, so my recent browsers actually play music again.

Since the last time I hacked on this, I actually added my 1500+ freshly ripped cd’s, in FLAC format – which browsers don’t actually support.

So, first off, I added an option for the scheduler – responsible for picking tracks, and picking audio files to represent them – to filter by extension. It’s not ideal, but it will do for now, and I punched that filter through the levels of abstraction in DAD. I now start it filtering on .mp3 and .oga, and so Chrome can play back all the tracks the scheduler throws at it.

The web-based player just loads tracks and timing info from the scheduler relative to page load time. I’ve been wanting to make that absolute for a while, so I did just that – the player server schedules tracks for epoch seconds now through websockets.

I had an entertaining half hour listening to the awesome echo effects obtained by having three chrome pages simultaneously playing the jukebox schedule – each page being slightly out of sync with the others.

As I’ll be wanting to use a smallish computer for music playback using a browser, I adapted the code to not use localhost any longer, but do everything with relative URL’s. Voila – the laptop now plays music too, a little bit more out of sync, and of course through its own speakers, adding to the eerie effect.

As an encore, I wanted to stumble my way through some jquery code, to which I’m a certified newbie. I want a nice background slideshow related to the current artist, and I pulled together echonest and bgstretcher-2 as an experiment.

That seems to work relatively well, except that the slideshow plugin doesn’t let you reload a new set of images to cycle through. And some of the other ones I tried instead after that seemed to have the same problem.

Oh well, it’s a start. If anyone knows of a good jquery background slideshow plugin that lets me update the list of url’s for images at any time, let me know!

March 17, 2012 11:06 PM

March 15, 2012 12:31 PM

After last week’s Linode incident I was getting a bit more worried about security than usual. That coincided with the fact that I found I couldn’t run puppet on one of my linodes, and some digging turned up that it was because /tmp was owned by uid:gid 1000:1000. Since I didn’t know the details of the breakin (and I hadn’t slept more than 4 hours for two nights, one of which involving a Flumotion DVB problem), I had no choice but to be paranoid about it. And it took me a good half hour to realize that I had inflicted this problem on myself – a botched rsync command (rsync arv . root@somehost:/tmp).

So I wasn’t hacked, but I still felt I needed to tighten security a bit. So I thought I’d go with something simple to deploy using puppet – port knocking.

Now, that would be pretty easy to do if I just deployed firewall rules in a single set. But I started deploying firewall rules using the puppetlabs firewall module, which allows me to group rules per service. So that’s the direction I wanted to head off into.

On saturday, I worked on remembering enough iptables to actually understand how port knocking works in a firewall. Among other things, I realized that our current port knocking is not ideal – it uses only two ports. They’re in descending order, so usually they would not be triggered by a normal port scan, but they would be triggered by one in reverse order. That is probably why most sources recommend using three ports, where the third port is between the first two, so they’re out of order.

So I wanted to start by getting the rules right, and understanding them. I started with this post, and found a few problems in it that I managed to work out. The fixed version is this:
UPLINK="p21p1"
#
# Comma seperated list of ports to protect with no spaces.
SERVICES="22,3306"
#
# Location of iptables command
IPTABLES='/sbin/iptables'

# in stage1, connects on 3456 get added to knock2 list
${IPTABLES} -N stage1
${IPTABLES} -A stage1 -m recent --remove --name knock
${IPTABLES} -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

# in stage2, connects on 2345 get added to heaven list
${IPTABLES} -N stage2
${IPTABLES} -A stage2 -m recent --remove --name knock2
${IPTABLES} -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

# at the door:
# - jump to stage2 with a shot at heaven if you're on list knock2
# - jump to stage1 with a shot at knock2 if you're on list knock
# - get on knock list if connecting t0 1234
${IPTABLES} -N door
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2
${IPTABLES} -A door -m recent --rcheck --seconds 5 --name knock -j stage1
${IPTABLES} -A door -p tcp --dport 1234 -m recent --set --name knock

${IPTABLES} -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
${IPTABLES} -A INPUT -p tcp --match multiport --dport ${SERVICES} -i ${UPLINK} -m recent --rcheck --seconds 5 --name heaven -j ACCEPT
${IPTABLES} -A INPUT -p tcp --syn -j door

# close everything else
${IPTABLES} -A INPUT -j REJECT --reject-with icmp-port-unreachable

And it gives me this iptables state:

So the next step was to reproduce these rules using puppet firewall rules.

Immediately I ran into the first problem – we need to add new chains, and there doesn’t seem to be a way to do that in the firewall resource. At the same time, it uses the recent iptables module, and none of that is implemented either. I spent a bunch of hours trying to add this, but since I don’t really know Ruby and I’ve only started using Puppet for real in the last two weeks, that wasn’t working out well. So then I thought, why not look in the bug tracker and see if anyone else tried to do this ? I ask my chains question on IRC, while I find a ticket about recent support. A minute later danblack replies on IRC with a link to a branch that supports creating chains – the same person that made the recent branch.

This must be a sign – the same person helping me with my problem in two different ways, with two branches? Today will be a git-merging to-the-death hacking session, fueled by the leftovers of yesterday’s mexicaganza leftovers.

I start with the branch that lets you create chains, which works well enough, bar some documentation issues. I create a new branch and merge this one on, ending up in a clean rebase.

Next is the recent branch. I merge that one on. I choose to merge in this case, because I hope it will be easier to make the fixes needed in both branches, but still pull everything together on my portknock branch, and merge in updates every time.

This branch has more issues – rake test doesn’t even pass. So I start digging through the failing testcases, adding print debugs and learning just enough ruby to be dangerous.

I slowly get better at fixing bugs. I create minimal .pp files in my /etc/puppet/manifests so I can test just one rule with e.g. puppet apply manifests/recent.pp

The firewall module hinges around being able to convert a rule to a hash as expressed in puppet, and back again, so that puppet can know that a rule is already present and does not need to be executed. I add a conversion unit test for each of the features that tests these basic operations, but I end up actually fixing the bugs by sprinkling print’s and testing with a single apply.

I learn to do service iptables restart; service iptables stop to reset my firewall and start cleanly. It takes me a while to realize when I botched the firewall so that I can’t even google (in my case, forgetting to have -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
) – not helped by the fact that for the last two weeks the network on my home desktop is really flaky, and simply stops working after some activity, forcing me to restart NetworkManager and reload network modules.

I start getting an intuition for how puppet’s basic resource model works. For example, if a second puppet run produces output, something’s wrong. I end up fixing lots of parsing bugs because of that – once I notice that a run tells me something like
notice: /Firewall[999 drop all other requests]/chain: chain changed '-p' to 'INPUT'
notice: Firewall[999 drop all other requests](provider=iptables): Properties changed - updating rule
I know that, even though the result seems to work, I have some parsing bug, and I can attack that bug by adding another unit test and adding more prints for a simple rule.

I learn that, even though the run may seem clean, if the module didn’t figure out that it already had a rule (again, because of bogus parsing), it just adds the same rule again – another thing we don’t want. That gets fixed on a few branches too.

And then I get to the point where my puppet apply brings all the rules together – except it still does not work. And I notice one little missing rule: ${IPTABLES} -A INPUT -p tcp –syn -j door

And I learn about –syn, and –tcp-flags, and to my dismay, there is no support for tcp-flags anywhere. There is a ticket for TCP flags matching support, but nobody worked on it.

So I think, how hard can it be, with everything I’ve learned today? And I get onto it. It turns out it’s harder than expected. Before today, all firewall resource properties swallowed exactly one argument – for example, -p (proto). In the recent module, some properties are flags, and don’t have an argument, so I had to support that with some hacks.

The rule_to_hash function works by taking an iptables rule line, and stripping off the parameters from the back in reverse order one by one, but leaving the arguments there. At the end, it has a list of keys it saw, and hopefully, a string of arguments that match the keys, but in reverse order. (I would have done this by stripping the line of both parameter and argument(s) and putting those on a list, but that’s just me)

But the –tcp-flags parameter takes two arguments – a mask of flags, and a list of flags that needs to be set. So I hack it in by adding double quotes around it, so it looks the same way a –comment does (except –comment is always quoted in iptables –list-rules output), and handle it specially. But after some fidgeting, that works too!

And my final screenshot for the day:

So, today’s result:

• fixes for the recent module
• an implementation of –tcp-flags
• some small fixes for firewallchain
• A feature integration branch pulling all of these together

Now, I have a working node that implements port knocking:
node 'ana' {

$port1 = '1234'
$port2 = '3456'
$port3 = '2345'

$dports = [22, 3306]

$seconds = 5

firewall { "000 accept all icmp requests":
proto => "icmp",
action => "accept",
}

firewall { "001 accept all established connections":
proto => "all",
state => ["RELATED", "ESTABLISHED"],
action => "accept",
}

firewall { "999 drop all other requests":
chain => "INPUT",
proto => "tcp",
action => "reject",
}

firewallchain { [':stage1:', ':stage2:', ':door:']:
}

# door
firewall { "098 knock2 goes to stage2":
chain => "door",
recent_command => "rcheck",
recent_name => "knock2",
recent_seconds => $seconds,
jump => "stage2",
require => [
Firewallchain[':door:'],
Firewallchain[':stage2:'],
]
}

firewall { "099 knock goes to stage1":
chain => "door",
recent_command => "rcheck",
recent_name => "knock",
recent_seconds => $seconds,
jump => "stage1",
require => [
Firewallchain[':door:'],
Firewallchain[':stage1:'],
]
}

firewall { "100 knock on port $port1 sets knock":
chain => "door",
proto => 'tcp',
recent_name => 'knock',
recent_command => 'set',
dport => $port1,
require => [
Firewallchain[':door:'],
]
}

# stage 1
firewall { "101 stage1 remove knock":
chain => "stage1",
recent_name => "knock",
recent_command => "remove",
require => Firewallchain[':stage1:'],
}

firewall { "102 stage1 set knock2 on $port2":
chain => "stage1",
recent_name => "knock2",
recent_command => "set",
proto => "tcp",
dport => $port2,
require => Firewallchain[':stage1:'],
}

# stage 2
firewall { "103 stage2 remove knock":
chain => "stage2",
recent_name => "knock",
recent_command => "remove",
require => Firewallchain[':stage2:'],
}

firewall { "104 stage2 set heaven on $port3":
chain => "stage2",
recent_name => "heaven",
recent_command => "set",
proto => "tcp",
dport => $port3,
require => Firewallchain[':stage2:'],
}

# let people in heaven
firewall { "105 heaven let connections through":
chain => "INPUT",
proto => "tcp",
recent_command => "rcheck",
recent_name => "heaven",
recent_seconds => $seconds,
dport => $dports,
action => accept,
require => Firewallchain[':stage2:'],
}

firewall { "106 connection initiation to door":
# FIXME: specifying chain explicitly breaks insert_order !
chain => "INPUT",
proto => "tcp",
tcp_flags => "FIN,SYN,RST,ACK SYN",
jump => "door",
require => [
Firewallchain[':door:'],
]
}
}

Lessons learned today:

• watch iptables -nvL is an absolutely excellent way of learning more about your firewall – you see your rules and the traffic on them in real time. It made it really easy to see for example the first nc command triggering the knock.
• Puppet is reasonably hackable – I was learning quickly as I progressed through test and bug after test and bug.
• I still don’t like ruby, and we may never be friends, but at least it’s something I’m capable of learning. Puppet might just end up being the trigger.

Tomorrow, I need to clean up the firewall rules into something reusable, and deploy it on the platform.

March 04, 2012 10:32 PM

Duitsland is goed bezig.

Dat Europa het al bij al nog niet dramatisch slecht doet in deze crisis, hebben we aan de Duitsers te danken. Zij trekken de cijfers omhoog. Het is er ook aan te merken de laatste tijd. Merkel is precies de officieuze president van Europa en dat zullen die ‘luie’ Grieken geweten hebben!

Andere lidstaten beginnen daarover te morren en het is blijkbaar zo erg dat de Duitse minister van buitenlandse zaken het nodig acht om een charmeoffensief in te zetten. Hij wil daarmee ‘de angst voor een oppermachtig Duitsland wegnemen’ stond vandaag in Metro. Als je vriendjes vinden dat je teveel de baas speelt, dan kan je gewoon wat minder bazig beginnen doen of je kan hen uitleggen dat er goede redenen zijn waarom jij de baas mag spelen. Dat charmeoffensief neigt een beetje naar de tweede optie en daar krijg ik het benauwd van.

Maar het is nog erger. Sinds haar strenge aanpak van Griekenland doet Merkel het ongelooflijk goed bij opiniepeilingen in de Heimat. De Duitse bevolking voelt blijkbaar wel wat voor een groot en sterk Duitsland.

En het zijn niet alleen de Duitsers. Wij Belgen voelen ook wel wat voor sterke Duitsers. Vandaag viel het mij plots op dat twee automerken een Duitstalige baseline (is baseline de juiste term?) voeren in België :

1. Opel: Wir leben autos
2. Audi: Vorsprung durch technik
3. Update via @gewoonstijn: VW: Das auto

Volgens mij is dat iets speciaals. Vroeger was dat toch niet? Een normale baseline is in de taal van uw publiek of in het Engels. Een andere taal heb ik nog niet veel gezien. Zeker in Wallonië ligt dat gevoelig (zelfs voor Engels). Maar ook daar gebruiken ze een Duitse baseline. Opel doet het ook in Frankrijk, Audi niet.

Stel u voor dat Peugeot of Citroën afkomen met een Franse baseline in Vlaanderen. Zou dat goed werken?

Ik denk niet dat die automerken de voorposten zijn voor een Duitse invasie (Opel is trouwens Amerikaans). Maar het wil ongetwijfeld wel zeggen dat die merken marktonderzoek hebben laten doen en dat daaruit gebleken is dat de Belgische bevolking positieve gevoelens heeft bij Duitse baselines.

Op zijn minst kunnen we spreken van een groeiende populariteit van de Duitse cultuur (en moest ik niet zo lui zijn, ik zou waarschijnlijk parallellen kunnen vinden op andere vlakken).

Ik ben alvast compleet paranoïde en volgens mij komt het er op neer dat Duitsland de zeep op de grond heeft gegooid en dat wij ons allemaal langzaam maar zeker aan het bukken zijn om ze op te rapen :-)

Update: Michel (in de comments) en Stijn (op twitter) merken terecht op dat ik paranoïde ben, want dat de Duitse merken al langer Duits gebruiken in hun communicatie.

March 01, 2012 09:56 AM

Well, this sure has been a long time in the making.

Fluendo and Collabora have a checkered past which I won’t get into, but on paper it has always made sense for these two companies to collaborate and making GStreamer work commercially. One company specializes in products, the other in consulting (I’m sure you can figure out which is which), and complement each other perfectly to make GSstreamer more successful commercially.

I personally have always believed that we need to get GStreamer to other platforms and make them as easy to use as possible. Windows was an obvious target in the past, and now Android is another. There is a big difference between a successful open source project, and a commercially successful one. Flumotion’s Andoni Morales who came with me to the GStreamer 0.11 hackfest in Malaga is going to be working on this one SDK to rule them all.

Christian beat me to it in the blogosphere, but the word is now officially out! Feel free to read Fluendo’s press release.

February 27, 2012 12:01 PM

Today I needed an xml diff tool. There seem to be an xmldiff and a diffxml, neither of them packaged by Fedora at the moment. I found an old src.rpm for Fedora 6 for xmldiff 0.6.8

The src.rpm doesn’t rebuild as is for various reasons: its %check stage imports unittest2, so I disabled check. Then it didn’t find debuglist.files, so I disabled building a debug package. And then it found an installed egg file which it didn’t like, so I disabled checking for installed files.

Since I’m going to forget how I did this in the future when I will need this again for some obscure reason, and because if you ever build rpms you may find this useful, here is the command that did it for me for Fedora 15:
rpmbuild --rebuild -D '%check exit 0' -D 'debug_package %{nil}' -D '_unpackaged_files_terminate_build exit 0' xmldiff-0.6.8-1.fc6.src.rpm

Now, back to comparing xml files.

February 10, 2012 12:56 PM

I’m in the quiet town of Malaga these three days to attend the GStreamer hackfest. The goal is to port applications over to the 0.11 API which will eventually be 1.0 There’s about 18 people here, which is a good number for a hackfest.

The goal for me is to figure out everything that needs to be done to have Flumotion working with GStreamer 0.11. It looks like there is more work than expected, since some of the things we rely on haven’t been ported successfully.

Luckily back in the day we spent quite a bit of time to layer parts as best as possible so they don’t depend too much on each other. Essentially, Flumotion adds a layer on top of GStreamer where GStreamer pipelines can be run in different processes and on different machines, and be connected to each other over the network. To that end, the essential communication between elements is abstracted and wrapped inside a data protocol, so that raw bytes can be transferred from one process to another, and the other end ends up receiving those same GStreamer buffers and events.

First up, there is the GStreamer Data protocol. Its job is to serialize buffers and events into a byte stream.

Second, there is the concept of streamheaders (which is related to the DELTA_UNIT flag in GStreamer). These are buffers that always need to be send at the beginning of a new stream to be able to interpret the buffers coming after it. In 0.10, that meant that at least a GDP version of the caps needed to be in the streamheader (because the other side cannot interpret a running stream without its caps), and in more recent versions a new-segment event. These streamheaders are analogous to the new sticky event concept in 0.11 – some events, like CAPS and TAG and SEGMENT are now sticky to the pad, which means that a new element connected to that pad will always see those events to make sense of the new data it’s getting.

Third, the actual network communication is done using the multifdsink element (and an fdsrc element on the other side). This element just receives incoming buffers, keeps them on a global buffer list, and sends all of them to the various clients added to it by file descriptor. It understands about streamheaders, and makes sure clients get the right ones for wherever they end up in the buffer list. It manages the buffers, the speed of clients, the bursting behaviour, … It doesn’t require GDP at all to work – Flumotion uses this element to stream Ogg, mp3, asf, flv, webm, … to the outside world. But to send GStreamer buffers, it’s as simple as adding a gdppay before multifdsink, and a gdpdepay after fdsrc. Also, at the same level, there are tcpserversink/tcpclientsrc and tcpclientsink/tcpserversrc elements that do the same thing over a simple TCP connection.

Fourth, there is an interface between multifdsink/fdsrc and Python. We let Twisted set up the connections, and then steal the file descriptor and hand those off to multifdsink and fdsrc. This makes it very easy to set up all sorts of connections (like, say, in SSL, or just pipes) and do things to them before streaming (like, for example, authentication). But by passing the actual file descriptor, we don’t lose any performance – the low-level streaming is still done completely in C. This is a general design principle of Flumotion: use Python and Twisted for setup, teardown, and changes to the system, and where we need a lot of functionality and can sacrifice performance; but use C and GStreamer for the lower-level processor-intensive stuff, the things that happen in steady state, processing the signal.

So, there is work to do in GStreamer 0.11:

• The GStreamer data protocol has not really been ported. gdppay/depay are still there, but don’t entirely work.
• streamheaders in those elements will need adapting to handle sticky events.
• multifdsink was moved to -bad and left with broken unit tests. There is now multisocketsink. But sadly it looks like GSocket isn’t meant to handle pure file descriptors (which we use in our component that records streams to disk for example)
• 0.11 doesn’t have the traditional Python bindings. It uses gobject-introspection instead. That will need a lot of work on the Flumotion side, and ideally we would want to keep the codebase working against both 0.10 and 0.11 as we did for the 0.8->0.10 move. Apparently these days you cannot mix gi-style binding with old-style binding anymore, because they create separate class trees. I assume this also means we need to port the glib2/gtk2 reactors in Twisted to using gobject-introspection.

So, there is a lot of work to be done it looks like. Luckily Andoni arrived today too, so we can share some work.

After discussing with Wim, Tim, and Sebastien, my plan is:

1. create a common base class for multihandlesink, and refactor multisocketsink and multifdsink as subclasses of it
2. create g_value_transform functions to bytestreams for basic objects like Buffers and Events
3. use these transform functions as the basis for a new version of GDP, which we’ll make typefindable this time around
4. support sticky events
5. ignore metadata for now, as it is not mandatory; although in the future we could let gdppay decide which metadata it wants to serialize, so the application can request to do so
6. try multisocketsink as a transport for inside Flumotion and/or for the streaming components.
7. In the latter case, do some stress testing – on our platform, we have pipelines with multifdsink running for months on end without crashing or leaking, sometimes going up to 10000 connections open.
8. Make twisted reactors
9. prototype flumotion-launch with 0.11 code by using gir

That’s probably not going to be finished over this week, but it’s a good start. Last night I started by fixing the unit tests for multifdsink, and now I started refactoring multisocketsink and multifdsink with that. I’ll first try and make unit tests for multisocketsink though, to verify that I’m refactoring properly.

January 26, 2012 10:16 AM

The application is build at SDK version 10, so it should work on any Android 2.3 or higher. It might possibly work on lower versions as well. After installation of the application and the first run on your telephone, you will need to login. The app authenticates with a Drupal user and only stores the endpoint URL and the session cookie. The session cookie is send when uploading an image so we know exactly who's uploading. Besides selecting from the app, you can also go to your Image gallery, select an image and use the Share menu to drop the image to the application.

The layout of the application as the code is relatively simple, it probably doesn't follow the best practices of Java programming, so be gentle in case you start reviewing, I'm still learning :)

The Drupal part

The Drupal module - D7 only - is simple as well, but was less hard to develop. While I could 've used a combination of services and other modules, I decided to write a simple module with a single menu callback that accepts a request that can either authenticate a user or create a node with title, image and other keys. Once enabled, you need to go to 'admin/config/media/drupapp' and start configure following items:

• The endpoint. Defaults to 'drupapp', but you can change this to your likings.
• The image field name: this will immediately select the content type that will be used to save the node.
• Published status: default status, handy if you want to review after upload.
• Debug option: log the complete $_POST variable through watchdog for testing purposes.

You will need to grant permission on the permissions page as well. Best practice is to create a new role which only has the "upload images via app" permission, and that role does not necessarily need a permission to create nodes.

Combine the 2 technologies and you get Drupoid. What's in name right ? You can browse, fork and/or download the code at http://github.com/swentel/Drupoid. Feel free to modify it to your own needs. The app in its current version will never be made available on the Android market as it's really a personal project. But it might serve as a nice start example for your own adventures in mobile application land.

You can also see some pictures from the screens at http://realize.be/mobile or an installation guide at http://www.slideshare.net/yoroy/drupoid.

January 18, 2012 09:38 PM

January 13, 2012 11:04 AM

Toen ik daarnet naar het station fietste, passeerde ik een stoere grote man met kort haar, type buitensmijter*. Hij was eind de dertig, droeg een spijkerbroek, een lederen vest en er krulde een tattoo uit zijn t-shirt (daar waar je borsthaar zou verwachten). Dat alles werd bekroond met een gouden kettingske en wat littekens.

Vrij cliché en dus vrij onopvallend, ware het niet dat hij een belachelijk klein hondje aan een leiband meesleurde. Het hondje had duidelijk geen zin om te volgen en probeerde af te remmen, maar de man stapte onverstoorbaar verder, blik op oneindig.

Na het stallen van mijn fiets, kwam ik de man opnieuw tegen in de centrale gang van het station. Het hondje was nog steeds vruchteloos aan het afremmen en op het moment dat ik hen zie, blijkt ook waarom: het hondje begint te kakken. Pas 30 meter uitgerekte drol later merkt de man het op en wanneer hij snapt wat er gebeurd is (het duurt drie volle seconden), gaat hij volledig uit de bol. Hij begint te stampvoeten en roept dingen als “WE GAAN TE LAAT KOMEN” en “NU MOE KIK DA HIER GODVERDOMME OPKUISEN OFWA?”.

De bodybuilder breekt. Hij kan het niet aan. Het voelt voor mij al raar aan om DIE man met DAT hondje in het station te zien en het gevoel is blijkbaar wederzijds. Hij is ver buiten zijn comfortzone. Hij kan de omgeving zijn wil niet opleggen en wordt overmand door stress. Hij verliest zijn ‘cool’. Geconfronteerd met dat falen, wordt het uiteraard alleen nog maar erger en het hondje moet het ontgelden in een scheldtirade.
Ik zou durven zweren dat ik hem bijna zie wenen.

Best wel zielig om te zien. Elke vezel in mijn lijf wil hem gaan helpen. Maar ik durf niet! Het is en blijft een boom van een vent die zichzelf aan het verliezen is. En de hond stinkt. Naar kak.

Uiteindelijk ben ik toch gaan helpen natuurlijk. Bleek dat hij iemand moest afhalen. Zijn moeder. Het was haar hondje. Het is allemaal nog goed afgelopen. Ik mag vanaf nu gratis binnen in de CherryMoon.

* gelijk @PabloPoublon op Facebook/twitter

January 12, 2012 09:35 PM

January 05, 2012 10:09 AM

Every new year is a time of cleaning. After getting back to Inbox 0, my next target is my mailing list subscriptions.

It must be something psychological, but I cannot bring myself to unsubscribe from some of these mailing lists. I don’t check on them daily, but once in a while it’s darn useful to search through my local copy of mails on, say, selinux, and find solutions for a problem I’m having.

However, all this mailing list mail brings me a lot of headache. My email client is slow, and I would want it to be fast for the real mail I’m getting (from actual people, needing actual work). It’s hard to track the mails that matter – all my list mail gets put into folders automatically with some procmail magic, but it also means that some of the things I should be paying more attention to are just another bold folder in Evolution somewhere down the mail tree. And lastly, the server where I host my mail shared with friends gets too much traffic, and syncing 3 different evolutions over IMAP with it is a big part of the burden.

I vastly prefered the newsreader model of old, and I think the de facto standard of mailing lists really is a mistake. But I’m not sure what to replace it with.

What I want:

1. have selected mailing list archives be available on my machines, locally
2. have them synced/updated automatically
3. have them out of the way of my normal mail usage unless when I need them

I’ve been considering getting a separate email account just for email lists for this purpose, although I don’t look forward much to having to change all my subscriptions, and would first like to hear from other people how this approach works out for them.

There used to be a push towards web-based mailing list subscriptions, but I don’t know if anyone is really seriously using that, and I would like to have the option of reading these mailing list archives offline.

How do you separate your ‘real’ mail from your mailing list mail? How do you handle them?

January 02, 2012 04:00 PM

Zoals elk jaar heeft Netlash een overzicht gemaakt van de Internet trends voor 2012, zoals voorspeld door 39 mensen die daar hun mening over kwijt wilden. Het ziet er weer zeer mooi uit en dan ben ik fier dat ik een bijdrage mocht leveren. Op zich wil dat niets zeggen, want iedereen die dat wil, mag een bijdrage leveren, maar het ziet er mooi uit en dat is cool.

Je kan het hieronder lezen of downloaden (en als dat niet werkt, kunt ge het op de blog van Netlash vinden).

December 30, 2011 10:32 PM

Every few weeks I have to spend an hour figuring out exactly the same non-googleable thing I’ve already needed to figure out. So this time it’s going on my blog.

The problem is simple: given an input file listing paths, one per line, which probably contain spaces – how do I run a shell command that converts each line to a single shell argument ?

Today, my particular case was a file /tmp/dirs on my NAS which lists all directories in one of my dirvish vaults that contains files bigger than a GB. For some reason not everything is properly hardlinked, but running hardlink on the vault blows up because there are so many files in there.

Let’s see if wordpress manages to not mangle the following shell line.

perl -p -e 's@\n@\000@g' /tmp/dirs | xargs -0 /root/hardlink.py -f -p -t -c --dry-run

December 30, 2011 06:18 PM

I don’t want to simply be complaining about Nokia’s sad Linux story ending.

It’s obvious that things aren’t going to get better though for Maemo device owners.

Here are two things that caused me trouble over the last few months, and the fix that did it for me, in case you were suffering from the same problems.

The Facebook photo sharing functionality just stopped working for me. It would upload the file, then give me an error without specifying any reason. I had simply assumed the Facebook API had changed, and since afaik this plugin is sadly closed-source (what on earth possessed Nokia to make a social media sharing component closed to begin with is beyond me – what kind of IP secrets could you possibly have in there?) I thought I would have to do without for now. And it really is a hassle to manually copy photos off then share them from the desktop.

Turns out that I simply had to re-authenticate the Sharing Account in Settings. No idea why – maybe Facebook changed some authentication system in the last few months ? Now it works again.

(As a side note, this plugin’s behaviour is really annoying when it comes to uploading photos. It will always try to upload as soon as you connect to a network, although usually all you get is access to some web page on which you have to authenticate, usually by paying, to get on to the net. The sharing plugin already blasts photos at facebook, then fails, gives you a non-useful error message, and then sits there forever without any option to retry. All you can do is cancel the transfer, in which case you will have to re-upload the photos from your library. After some time I figured out that a reboot caused it to retry all pending uploads on the next network connect, and then after that I figured out a kill of a sharing manager process did the same thing. But really, Nokia engineers – a simple ‘retry’ button was too hard ?)

For the last two weeks my GPS stopped getting a lock completely. This wreaked havoc on my barriosquare/foursquare checkins as well, which simply don’t register without a GPS lock (yes I still have a half-done port of bsq to the new foursquare API, but last time I tried I was still stuck on the simply terrible browser coming with the device that seems to be unable to properly complete SSL requests in emulator mode).

I stumbled across this page and simply changed the AGPS server to google’s. Worked like a charm on the next connect. Nokia, I don’t know what you did to your AGPS server – surely other phones you have are using it too, not just the Maemo ones ?

I swore I wasn’t going to buy an N9 because there’s no point in buying an EOL’d phone if I plan to develop for it. The reviews when it actually came out almost persuaded me to get it, and the five minutes I got to play with Luis de Bethencourt’s phone got me really close. I’ve even seen plans in Belgium offering this phone! But really, there isn’t much point if Nokia isn’t going to support this phone any more and services are just going to get worse, and important parts of the stack remain closed and thus unfixable down the road.

This month’s Android course at work at least got me familiar developing for the phone and I was actually impressed by Eclipse this time around, and while Java still seems like a bitch to program in, the whole emulator setup is easy to use… Who knows, my next phone may in fact be an Android.

In the meantime, it’s nice to see that some of my fixes go noticed. That motivates me to possibly fix that other annoyance in erminig-ng – all-day events triggering an alarm at midnight and waking me up :)

December 29, 2011 09:35 AM

O Fnac, jij duivelse winkel. Hoe kan ik weerstaan aan al die mooie boeken in uw rekken? Zo interessant, zo uitnodigend zijn de selecties die op uw koppen liggen!

Maar ik heb geen tijd/energie om boeken te lezen! Ons boekenkast ligt vol met boeken die we al gekocht hebben, maar nog niet gelezen. En dan zwijg ik nog over de boeken op de e-reader. Dus ik moet mij beperken tot het kopen van kerstcadeautjes voor anderen.

Het is een drama en het doet pijn, maar ik neem mij voor om ze later allemaal te lezen! En ik hoop dat ‘later’ al over 2 jaar begint.

Dit gezegd zijnde heb ik toch een boek gelezen afgelopen zomer. Hoera! Dat wou ik allang eens melden, maar het was er nooit van gekomen, dus doe ik het nu.

Een boek gelezen dus. En het was dan nog een voltreffer ook. Dubbel hoera! “De eenzaamheid van de priemgetallen”. Prachtig geschreven en soms herkenbaar (incl. referenties naar The Cure). Beetje hard misschien, maar op een mooie manier.

Daarna ben ik in “In Europa” begonnen en voorlopig lukt het om dat stukje per stukje te lezen. Die techniek werkt wel voor dat boek. Het is niet fantastisch goed geschreven, maar goed genoeg en enorm interessant. Ik stond er van versteld hoeveel ik vergeten was uit de geschiedenislessen. De opgefriste kennis helpt mij om de huidige internationale politieke vraagstukken beter te kaderen. Het helpt zelfs om de tijdsgeest te vatten.

December 22, 2011 10:59 PM

I haven’t been too good this year at removing friction from my workflow. Today I wanted to change that. And the random friction thrown my way today has to do with ssh.

You see, somewhere along the line I read that it is a good idea to create separate keys for separate identities. So I have an identity for all work-related stuff (which I consider ‘ring 1′: it’s unlikely to change but everyone can get fired or change jobs), one for personal stuff on machines I actually control (‘ring 0′: they’d have to pry it out of my dead hands), another for my ‘public online default’ identity (‘ring 2′: I can always pull a whytheluckystiff and pull myself of the net and reinvent myself), and then per-project identities (‘ring 3′: I may lose interest in being a fedora or gstreamer contributor without massive changes in my personality).

I started splitting ring 3 per project when it made sense – for example, Fedora recently enforced a key change even if your account wasn’t compromised and even if you already have a strong passphrase on your key (like I had), and of course a massive flamefest ensued. I shrugged and decided to split off a new key and set that on all my machines.

But the problem is, this whole tower of ssh doesn’t really work well in practice. I chose a long passphrase for the new fedora keys, but obviously I do not want to type that every time I clone a package or commit changes. So I use ssh-agent. In theory, ssh-agent adds your keys and asks you for the passphrase once, and is then able to offer those identities to the other side.

The problem is a lot of ssh servers out there only give you a few tries. So your ssh agent will offer identity by identity until it gets refused. If my fedora identity was added as the fourth identity I lose – I can’t clone a package.

Specifying IdentityFile in the ssh config is useless. It is poorly documented, but IdentityFile files actually come after your ssh-agent identities. So your agent blasts all the wrong keys at the host first, and you get denied.

So you can specify IdentityOnly to make sure that only the identity file you want is being used. Sadly in that case it will not use the ssh-agent at all, so it will ask you for the password to your key file – the whole reason you want agents to be used in the first place.

Now obviously ssh has all the pieces it needs to Do The Right Thing. If my config says to use this identity and this identity only, ssh should be able to request ssh-agent to present that identity, and that identity only, and make the login happen without any password.

Surely I must be missing something obvious. Surely one of you uberhackers out there has set up the same thing as me. Why don’t you comment about it here and help the rest of us?

December 22, 2011 11:57 AM

December 08, 2011 02:06 PM

Sommige mensen hebben schrik van Google omdat die achter uwe rug uw privacy schenden. Ha! Ze zouden schrik moeten hebben van MIJ! Ik ben een topspion.

Vandaag zat ik op de trein in te dutten toen ik mijn buurman tegen zijn maat hoorde vertellen over zijn ex-job. Het ging daar te traag naar zijn goesting. Ze zaten daar allemaal de krant te lezen. Hoe meer hij vertelde over dat bedrijf, hoe meer ik de werkomgeving van een (hard werkende) vriendin herkende.

Ik kan het uiteraard niet zomaar laten gebeuren dat er iemand slecht spreekt over een vriendin, maar ik was ook nog niet 100% zeker dat het over hetzelfde bedrijf ging.

De enige oplossing was om stiekem een foto te nemen van die gast en door te sturen naar die vriendin. En ja hoor, 10min. later krijg ik de bevestiging en zijn naam. Om het dramatisch effect te verhogen, heb ik dan gewacht tot net voor ik moest uitstappen om hem aan te spreken:

- “Excuseer, Dieter, je hebt de groetjes van Florence Verstraeten. Ze laat weten dat ze nooit de krant leest op het werk.”

- “Ah, euh, maar hoe…”

- “Ja manneke, in ‘t vervolg twee keer nadenken alvorens ge slecht spreekt over een ander he. De staatsveiligheid is overal!”

En dan ben ik afgestapt.

Ik heb dat nog al gedaan en ik vind dat de max. Niet omdat ik mensen bang wil maken, maar omdat het ook bewijst dat de wereld klein is. De vorige keer dat het mij overkwam, was 9 maand geleden.Ik kijk al uit naar mijn volgende ervaring ergens volgende zomer!

[Ik heb mogelijks een beetje overdreven, maar ik heb wel een foto doorgestuurd, bevestiging gekregen en een mens aangesproken. Maar misschien niet zo offensief :-)]

December 07, 2011 03:09 PM

Het overkomt mij regelmatig dat ik met iemand aan het praten ben op de trein (of op het perron) en dat die persoon achterblijft in het gedrum bij het op- of afstappen. Ik heb dat dan niet door en zet mijn gesprek verder met een wildvreemde die toevallig naast mij loopt. Dat is schaamtelijk.

Ik probeer er mij tegen te wapenen door zoveel mogelijk oogcontact te zoeken met mijn gesprekspartner tijdens drukke momenten op de trein. Maar desalniettemin staat er bij het volgende oogcontact toch soms een wildvreemde naast mij.

Het grappige is, dat het andere mensen ook overkomt. Ik zie dat gebeuren. En heel soms (1 keer om de 6 maand, schat ik) ben ik de wildvreemde met wie iemand anders per abuis een gesprek verderzet.

En nu komt het spel! Mijn doel is om dat gesprek zo lang mogelijk aan de gang te houden, zonder dat die mensen dat doorhebben. Ik maak dan instemmende geluidjes en knik met mijn hoofd en heel soms geef ik zelfs antwoorden op vragen. Dat gaat goed met vragen waarbij het antwoord al in de vraag zit verwerkt:

- “Vond je dat ook niet overdreven?”

- “Ja, dat vond ik ook overdreven!”.

Mijn record is 150m.

December 05, 2011 04:45 PM

Het is nu al een kleine tien jaar dat ik pendel tussen Gent en Brussel. Dat levert af en toe wel eens een anekdote op. En ik denk dat ik niet de enige ben. Er moeten toch nog mensen zijn die straffe NMBS verhalen hebben?

En wie leest er het allerliefst over treintoestanden? De pendelaars!

Daarom stel ik voor dat we allemaal samen een boek schrijven voor onszelf. Een boekje over het pendelen met de NMBS: de treinen, de stations, de rare kwieten op de trein, de vertragingen, … Een bundeling van anekdotes. Niet noodzakelijk literatuur, maar vrolijk leesbare lectuur.

Alle stukjes moeten voldoen aan een bepaald format:

• Het moeten korte anekdotes zijn (max. twee tot drie pagina’s), luchtige tussendoortjes.
• Ze mogen niet zeurderig of zagerig zijn, we willen geen NMBS klachtenboek maken.
• Er moet ergens een grappige of verrassende twist in zitten.

En we gaan niet alleen teksten in dat boekske zetten. Er komen ook foto’s in. Het gaat 50/50 zijn, want we willen ons boekske ook aan mensen verkopen die niet willen lezen, maar wel willen kijken.

Geen typische treinfoto’s natuurlijk, maar originele grappige getuigenissen van het pendelaarsleven.

Ik zal een website maken waarop we alles kunnen verzamelen. Ik zorg ook voor de juridische onderbouwing van het geheel, zodat het een beetje eerlijk verloopt. Alle winst wordt evenredig verdeeld onder iedereen die een bijdrage geleverd heeft aan het boekje.

Deadline is tegen de zomer, zodat we nog een uitgever kunnen vinden die ons boekje kan uitgeven tegen dat het boekenbeurs is. Ah ja! Het is wel de bedoeling om daar geld aan te verdienen natuurlijk :-).

De kwaliteit van de stukjes zal ook wel beoordeeld worden op hun kwaliteit. Hoe we dat gaan doen, dat weet ik nog niet. Maar we kunnen niet alles zomaar opnemen, want we willen geld verdienen, dus het moet goed zijn. Ik ben niet zo’n fan van stemmingen op een website om kwaliteit te bepalen , want dat gaat er niet altijd even eerlijk aan toe. Misschien laten we gewoon de uitgever kiezen (als we die vinden).

De eerste druk verkopen we integraal aan METRO, die het dan gratis mogen verdelen samen met hun krantje. Dan zijn we toch al zeker van een BEETJE winst.

Goed, wie doet er mee? Ik moet tien deelnemers vinden voordat ik het zie zitten om een website op te straten hiervoor :-).

November 28, 2011 10:41 PM

Het is maandagochtend en mijn trein heeft (uiteraard) vertraging. Twintig minuten nu al. De vertraging is pas ingetreden na het vertrek, dus ik zit wel warm, maar ik zit ook gevangen in deze overvolle trein. Ik kan niet snel nog even een krant gaan kopen en die dan gezellig lezen in de Panos.

Ik kan alleen maar uit het raam staren naar de mist boven de velden. En luisteren naar mijn buurman.

Die heeft een vals gebit en hij zit het de hele tijd te verplaatsen in zijn mond. Denk ik. Ik heb geen ervaring met valse tanden, maar er beweegt iets in zijn mond. Misschien klikt hij het de hele tijd los en vast met zijn tong. Misschien is hij op zoek naar een laatste restje ontbijt. Wat het ook is dat hij doet met die tanden, het maakt alleszins een irritant knarsend geluid dat mij nog dieper raakt dan piepschuim.

Dat duurt nu al meer dan twintig minuten, die mens is verslaafd aan zijn tanden. En het zijn niet de tanden alleen. Hij rekt zich ook uit en maakt daarbij geluidjes alsof hij alleen in zijn slaapkamer zit. Schaamtelijk. Ge moogt u uitrekken op de trein, maar IN GODSNAAM, hou het stil. Uitrekgeluidjes lijken sterk op sexkreuntjes en dat is een brug te ver op de trein.

Het werkt zo hard op zijn zenuwen dat ik zin heb om hem vol in het gezicht te slaan. En djoef op zijn muil.

En dan realiseer ik het mij! Misschien kan die mens daar niet aan doen! Dit is waarschijnlijk een inherent irritante man en hij heeft in het verleden als eens op zijn muil gekregen omdat hij zo irritant is!

Vandaar de valse tanden.

November 28, 2011 03:46 PM

Twee weken geleden heb ik een banaan meegenomen naar het werk. Ik steek die dan in het (weinig gebruikte) voorste compartiment van mijn rugzak.

Vandaag heb ik hem teruggevonden. Het is te zeggen, de securityman van de Mediamarkt, die per se mijn rugzak wou controleren, heeft hem teruggevonden.

Ik moest daarmee lachen.

Hij niet.

November 28, 2011 09:47 AM

Ergens in mijn middelbare school carrière heb ik “The Mist” van Stephen King gelezen. Dat gaat over een mist die zo dicht is dat de mensen niet meer uit hun huis kunnen/durven komen. Er komt ook wel een monster aan te pas, maar het punt is dat die mist echt potdicht is. Als je je arm uitstrekt, kan je de tippen van je vingers niet meer zien. Griezelig!

Die mist uit dat boek is altijd een beetje mijn maatstaf geweest voor mist.

Gisterennacht reed ik terug naar huis door een behoorlijk dichte mist. Misschien wel de dichtste mist die ik ooit al heb meegemaakt. Maar ik kon nog makkelijk dertig meter voor mij uit zien. Een flauw stoompje in vergelijking met The Mist (en een monster was al helemaal niet te bespeuren). Ik begon mij dan ook af te vragen waar Stephen King zijn inspiratie had gehaald voor zo’n dikke mist, want die bestaat niet eens in het echt! (monsters bestaan wel)

Vandaag heb ik ontdekt van waar die inspiratie kwam. Ik reed met de fiets naar huis door een relatief dichte mist. Maar het was ook koud en ik had mijn bril op. Langzaam maar zeker begonnen de glazen van mijn bril te bewasemen door mijn adem. Ik had dat niet direct door en even dacht ik dat de mist wel heel erg dicht was geworden. Ik ben zelfs gestopt uit schrik om tegen het monster te botsen.

En ja hoor, Stephen King draagt een bril.

November 21, 2011 10:08 PM

A new Fedora, a new decision on which machines to upgrade. Usually I try to stagger the three machines I use most – my work desktop, my home desktop, and my laptop. I had updated work machine and laptop to F-15 when it came out, and kept my home desktop at F-14.

I actually have two or three root partitions on each of those machines, and I typically do a fresh install on a separate root, so I can try things, poke around, and make sure everything I will need works. When I do the install, I don’t mount my /home partition, because I don’t want to have the new version upgrade things for me on my user config.

I have a pretty long checklist by now that I go through on each install/upgrade, installing the packages I use a lot, setting up specific configuration, copy over ssh keys, …

I actually liked F-15 a lot, and though GNOME 3 has its issues (which I still want to document in a separate post), I overall enjoyed the experience. At home, I noticed myself using the windows key or moving my mouse to the top left corner expecting something to happen.

That is how you know you really are ready for GNOME3.

So I thought, what the heck, let’s get to upgrading all of them. I started with my laptop, as usual. That mostly went fine, except for hurdle number one. My laptop actually has /home encrypted. And I did not add it to my custom layout in anaconda. So, the system dropped me in a rescue shell after booting. It took me quite a while to figure out that I had to copy over /etc/crypttab from the old system. After that, things worked again.

Arguably, hurdle #1 may not be Fedora’s fault. Maybe normal users don’t encrypt home drives, or use custom partitioning like I do (although on a few fedora upgrades this saved my bacon when it turned out certain things I needed didn’t work in the new Fedora, like VMWare)

And yes, GNOME 3.2 is a slight improvement. Enough to make a difference at least. All the usual applications seem to work, so I can now mount my old /home directory.

That’s when I ran into hurdle number 2: the default uid/gid numbering change. My thomas user now was 1000:1000 as opposed to 500:500 on all my machines before Fedora 16.

In this day and age, I still have to shell it up to fix things like that:
find / -uid 500 -exec chown 1000 {} \;
find / -gid 500 -exec chown :1000 {} \;
If I had less shame I’d tell you how embarassing it is if you do this for a few users on your system, and start thinking “let’s put this in a for loop”, and because it’s already 1 AM you start doing things like
for a in 0 1 3; do find / -uid 50$a -exec chown 10$a {} \; ; find / -gid 50$a -exec chown 100$a {} \; done
Note how I got the number of 0′s wrong in the first find, and how I actually forgot the : in the second. You can imagine how amusing it is to fix the effect of those commands.

But I’m a shameful person so I won’t tell you about this bit. Instead, suffice it to say that this took a long time.

Ok, so now /home is mounted on the laptop, and for the most part things worked fine.

On comes the weekend, so I turn to the home machine. I tend to keep the work machine for last, because I don’t want to spend work time on fixing distro problems. And I usually take a whole weekend to upgrade at home. The home machine turned out to be more of a problem. I ran headlong into hurdle number three. You see, there is this new thing called GPT for your partition table, and it is now the default, and it means that fdisk will no longer work, and now you should use gdisk (which sadly is not installed on the rescue bit of the install DVD, boo!), and this is all so we can have grub2, which is supposed to be better or something.

I’m sure one day I will be thankful. But on my home machine, I didn’t know any of this, and just had anaconda tell me something about the boot image being too large and there was no space for it and my system may not boot. (I am not sure why I did not run into this problem on my laptop – presumably, looking at the disk layout now, because I kept the original install, which includes Windows, and just shrunk that and added linux – so it’s probably the windows thing doing the booting). And sure enough, the Fedora 16 install did not boot. It dropped me into my friend, the shell.

So here’s the thing. This new way of doing things needs more space than your average MBR, so you actually need to create a primary partition for this, and it needs to be in the first 2 TiB. So you know what time it is now. It’s resize-o-clock time – I get to learn the joys and mysteries of shrinking ext4-on-software-raid so I can make space for this new partition, which doesn’t need to be big, apparently 5 MB is more than enough. Aren’t I happy now that I stubbornly stuck to having a /boot partition as the primary one on my machines, so I can just shrink that a little?

So shrinking an ext partition I already had down pat. I learnt about shrinking software raid partitions, and again I got into the land of not understanding which of the many types of numbers (sectors ? blocks ? bytes ? cilinders ? Mebi vs Mega ?) are understood the same way by the tools, or not understanding how much of those numbers you need to count extra because of the layer of indirection being added (encryption on logical volume on LVM on software RAID anyone ?). So to be safe I end up shrinking 10% on each layer of the onion as I go deeper – then let the tools handle growing to the maximum space again, since that’s the one thing they’re usually decent at.

But you know, if I’ve done all this, I want to get it right. I don’t want a stinking BIOS boot partition sitting after my /boot partitions. That’s not how F16 sets it up by default. But I have never actually moved a partition. So, download gparted, look at it, figure out how it can let me do that, make sure I ask it to count by cylinders so it doesn’t leave gaps, be puzzled at why it doesn’t let me fractions for MiB sizes of partitions, and work around it in some other way. And so I finally have those two software raid /boot-wearing partitions where I want them – sitting right behind this new BIOS partition.

I create a new partition in fdisk (which is what I’m used to), but I can’t actually set the partition type to EF02, which has four characters where I expect two. But really that is what BIOS BOOT should be.

And now the internet tells me I need to set some flag on it using a tool called parted – some flag called bios_grub. Except when I type that magical command that sets the flag, it tells me it can’t exist:
[root@otto ~]# parted /dev/sda set 6 bios_grub
parted: invalid token: bios_grub
Flag to Invert?

Isn’t this tool nicely written for only the writer of the tool instead of for human beings? Of course I don’t know this when it barfs this at me, but at the end of this story I figured a bunch of things out that this tool could have told me.

You see, invalid token just means that it doesn’t accept the flag named bios_grub. I know this because I’m a programmer so I know the programmer used a token parser – a thing normal people shouldn’t have to know about. What’s that you’re asking? Flag to Invert? How about the Belgian flag, I would quite like to see the colors go in the opposite direction. No, that’s a prompt to choose a different flag to invert than bios_grub. Apparently bios_grub is a flag, not a setting, and I’m trying to invert it, instead of setting it. Can you tell me what flags you do know about, dear parted ?

(parted) help set
set NUMBER FLAG STATE change the FLAG on partition NUMBER

NUMBER is the partition number used by Linux. On MS-DOS disk labels,
the primary partitions number from 1 to 4, logical partitions from 5
onwards.
FLAG is one of: boot, root, swap, hidden, raid, lvm, lba, hp-service,
palo, prep, msftres, bios_grub, atvrecv, diag, legacy_boot
STATE is one of: on, off

Wait, what ? You do know about bios_grub ? But you don’t let me set it ?

I seriously spent 30 minutes on trying to figure that one out.

In the end, it’s because a) I should run gdisk b) parted won’t let you set that flag on a normal MBR drive c) gdisk should convert to using GPT and d) the messages gdisk prints by default are SUPER scary and the docs say that this is intentional to keep away stupid Windows users (I am not making this shit up). Well, that’s why I use software RAID, isn’t it ? How about we take our chances, dive in deep, and let this gdisk thing do the conversion to GPT on the first disk. Gulp.

OK, I got lucky. That actually worked. I can now create this partition, with the proper flag set. While I’m at it, why don’t we try this ‘sort partitions’ option in gdisk so that this new partition, which is now at the start, but listed as number 4 out of 4, shows up as number 1. Sure, it will renumber all other partitions, but let’s just hope that most things use UUID’s and labels and what not by now, and if not I should be able to figure things out.

In what feels like Day 5 in a two-day weekend, the system now boots! I actually see a new grub (wait, why is that text-mode only again ? Fedora guys, you spent years to make everything look graphical, because that was some huge important feature that mostly got in my way when it took longer than it was supposed to and I had no way to see why except reboot and remove quiet and rhgb from the options) and now you suddenly let grub2 take that back from you? Show us some spine, please), and the system shows me plymouth again. Until it doesn’t anymore, and drops me into a terminal screen.

Hurdle number four. Can you guess what it is ? Go on, take a stab. If you’ve updated your system, I’m sure you know the answer. I’ll give you some whitespace to think about it…

SELinux. Riding in to relabel my file system to save it from the evil people out there. And sure, it warns me. This may take a long time. And then it proceeds to throw asterisks in my face. Lots of asterisks. It’s not the first time this happens. But every time it does, I cannot help but wonder one thing.

Who thought it was a great idea to throw asterisks at the user? How many asterisks am I supposed to expect? Never mind that you can’t actually count them unless you glue your eyeball at the screen, because there are so many they actually scroll off at the top. You know, if you squint hard enough, you can see the maniacally laughing face of the programmer who thought this was a nice way of showing progress. Never mind that tools like fsck can show a progress bar that actually means something (if you trick it into sending data to file descriptor 0) in a sensible way – one line on the console, and visible progress towards an end goal of 100%.

If only I could guess what a long time is going to end up being. Is it a ‘get a drink’ amount of time? Or ‘watch some dexter’. Or nookie time? Or, get the hell out of the house and do all the shopping for the next three hours because there’s no way you’ll be doing anything useful with this system for that long?

So I do all of those things, twice, and one even four times times (I won’t tell you which but I ended up having to pee a lot), and I come back, and the system has rebooted, and there’s actually a GUI asking me to log in.

You know, this Fedora 16 better be frigging spectacular after this six day weekend.

I log in, follow my standard upgrade checklist, try out some of my tools. Media keys don’t seem to work as before for my prototype music player (it flashes a nasty forbidden sign at me), and even though I set up to have nothing happen on inserting audio CD’s (because my LEGO robot is inserting CD’s into an external drive about fifty times a day), Rhythmbox craps on and FORCES me to select which of the many CD’s with exactly the same name that audio CD might be. So, par for the course so far.

Maybe a reboot will fix that, it may not know about those settings until I have everything installed and upgraded. And if I reboot, I’d better convert my second drive to GPT and fix my /boot and set that flag and all that. So I do. And for some reason I can’t figure out how to tell software raid that sda2 and sdb2 (which are both still perfectly mountable as ext file systems and were part of the previous RAID-1 /boot array before I resized them) really are a software raid. So there’s this point where I’ve wasted more time on trying that then it would have taken me to actually manually type every byte on that /boot partition, and I just give up and recreate a software raid on those two partitions and copy stuff over.

And then I reboot. And won’t you know it. Effing goddamn selinux relabel all over again. In fact, this way too long entry was typed completely in less than half the time selinux took to complete some work it had already done an hour ago.

I better have a working system after this last relabel finishes. Now excuse me while I go make some comfort food, potatoes and beans and runny eggs with butter sauce. I’m going to eat it while my good friend Dexter comes back from a long holiday. It’s the only thing that is going to get me out of this weekend funk. And you know who I will be thinking about every time my friend Dexter tells me of a problem he solved…

November 13, 2011 01:03 PM

Following up on last night’s post, I updated my package repositories with all packages I currently work on before sending them off to Fedora.

The repository is updated with chromaprint, gstreamer-chromaprint, longomatch, and some of my own projects: moap, mach, morituri, and savon.

November 10, 2011 08:38 AM

Another November, another Fedora. 16 came out, so it was time to update mach again.

And today I thought, is there any reason mach isn’t 1.0 yet ? Am I going to do anything more to this piece of code before I want to call it that ?

And the answer is, no. It’s the first Python application I’ve written, and I’m not particularly proud of the code, but I’m happy I’ve made good use of it for so long, and that it helped push packaging approaches forward and sparked ideas for the Fedora build system.

Since I didn’t like the original code for mach2 (there was a version 1 which was Makefile-based), I started a rewrite with unit tests, better code layout, decent classes for abstracting distro-specific stuff, and so on.

The experience of how mock was created based off mach2 was a slightly sour one however, so I wasn’t really motivated to finish the mach3 rewrite. Sometimes that’s the drawback of open source – sure, forking is specifically allowed, so don’t whine about it when it happens. But when it’s done gratuitously, with no serious attempt at collaborating, it doesn’t feel like it’s in the spirit of open source.

Anyway, that was a long time ago. mach2 as it is today, is done. It really only needs updating for newer versions. As long as it works for me, it’s unlikely I will continue mach3, but who knows?

Enjoy the release!

November 09, 2011 10:56 PM

Ik ben 34 jaar door het leven gegaan zonder de echte betekenis van het woord “saignant” te beseffen. Voor mij was dat gewoon een woord dat op een bereidingswijze was geplakt en ik had nooit de link gelegd met het werkwoord “saigner”. Ik wist nochtans dat saigner het Frans is voor bloeden, maar het heeft dus heel lang geduurd vooraleer mijne frank viel. Het is nochtans heel logisch.

Dat heeft grote implicaties! Bijvoorbeeld voor mijn begrip van het woord “seigneur” of “monseigneur”. Heeft dat ook iets met bloed te maken? Natuurlijk niet, want het is anders geschreven en het gaat over de baas van een seigneurie. Maar in mijn hoofd gaat monseingeur altijd bloeden vanaf nu. Één keer per maand zal ik mijn loetje dan ook aanspreken met het koosnaampje “monsaigneur”. Of moet dat dan “masaigneuse” zijn?

November 08, 2011 08:57 AM

Tips voor bedelaars zijn een specialiteit van mijn blog en vandaag heb ik er nog eentje in de aanbieding. Deze tip heb ik niet zelf bedacht, maar is gestolen van Thomas.

Bedelaars aan grootwarenhuizen zouden een verzameling moeten maken van kortingsbonnetjes en coupons. Die kunnen ze knippen uit kranten (vuilbak van het station zit vol met dagverse kranten) of uit reclamefolders (uit de brievenbus recupereren van leegstaande huizen) en dan mooi per categorie op een groot bord hangen.

Als je daarmee aan de ingang van een grootwarenhuis gaat staan, heb je gegarandeerd succes! Zelf bonnetjes uitknippen, bijhouden en op het juiste moment gebruiken is zo een gedoe dat het bij veel mensen fout loopt. Het is veel makkelijker als je een selectie van bonnetjes zou kunnen maken bij het binnengaan van een supermarkt! Op dat moment weet je al wat je gaat kopen en kan je veel efficiënter selecteren. Ik zou zeker een deel van de winst willen delen met een bedelaar in ruil voor deze service.

Bedelaars aan een Delhaize hebben het al helemaal makkelijk om kortingsbonnen te sparen. Elke klant krijgt gepersonaliseerde kortingsbonnen bij elk kasticket. Maar dat is zo een belachelijk onhandig systeem dat de lege winkelkarretjes en volle vuilbakken op de parking uitpuilen van die bonnen!

November 04, 2011 08:38 PM

Onze nieuwe poetsvrouw is een dikke zwarte negerin.

Dat bedoel ik helemaal niet pejoratief, want ze poetst heel goed. Maar als ik haar zie, met haar ingewikkeld in elkaar zittende kleurrijke kleding, denk ik direct “dat is nu eens een echte dikke zwarte negerin”. Niet dat ze veel te dik is, ze heeft het gewicht dat past bij haar uitstraling en ze is leniger dan je zou verwachten, het is gewoon zo een echte Afrikaanse mama.

Ze praat ook van dat heerlijk Negerfrans. Ik stel haar soms dwaze vragen, alleen maar om haar te horen antwoorden. En als ze haar GSM beantwoord, luister ik stiekem mee. Heerlijk taaltje. Frans is al melodieus, maar NegerFrans is pas echte muziek.

Liliane brengt kleur in ons huis. Daarnet nam ze de stofzuiger mee naar boven. Ze droeg hem niet in haar hand, maar hij balanceerde op haar hoofd. Handig op onze smalle trap.

November 04, 2011 08:22 PM

Gisteren zijn we de perfecte reclame voor Velux dakvlakramen tegengekomen.

Het was grijs weer en we reden door een straat met grijze rijhuizen toen we plots een kleurrijke ballon boven de straat zagen hangen.

De ballon hing vast aan een lang rood lint en dat lint eindigde in een hand dat door een openstaand dakvenster naar buiten stak.

Dat was zo’n mooi zicht. Het was alsof dat grijze huis een hand had en een ballon vasthield. Het maakte de straat ineens een stuk minder grijs.

Het was ook een leuke gedachte dat er in die grijze huizen toch kleurrijke mensen wonen en dat hun vrolijkeheid soms een beetje ontsnapt door het dakraam.

October 30, 2011 02:24 PM

I’m in Prague right now for the second GStreamer conference. Prague is as pretty as I remember it from eighteen years ago when I was still in high school and we had our yearly school trip.

It’s great to see a mix of familiar and new faces again. 11 years ago GStreamer was made public, and I joined a year later around the 0.1.1 release if I recall. And now it’s this huge living breathing thing.

Tomorrow I will be giving a talk about Flumotion here, at 12.00 in the main room. If you’re interested in GStreamer beyond mere playback, this talk is for you. The only sad part is that my good friend Jan Schmidt will be talking about Bluray at the same time, but I’m relying on Ubicast to record it properly so I can see it later!

October 24, 2011 01:28 PM

October 15, 2011 09:05 AM

October 11, 2011 07:38 AM

We had a problem with one of the encoders producing artifacts under certain conditions. It was hard to reproduce, but it usually happened on cartoons, so some of the web developers helped the core team out to see where they were triggered and spent half an hour watching anime cartoons looking for artifacts.

The boss walked past when one of them was watching the cartoon. A week later, he informed the development manager that his guys were watching cartoons on the job. It wasn’t his business, of course, but the boss thought he should know.

So the development manager, in his next sitdown with the developer, said: “Don’t get upset, but I wanted to let you know that the boss has caught you watching anime at work…”

Needless to say the developer was rightfully upset, wondering how the boss could possibly think he was stupid enough to be watching cartoons for fun in plain sight at work…

September 26, 2011 07:42 PM

September 26, 2011 08:30 AM

Our company has a history of working with interns, thanks to our marketing manager. One day, our Operational Manager got an intern. He’s easy-going and gets along with everyone in the company. The intern came for her first day, and joined him in a bunch of meetings as he took the time to explain what sort of things the Operations department actually does.

At the end of the day, he spoke the now-famous words “Espero verte mañana” – I hope to see you tomorrow!

He didn’t. She never came back!

September 19, 2011 06:39 PM

Last week one of our developers said goodbye to another developer, including the following advice:

Make sure you self.setMood(moods.happy) and enjoy Paris as your virt_base and errrm it improves your self.uiState of mind.

See, I find that amusing.

September 19, 2011 02:33 PM

After ripping over a 1000 CD’s perfectly, and having problems on a few (bad discs, weird audio, a few small niggles to fix), I ran into a fun failure.

Apparently, the file name in u’morituri/Sufjan Stevens – Illinois/02. Sufjan Stevens – The Black Hawk War, or, How to Demolish an Entire Civilization and Still Feel Good About Yourself in the Morning, or, We Apologize for the Inconvenience but You\’re Going to Have to Leave Now, or, “I Have Fought the Big Knives and Will Continue to Fight Them Until They Are Off Our Lands!”.flac’ is too long for my NAS.

Thank you Mister Sufjan. In your honour, I added a function to morituri to shrink the filename to a power of two minus one, below either the given length or 128 characters, whichever is less. For now the algorithm splits on spaces and changed the file name to morituri/Sufjan Stevens – Illinois/02. Sufjan Stevens – The Black Hawk War, or, How to Demolish an Entire Civilization and Still Feel Good About Yourself.flac

That is good enough for me… I was worried I had to teach this one tiny function about keeping quoted pieces together, or how comma’s work, or how ‘or, ‘ works, and so on, just to satisfy my crazy sense of aesthetics.

September 18, 2011 03:15 PM

One day we got a call from one of our customers across the globe who stream a wildlife channel. They told us that, instead of a broadcast of their channel, which was embedded on their home page, there was now an adult channel in its place. A very different kind of wildlife channel… Could we fix it as soon as possible?

I was pretty worried, because if it were our mistake that would be pretty embarassing. So I asked Zaheer, our resident DVB expert at the time, to investigate.

After a lot of debugging and head scratching he told me that there was nothing wrong on our side – the channel that was configured to capture was in fact an adult channel. In the end, we looked up the satellite channel names again, and then we noticed that our customer’s channel was not in the place where it used to be. Apparently it had changed PID.

When we told them what happened, they told us, ‘Oh yes, you are right. We got a notice from our satellite provider a few months ago that the number would change but we forgot all about it…’ They didn’t think to warn us.

I was just happy it wasn’t our fault after all!

September 12, 2011 07:30 PM

September 12, 2011 09:00 AM

Our data center has some satellite dishes on it allowing us to encode any of the signals for our customers. One day we had a problem with one of our dishes, and soon after a customer called to one of our sales people. Her answer ?

“We’re sorry, one of the satellites fell down. But don’t worry, tomorrow we’re launching a new one!”

The customer was so impressed that he thanked her for our dedication and hung up.

We never had the balls to tell this customer we don’t actually run a space agency.

September 04, 2011 09:29 AM

Attachment	Size
Booklet front	297.26 KB
Booklet instructions	979.05 KB

August 18, 2011 08:00 AM

August is a great month for clearing the decks in Spain. Half the people at work are on holiday, and by the time they get back the other half goes away. It’s a great month for making progress on all those things that’ve been lying around for months not getting done.

Same in my spare time. After lots of changes in my life in the last year, I’ve been settling down again, in a new apartment, in a new life, and I am finally in a place where I can do some hacking again. And it’s a great feeling, to be back in a flow state and fixing problems.

Tonight I wanted to direct my hacking attention to my venerable N900 phone. First of all, I wanted to figure out why mushin, my couchdb-based GTD application, was not showing the right results for my shopping list. Turns out I had a broken svn snapshot running on the phone, and that was easy to fix. In the process I set up scratchbox again on my new laptop.

Then I directed my attention to erminig-ng, an application that syncs between the maemo calendar and Google calendar. It had stopped working a few months ago, giving a traceback during sync. I first tried reproducing the problem in scratchbox, but I couldn’t even hit the bug after copying the sqlite databases for both erminig and calendar.

I ended up adding some well-placed prints and

import code; code.interact(local=locals())

statements on the phone’s file system so I could inspect the objects provoking the crash. It turns out there was a simple call somewhere that overwrites an event’s alarm time setting, but it does so with an int when the code in gdata expects a string.

Now, to submit the fix, I have the same problem as last time I submitted a fix. erminig-ng is a fork of erminig. erminig has a website that has no updates since 2008. The version number on that site is higher than the version number of erminig-ng, but erminig-ng is packaged as erminig, which is confusing for people. The bug tracker on maemo garage that the package for erminig-ng links to is only for erminig, and people get confused, so instead they’re reporting all their problems in a long maemo talk conversation. It looks like the maintainer for erminig-ng is the same as for erminig, so I’m not sure why this confusion is there. It looks like it would b easy to clear up. The code is not in any public repository as far as I can tell, and people are fixing things here and there with patch files in the maemo talk thread giving instructions on how to patch files on the root file system. A mess, really. It should probably just go on github somewhere, but I want to hear from the maintainer first.

So, I slurped the latest 0.2.12 release source into my svn repository, forked it to a trunk directory, and started integrating my patch. All .py files are in the root, I had to fight the urge to not clean it up properly and put it nicely in separate directories, because that would make merging patches back harder. I added a test case for the bug I was hitting as well – there were no tests so far in the package.

After my bug was fixed, and the test worked, I rebuilt a package for my phone. That led me to updating my maemo repository, adding gpg signing to it, adding metadata info to it, and doing everything needed to make it easy to update and not trigger any errors on the phone when upgrading. Now it looks like a nice proper maemo repository (and even apt-file should work) – but if any debhead wants to give it a looksee to see if anything’s wrong feel free!

So, now erminig 0.2.12.2 is available from my repository, works on my phone, and is now again properly syncing my calendar information. I dropped a line in the forum thread for a person who was probably running into the same bug as me.

Next on my radar – looking at barriosquare to see if an update to the foursquare v2 API is doable.

I felt so guilty for not doing enough maemo stuff over the last year to try and get an N950. Now I regret not applying – it’s going to be the last phone in the series (yes, I’m one of those who think Nokia is making a huge mistake), the focus of what’s left of Maemo will be on that phone, and I won’t be able to update the programs I use on Maemo, and my phone will slowly become obsolete. At which point I probably have no choice but to switch to Android, the non-Linux Linux system. I don’t know many people anymore who work for Nokia, but if any of them are listening and still want to help me get an N950, feel free to let me know.

And the kicker of my patchwork ? It turns out that, when I fixed a bug in the previous version, I had already forked off 0.2.11 into my repository at a different location. And in that checkout, I also added a unit test, a similar-but-different hack for not being able to load hildon as a module, and a HACKING and TODO file with very similar notes as today.

Hey, at least I’m consistent. And maybe I should throw this stuff on github after all, so I can use git’s branch powers to track the upstream releases, and the work I did that did not get taken upstream…

August 10, 2011 09:04 PM

One of the key concepts in my rewrite of DAD is that it should be possible to relate the same track across different files and computers. I have copies of files, and different encodings of the same track, spread across machines. Various applications I use for playback seem to exist in isolation on each machine, and so I tend to rate only occasionally knowing that my ratings aren’t centralized. And I get annoyed when banshee detects three copies of an album, and then orders them by track number, playing each track three times before moving on to the next one.

The logical way to do is is through acoustic fingerprinting. These are algorithms that extract certain features from an audio file and calculate an algorithm-specific ‘fingerprint’ for it. Usually, these fingerprints are not identical across different encodings of the same file, so you can’t look up twins in a list; but the fingerprints can be compared to each other and a ‘difference’ within a certain confidence interval calculated.

Most fingerprinting algorithms have a library that calculates a fingerprint and then submits it to a complimentary web service where it can quickly compare it to find twins.

In the past, either the client library/application or the web service (or both) was not open enough to be of interest for most Free Software people.

But recently, someone in the #morituri channel mentioned acoustid which only consists of open components. So, that seemed interesting enough to try out!

The chromaprint client-side library consists of a library, a sample application (linked against FFMPEG), and a python module with some sample scripts.

There is also a gst-chromaprint GStreamer plug-in on github. (As a side note, amazing to see that GStreamer plug-ins these days come for free! I recall the days when we had to the work ourselves to write GStreamer plug-ins for libraries)

So, after giving them a quick test run, I packaged up the whole set and it’s now available for Fedora 14 and 15 in my package repositories

The chromaprint-tools package contains fpcalc and you need to enable rpmfusion-nonfree to get its ffmpeg dependency.

And after that, I created a Task in DAD for chromaprint, and now I have:

$ dad analyze chromaprint /opt/davedina/audio/albums/Afghan\ Whigs\ -\ Gentlemen/Afghan\ Whigs\ -\ Debonair.ogg
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
/opt/davedina/audio/albums/Afghan Whigs - Gentlemen/Afghan Whigs - Debonair.ogg:
Found 1 results
- Found 4 recordings.
- musicbrainz id: 62b2952a-4605-4793-8b79-9f9745ea5da5
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: 8ff78e73-f8bd-4d78-b562-c3e939fb93fb
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: a0d5ced6-43e8-450a-bf11-94f1f4520b92
- artist: The Afghan Whigs
- title: Debonair
- musicbrainz id: d01ac720-874c-48d6-95c6-a2cb66f9d5d0
- artist: The Afghan Whigs
- title: Debonair

Sweet…

Now it’s time to dump that in the couchdb database backend, and start identifying duplicate tracks.

Acoustid seems to be a relatively young project, but its maintainer is very active on the mailing list and it’s filling a hole in the open world that I’m happy to see filled! Thank you Lukas.

August 09, 2011 07:55 PM

[root@ana ~]# rpm -Uhv /home/thomas/rpm/RPMS/x86_64/gstreamer011-*
Preparing... ########################################### [100%]
1:gstreamer011 ########################################### [ 33%]
2:gstreamer011-devel ########################################### [ 67%]
3:gstreamer011-debuginfo ########################################### [100%]

Sweet!

August 06, 2011 12:24 PM

Over the past few years I’ve been quietly exploring ideas for my ideal music application. When I lived together in that great house in Gent, we had a hacky set of PHP code that let us import music, rate it, and have it play back. It worked for our purposes, but it was a collection of hacky PHP code and hacky Perl code.

Now I’m not saying I got that much better at coding, but I’m sure I improved a little bit. I’ve always put off actually writing the damn code to replace it, and hence I have a bunch of separate music collections – the music I was listening to in that house (properly rated, but very outdated), random collections of downloads, and now the collection of CD’s I bought ever since leaving that house that never quite made it into my computer and are now being imported by the Lego robot.

Over a year ago, I re-implemented the mixing backend on top of GNonLin, which for the most part works as long as I don’t actually dereference tracks played – somethign to figure out at some point. I have ideas about a pure web-based mixing backend as well, but I need to learn modern stuff like JQuery first.

But the missing key really was something that handles the database part well enough, because my application should work distributed – it should manage my tracks on all my devices, including all my computers, and be able to figure out that some crappy mp3 of a song on my laptop is the same song as the flac version at home on my NAS. So if I rate that crappy mp3 on my laptop, I want that taken into account when my home machine creates a mix.

And for me, CouchDB promised to fill that niche. Except of course that I spent the last year figuring out how I can marry CouchDB’s approach to replication with my natural desire to denormalize. It turns out that’s possible with CouchDB, but it involves doing a lot of client-side caching (and invalidating/changing on change notifications) and is already pretty slow when I do it for my 14000 test tracks.

So, I’ve decided to experiment in a world where normalization is not needed, and I’m just going to pick one central concept (The ‘track’), store as much related data into that document as possible (on each computer, the fragments of audio files that represent that track; its ratings; what album it’s on; which artists made the track), treat some of those values as caches for the last known value from parent documents, and just go for speed first and see how that goes.

Yes, I am going to relax about not having everything perfect on the inside, so I can move on and write some more code that I can actually use.

I enjoyed a lot trying to shoehorn CouchDB into my relational wordview, but I want to see what life is like on the other side.

Before I was also very focused on migrating my old data (from the music I had when I was in the house in Gent) and its ratings. That’s still important to me, but I think right now I’d more enjoy having something that lets me listen to and rate new music. When I originally wrote DAD I didn’t expect to be getting so much music that wasn’t from CD’s. That’s obviously not the case anymore, and I’m probably one of the last maniacs still buying CD’s and worrying about getting them sample-perfect onto my NAS. In today’s reality I need to deal with having the same track fifteen times, in various qualities, and I wish my computer handled that for me.

As part of this shift in approach, both in how I use CouchDB and what music I now want to listen to, I’m going to build the code from the opposite side I’ve been doing, focusing on smaller building blocks and getting the experience right. Step one will be collecting the right data about audio files, splitting them into individual fragments, and loading music in two passes into the databases. I’ll focus on having small tools that show that the application can add tracks quickly and start playing them, filling in the more costly information later, and show that the GUI frontend can update these in realtime in the database view.

And, as usual, I like to shoehorn in a use for my python command class, so I’ll be using that as a collection point for these little tools as I work my way up.

After plugging in the right plumbing, in twenty minutes I had this on top of my old code:

$ dad analyze level /mnt/nas/media/davedina/audio/albums/Nirvana\ -\ In\ Utero/Nirvana\ -\ All\ Apologies.ogg
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
Successfully analyzed file /mnt/nas/media/davedina/audio/albums/Nirvana - In Utero/Nirvana - All Apologies.ogg.
2 fragment(s)
- fragment 0: 0:00:00.000000000 - 0:03:50.230204081
- peak 0.240 dB (105.672 %)
- rms -14.199868248342282 dB
- peak rms -8.913940439528652 dB
- 95 percentile rms -12.001385041642244 dB
- weighted rms -14.202287606952533 dB
- weighted from 0:00:01.205986394 to 0:03:39.612879818
- fragment 1: 0:23:59.107482993 - 0:31:32.227482993
- peak 0.526 dB (112.876 %)
- rms -14.742109190444983 dB
- peak rms -8.729096757819718 dB
- 95 percentile rms -11.56951163744373 dB
- weighted rms -14.742603253857133 dB
- weighted from 0:23:59.223582765 to 0:31:18.498684807

In case you were wondering, this shows the code correctly determining that the ‘All Apologies’ track on the In Utero CD contains in fact two songs. It always annoys the hell out of me when any of the music players I use doesn’t play anything for 20 minutes just because Kurdt thought that would be amusing all those years ago.

(In case you were really astute, you may have noticed that this code claims that the peak of these fragments is over unity, which would be weird and wrong you would think. Monty could give you a long and interesting explanation on how that is in fact natural and every time I read it I still don’t get it, even with my audio engineering background, and I still don’t know if this apparent peak level is a bad thing, but in practice my playback code auto-levels anyway and consistently reduces volume on tracks, so I don’t think it matters anyway…)

August 06, 2011 12:07 AM

This week at work we ran into a problem where one of our Python processes was consuming close to 3 GB of memory because it’s not properly cleaning up a list. Because of other bugs this process could not be easily restarted without triggering other problems, so our core team asked for some suggestions and I told them “Why don’t you try cleaning up the Python list using GDB and the Python C API ?” I had a vague recollection of someone on our team doing something like this a few years ago.

I also asked them to blog about it, because there aren’t that many resources readily findable on the subject.

So here is Andoni’s take on the problem.

If any Pythonista can suggest how he could have avoided the segfault during garbage collection, please let us know!

August 05, 2011 10:47 AM

August 04, 2011 07:02 AM

July 30, 2011 10:47 PM

July 20, 2011 07:11 AM