planet.freedesktop.org

Dave Airlie: initial kms for radeon r500 atombios pushed

Fri, 25 Jul 2008 23:08:42 +0000

AMD today announced the source code to that atom parser from their kgrids project, I've been playing with this code for a few weeks to see if we could use it for kernel modesetting (kms).

So I've just pushed my initial radeon kms code to the "gem-modesetting" branch of the drm git tree.
The corresponding DDX code is in the radeon-gem-ms branch of my personal xf86-video-ati repo.

This contains 2 major things - and doesn't contain lots of other major things:

1. GEM style API to userspace layered on top of TTM internals.
- Only really does pinned allocations - no superioctl support yet, working with Jerome on tying that down.

2. atombios parser from AMD + modesetting code.
- This only support rs690 and r500 devices so far. We don't have r600 code in the drm tree properly yet,
and legacy bios is TODO.

So the major TODOs are:

add support for modesetting on legacy cards (r100/r200/r300/r400) - ported from -ati driver. We need a fixed point
implementation of the bandwidth calculation code.

proper command submission - Jerome is looking into this and I think we have a solid plan. for now I'm just abusing the old
style command submission routines with GEM ioctl.

3D - cmdbuf work on the Mesa drivers needed, RH has some in-progress code.

EXA pixmaps - still using one big EXA offscreen area, probably need to move to EXA pixmaps for DRI2 and also to use the superioctl properly.

The Irregular Radeon Development Companion: new ATOM parser

Fri, 25 Jul 2008 22:17:17 +0000

Alex Deucher (agd5f) just announced the following on the X.Org mailing lists:


AMD is pleased to release a new atom parser which is more suitable for
inclusion in the Linux kernel.
You can download the new ATOM parser here:
http://www.x.org/docs/AMD/kgrids-atom.zip

Enjoy!

Alex Deucher

So what does this mean? This means that AMD is working on getting drm modesetting (also refered as kernel modesetting) to work with Atom-BIOS based cards (R500+). This is great news since the original Atom-BIOS parser (currently used in xf86-video-ati and the atom-support branch of xf86-video-radeonhd) was never targeted for inclusion in the Linux kernel.

Right now you can't really test the code. You got to wait until it is picked up either by the DDX drivers (xf86-video-ati or xf86-video-radeonhd) or by the DRM modesetting approaches in progress for radeon right now. It's most likely that the parser will no be picked up within the next few days but within a short timeframe.

Lennart Poettering: String Pools

Fri, 25 Jul 2008 21:32:00 +0000

In part 2.4.3 of Ulrich Drepper's excellent How To Write Shared Libraries (which unfortunately is a bit out-of-date these days) Ulrich suggests replacing arrays of constant strings by a single concatenated string plus an index lookup table, to avoid unnecessary relocations during startup of ELF programs. Maintaining this string pool is however troublesome, it is hard to read and difficult to edit. In appendix B Ulrich lists an example C excerpt which contains some code for simplifying the maintaining of such strings pools, after an idea from Bruno Haible. In my opinion however that suggestion is not that much simpler, and requires splitting off the actual strings into a seperate source file. Ugly!

Some Free Software uses string pools to speed up relocation, e.g. GTK+. Some development tools like gperf contain support for string pools.

All solutions for string pool maintaining I could find on the Internet were not exactly beautiful. Either they were completely manual, manual plus a validity checking tool, or very very cumbersome. Googling around I was unable to find a satisfactory tool for this purpose^[1].

After Diego Petteno complained about my heavy use of arrays of constant strings in libatasmart I sat down to change the situation, and wrote strpool.c, a simple parser for a very, very minimal subset of C, written in plain ANSI C. It looks for two special comment markers /* %STRINGPOOLSTART% */ and /* %STRINGPOOLSTOP% */, moves all immediate strings between those markers into a common string pool and rewrites the input with the strings replaced by indexes. Code accessing those strings must use the special _P() macro. With these minimal changes to a source file, passing it through strpool.c will automatically rewrite it to a string-poolized version. The nice thing about this is that the necessary changes in the source are minimal, and the code stays compilable with and without passing it through the strpool.c preprocessor.

Here's an example. First the original non-string-poolized version:

static const char* const table[] = {
	"waldo",
	"uxknurz",
	"foobar",
	"fubar"
};

static int main(int argc, char* argv[]) {
	printf("%s\n", table[2]);
	return 1;
}

For later use with strpool.c we change this like this:

#ifndef STRPOOL
#define _P(x) x
#endif

/* %STRINGPOOLSTART% */
static const char* const table[] = {
	"waldo",
	"uxknurz",
	"foobar",
	"fubar"
};
/* %STRINGPOOLSTOP% */

static int main(int argc, char* argv[]) {
	printf("%s\n", _P(table[2]));
	return 1;
}

When passed through strpool.c this will be rewritten as:

/* Saved 3 relocations, saved 0 strings (0 b) due to suffix compression. */
static const char _strpool_[] =
	"waldo\0"
	"uxknurz\0"
	"foobar\0"
	"fubar\0";
#ifndef STRPOOL
#define STRPOOL
#endif
#ifndef _P
#define _P(x) (_strpool_ + ((x) - (const char*) 1))
#endif

#ifndef STRPOOL
#define _P(x) x
#endif

/* %STRINGPOOLSTART% */
static const char* const table[] = {
	((const char*) 1),
	((const char*) 7),
	((const char*) 15),
	((const char*) 22)
};
/* %STRINGPOOLSTOP% */

static int main(int argc, char* argv[]) {
	printf("%s\n", _P(table[2]));
	return 1;
}

All three versions can be compiled directly with gcc. However, the version that was passed through strpool.c compresses the number of relocations for the table array from 4 to 1. Which isn't much of a difference, but the larger your tables are the more relevant the difference in the number of necessary relocations gets.

A more realistic example is atasmart.c which after being preprocessed with strpool.c looks like this. In this specific example the number of necessary startup relocations goes down from > 100 to 9.

I am note sure if the parser is 100% correct, but it works fine with all sources I tried. It even does suffix compression like gcc does for normal strings too.

Footnotes

[1] Or maybe I just suck in googling? Anyone has a suggestion for such a tool?

Christian Schaller: Back from Turkey

Fri, 25 Jul 2008 15:35:18 +0000

So a little later than ‘everybody else’ I am now back from Turkey. Wim and I had both decided we needed some vacation and since we where going to Turkey for GUADEC it would be a great opportunity for us to travel around and see the area. Our original plan included climbing Mount Arrarat and visiting Georgia and Armenia, but trouble getting climbing permit combined with some german tourists getting kidnapped on Mount Arrarat by Kurd sepratists got us to adjust our plans. We ended up instead travelling down to cities such as Urfa (Edessa) and Harran, spending some days in Cappadocia and finally visiting Pamukkale and Ephesos. Had a blast of a time although Wim seemed a little less enthusiastic about females of the Korean and Taiwanese variety than me. Could of course just be that ‘Hello Kitty’ panties fail to get his sap boiling

For anyone travelling to Turkey I think spending 2-3 days in Göreme is an absolute must. It is a charming backpacker town with a lot of the hotels carved out of volcanic rock. A little gimmicky in the sense that the locals no longer live in the rocks for the most part, but the area is something which just have to be experienced.

Although we enjoyed the food I think it is safe to say that neither myself or Wim will be eating salad with cucumber and tomato, sprinkled with a little lemon anytime soon. 10+ days in a row is enough for a while

One thing I started wondering about while travelling around was the enormous expectations by local people for what an EU membership would mean for the people of Turkey, ranging from the EU solving all minority rights issues for Kurds, Armenians and Alavis, west European living standards and salaries for everyone, to solidifying Turkeys secular traditions. While the EU will for certain help push Turkey in the right direction on these issues, I can’t help but wonder if a future EU membership for Turkey might end up being a big lettdown compared to the expectation level I experienced.

Dave Airlie: gdbing the X server with xkb ... finally..

Fri, 25 Jul 2008 03:42:33 +0000

So for a long time if you ran X inside gdb, gdb would die badly when X forked xkbcomp. X hackers have worked around this by either running -kb or attaching gdb after X is started for many years.

I finally found it by accident this morning, I was working on some modesetting things, and had been gdb'ing the X server fine, when suddenly it started doing the wierd hang, I realised I'd just enabled direct rendering code and initialised the lock.

Now the kernel has this really really really really dirty hack for the DRM that I'm not 100% sure we really need anymore. The original drm designers decided that if a DRI client held the lock then if it died it might crash the GPU, so they got this special signal blocking code into the Linux kernel which we use to block certains signals across lock hold. When the lock drops, the process gets the signal. Of course one of these signals was one gdb was using to stop the X server and the X server always holds the lock when running so it got confused when an blockable signals somehow managed to block itself :)

Now the fixes for this are

a) kill the notifier stuff, I'm not sure it ever mattered, I at least could kill it for modesetting drivers as we have a lot more
control over the GPU from the kernel.
b) Have the X server drop the DRI lock around certain operations like xkb execing xkbcomp etc.. this seems worse.

What I've done for now in drm git is just not set the notifier up for the "master" process, i.e. the X server. This will at least allow people to debug their X servers with xkb.

John (J5) Palmieri: Interesting news out of the Intel camp

Thu, 24 Jul 2008 20:48:11 +0000

It seems that Moblin will be switching from Ubuntu to Fedora Linux as their base operating system. I’m interested in finding out the underlying reasoning for such a move. The stated reason is because they wanted to use RPM instead of DEB. I can’t quite buy that but perhaps that is because having come from both camps I think that packaging is an implementation detail that too many people put way too much stock in. This has the effect of causing unnecessary emotional splits within the community resulting in animosity which often overshadows real threats.

The second reason given, which has to do with building a community is pretty broad but more believable. Fedora has made huge strides while also sticking to its guns in the freedom department and being valuable upstream contributors. It may be that we sacrificed short term gains which can be gotten via a bit more differentiation, or out of the box “just works” on closed hardware but as companies are being convinced to open up their specs and open drivers are being written, a large portion of which is being done by Fedora developers working upstream, little of the short term gains matter much.

I suspect the real reason is somewhere in the community vein, staring with the Kernel and X team developers who work tirelessly making sure their work is fit for upstream consumption and can be supported in the long term. Following their lead the rest falls naturally out of that single notion of moving Linux forward as a whole. Kudo’s to all my Fedora friends - keep moving forward.

[read this post in: ar de es fr it ja ko pt ru zh-CN ]

Thomas Vander Stichele: GTK question

Thu, 24 Jul 2008 16:23:30 +0000

We have a lot of text labels in Flumotion where we put mid-sentence newlines just so that once in a while the line gets broken and doesn’t push the containing widget out wider than we want it to be.

Obviously this is a bogus way of doing things, but even our resident GTK+ wizard, Johan Dahlin, didn’t have a ready answer for what the right way to do this is. So - how should we do it so that text gets flown correctly without mid-paragraph newlines and without ultra-wide widgets ?

The answer is probably superobvious.

Lennart Poettering: PulseAudio 0.9.11 released

Thu, 24 Jul 2008 13:32:00 +0000

I just relased PulseAudio 0.9.11.

It's an awesome release. To learn more about why, read the linked email, and this and maybe this blog story.

Bastien Nocera: Totem article: "Movie Magic"

Thu, 24 Jul 2008 13:23:00 +0000

Philip will be happy. The author of this article for Linux Magazine UK is full of praise for the YouTube plugin. Good work Philip!

Peter Hutterer: Input configuration in a nutshell.

Thu, 24 Jul 2008 10:30:28 +0000

Once upon a time, we had a number of free operating systems. Linux, BSDs, emacs, XFree86, you name it. Then one of them chose to be less like one and become a windowing system instead (which was supposed to it's job from the beginning, albeit it kinda forgot about it halfway there). The principle of DTRT was embraced and the almighty xorg.conf file was given the handshake and shown the door. These days, X starts without a config file these days, and everything just works.

Most of the time. When it doesn't it's usually ugly. So here's a guide on how to configure input devices with recent servers. This guide includes

a quick overview of the evdev input driver
how to set up input hotplugging.
what to do in the xorg.conf

First, if you're not running Linux you can stop reading. Keep your config file, make sure you reference your input devices in the ServerLayout section, and start porting the evdev kernel module to your OS of choice.

A quick overview of the evdev input driver
xf86-input-evdev is the hotplugging-capable input driver to be used for mice and keyboards (and other devices, if appropriate). One main difference is that it uses devices files such as /dev/input/event0 instead of /dev/input/mouse0 (like the mouse driver does for example). The other big difference is that it can query the kernel about the capabilities about each device and set up itself correctly. So if you point a device to /dev/input/event0, it will automatically set itself up as mouse or keyboard, as appropriate.
The second important thing is that it puts a grab on the device file (not the same as a pointer/keyboard grab in X!). This grab gives us two benefits: the device will not send events to the collector devices (e.g. /dev/input/mice) anymore. And if anyone else tries to grab the device they will fail. So we can ensure that only one process receives events from this device.

What does all this mean to us now?
We can change our mice and keyboards in the xorg.conf to use the evdev driver instead. As device, we can either specify /dev/input/eventX, but these numbers may change on reboot. Look into the /dev/input/by-id and /dev/input/by-path directory. I recommend to always use /dev/input/by-id/MyMouseName-event-mouse as device instead of the eventX files. Note that you'll find MyMouseName-mouse and MyMouseName-event-mouse, of which only the latter will work with evdev. Ignore the former.
The big benefit is simply that you can configure multiple devices quite easily now. Simply set up InputDevice sections for each device and the X server will treat them as different devices. This is quite useful if you're using MPX.

How to set up input hotplugging
Writing xorg.conf sections is not only boring, it's also quite pointless in most cases. So let's let the server find input devices automagically. The two ingredients are HAL and DBus. At startup, the server queries HAL over DBus for a list of devices and adds them one-by-one.

All you need to do is configure HAL so the server adds the devices with the right options.
You must merge the key input.x11_driver. Look at x11-input.fdi. This file should reside in /etc/hal/fdi/policy/, and it tells HAL to merge the evdev driver for each mouse and keyboard It also merges some Xkb options for keyboards.
Important: the server will not add devices unless they have input.x11_driver set. If you end up with no devices at all, then your HAL setup is probably broken.

To test your installation, run hal-device to obtain a list of all devices and look for the input.x11* keys. If they are not present, modify the fdi file and restart HAL.

Your X server is now hot-plug capable. It will add all mice and keyboards automatically, even if you add/remove them at runtime. There's modified fdi files floating around to add hotplug for wacom tablets and synaptics touchpad, finding them is left as an exercise for the reader.

What to do with the xorg.conf
We have two configuration files now. Yay. One is xorg.conf, the other one is the fdi file. And they both don't know about each other, leading to interesting results. The server parses xorg.conf first, then gets the devices through HAL. And here we have a big culprit: if add a InputDevice section with the mouse driver, it will conflict with the evdev driver. Evdev grabs the files, remember? So when all mice are added through the HAL evdev hotplugging mechanism, all mice stop sending to the /dev/input/mice file. So although you still have a valid input device in the server, it will never emit any events.
The only solution here? Either stop hotplugging alltogether by adding Option "AutoAddDevices" "off" in your ServerLayout, or modify the fdi file that it doesn't merge input.x11_driver for the specific mice you don't want hotplugged (or all mice).

Troubleshooting list:

I have no input devices and compiled the server myself. Install libhal-dev and libdbus-dev, re-run configure and recompile.
I have no input devices. HAL setup needs to be configured. Make sure the x11-input.fdi file is in /etc/hal/fdi/policy, and that hal-device lists mice and keyboards with input.x11_driver = "evdev".
My xorg.conf settings are ignored. HAL adds your devices without settings, and evdev grabs all devices away. Modify the fdi and add your settings there, or add Option "AutoAddDevices" "off" to your ServerLayout. Note that this will stop input hotplugging.
I'm getting a "Grab failed" in the logs. You have two evdev input devices pointing to the same device file. This usually only happens if you have a xorg.conf entry with an evdev, and then HAL tries to add the same device again. You can ignore this warning.
I have devices in my xorg.conf but still don't get any devices. Your ServerLayout doesn't reference the input devices and your HAL setup is broken. Add Option "AllowEmptyInput" "off" to your ServerLayout or simply reference the InputDevice sections in the ServerLayout. While you're at it, you may also want to configure HAL.
My touchpad is slow. Check the logs if the touchpad is added with the evdev driver. If so, modify the fdi file to merge synaptics as input.x11_driver, or add a InputDevice section in your xorg.conf (must be referenced by the ServerLayout).
Evdev doesn't support option "XYZ". Add a feature request to X.Org Bug 16699. Bugs with patches get preference.

Simos Xenitellis: Layout editor keyboard

Thu, 24 Jul 2008 06:42:31 +0000

This is a screenshot of the keyboard for the layout editor. The keyboard is a widget which is composed of individual widgets of each key.

I did not use glade-3 for the keyboard at this time. Although it is possible to create custom widgets in Python and install them in Glade, the current distributed packages are missing something, thus it would be messy when others try to use the editor. It’s a good experience to do all by hand anyway.

When creating a layout, you drag and drop characters on the keyboard. The editor shows a table with characters though it would be possible to drag characters from gucharmap as well.

The next step is to get an intuitive UI so that when you drop a character on a key, the key expands (a popup appears) showing the available four positions to receive the character.

Daniel Stone: some metrics

Thu, 24 Jul 2008 00:15:18 +0000

Make of them what you will.

daniels@psyence:~/x/xorg/xserver% wc -l **/*.[ch] | tail -1 730420 total daniels@psyence:~/x/xorg/xserver% git diff -p xorg-server-0_99_1.. | diffstat | tail -1 2747 files changed, 178062 insertions(+), 628051 deletions(-) daniels@psyence:~/x/xorg/xserver% echo $((628051-178062)) 449989

David Zeuthen: Linux Plumbers Conference CFP extended!

Wed, 23 Jul 2008 17:46:37 +0000

I rarely blog these days, doing the Twitter thing instead and all. Anyway. The Call for Papers for the Linux Plumbers Conference in September in lovely Portland, Oregon has been extended until July 31st 2008. It’s a conference about the core infrastructure of Linux systems: the part of the system where userspace and the kernel interface. It’s the first conference where the focus is specifically on getting together the kernel people who work on the userspace interfaces and the userspace people who have to deal with kernel interfaces. It’s supposed to be a place where all the people doing infrastructure work sit down and talk, so that each other understands better what the requirements and needs of the other are, and where we can work towards fixing the major problems we currently have with our lower-level APIs.

I’m running the “Desktop Integration” microconf at Plumbers. I have two goals for the microconf. One is to be informative about what’s going on in the dusty hallways between the Kernel and the Free Desktop. Which I think is important at a place like Plumbers where we’re going to a lot of people working on similar problems present. The other goal is to actually try and make some headway on actual problems that require people from multiple communities working together. Such as some of the work Jon is focusing on, e.g. how to make fast-user-switching, multi-seat and terminal services Just Work(tm).

The conference is expected to sell out pretty quickly so it’s a good idea to go ahead and register for Plumbers Conference instead of procrastinating about doing it!

See you in Portland!

Adam Jackson: Words, words, words

Wed, 23 Jul 2008 14:34:49 +0000

The Fedora 10 name elections are open. My totally awesome name suggestion for F9 (Bathysphere) lost by a slim margin, so now I'm lobbying harder for decent names. So here's what you should vote for:

Whiskey Run, because whiskey is awesome, and the potential for a whiskeyrunner theme in the artwork is excellent.

Saltpetre, because it's a double-whammy connection with Sulphur (British variant spelling, and component of gunpowder), and gunpowder -> steampunk art -> awesome.

Farnsworth, because Futurama.

Cambridge, because Red Hat Linux 10 would have been Cambridge, and we finally made it to version 10 of something.

Every time you vote for something else, God detonates a kitten.

Lennart Poettering: Linux Plumbers Conference CFP Extended!

Wed, 23 Jul 2008 11:39:00 +0000

The Call for Papers for the Linux Plumbers Conference in September in Portland, Oregon has been extended until July 31st 2008. It's a conference about the core infrastructure of Linux systems: the part of the system where userspace and the kernel interface. It's the first conference where the focus is specifically on getting together the kernel people who work on the userspace interfaces and the userspace people who have to deal with kernel interfaces. It's supposed to be a place where all the people doing infrastructure work sit down and talk, so that each other understands better what the requirements and needs of the other are, and where we can work towards fixing the major problems we currently have with our lower-level APIs.

I am running the Audio microconf of the Plumbers Conference. Audio infrastructure on Linux is still heavily fragmented. Pro, desktop and embedded worlds are almost completely seperate worlds. While we have quite good driver support the user experience is far from perfect, mostly due because our infrastructure is so balkanized. Join us at the Plumbers Conference and help to fix this! If you are doing audio infrastructure work on Linux, make sure to attend and submit a paper!

See you in Portland!

Bastien Nocera: Better late than never

Wed, 23 Jul 2008 10:01:00 +0000

Back in the day, I used my paycheck to buy the top-notch MP3 player that was the Rio500. Unfortunately, I forgot it in the back pocket on my plane seat when flying over to Raleigh for my Red Hat induction. And then I used one of my first new job paycheck to buy a second generation (and very very expensive) iPod.

You could say that hacking on Walk500, a front-end to that great player is the reason why I'm hacking on GNOME these days.

I couldn't bear the thought anymore, and bought a Rio500 on eBay for $5. Hacking on it half-an-hour at a time, I cleaned up the code. The latest release of the modern era is available on SourceForge.

Matthias Hopf: Finger(s) Crossed

Wed, 23 Jul 2008 09:57:47 +0000

If the doctor is content with the situation the metal in my finger is removed today. Then I'll be able to type normally again, yippee! Ok, after some painful weeks stretching and strengthening the muscles... ;-)

Wish me luck.

Thomas Vander Stichele: I was almost home scot-free

Tue, 22 Jul 2008 21:17:19 +0000

Don’t know anymore how I landed on Five Geek Social Fallacies but I was feeling good about myself not feeling very afflicted with any of them. That is, until I hit number 5.

Yes, I admit to having bouts of ‘oh-you-didn’t-invite-me-what’s-wrong-with-us’, and its inverse, ‘oh-my-god-there-are-not-enough-chairs-for-all-the-people-I-should-be-inviting-I-will-end-up-on-a-lot-of-no-invite-lists’.

If you saw me in either mode at some point in life feel free to let me know if you notice any improvement at all on this front.

Jesse Barnes: Waiting for vblank...

Tue, 22 Jul 2008 19:29:11 +0000

Will the vblank-rework branch ever be ready to merge upstream? With some recent work by Michel and myself, I really hope so.

It’s a little distressing that such a simple thing could cause so much trouble. The motivation for reworking vblank in the DRM branch was easy enough: save CPU power by turning off interrupts when possible. Not interrupting the CPU lets it sleep deeper and longer, potentially saving quite a bit of power. Getting rid of the 60 or so vertical blank event interrupts per second when they weren’t needed seemed like a logical first step, and so vblank-rework was born.

Being a good citizen in Linux land often means improving whole subsystems rather than stuffing a bunch of fancy features into individual drivers. Working that way can be harder, but it spreads the benefits wider, and improves Linux as a whole. So my efforts in the vblank-rework branch targetting the generic DRM vblank code, improving the driver APIs and making sure that all drivers could benefit from the new infrastructure, allowing them to disable interrupts when not needed. However, that’s where the “harder” part comes in: every driver needed an update. At first I thought, “Hey this is a neat, new set of APIs, surely everyone will want to use them, I’ll just convert the i915 driver (after some initial work on a radeon based laptop).". Unfortunately, the DRM drivers are in dire need of attention, so after several months of waiting I ended up converting all the drivers myself, though only a few like radeon and i915 actually implement the API fully enough to disable interrupts.

So far, so good. I figured everything was fine and the shiny new branch was ready, so I merged it into the master DRM branch in preparation for an upstream push. Then tragedy struck: Michel’s sharp eyes and testing discovered a potentially fatal bug. While many GPUs provide a frame count register we can use to keep the vblank count accurate (necessary since OpenGL extensions expose an absolute count to applications for some reason), they’re typically only updated at the leading edge of vactive. This means that your application may wakeup at vactive time instead of vblank time, causing ugly tearing; exactly what you’d like to avoid! After a bit of back and forth and a couple of false starts (trying to work around the problem with solutions that turned out to be racy), we decided to go back to using the atomic counter (which is only updated at interrupt time) for wakeups, rather than the hw register, using the latter for keeping the counter accurate across interrupt disable periods.

Which brings us to last week. I hacked up the scheme described above and started testing. As I found and fixed bugs (well actually Michel probably found most of them), I discovered that the API could actually be simplified a bit, and some of the code to compensate for corner cases was no longer necessary, so both the wraparound compensation logic in the pre/post modeset ioctl and the funky accounting we tried to do there could be removed. The result, I hope, is ready for upstream finally.

So where does that leave us, API-wise? Well, on the userland front we have a new ioctl, DRM_IOCTL_MODESET_CTL, with _DRM_PRE_MODESET and _DRM_POST_MODESET arguments. It should be called with _DRM_PRE_MODESET in userland drivers prior to any activity that resets the hw frame counter (typically mode setting). When the mode set completes, it should be called again with _DRM_POST_MODESET. These calls tell the kernel to account for any lost events so that the vblank count exposed to applications can stay accurate.

On the driver front, there are a few different calls and callbacks:

drm_vblank_get - increase the refcount on the vblank counter

This call just tells the core code that the caller is actively using the vblank counter for something, e.g. scheduled buffer swaps or a blocking vblank wait call.
drm_vblank_put - decrease the refcount

Tell the core you’re done with the vblank counter. When the refcount reaches 0, the kernel knows it can disable the interrupt at some point in the future.
drm_vblank_init - initialize the core vblank code

Should be called at driver load time or IRQ init ioctl time to init the core.
driver.get_vblank_counter - return the current hw frame count

Used by the core code to keep the count accurate across interrupt enable/disable periods.
driver.enable_vblank - enable vblank interrupts on a given CRTC

Used by the core to enable interrupts when the refcount increases.
driver.disable_vblank - disable vblank interrupts on a given CRTC

Used by the core to disable interrupts after a timeout period if the refcount is 0.

With a few simple changes, a given DRM driver can support the new scheme to save power. If you find bugs or have issues with the new APIs, let me know and/or file a bug at bugs.freedesktop.org.

Ian Romanick: Love the media

Tue, 22 Jul 2008 19:15:45 +0000

I guess I must be awesomely famous or something (probably "or something"). My new job has already been mentioned on Phoronix.com. Unfortunately, the work that I will be doing to parallelize Mesa won't be for anything as glamorous as Larrabee.

In fact, it will be for the complete opposite end of the spectrum. We're seeing more and more chips, like Intel's own Atom, that trade wide, out-of-order pipelines for simpler, in-order cores with multiple threads per core. On those types of architectures, especially when hardware TNL is not available, Mesa's performance is really, really bad.

There is room for a lot of improvement even on more traditional systems with hardware TNL and processors with good single-threaded performance. It is easy to find apps where lot of time is spent in the app, a lot of time is spent in the driver, and only one CPU is utilized. Sure, the app developer could multi-thread the app, but why? That means multi-threaded optimization needs to be done in N places (for N apps) instead of just one.

Anyway...I guess I should get back to work now...

Oliver McFadden: To Hong Kong and Beyond.

Tue, 22 Jul 2008 14:22:21 +0000

We arrive in Hong Kong about a day ago, after a insanely long flight. I don't know whether it's just Air New Zealand who provides all of 4" of leg room on Economy flights, or all airlines... It really sucks. I feel sorry for those who were continuing on to London Heathrow without any break.

I didn't manage to sleep on the flight so I was only barely awake during the morning, but we still had a good look around some of the shops and malls. We didn't have time to buy anything yet; the place is massive.

I'll post some photos later tonight, or if not, they will be on Steve's Live Journal: http://dragonflamezor.livejournal.com/

Thomas Vander Stichele: modern life breaks down

Tue, 22 Jul 2008 09:32:10 +0000

when after 10 minutes of brushing your teeth and waiting for the short judder to indicate that you can stop brushing, you realize you’re not actually holding an electric toothbrush.

I guess I don’t need to brush my teeth for the next 3 days.

Keith Packard: X output status july 2008

Tue, 22 Jul 2008 01:25:43 +0000

Ok, so I didn’t get a lot of time for coding last week. And, this week there’s OSCON, so coding time will be short again. I figured I should spend some time writing up a brief report about where X output is at today.

Output Hotplug

I think this stuff is fairly solid these days, although we don’t have much in the way of auto-detection of monitor connect/disconnect. There are two reasons here:

The hardware notifies the operating system via an interrupt. Given mode setting code in user space, dealing with interrupts is a huge pain and hence hasn’t been hooked up yet (see below).
Analog outputs (VGA, TV) do detection using impedance changes in the output signal path. This means we have to keep them active if we want to detect a connection. That takes a lot of power (about 1W to light up the VGA output without a monitor connected). What we could do is detect when a monitor was unplugged; that’s free.

There are a few other random improvements that are coming soon, like CEA additions to the EDID parsing code. These are additional data blocks that follow the standard EDID data and are used for ‘consumer electronics’ devices. Supporting these should make more HDMI monitors ‘just work’.

Initial Mode Selection

Detecting connected monitors is fine, but one thing we haven’t really solved is what to do when you have more than one connected when the server starts. My initial code would pick one ‘primary’ monitor, light that up at its preferred size and then pick modes for the other monitors which were as close as possible to the primary monitor size without being larger. Obviously, I liked that as it meant my laptop always came up looking correct on the LVDS and my external VGA would show most of the screen.

However, this was reported to confuse a lot of users. I can imagine that starting the X server with one of the outputs connected but not turned on would make for some ‘interesting’ support calls. So, now the X server picks a mode which all outputs can support and uses that everywhere. Sadly, this means that my laptop panel gets some random scaled mode (usually 1024x768) which looks quite awful.

I think we need something better than either of these choices, but I’m not quite sure how it should work.

Kernel Mode Setting

A bunch of people, including Jesse Barnes and Dave Airlie, have been hacking to move the output configuration code into the kernel. This will solve lots of little problems, like how to display kernel panic messages, and how to deal with interrupts for output hotplug.

This code is up and running fairly well these days, but depends on a kernel memory manager to deal with frame buffers. The integration of GEM into the kernel is blocking this work, but I’m hopeful that this will be sorted out in the next couple of weeks.

GEM — the Graphics Execution Manager

Work here was stalled for a few weeks while we sorted out memory channel interleaving issues. Now things are moving again, and we’re working on getting it stable enough to merge into master. That means fixing a few more critical bugs that the Intel QA team has identified.

One of these bugs is that our GL conformance tests weren’t working right; that turned out to be caused by tests reading back data from the frame buffer one pixel at a time. Our read-back path passed through the GEM memory domain code to pull objects back from GTT space to CPU space. That meant flushing the front, back and depth buffers from the CPU cache. With each of those at 16MB, reading a single pixel took long enough that the tests would time-out. Increasing the timeouts to ‘way too long’ is making them run, but tests which would complete in a few hours are now taking days.

We’ve got two different plans for fixing the read-back path:

Use pread to access precisely the data we need. This would involve flushing a single cache line for the tests above.
Mapping the back buffer through the GTT. This would eliminate the need to clflush anything as the GTT mappings are write combining and so reads bypass the cache.

Eric is working on the former, and I’m working on the latter. More news later (this week?) when we see which one wins.

Composite Acceleration

With Owen Taylor’s change to the glyph management code in the server, Eric and Carl were able to change the driver to batch multiple glyph drawing operations into a command buffer. Once Carl had this working, we went from 13000 glyphs/sec to 103000 glyphs/sec. Obviously we’re hoping for even larger improvements as a pure software solution is well over 1 million glyphs/sec. Even still, 103000 glyphs/sec is enough to make my desktop vastly more usable, and using the software path means losing a lot of other useful acceleration.

DRI2 — Redirected Direct Rendering

Right now, direct rendered GL applications (which is the fastest way we can do GL at present) get drawn to a giant screen-sized back buffer and then copied from there to the screen at swap buffers time. Because everyone shares the same back buffer, you get to clip your drawing as if you were drawing directly to the screen. While this normally doesn’t matter much (aside from some performance costs associated with lots of clip rectangles), when you’re running a compositing manager (like compiz), the 3D applications end up ignoring the per-window offscreen pixmap and spam their output directly to the real frame buffer.

DRI2, written by Kristen Høgsberg, solves this by changing how direct rendering works and giving everyone a private back-buffer to draw to. Now, at buffer swap time, that private back-buffer can be copied to the window’s pixmap and compiz is happy.

This work has been around for a few months, but depends on a TTM-based memory manager. That dependency isn’t very strong, and krh has promised to fix it shortly. Once that’s done, getting the GEM driver to support DRI2 won’t take long, and we’ll have our fully composited desktop running. With luck, that’ll happen before September.

Final Words

As you can see, we’re nearing the end of our long X output rework saga, with most of the pieces falling into place in the next month or two.

Alberto Ruiz: Back from GUADEC '08, back to Dublin

Mon, 21 Jul 2008 12:12:55 +0000

Okay, it's been a week already, but it was not only the back from GUADEC for me, but coming back to Dublin at the Sun offices as a full time employee.

GUADEC has been great time, as usual, I would like to thanks Baris and the whole Istanbul team for their excellent job.

Others have summarized the contents pretty well, so I won't go through that. However, which is been pretty clear to me is that this GUADEC has been an interesting start point towards leadership strength on the community.

First, Stormy Peters as our Executive Director, this is probably the best decision that the board has made since I've got involved in GNOME, I'm pretty sure she's going to make a difference in the project's direction and focus, plus getting in touch with other parties to make them embrace our community and technology more and better than before. Go Stormy!
Second, Gtk+ 3.0, okay, a lot of people have complained about certain details about the plan, however, we have a plan that can be discussed, and we have people committed to accomplish it, that is way far better than no having plan and no people at all. Big kudos to the Imendio guys and other Gtk+ heads for pushing this discussion further.
And third, the GNOME release team for putting the 3.0 release with a better long term strategy for the desktop in the table. Again another leadership show that I highly appreciate.

My talks went pretty well, however, the attendance to both the APOC and the Gtk+ one were pretty low since unfortunately both of them were on the after hours track. It was specially sad that no core Gtk+ hackers were around for the last one, I might extend the topic a little further and even propose a keynote next year as suggested by some Igalians (which were pretty much the 90% of the audience :-( ).

And last but not least, the board have decided to go for Gran Canaria as next years' GUADEC, I'm pretty thankful to the board for trusting our proposal and I'll make sure that next year is at least as good as the previous ones. If you have any suggestions, feedback or if you want to volunteer for helping, don't hesitate to give us a shout at the guadec's mailing list.

Sergey V. Udaltsov: Misc

Mon, 21 Jul 2008 09:11:04 +0000

Regarding recent discussions related to (D)VCS for GNOME, I think people might fight interesting results of the voting on highly popular Russian Linux site (analog of Slashdot). The comments is all in Russian, but everyone can read and interpret the results… (Just in case you care to follow the link, “Не использую”=”Do not use”). May be, staying with the current Subversion is not a bad idea, after all?;)

Yesterday, I reformatted internal n810’s flash from FAT32 to ext3. Why? Because root partition is too small and crouded. Now, I have several Megs of canola2 covers, this is too much for my small root…

BTW, I could not find the way to make canola use non-default place for disk covers (and symlinking to FAT32 partitions does not help - there are some issues with Russian dirnames).

Younes Manton: Up and running on real hardware

Mon, 21 Jul 2008 01:53:16 +0000

I reached a nice milestone today: working playback on my Geforce 6200. Most of the work went into the winsys layer, with some bug fixes and workarounds in other places, but everything is up and running now. Unfortunately the output isn't perfect, there is some slight corruption here and there. I'm guessing it has to do with some dodgy assumptions I made about shader arithmetic (rounding, saturation, etc) that SoftPipe went along with but the GPU didn't. The other issue is that there is some severe slowdown when any 2D drawing happens on the rest of the desktop. I'm guessing this may be due locking when copying the backbuffer to the window, or maybe I'm completely soaking up the CPU.

Currently nothing is optimized, I'm not even turning on compiler optimization, and I have a really slow prototype IDCT implementation performed on the CPU in place of the hardware version, so I'm sure I'm eating up a lot more CPU time than I will be by the end of the summer. I have a lot of different ideas on optimization that will target CPU usage and GPU fillrate usage, but given that I get almost full speed playback currently, I'm pretty confident that I'll be able to get HD playback by the end of SoC.

As far as the winsys goes, I was able to use most of the current Nouveau winsys. Unfortunately the DRI stuff is buried within Mesa right now, so I had to extract a lot of things and create a standalone library to handle screens, drawables, the SAREA, etc. to be able to use DRI without including and linking with half of Mesa. The winsys interface is also simpler than Mesa's; there are only a few client calls, the backbuffer is handled in the state tracker, and the winsys doesn't have to create or call into the state tracker. It took me a while to realize why the Mesa winsys was set up the way it was, and that I could simpify things on my end.

Here are some screen grabs:

Dave Airlie: technology fail weekender...

Sun, 20 Jul 2008 23:14:08 +0000

So this weekend was a total technology fail..

so on Sat night Gia rented a DVD to watch on a laptop, I was all should be no problem with this, armed with my livna (and livna-development) repositories, I started off on laptop 1, Thinkpad T60P. totem pulled an immeditate choke with some useless error message. mplayer choked similiarly with a nice bonghits in the libdvdread error message. This I tracked down to the region not being set correctly, so I set the drive region to region 4, and then mplayer failed even harder in a tight loop with kill -9 the only way out. It looked like bad sectors or something in the kernel.

So I went and tried it on laptop number 2, my Dell Insprion 6000, this machine is the oldest piece of kit I own and has been steadily on it way out the door (backlight flickers on/off due to dodgy connections internally). However it is also the only machine I have at home with a non-Linux OS on it, it has Windows XP. So I booted XP on it, stuck the disk in and it played fine, I then booted Linux on it and had the same problem as on the the other laptop. So I decided to let Gia use Windows XP to watch the DVD, I then unplugged the laptop before realising the battery was sitting on the bench (as it overheats a lot instead of charging nowadays.. and it isn't one of the goes on fire batteries either..). So it appears cutting the power wasn't such a good idea, as it appears the hard disk heads crashed or some such fail, as attempting to boot XP on the laptop after that gave a bad boot bluescreen and mounting the XP ntfs partition from Linux went into an ATA reset cycle.

So I had to watch half the rugby (it wasn't a great game in the end), and then watch some movie with Matthew McConaughty in it, at least there was scuba diving in it, so it could have been worse.

Then on Sunday on our way to the cinema, I drops my iphones on the grounds. Oh noes. My iphone volume buttons have been busted for a month or two in any case, but now the lock/power button decided to stop being useful. Of course my iphone is a US phone where I'm sure the warranty has been invalidated six ways from Tuesday at this point. So I'll probably have to go opens it ups and play with its insides. I've damaged a lot of phones in my life but the iphone has actually been the quickest to get broken, and considering I don't drink nearly as much as I used to this isn't a good showing for it.

Lennart Poettering: Oh, Solomon! He has outdone you!

Sun, 20 Jul 2008 21:29:00 +0000

Wide-angle lenses are a great invention.

Jon Phillips: Thanksgiving trip visiting Gwen and Pat

Sun, 20 Jul 2008 05:02:59 +0000

.flickr-photo { border: solid 2px #000000; } .flickr-yourcomment { } .flickr-frame { text-align: left; padding: 3px; } .flickr-caption { font-size: 0.8em; margin-top: 0px; }

Thanks giving trip- visiting Gwen and Pat 3, originally uploaded by deerfang.

This is Gwen Widmer, Patrick Clancy, me, and . I studies with Pat at Kansas City Art Institute for undergrad and worked with Pat and Gwen on some early startups. You guys already know the connection with Lu.

I should write a book about all the great people I have considered mentors (and friends) in the past. I think post iSummit, I will take a few days off to plan out the next 10 years or so

Simon McVittie: Implementing D-Bus services with telepathy-glib

Sat, 19 Jul 2008 13:08:35 +0000

(Part 1 of an ongoing series: shiny things in telepathy-glib)

16:46 < epmf> tp-glib is made of magic

—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Introduction to telepathy-glib

telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-)

My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September.

This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months.

The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!

Some background: service-side code generation

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong.

dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time.

Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks.

Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:

copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
pray that the diff process hadn't randomly exchanged functions' implementations
update the resulting code in src so it worked
check the whole mess back into darcs
hope that nobody else had made changes that would result in darcs conflicts

If you haven't yet gathered, this was a bit of a nightmare.

Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.

How telepathy-glib fixes code generation

telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib:

commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Frécinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200

    Interface support for bindings.

Instead of running dbus-binding-tool or creating GObjects with D-Bus interfaces, we put the entire Telepathy spec through glib-ginterface-gen.py, a distant descendant of Gabble's gengobject.py. For each interface in the spec, we generate a GInterface that mirrors the D-Bus interface. Any GObject that implements the GInterface automatically gets the D-Bus interface if it's exported onto D-Bus.

There are a few non-obvious refinements, though:

The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns.

For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened.

This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done).

For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.
The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.
If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.
While we're generating code anyway, we generate some static inline wrapper functions which wrap dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().
Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.

Not all interfaces are stable enough to be included in telepathy-glib's stable API and ABI, so some of our other projects include a copy of the telepathy-glib code-generation tools, and generate their own "mini-telepathy-glib" (traditionally in a /extensions/ directory) for their implementation-specific (drafts of) interfaces. This isn't yet as polished as it ought to be (mainly because we don't want to freeze the augmented-introspection-XML format that we write the Telepathy spec in, because it's rather ad-hoc and hackish in places) but it works quite well in practice.

To see what this looks like in practice, have a look at the examples in telepathy-glib, or at a Telepathy connection manager like telepathy-gabble.

Mixins and base classes

Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces.

There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does.

The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P

The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.

John (J5) Palmieri: Birthday Bash Done - Houston We Have Liftoff

Sat, 19 Jul 2008 08:01:02 +0000

All in all pretty good for a last minute event. It ended up being a night at the good life bar as it was too much fun to leave. People ended up rotating in and out all night and we all had a blast. My dinner was pretty good. The spinage was amazing but the steak was a little overdone. The vodka martinis were also good but I would stay away from the mixed drinks as they didn’t seem to be of the same quality. I still prefer Om for martinis and cocktails but overall Good Life was pretty good. Dancing was fun on such an intimate dance floor and we even had a show of the local hip hop dance crew showing off their moves. Well it is 4 o’clock here - a bit late for this old man. I need to pack for my trip to Aruba on Sunday and am meeting someone for dinner tomorrow night. I will be mostly absent for the next week as I try to relax and soak in some sun and beach but I will be checking e-main every once in awhile and my phone will still work for emergencies.

[read this post in: ar de es fr it ja ko pt ru zh-CN ]

Lennart Poettering: PulseAudio FUD

Fri, 18 Jul 2008 15:02:00 +0000

Jeffrey Stedfast

Jeffrey Stedfast seems to have made it his new hobby to bash PulseAudio. In a series of very negative blog postings he flamed my software and hence me in best NotZed-like fashion. Particularly interesting in this case is the fact that he apologized to me privately on IRC for this behaviour shortly after his first posting when he was critizised on #gnome-hackers -- only to continue flaming and bashing in more blog posts shortly after. Flaming is very much part of the Free Software community I guess. A lot of people do it from time to time (including me). But maybe there are better places for this than Planet Gnome. And maybe doing it for days is not particularly nice. And maybe flaming sucks in the first place anyway.

Regardless what I think about Jeffrey and his behaviour on Planet Gnome, let's have a look on his trophies, the five "bugs" he posted:

Not directly related to PulseAudio itself. Also, finding errors in code that is related to esd is not exactly the most difficult thing in the world.
The same theme.
Fixed 3 months ago. It is certainly not my fault that this isn't available in Jeffrey's distro.
A real, valid bug report. Fixed in git a while back, but not available in any released version. May only be triggered under heavy load or with a bad high-latency scheduler.
A valid bug, but not really in PulseAudio. Mostly caused because the ALSA API and PA API don't really match 100%.

OK, Jeffrey found a real bug, but I wouldn't say this is really enough to make all the fuss about. Or is it?

Why PulseAudio?

Jeffrey wrote something about 'solution looking for a problem' when speaking of PulseAudio. While that was certainly not a nice thing to say it however tells me one thing: I apparently didn't manage to communicate well enough why I am doing PulseAudio in the first place. So, why am I doing it then?

There's so much more a good audio system needs to provide than just the most basic mixing functionality. Per-application volumes, moving streams between devices during playback, positional event sounds (i.e. click on the left side of the screen, have the sound event come out through the left speakers), secure session-switching support, monitoring of sound playback levels, rescuing playback streams to other audio devices on hot unplug, automatic hotplug configuration, automatic up/downmixing stereo/surround, high-quality resampling, network transparency, sound effects, simultaneous output to multiple sound devices are all features PA provides right now, and what you don't get without it. It also provides the infrastructure for upcoming features like volume-follows-focus, automatic attenuation of music on signal on VoIP stream, UPnP media renderer support, Apple RAOP support, mixing/volume adjustments with dynamic range compression, adaptive volume of event sounds based on the volume of music streams, jack sensing, switching between stereo/surround/spdif during runtime, ...
And even for the most basic mixing functionality plain ALSA/dmix is not really everlasting happiness. Due to the way it works all clients are forced to use the same buffering metrics all the time, that means all clients are limited in their wakeup/latency settings. You will burn more CPU than necessary this way, keep the risk of drop-outs unnecessarily high and still not be able to make clients with low-latency requirements happy. 'Glitch-Free' PulseAudio fixes all this. Quite frankly I believe that 'glitch-free' PulseAudio is the single most important killer feature that should be enough to convince everyone why PulseAudio is the right thing to do. Maybe people actually don't know that they want this. But they absolutely do, especially the embedded people -- if used properly it is a must for power-saving during audio playback. It's a pity that how awesome this feature is you cannot directly see from the user interface.^[1]
PulseAudio provides compatibility with a lot of sound systems/APIs that bare ALSA or bare OSS don't provide.
And last but not least, I love breaking Jeffrey's audio. It's just soo much fun, you really have to try it! ;-)

If you want to know more about why I think that PulseAudio is an important part of the modern Linux desktop audio stack, please read my slides from FOSS.in 2007.

Misconceptions

Many people (like Jeffrey) wonder why have software mixing at all if you have hardware mixing? The thing is, hardware mixing is a thing of the past, modern soundcards don't do it anymore. Precisely for doing things like mixing in software SIMD CPU extensions like SSE have been invented. Modern sound cards these days are kind of "dumbed" down, high-quality DACs. They don't do mixing anymore, many modern chips don't even do volume control anymore. Remember the days where having a Wavetable chip was a killer feature of a sound card? Those days are gone, today wavetable synthesizing is done almost exlcusively in software -- and that's exactly what happened to hardware mixing too. And it is good that way. In software mixing is is much easier to do fancier stuff like DRC which will increase quality of mixing. And modern CPUs provide all the necessary SIMD command sets to implement this efficiently.

Other people believe that JACK would be a better solution for the problem. This is nonsense. JACK has been designed for a very different purpose. It is optimized for low latency inter-application communication. It requires floating point samples, it knows nothing about channel mappings, it depends on every client to behave correctly. And so on, and so on. It is a sound server for audio production. For desktop applications it is however not well suited. For a desktop saving power is very important, one application misbehaving shouldn't have an effect on other application's playback; converting from/to FP all the time is not going to help battery life either. Please understand that for the purpose of pro audio you can make completely different compromises than you can do on the desktop. For example, while having 'glitch-free' is great for embedded and desktop use, it makes no sense at all for pro audio, and would only have a drawback on performance. So, please stop bringing up JACK again and again. It's just not the right tool for desktop audio, and this opinion is shared by the JACK developers themselves.

Jeffrey thinks that audio mixing is nothing for userspace. Which is basically what OSS4 tries to do: mixing in kernel space. However, the future of PCM audio is floating points. Mixing them in kernel space is problematic because (at least on Linux) FP in kernel space is a no-no. Also, the kernel people made clear more than once that maths/decoding/encoding like this should happen in userspace. Quite honestly, doing the mixing in kernel space is probably one of the primary reasons why I think that OSS4 is a bad idea. The fancier your mixing gets (i.e. including resampling, upmixing, downmixing, DRC, ...) the more difficulties you will have to move such a complex, time-intensive code into the kernel.

Not everytime your audio breaks it is alone PulseAudio's fault. For example, the original flame of Jeffrey's was about the low volume that he experienced when running PA. This is mostly due to the suckish way we initialize the default volumes of ALSA sound cards. Most distributions have simple scripts that initialize ALSA sound card volumes to fixed values like 75% of the available range, without understanding what the range or the controls actually mean. This is actually a very bad thing to do. Integrated USB speakers for example tend export the full amplification range via the mixer controls. 75% for them is incredibly loud. For other hardware (like apparently Jeffrey's) it is too low in volume. How to fix this has been discussed on the ALSA mailing list, but no final solution has been presented yet. Nonetheless, the fact that the volume was too low, is completely unrelated to PulseAudio.

PulseAudio interfaces with lower-level technologies like ALSA on one hand, and with high-level applications on the other hand. Those systems are not perfect. Especially closed-source applications tend to do very evil things with the audio APIs (Flash!) that are very hard to support on virtualized sound systems such as PulseAudio [2]. However, things are getting better. My list of issues I found in ALSA is getting shorter. Many applications have already been fixed.

The reflex "my audio is broken it must be PulseAudio's fault" is certainly easy to come up with, but it certainly is not always right.

Also note that -- like many areas in Free Software -- development of the desktop audio stack on Linux is a bit understaffed. AFAIK there are only two people working on ALSA full-time and only me working on PulseAudio and other userspace audio infrastructure, assisted by a few others who supply code and patches from time to time, some more and some less.

More Breakage to Come

I now tried to explain why the audio experience on systems with PulseAudio might not be as good as some people hoped, but what about the future? To be frank: the next version of PulseAudio (0.9.11) will break even more things. The 'glitch-free' stuff mentioned above uses quite a few features of the underlying ALSA infrastructure that apparently noone has been using before -- and which just don't work properly yet on all drivers. And there are quite a few drivers around, and I only have a very limited set of hardware to test with. Already I know that the some of the most popular drivers (USB and HDA) do not work entirely correctly with 'glitch-free'.

So you ask why I plan to release this code knowing that it will break things? Well, it works on some hardware/drivers properly, and for the others I know work-arounds to get things to work. And 0.9.11 has been delayed for too long already. Also I need testing from a bigger audience. And it is not so much 0.9.11 that is buggy, it is the code it is based on. 'Glitch-free' PA 0.9.11 is going to part of Fedora 10. Fedora has always been more bleeding edge than other other distributions. Picking 0.9.11 just like that for an 'LTS' release might however be a not a good idea.

So, please bear with me when I release 0.9.11. Snapshots have already been available in Rawhide for a while, and hell didn't freeze over.

The Distributions' Role in the Game

Some distributions did a better job adopting PulseAudio than others. On the good side I certainly have to list Mandriva, Debian^[3], and Fedora^[4]. OTOH Ubuntu didn't exactly do a stellar job. They didn't do their homework. Adopting PA in a distribution is a fair amount of work, given that it interfaces with so many different things at so many different places. The integration with other systems is crucial. The information was all out there, communicated on the wiki, the mailing lists and on the PA IRC channel. But if you join and hang around on neither, then you won't get the memo. To my surprise when Ubuntu adopted PulseAudio they moved into one of their 'LTS' releases rightaway ^[5]. Which I guess can be called gutsy -- on the background that I work for Red Hat and PulseAudio is not part of RHEL at this time. I get a lot of flak from Ubuntu users, and I am pretty sure the vast amount of it is undeserving and not my fault.

Why Jeffrey's distro of choice (SUSE?) didn't package pavucontrol 0.9.6 although it has been released months ago I don't know. But there's certainly no reason to whine about that to me and bash me for it.

Having said all this -- it's easy to point to other software's faults or other people's failures. So, admitting this, PulseAudio is certainly not bug-free, far from that. It's a relatively complex piece of software (threading, real-time, lock-free, sensitive to timing, ...), and every software has its bugs. In some workloads they might be easier to find than it others. And I am working on fixing those which are found. I won't forget any bug report, but the order and priority I work on them is still mostly up to me I guess, right? There's still a lot of work to do in desktop audio, it will take some time to get things completely right and complete.

Calls for "audio should just work (tm)" are often heard. But if you don't want to stick with a sound system that was state of the art in the 90's for all times, then I fear things *will have* to break from time to time. And Jeffrey, I have no idea what you are actually hacking on. Some people mentioned something with Evolution. If that's true, then quite honestly, "email should just work", too, shouldn't it? Evolution is not exactly famous for it's legendary bug-freeness and stability, or did I miss something? Maybe you should be the one to start with making things "just work", especially since Evolution has been around for much longer already.

Back to Work

Now that I responded to Jeffrey's FUD I think we all can go back to work and end this flamefest! I wish people would actually try to understand things before writing an insulting rant -- without the slightest clue -- but with words like "clusterfuck". I'd like to thank all the people who commented on Jeffrey's blog and basically already said what I wrote here now.

So, and now I am off hacking a bit on PulseAudio a bit more -- or should I say in Jeffrey's words: on my clusterfuck that is an epic fail and that no desktop user needs?

Footnotes

[1] BTW 'glitch-free' is nothing I invented, other OS have been doing something like this for quite a while (Vista, Mac OS). On Linux however, PulseAudio is the first and only implementation (at least to my knowledge).

[2] In fact, Flash 9 can not be made fully working on PulseAudio. This is because the way Flash destructs it's driver backends is racy. Unfixably racy, from external code. Jeffrey complained about Flash instability in his second post. This is unfair to PulseAudio, because I cannot fix this. This is like complaining that X crashes when you use binary-only fglrx.

[3] To Debian's standards at least. Since development of Debian is very distributed the integration of such a system as PulseAudio is much more difficult since in touches so many different packages in the system that are kind of private property by a lot of different maintainers with different views on things.

[4] I maintain the Fedora stuff myself, so I might be a bit biased on this one... ;-)

[5] I guess Ubuntu sees that this was a bit too much too early, too. At least that's how I understood my invitation to UDS in Prague. Since that summit I haven't heard anything from them anymore, though.

John (J5) Palmieri: Birthday in Boston

Fri, 18 Jul 2008 00:48:27 +0000

I thought I was going to be on vacation for my birthday this year but it turns out I’m leaving the day after so I decided to do something this Friday to celebrate with people in Boston. I’m going to have a small dinner with close friends at Good Life Bar around 8:00 and then perhaps some drinks at Jacob Wirth. In any case I’ll be microblogging my whereabouts on identi.ca so use it as an excuse to come down, drink a few beers and have a good time.

[read this post in: ar de es fr it ja ko pt ru zh-CN ]

Jon Phillips: Cantocore.com Launched + Home Concert Preview Next WED

Thu, 17 Jul 2008 23:44:39 +0000

Over at Fabricatorz we launched the Cantocore.com website and are continuing to push on raising funds to fund selected artists projects. If you are in SF, we invite you to come to the Home Concert Preview next WED from 7-9. Tickets to the event are $50 and may be purchased at the door, or preferably, through paypal or contact jhoover.charles@gmail.com to process payment.

NOTE: If there is some other contribution to Cantocore by writing about the shows, contributing to the blog, helping record the event, please contact me to discuss bypassing ticket price.

Good friend Christopher Willits will be performing along with Chinese musician Ma Jie“>Ma Jie. Wine and snacks will be served as well! From the RSVP list it will be a great mix of interesting folks from the area. We have a cap of 70 people, so please do RSVP today. More about the launch, project, and Cantocore.com Home Concert Preview:

Hi all, welcome to a new project which is a collaboration between Fabricatorz and Garage Biennale. Here are two levels of scale about the project:

“Art show in Guangzhou, China in September 2008 and then in San Francisco November 2008.”

And, a few more sentences…

“Garage Biennale and the Fabricatorz are bringing you Cantocore, a research project investigating contemporary art and culture between Canton (Guangdong) and cities around the world. The initial focus is a contemporary art exhibition with two versions of the same show in San Francisco and Guangzhou, China.”

There is a solid line-up of artists working on projects at present for both shows with the main concept being the ideas of import and export where projects in both locations are the same, but different versions.

Lu re-blogged the Cantocore Home Concert Preview we are doing next WED in SF, and so, I’m pushing out this post about the Cantocore project which is a contemporary art show that Lu, myself and Justin Hoover are pushing out this early September and November. The first instance of Cantocore is to generate some funds to help in production of the artwork. No one is getting paid from the raising of monies to support art production, so we are putting on a concert to preview the artwork for the show.

Lu wrote about here:

What it will take to make an art show. You will need three things: Artists, Space and Money! Sometimes these three things don’t come so easy, don’t they?

These are the things we are pushing right now for Cantocore exhibition, a show in both Guangzhou China and San Francisco this fall. There is a home concert preview party coming Wednesday 7/23 to fund the production of the shows. I am not a very good sales woman, but really want to make this one works!

Visit Cantocore site for information about the show, artists, and detail about the preview concert. Performing musicians for the the preview concert are Ma Jie and Christopher Willits.

And then there is a follow-up post on the Cantocore website:

Cantocore Home Concert Preview is coming up next Wednesday July 23rd, 7-9 PM in San Francisco. Right now our RSVP deadline for this event is extended to Monday, July 21st. And tickets are on sale right now for $50 USD! All money goes to the production of the Cantocore exhibition in Guangzhou and San Francisco. And, all contributions will be rewarded with praise, promotion in printed materials and on this website.

This event is a preview of the coming exhibitions, and also a home concert featuring Chinese traditional instrument musician Ma Jie and electronic musician Christopher Willits.

To reserve your tickets please do one of the following:

Call Justin at 415-425-1647
Email: jhoover.charles@gmail.com
Or use Paypal option

We are eager to raise some funds from those interested in the ideas which will gain both a plug on the http://cantocore.com website and then in our printed material. Feel free to ask questions here on this blog post publicly, or send us email about this.

Please participate in this project and if in SF, RSVP by next MONDAY, July 21 to come to the Cantocore Home Concert Preview.

Donnie Berkholz: Early coverage of Gentoo 2008.0

Thu, 17 Jul 2008 06:05:14 +0000

Here’s links to a few places that posted announcements for 2008.0. Go there and get involved by voting and commenting!

Gentoo 2008.0 makes the Digg frontpage

Reddit
DistroWatch (original announcement)
LWN.net
TuxMachines
Phoronix
OSNews
OStatic
Heise
Linux Format
Clubic (Warning: French. English translation)
FOSSWire
DesktopLinux.com
Linux Magazine (UK)
InternetNews.com — The only one to do any real reporting & interviewing
Open Source Pixels
Techgage
LinuxNews.pl (English translation)
Programas Livres (Warning: Portuguese. English translation)
Linux Journal
Linux1.no (English translation)
PettiNix (Warning: Italian. English translation)
Linux.org.ru (English translation)
ZDNet.co.uk
DistroWatch Weekly

There’s also a ton of talk about Gentoo 2008.0 happening on Twitter, which I’m following through the Summize search engine with a search for gentoo. I’m also following blogs talking about Gentoo, and I have a news search for Gentoo that I expect to pick up more later once more journalists pick up the news.

Update 1: Added Phoronix, thanks to Denis Dupeyron (Calchan).

Update 2: Added OSNews

Update 3: Added OStatic, Heise, Linux Format, Clubic, FOSSWire, DesktopLinux.com, Linux Magazine (UK)

Update 4: Added InternetNews.com

Update 5: Added Open Source Pixels, Techgage

Update 6: Added LinuxNews.pl, Programas Livres

Update 7: Added Linux Journal, Linux1.no, PettiNix, Linux.org.ru

Update 8: Added ZDNet.co.uk, DistroWatch Weekly