OPF Blog: Cargo Cult Standards

A new OPF blog entry: Cargo Cult Standards. Reproduced below...

I’ve been keeping an eye on the #anadp11 Twitter backchannel, particularly the discussion about standards. I’m not there, so I don’t know what points have been made, but I want to try to head off a common misconception. Standards are wonderful things, but the standard itself is not enough. It’s the social consensus built by the standardisation process that counts, and that ensures the standard will be adopted and sustained. If we focus on the standard, but not on the social process (and thus build the standard in isolation), we may end up in a position where we have too many nearly-identical standards, none of which are all that widely used. Oh look, we’re already there.

Of course, getting a diverse community to agree on anything is hard work, and we need to be able to push forward without waiting for every decision to be signed off by everyone. But complex but successful standards are created all the time, and we can learn from the existing standardisation processes and adopt them as appropriate.

For example, Google caught some flack recently for their decision to drop h.264 video support and move to the WebM VP8 codec. This codec is published under an open licence, with an open source reference implementation, and critically, is royalty free. But the standard itself was built entirely in secret by a company Google bought out, with no wider consultation, and there is no clear way for any other vendors or interested parties to influence its future development. Google’s critics pointed this out, noting that although the h.264 standard may be behind a paywall, the process of developing the standard is very well defined and designed to take into account the needs of the adoptive community. Sadly all these means that HTML video remains only partially standardised, which makes digital preservation of web video significantly more difficult than it needs to be.

I suspect neither the Google nor the h.264 approach is quite right for the digital preservation community, as we probably need both the process and the standard to be open. Perhaps the W3C approach would be a better fit.

Of course, shifting the focus onto the community will mean changing the way we do things. It will mean working in the open first, not pushing out yet another WORN* document just as all the project money gets used up and everyone moves on. It may mean compromise, letting go of the odd sacred notion for the greater good. It also means some hard work trying to properly understand the existing standards to see how they might be adopted or modified, rather than the concentrating only on our own needs, succumbing to not-invented-here syndrome, and reinventing our own wheels.

For the standards-bearers, it means accepting that no standard is perfect, or indeed every really finished. This means making it clear that we are willing to change, and publishing clear information on how this would be done. This means being willing to relinquish some control, keeping the vision clear while widening the stakeholder base and adopting standardisation processes that scale. This may mean deprecating aspects of our homegrown standards – for example, should we be replacing PREMIS provenance records with constructs from the Open Provenance model? It may also mean trying to take part in bigger standardisation processes (like W3C or ISO), recognising when the associated risks are outweighed by the potential rewards.

It won’t be easy, but frankly, we have no choice. Without it, we’ll end up with yet more Cargo Cult standards, where we have all of the outward appearances of a healthy standardisation culture (the document ‘cargo’) without the social consensus we need to drive adoption and make any of them sustainable in the long term.

* WORN: Write Once Read Never.