10 questions to Michael Sparks, OS Guru at BBC (Part 2)

Part 1 of the article was published yesterday.

The following answers must be construed generally as personal opinions and preconceptions, and not in any shape or form an official BBC position. Michael Sparks does not speak for the BBC here despite working there, and despite speaking on behalf of the BBC at Linux World London.

6. How receptive and supportive was the BBC when Open source was first introduced to the organisation? Did you meet with any obstacles?

I wasn't working with the BBC when open source was first introduced or when the BBC first released some software as open source. It was probably introduced in several areas independently in different ways by people who didn't know that other people were using open source software in the organisation.

By the time I joined - 4 years ago - the BBC had been using Apache, Perl, Linux & Wikis in lots of locations for many years. The most notable piece of open source that the BBC had released at that point was a tool called "BETSIE" which produces text only versions of web pages (as used on the news web pages amongst others) and that had been released as open source many years ago.

However, when I joined the BBC, there was a definite move inside BBC R&D (as it was called then) by a moderate number of engineers to look to see whether what we were developing could or should be released as open source (However, as far as I'm aware nothing had been formally released as open source at that time by BBC R&D).

As a result when we started work on Kamaelia, I was in the right place at the right time - my immediate management was extremely supportive, and Kamaelia was started on the understanding that the project would be released as open source when there was a core that could be released.

When the time came for release however, there was an interesting issue. The only project that BBC Research had released in a manner with buy-in from senior management as open source at that point in time was Dirac. At that point in time the licensing fees for MPEG-4 hadn't stablised and it was looking like it would be hideously expensive for broadcasters, and the opportunity to take Dirac further for independent standardisation allowing the industry to effectively "start again" was appealing (at least this is my understanding).

Some people took this release of Dirac as the BBC going "we're going into head to head competition with X", since that makes a good story, when in fact we'd've been delighted if the people who were claimed to be competing with us collaborated with us. Most notably it was a news report around Linux World that caused the biggest problems due to the misconception by the media of the BBC seeking to compete rather collaborate.

As a result when Kamaelia came to be released, the release process was paused while some formal criterion were drawn up - based on the experience (good and bad) with Dirac. After this Kamaelia was measured against those. This took a rather long period of time since the BBC does recognise that once you've released, you've released - it can't be clawed back. This was frustrating on a personal level, but makes sense on an organisational level.

Since then the level of support I have had from the organisation has been great. When we needed a contributor agreement for accepting contributions from the community for example, our legal team were great and extremely supportive. Coming from an internet and open source background it took longer than I anticipated, but in retrospect no longer than it would normally take in any public organisation.

7. What kind of restrictions does BBC encounter as a public organisation that has to deal with Intellectual property?

We're required under the current charter to exploit any work to the greatest benefit to the license fee payer. (The wording in the new charter is more detailed than the current, but the spirit appears to remain the same)

Some approaches - such as patenting - are commonly understood, and accepted mechanisms for protecting the investment made in research in an area. (Bear in mind the bulk of R&D by the BBC over the past 70-80 years has been in the hardware arena). These patents can then be licensed which over time can be a source of income to the BBC. NICAM stereo is a good example of licensing IPR which benefitted the audience both in terms of increased audio quality and in terms of a modest revenue stream to offset R&D costs to the license fee payer.

Another standard approach inside the BBC has been the development of open standards - like PAL, like DVB, etc. The creation of an open standard encourages multiple vendors which in the end tends to reduce unit costs, which has the general benefit of reducing costs both on the high street and on the broadcast supply side.

Open source fits very much in the arena of creating systems which have the potential for similar benefits as open standards, the benefits of technology transfer into industry that patents provide (often without the income, but dual licensing can keep that door open), and the benefit of publication for prior art purposes that patenting provides. It also fits into the research/scientific agenda of publishing your work in a repeatable fashion so that others can duplicate your work and feedback on your approach & results.

As a result these arguments fit very well with the old charter's wording. The new charter goes further than the old charter and explicitly states that the BBC should be looking to further open standards, and but looks to leave the door open for traditional style "patent & license" style exploitation where it would most benefit the license fee payer & the industry as a whole. (At least this is my interpretation of it)

There is one little foible however that is interesting. Whilst my personal preference in licensing is the BSD license, this license is problematic for the BBC, generally speaking.

If we allow others to close a piece of research code, which perhaps only a supplier to the BBC is interested in, then we give an asymmetric benefit to that supplier. They would be able to invest little, take close the code and then sell it back in a fashion that might not quite suit the BBC. As a result it's important for the BBC to use licenses that require any changes to BBC code to be shared back - as a quid-pro-quo if you like.

Also, the BBC needs to take software patent issues seriously. Whether or not they should legally exist in Europe, in practice they do. As a result an open source license that takes into account patents is similarly important.

Also, software the BBC produces sits in a software eco-system. Some software is proprietary, and some is open source. The BBC has no place making a political judgement as to which is more valid than the other, and as a result beyond protecting the interests of the license fee payer, if we release code as open source it should be non-discriminatory of the potential userbase.

As a result this is why Kamaelia uses the Mozilla Tri-license. For minimal compliance, if someone merely uses Kamaelia, they don't have to release their code. If they change any code we supply and redistribute that changed code, they have to supply the changes. However, they don't have to share the source of components they wrote. The use of the tri-license also means that as well as interoperating with proprietary systems, Kamaelia can integrate with GPL systems cleanly.

Finally, there is the very real question of market distortion. By merely existing, the BBC distorts the market. Anyone who's seen television in a country without a public service broadcaster can see what a positive influence this can be, but likewise the dangers of market distortion can be clear.

Similarly, if you release code in the right way you can generate extra wealth in the market place for all (if everyone can use it you're all wealthier). If you release it badly, you risk distorting the market in a bad way, and its worth remembering that if you do this, it often means you're affecting real people with real jobs and real families.

Please remember here though, that in all the above, this is all from my personal perspective, but I'd hope in this area my views aren't too far off the BBC view for hopefully obvious pragmatic reasons. (However, if they are, I'm sure someone will tell me :-)

8. What are, from your perspective, the main differences that exist between Open source implementation in a private organisation and a public one like BBC?

From my personal perspective, the key one comes from this simple point: who paid for it?

In a private organisation, the people who paid for it are those who own the company - either the founders or shareholders depending on the structure. In that case the purpose of software development is to either produce a product for sale, or to enable a product to be sold (being that product services, or some other form of goods - like films, books, toys, cars).

If the purpose is to sell software, then you are almost forced down the dual licensing route - "these rights and abilities you get for free, these you pay for". This works very well for some companies - most notable examples are Trolltech, MySQL, and Sleepcat. Report Lab is also a very nice example of this.

It's worth nothing also that in each of these examples there are real benefits beyond just software ownership and the use of the tool that come from purchasing the software license. You can have the ability to close the code with some, with others its essentially tied to a support contract, better (closed) tools that exist due to the open code, better access to documentation and so on.

Selling open source software through dual licensing however is proving to be the exception rather than the rule. The more common way that people are benefitting from open source developement in businesses is not through selling the software, but through selling a service because the software exists.

The least thought about example here is the internet itself. There are thousands of ISPs that exist that sell a service that only exists because TCP/IP exists, because the web exists, because email exists and because all 3 have a low barrier to entry - largely due to the underlying infrastructure being built from open source.

ie The creation of open source software can create entirely new markets upon which companies can operate. Microsoft famously was ignoring the internet before doing a major U-turn, and now they're looking to build their "Live" services ontop of a platform (the TCP/IP based internet) which is dominated by open source at a software level.

On this level of selling a service or product because of an open source product's existance, you then get into the issue of use of open source software vs proprietary. In a business this decision almost always comes down to "is this the most cost effective solution to the problem over the expected time period of the use of the solution". If it's an ad-hoc system for transferring files from a to b, maybe something unsupported, understood by 1 admin is fine.

If it's a system to be used daily for copying entire file system trees for mirroring, is something like rsync what's desirable, or is it more desirable to hire a cheaper systems administrator who just uses a product that can be bought and supported off the shelf? If it's a large scale backup system, is it worth rolling your own system or is it worth getting a supported solution with a "just in case" support contract with a data recovery firm? Whether the solution is open source or not factors in on that basis normally as a cost saving more than anything else.

Even here though, a business may choose to buy a product or service based on open source software - perhaps a caching appliance based on Squid for example. This then gives them the benefit of liasing with another company for support and updates, and the benefits of a widely distributed development team for a niche product. (Though perhaps lacking in the performance than some products can have in a niche area)

Regarding a public service organisation however, the differences are interesting. You still have all the benefits of a private company, but you have some other issues as well. If you're talking about about merely using open source software, then you have the issue "should the public organisation use open source in order to encourage other organisations to do the same".

My personal feeling is that motivation would be political, and whilst a government could make such a mandate, it's not for any public body to make that decision. That is as I say, a personal feeling.

If however the purpose behind using open source software is to promote an open standard, then than is clearly within the BBC's remit (especially with the new charter), and as a result there's little question in my mind - under that circumstance the BBC should use software that promotes open standards. At the end of the day they tend to reduce costs for the license fee payer.

Again, this is personal opinion.

The people who paid for the work are the public.

This means that the public has invested their finances in your work area.

There is a school of thought that says "the public paid for it, they should have access to it". If you apply this to software this implies that release as open source is a naturally good idea. In reality the answer to that question is more complex, but it's an interesting issue.

An interesting correlation for the private sector is this. At my time working with Inktomi, Inktomi's ability to sell one of its core product's - a caching product called "traffic server" went up when it was able to support Linux.

Because the company was supporting an open source application it's revenue increased. Internally Inktomi also used Linux for a variety of servers and services, and used open source wiki software for internal collaboration, and even use CVS - an open source SCM system for source code collaboration. They made money due to using open source software.

Contributing patches back to any of these things - the Wiki code, to Linux, to CVS made Inktomi more effective as an organisation, since the tools it used improved.

9. What is the open source project/application/solution that has impressed you the most in the past year?

The Dojo Toolkit. Someone has finally produced a cross platform wysiwyg editor for HTML that sits in a browser. It's a tiny little thing, but incredibly important.

10. What are your views on the General Public License v3.0

OK, let me preface this with I'm not a lawyer, and these are personal views not the views of the BBC. I know I've said that repeatedly in my answers here, but I really want to make that clear. (I'm not sure the BBC *has* a view on the GPL v3!) Any mistakes with my understanding of copyright law are all mine, though you're welcome to make the same mistakes!

I'm a fan of the BSD license for release of my own work outside of work time (at work I'm constrained in other ways). I prefer to release my code and let others do what they like with it.

As a a result, to me personally it's not _directly_ relevant. However, I am interested in software licensing, and I do understand the aims of the Free Software Foundation, even if I feel at the end of the day it should be down to individual code creators (individuals or companies) to decide their own licensing terms (rather than everything GPL).

To me the GPL v3 is an important development for the free software movement, and I think it goes a long way to furthering their goals in the modern world.

However, the GPL is very much based around the idea that you as _a user_ must be able to replicate and change the applications you use for your own purposes (This is why the GPL constrains developers to not distribute derivative works under anything else).

This raises questions like "what is an application".

It is also, by definition being a copyright license, only triggered when you use the code in a manner restricted by copyright. Mere execution of code is allowed by copyright law without a license for example and does not trigger the license, where as distribution does trigger a license. As a result putting code up as a tar ball on a website triggers the license, whereas merely running it as a CGI doesn't. (all as I understand it, I'm not a lawyer :)

And this brings us neatly to the problem with the GPL version 3. GPL version 3 does not protect the user's ability to inspect, change, modify and run a modified version of the application as it should given it's basic goals with a number of modern applications. It is also questionable whether it can.

The sorts of systems I'm referring to are web services, especially those powered by large backend databases of data you cannot access.

Tim O'Reilly has argued about this publicly in the past with Richard Stallman many years ago, and recently restarted the debate in this area with the inflammatory stance that Open Source licenses are obsolete.

By this he's saying that the freedoms these licenses seek to preserve are not protected with the new style of applications that are emerging. He's not saying that they don't protect your traditional applications adequately (they do), he's not saying they're bad. He's saying they're incomplete, and I also suspect he's worried that they can't be complete.

I also happen to agree with his point of view here, and its best taken by example. I use Kmail as my email client. My ability to dive into the source code for it is protected on a number of different layers in different areas.

Some parts are LGPL, some parts are GPL. As a result I can get hold of the source code, and change anything I like, recompile it and re-run it as a fully functional application, using the same data I had beforehand.

Can I download the source to google mail? If I could, would I have the hosting capabilities ? If I did, would I have access to the same spam busting database ? This latter point is an interesting one - I receive a huge amount of spam. Normally with my mail app I could take the spam handling system from one and incorporate it directly into another. Can I do that with google mail?

Even if they released the source code?

What about google earth? How about google docs? And the killer app itself - what about google search? I can download ht://dig and change it. There's dozens of other search apps I could use and I can modify those. Can I download the source to a search engine recompile it, run it locally and expect the system to provide me the same or better results? No.

This isn't a problem for Google and I pick them here only because the people I've met at Google have been nice people. However, it does show an interesting trend. The proprietary app of the latter quarter of the twentieth century was a compiled program that ran on your own local computer that you owned and accesses locally but you didn't have access to.

It's rapidly beginning to look like the proprietary software of the first quarter of the 21st century will be those who produce networked applications that everyone wants to use, where the code is hosted and run remotely and all you interact with is the effects of the running application. No source code is distributed, no license is triggered, and with the secondary nature of *even if the source was distributed you still couldn't realistically use the application*.

So, whilst the GPL v3 does try to protect the user against proprietarising data (DRM), it seems to miss that many applications used by huge swathes of the population are rapidly becoming proprietarised by being run as services.

Whether the solution to this will be to reuse the same trick of taking an intellectual monopoly grant and use it against itself as the GPL does, or something new either based on a legal premise or based on a technical alternative, I think is something yet to play out.

However I think the GPL v3 is at risk of completely missing this risk to their target audience, and hopefully they will listen to the likes of Tim O'Reilly when he says their licenses are obsolete, rather than just say "we won't run their software on our computers". It's too late - the internet really is the computer for many people now for growing numbers of applications.

As I say with regard to this, my answers here are strictly personal, and do not reflect BBC opinion on any subject, except perhaps co-incidentally, and if they differ wildly from BBC opinion, its worth remembering that Research Engineers are required to think outside the box :-)