I’m giving a talk today at the Open Source, Open Development, Open Innovation workshop in Oxford:

Open source software is more than just a licence, it is also a software development methodology that allows companies to share resources and collaborate on non-core parts of their software/service offering. When managed well, open development enables a reduction in cost, and an increase in innovation as a result of the convergence of the best minds in the problem space. In this presentation Bertrand Delacretaz will describe how Day Software has embraced open development by positioning itself as the leaders in both open standards and open source software. We will examine how Day’s active engagement with 25 open source projects and numerous standards groups has enabled the company to become a world leader in their market and in the open source projects they participate in.

The funny thing is that the above abstract was written by Ross Gardler while waiting for my own version of it – and it says exactly what I was trying to say, only better ;-)

The event is covered by a live blog, and you can ask questions there.

To put it simply, my conclusion is that quick feedback from users and customers is key to open innovation – and open source. if done right, provides lots of feedback, fast.

Looking at the diversity of Apache Software Foundation communities, one can see a recipe for failure: people from different cultural backgrounds, different mother tongues, different employers, different timezones…all working together to create some of the best software on the planet? You must be kidding.

How can this very loose collage of disparate people pump out dozens of high-quality releases every year, often working better than more structured corporate teams? This “mystery” has been on my mind for a while, and I have identified four drivers that influence the way we use collaboration tools that play a major part in our success.

The first driver is a common vision amongst project members. The Biblical saying, “Without a vision, my people perish” is quite valid for our projects. Both using a central development mailing list for each project, and spending time to collectively define our project’s charter, helps us foster this common vision amongst project members. Every member should have the same answer to the “what are our goals?” question, so it’s important to get them to talk in a central place, where they all get the same information, as opposed to undocumented, one-to-one discussions.

Secondly, providing real-time status updates to project members is key in helping them stay on track. At Apache, this is implemented by the many events generated by our collaboration tools: commit events to indicate code changes, issue tracker events to provide updates about the status of bugs and new features, success/failure events from continuous build systems, and standardized ways of announcing releases so that other projects are informed. Project members subscribe freely to as few or as many event channels that they want to, so as to stay on top of things in near real-time, and without having to actively ask others about what happened. Status meetings? No need for those, as the information is flowing all the time.

The third success driver lies in enabling real-time help requests. In an immediate crisis of the “we need to deliver this by tomorrow” type, especially when working with a big team, you need to be able to ask for help without necessarily 1) knowing who specifically will help, and 2) bothering others with direct person-to-person requests, especially if they work in a different timezone. The key here is using issue trackers, where one web page stores key data and parts of the dialog that leads to resolving an issue. Posting an issue on the tracker, with sufficiently detailed instructions about how to reproduce the problem, along with attributes such as severity level, affected modules, etc., is the best way to expose a problem to the group quickly and with precision. Using an issue tracker also allows you to quickly and efficiently change priorities as well as re-assign issues and tasks – key elements that make all the difference in a crisis.

Finally, having searchable archives of this information allows new project members, or those returning after a period of absence, to learn what transpired and why things have been done in a certain way. Without self-service archives, new participants would have to talk to everybody to find out about the project’s history, past decisions, conventions, etc., which is neither efficient nor scalable. Most of our archives are automatically built as project activities progress: mailing lists are archived, source code control history is kept forever, and issue trackers write the full history of the project’s micro-decisions.

Combined with Apache’s principles of meritocracy and consensus-based decision making, these four collaboration drivers allow our project teams to work very efficiently, and, in many cases, even more so than structured teams that do not establish those central hubs of information exchange.

Does your project team foster a common vision and provide tools for real-time status updates, real-time help requests and self-service archives to its members? If yes, congratulations: you’re on a good track to becoming as effective as an Apache project!

Many thanks to Sally Khudairi for reviewing and copy editing.

See also my Open Source Tools are Good For You presentation, which discusses the tools that Apache projects use to implement this.

People thinking about creating an open source project might rightly consider hosting on one of the various hosting services available: Google Code, SourceForge, kenai, bitbucket and github come to mind. Quick and easy, create a repository or request some resources and you’re in business.

Incubating a project at the Apache Software Foundation (ASF) takes a lot more effort than just requesting a hosting space on one of those services, so why would you do that? One can perfectly host code on one of those services with an Apache License, so what’s the difference?

I think the big difference lies in the governance model, and in fact calling the ASF just a code repository is very wrong. Let’s discuss some key elements of that.

The Apache voting process has been tried and tested since 1999, or even earlier. This is one of the things that projects coming through the incubator have to learn, led by their mentors. Learning is usually very easy as people quickly see the benefits of those simple no-nonsense rules.

The ASF also provides a well-defined structure for managing projects, and the foundation as a whole, in a fair and consensus-driven way. One could argue that structure gets in the way, and sometimes it does, but when things go wrong having a well-defined way of getting back on track helps tremendously. And this structure leaves a lot of freedom to the project’s management committee (PMC), there’s a lot of room for adapting a project’s way of working to its community and goals.

Creating an Apache project is certainly not required for all open source projects (and the foundation couldn’t scale to thousands of projects right now anyway), but for the critical infrastructure parts of one’s business (what’s sometimes called “open core”), having an established governance model makes all the difference.

The governance model is just one of the benefits that Apache projects get – there’s also the visibility, brand recognition, nice build services, and other tools, and, last but not least, the many friends that you make along the way! As everybody now knows, there are no jerks at Apache!

We had a great NoSQL meeting yesterday evening colocated with ApacheCon. Thanks Jukka for organizing!

I was in track B for the second part, and found it very interesting to compare three different approaches to non-relational content storage: MarkLogic server, JCR and Pier Fumagalli’s Lucene+DAV technique.

I also quite liked Steve Yen’s “horseless carriage” way of looking at NoSQL. Defining things by what they are, as opposed to what they are not, sounds like a good idea.

I gave a short talk about JCR, find the slides below. Of course, as usual, they’re not as good as when I’m here to talk about them ;-)

I have just finished my slides for next week at ApacheCon. Though the topic of how to “survive” in our open source communities has been on my mind for a while, this is a totally new presentation, which is both great (in the blank slate sense) and a lot of work.

Having recently read Presentation Zen (very recommended if you do presentations and/or like beautiful books), I started adding full-screen pictures to the first few slides, and couldn’t stop! The presentation will then consist of me ad-libbing (or more precisely trying to tell stories) on a series of nice pictures grabbed from morguefile.com (don’t worry about that name).

I’ll post the slides here later, for now they are super secret, so you’ll just get the teasers…images courtesy of morguefile.com (update: slides added now).

Hope to see you next week! In any case I have collected a number of useful links in my delicious bookmarks, I’ll point people to them in the presentation.

I was at OpenWorldForum in Paris a few weeks ago, together with fellow Apache members Sander Striker and Emmanuel Lécharny.

My first impressions (apart from the fact that Paris is always nice – I knew that already) were that the French tend to wear suits and say “vous” (polite form of “you”) instead of “tu” (the familiar form) which I would tend to use in geeky circles. Cultural differences…

But more seriously, how well does the french-speaking world know the Apache Software Foundation? Not well, it seems to me. In most of our discussions people could associate the ASF with the Apache HTTP Server project, but not much more. 2′000 committers? 300 members? Really?

To help improve this, I hope that the ASF can take a more active role in the conference next year, I’ll bring this up next week at ApacheCon with our conference people.

In the meantime french-speaking folks are welcome to learn a bit more about it thanks to TiViPRO’s interviews of Emmanuel and myself, shot during the conference.

For me the major IKS project event this quarter is without a doubt the Workshop in Salzburg at the end of May that I blogged about before.

Talk is cheap and implementation is another story – we now have to implement the ideas discussed there, in my case mostly the search engine and user stories. We’ve got started on those, but most of the work is still ahead of us.

We have also received the first benchmark requirements, where we’ll have to evaluate our product’s semantic search abilities, and create some prototype extensions that allow us to better cover this area.

With so many CMS luminaries at the Salzburg workshop, we have started to create an IKS community – at least when it comes to agreeing on what IKS should be, and that’s great already. Now’s the time to move to the next level and actually start creating useful software for and with this community.

Well, not exactly dead, but today the Apache Software Foundation elected a new board of directors:

2009 board: Shane Curcuru, Doug Cutting, Justin Erenkrantz, Roy T. Fielding, Jim Jagielski, Geir Magnusson, Jr., Brian McCallister, Brett Porter, Greg Stein

Congrats to all! There were a lot of excellent candidates this time, and the new board looks like a nice mix of great people. Looking forward to what this year brings.

And thanks to the outgoing board – being part of it has been a rich learning experience for me. Hard work at times, but it’s great to have more insight about how the foundation works and to be able to help.

I’ll list the names of the 2008 board here, so that I can find them in 50 years when we talk about the good times:

2008 board: Justin Erenkrantz, J Aaron Farr, Jim Jagielski (chairman), Geir Magnusson Jr, William Rowe Jr, Sam Ruby, Henning Schmiedehausen, Greg Stein, Bertrand Delacretaz

Shane Curcuru has created a nice timeline of all board members from the beginning to 2008.

The ASF uses a (way too) large number of mailing lists for all its internal and project communications.

Having crosscutting discussions is quite hard – for example, many projects use OSGi these days, and the only way for them to share their OSGi experience would be to create yet another list, or to subscribe to all of each other’s lists, which means a lot more traffic to manage.

One of my current technical dreams is to have a single list for all of the ASF, using tags to define the audience and visibility of messages – a la Twitter hashtags.

A message about the maven-scr-plugin on the Sling list, for example, would be tagged

#sling #osgi #maven-scr-plugin #scr #public

so that people subscribing to the #osgi and #scr tags, for example, would see it.

Another obvious use case is to easily ignore all discussions about a given topic (like #budget maybe? ;-), in a reliable way and without losing other communications within the same group.

I’m not sure how to implement this today (particularly the access control part for things like the #asf-private tag), but that would in my opinion be a huge improvement on what we have now.

I’m on my way back from Salzburg where the Salzburg Research team organized a great meeting for the IKS project. Flawless organization as usual, thanks and congrats!

Today’s requirements workshop featured an impressive collection of very powerful brains (and nice people to hold them ;-) including, besides the usual IKS suspects, representatives from more than twenty CMS communities and companies.

I was a bit worried at first that IKS, being mostly in a requirements definition phase, didn’t have much to show to those people, but today’s brainstorming went very well, and the results exceed my expectations.

The most important result for me is agreeing to setup a prototype semantically enhanced search engine, that will use metadata and RDFa embedded in web pages to index content. This will provide the IKS community with a testbed for semantically enhanced websites, and allow us to demonstrate the usefulness of embedded semantic information by making full use of that for searching instead of just enhancing the display of search results. The extracted data might also be very useful for our academic partners to run experiments on real-life data that we’re familiar with. We might not have to write lots of code to setup such a search engine, but it’s important to have our own thing that people can also run behind firewalls, if needed to run experiments on private data.

The second result that I’m excited about is agreeing to work together on a prototype of a semantic rich text content editor, where you’ll get functions like insert person or insert company besides the usual insert link and insert image functions. This will allow us to start making our customers more aware of the importance of semantic markup, in a way that’s not too different from what they’re doing now.

Last but not least in my list of results-that-got-me-excited-about-all-this is agreeing on the creation of a list of simple user stories that demonstrate what IKS is about, in a very simple and understandable way, while allowing us to define use cases and features that might be challenging to implement today.

More complete information about the meeting should be available from the IKS project blog in the next few days, make sure to subscribe to that. For now Bergie (who suggested the semantic editor project) has been taking notes on Quaiku if you’re eager to learn more.

To take part in (or just follow) these projects, subscribe to the IKS mailing list which is going to be our communicatios hub.

Hope to see you there – in a week from now, as next week is my cycling-in-France/offline holiday. Looking forward to getting more familiar with the 29er before the next, more off-road trip in a few weeks.