Pragmatic validation metrics for third-party software components

October 22, 2010

Earlier this week at the IKS general assembly I was asked to present a set of industrial validation metrics for the open source software components that IKS is producing.

Being my pragmatic self, I decided to avoid any academic/abstract stuff and focus on concrete metrics that help us provide value-adding solutions to our customers in the long term.

Here’s the result, for a hypothetical FOO software component.

Metrics are numbered VMx to make it clear what we’ll be arguing about when it comes to evaluating IKS software.

VM1
Do I understand what FOO is?
VM2
Does FOO add value to my product?
VM3
Is that added value demonstrable/sellable to my customers?
VM4
Can I easily run FOO alongside with or inside my product?
VM5
Is the impact of FOO on runtime infrastructure requirements acceptable?
VM6
How good is the FOO API when it comes to integrating with my product?
VM7
Is FOO robust and functional enough to be used in production at the enterprise level?
VM8
Is the FOO test suite good enough as a functionality and non-regression “quality gate”?
VM9
Is the FOO licence (both copyright and patents) acceptable to me?
VM10
Can I participate in FOO’s development and influence it in a fair and balanced way?
VM11
Do I know who I should talk to for support and future development of FOO?
VM12
Am I confident that FOO still going to be available and maintained once the IKS funding period is over?

VM1 can be surprisingly hard to fulfill when working on researchy/experimental stuff ;-)

Suggestions for improvements are welcome in this post’s comments, as usual.

Thanks to Alex Conconi who contributed VM11.


Adobe, Day and Open Source: a dream and a nightmare

July 30, 2010

What does the acquisition of Day by Adobe mean for Day’s open source activities? Some people are disappointed by the lack of comments about this in the official announcements to date.

Thankfully, Erik Larson, senior director of product management and strategy at Adobe, commented on Glyn Moody’s blog post quite early in the frenzy of tweets and blog posts that followed yesterday’s announcement.

Quoting him:

…we are very excited for Day’s considerable “open source savvy” to expand Adobe’s already significant open source efforts and expertise. That is a strategic benefit of the combination of the two companies. I have personally learned a lot from David Nuscheler and his team in the past few months as we put the deal together.

Not bad for a start, but we’re engineers right? Used to consider the worst case, to make sure we’re prepared for it.

Me, I’m an engineer but also an optimistic, and I’m used to start with the ideal, happy case when analyzing situations. It helps focus my efforts on a worthy goal.

So let’s do this and dream about the best and worst cases. This is absolutely 100% totally my own dreams, I’m not speaking for anyone here, not wearing any hat. Just dreamin’, y’know?

The Dream

This is late 2011.

The last few months have more than confirmed that Day’s acquisition by Adobe, one year ago, happened for strategic reasons: a big part of the deal was filling up gaps in Adobe’s enterprise offering, but Day’s open source know-how and network have brought a lot of value as well.

Day folks have played an important role in expanding the open development culture inside Adobe; Photoshop will probably never be fully open source, but moving more key components of the Adobe technology stack to open source, and most importantly open development, has paid off nicely. In terms of reaching out to developers and customers, in getting much better feedback at all levels, and in terms of software quality of course. It’s those eyeballs.

The Apache Software Foundation’s Incubator has been quite busy in the last few months. The new platinum sponsor enjoys a fruitful relationship with the foundation.

With JCR moving to their core, Adobe’s enterprise applications are starting to reach a new level of flexibility. Customers are enthusiastic about being able to access their data via simple and standards-based interfaces. Enterprise-level mashups, anyone?

JCR is not just that minor content repository API pushed by that small swiss software vendor anymore: being adopted by a major player has made a huge difference in terms of market recognition (I’m sure my friends at Hippo, Jahia and Sakai, among others, will love that one). The added resources have also helped improve the implementations, and people love the book!

With this, Apache Jackrabbit and Apache Sling have reached new levels of community participation and quality. Although quite a few new committers are from Adobe, a number of other companies have also pushed their developers to participate more, due to the increased market visibility of JCR.

Adobe’s additional resources, used wisely to take advantage of the Day team’s strengths, have enabled them to fully realize the CQ5 vision. Everything is content, really.

As in all fairy tales, the former Day team and Adobe live happily ever after. (Editor’s note: this is not Disney, can we strike that one please?)

The Nightmare

This is late 2011, and I can hear the programmers complaining in their bland cubicles.

Aaarrggghhhhh.

The few Day folks who still work at Adobe did try to convince their management to continue on the open source and open development track. No luck – you can’t argue with an US company making 4 billion a year, can you?

CQ5 customers are too busy converting their websites to native PDF (this is about documents, right?) to realize what’s going on. The most desperate just switched to DrooplaPress, the newest kid on the LISP-based CMSes block. That won’t help business much but at least it’s fun to work with. If you love parentheses, that is.

Adobe’s competitors who really jumped on the open source and open development train are gone for good, it is too late to catch up. You should have sold you shares a year ago.

Luckily, Apache Jackrabbit and Apache Sling are still alive, and increased involvement of the “Benelux Gang” (ex-Day folks spread over a few Benelux content management companies) in those projects means there’s still hope.

You wake up wondering why you didn’t accept that job at the local fast food. Computers are so boring.

Coda

I know life is more complicated than dreams sometimes, but I like dreams much better than nightmares, and I’m a chronic optimistic. So you can easily guess which scenario I’m going to work towards!

I’ll keep you posted about what really happens next. Once I wake up, that is.

Just dreamin’, y’know?

Related reading

Open Source at Adobe by my colleague and fellow Apache Member Jukka Zitting.

Open innovation in software means Open Source, a recent post of mine.

See also my collected links related to the announcement at http://delicious.com/bdelacretaz/adobeday.


This is how we work at Apache

July 16, 2010

I just had to (re-)explain how the Apache way of working makes a difference by enabling a continuous flow of information between developers.

No more begging for reports, no more boring meetings where you only exchange information: who could say no to that?

Here it is for your enjoyment. This is the same thing that I’ve been saying in my recent talks on this topic, reduced to the bare minimum.

  • All technical discussions and decisions on public mailing lists.
  • Speak in URLs: if you reference something (discussion, vote,
    code…anything) include its URL, which must be permanent.
  • Shared code repository, commit early, commit often (as in: daily at least, from day one)
  • Commit events sent to mailing lists and/or RSS feeds to which people
    can subscribe.
  • Shared issue tracker and “if you’re working on something it must be
    an issue in the tracker”
    so that progress reports are automatic. Also generates mail/RSS events.
  • Commits are linked to tracker issue IDs – by speaking in URLs in your commit messages, mostly.
  • Automatic archiving of all this information, for self-service access.

All this is public and centrally accessible of course, so everybody
gets the same information.

The main reluctance that I see when trying to convince people to work
in this way is the fear of exposing your mistakes and initial bad
designs in public. My answer is to just get over it: you’d find tons
of such blunders if you were to analyze my work at Apache in the last
ten years, yet I’m reasonably alive and kicking.


Open innovation in software means open source

July 2, 2010

Here’s a “reprint” of an article that I wrote recently for the H, to introduce my talk at TransferSummit last week.

According to Henry Chesbrough[1], Open Innovation consists of using external ideas as well as internal ideas, and internal and external paths to market, to advance a company’s technology.

Software architects and developers are usually not short of ideas, but which of those ideas are the really good ones? How do you select the winning options and avoid wasting energy and money on the useless ones?

Feedback is the key to separating the wheat from the chaff. Fast and good quality feedback is required to steer any fast vehicle or sports device, and it works the same in software: without a performant feedback loop, you’re bound to fall on your face – or at least to be slower than your competitors on the road to success.

Innovation is not invention – it’s about value

In a recent blog post on the subject, Christian Verstraete, CTO at HP, rightly notes that innovation is not invention. Whereas the value of a new invention might be unknown, the goal of innovation is to produce value, often from existing ideas.

The output of our feedback loop must then be a measurement of value – and what better value for a software product than happy stakeholders? Other developers adopting your ideas, field testers happy with performance, experts suggesting internal changes which will make them feel good about your software’s structure. That kind of feedback is invaluable in steering your innovative software product in the right direction, quickly.

How fast is your feedback loop?

If you have to wait months to get that high-quality feedback, as you might in a corporate setting, your pace of innovation will be accordingly slow.

In the old world of committees, meetings and reports, things move at the speed of overstuffed schedules and overdue reports – slowly. In the new world of agile open source projects, fast and asynchronous Internet-based communication channels are your friends, helping people work at their own pace and on their own schedule, while collectively creating value quickly.

Open source organizations like the Apache Software Foundation provide standardised tools and best practices to foster efficient communications amongst project members. Shared source code repositories generate events to which project members can subscribe, to be informed immediately of any changes in modules that they’re interested in. Web-based issue trackers also use events and subscriptions to make it easy to collaborate efficiently on specific tasks, without requiring the simultaneous online presence of collaborators. Mailing lists also allow asynchronous discussions and decisions, while making all the resulting information available in self-service to new project members.

It is these shared, event-based and asynchronous communications channels that build the quick feedback loop that is key to software innovation. It is not uncommon for a software developer to receive feedback on a piece of code that they wrote, from the other end of the world, just a few minutes after committing that code to the project’s public code repository. Compared to a written problem report coming “from above” a few weeks later, when the developer has moved on to a different module, the value of that fast feedback is very high. It can feel a bit like a bunch of field experts looking over your shoulder while you’re working – scary but extremely efficient.

How good are your feedback “sensors”?

Fast feedback won’t help if it’s of low quality, and fortunately open source projects can also help a lot here. Successful projects can help bring together the best minds in the industry, to collectively solve a problem that benefits all of them. The Apache HTTP server project is one of the best examples, with many CTO-level contributors including a few that were involved in defining the protocols and the shape of today’s Web. If software developers (God forbid) were sold between companies the way soccer players are transferred between teams, we’d see millions of dollars flowing around.

Open source projects are very probably the best way to efficiently bring software experts together today. Industry associations and interest groups might fulfill that role in other industries, but developers like to express themselves in code, and open source projects are where that happens today.

You could of course hire experts to give feedback on your software inside your company, but it’s only a handful of companies who have enough money to bring in the level and number of experts that we are talking about – and that might well turn out to be much slower than the open source way of working.

What’s the right type of project?

Creating or joining an open source project that helps your business and attracts a community of experts is not that easy: the open source project space is somewhat crowded today, and those experts are busy people.

Judging from the Apache Software Foundation’s achievements in the last ten years, infrastructure projects have by far the highest success rate. If you can reduce (part of) your problem to a generalised software infrastructure that appeals to a wide range of software developers, those experts will see value in joining the project. Apache Hadoop is another very successful example of software architects and developers from different companies joining forces to solve a hard problem (large scale distributed computing) in a way that can benefit a whole industry. On a smaller scale, Apache Jackrabbit , one of the projects in which my employer is very active, brings together many experts from the content management world, to solve the problem of storing, searching and retrieving multimedia content efficiently. Those types of software modules are used as central infrastructure components in systems that share a similar architecture, while offering very different services to their end users.

Projects closer to the user interface level are often harder to manage in an open group, partly because they are often more specific to the exact problem that they solve, and also because it is often hard for people coming from different companies and cultural backgrounds to agree on the colour of the proverbial bike shed. An infrastructure software project can be well defined by an industry specification (such as JCR in Jackrabbit’s case), and/or by automated test suites. These are usually much easier to agree on than user interface mock-ups.

Where next?

I hope to have convinced you that open source projects provide the best feedback loop for innovative software. As a next step, I would recommend getting involved in open source projects that matter to you. There are many ways to contribute, from reporting bugs in a useful way, to writing tutorials, contributing new modules or extensions, or simply reporting on your use of the software in various environments.

Contributing, in any small or big way, to a successful open source project is the best way to see this high-quality feedback loop in action. You might also try to use the open source ways of working inside your company, to create or improve your own high-quality “innovation feedback loop”.

I cannot pretend to have the definitive answer to the “how do you select and execute the right ideas to innovate?” question. When it comes to software, however, the fast and high-quality feedback loop that open source projects provide is, in my opinion, the best selection tool.

[1] Chesbrough, H.W. (2003). Open Innovation: The new imperative for creating and profiting from technology. Boston: Harvard Business School Press


Open Innovation in Software means Open Source

December 7, 2009

I’m giving a talk today at the Open Source, Open Development, Open Innovation workshop in Oxford:

Open source software is more than just a licence, it is also a software development methodology that allows companies to share resources and collaborate on non-core parts of their software/service offering. When managed well, open development enables a reduction in cost, and an increase in innovation as a result of the convergence of the best minds in the problem space. In this presentation Bertrand Delacretaz will describe how Day Software has embraced open development by positioning itself as the leaders in both open standards and open source software. We will examine how Day’s active engagement with 25 open source projects and numerous standards groups has enabled the company to become a world leader in their market and in the open source projects they participate in.

The funny thing is that the above abstract was written by Ross Gardler while waiting for my own version of it – and it says exactly what I was trying to say, only better ;-)

The event is covered by a live blog, and you can ask questions there.

To put it simply, my conclusion is that quick feedback from users and customers is key to open innovation – and open source. if done right, provides lots of feedback, fast.


What Makes Apache Tick?

November 17, 2009

apachecon_us_09_twleung.jpgLooking at the diversity of Apache Software Foundation communities, one can see a recipe for failure: people from different cultural backgrounds, different mother tongues, different employers, different timezones…all working together to create some of the best software on the planet? You must be kidding.

How can this very loose collage of disparate people pump out dozens of high-quality releases every year, often working better than more structured corporate teams? This “mystery” has been on my mind for a while, and I have identified four drivers that influence the way we use collaboration tools that play a major part in our success.

The first driver is a common vision amongst project members. The Biblical saying, “Without a vision, my people perish” is quite valid for our projects. Both using a central development mailing list for each project, and spending time to collectively define our project’s charter, helps us foster this common vision amongst project members. Every member should have the same answer to the “what are our goals?” question, so it’s important to get them to talk in a central place, where they all get the same information, as opposed to undocumented, one-to-one discussions.

Secondly, providing real-time status updates to project members is key in helping them stay on track. At Apache, this is implemented by the many events generated by our collaboration tools: commit events to indicate code changes, issue tracker events to provide updates about the status of bugs and new features, success/failure events from continuous build systems, and standardized ways of announcing releases so that other projects are informed. Project members subscribe freely to as few or as many event channels that they want to, so as to stay on top of things in near real-time, and without having to actively ask others about what happened. Status meetings? No need for those, as the information is flowing all the time.

The third success driver lies in enabling real-time help requests. In an immediate crisis of the “we need to deliver this by tomorrow” type, especially when working with a big team, you need to be able to ask for help without necessarily 1) knowing who specifically will help, and 2) bothering others with direct person-to-person requests, especially if they work in a different timezone. The key here is using issue trackers, where one web page stores key data and parts of the dialog that leads to resolving an issue. Posting an issue on the tracker, with sufficiently detailed instructions about how to reproduce the problem, along with attributes such as severity level, affected modules, etc., is the best way to expose a problem to the group quickly and with precision. Using an issue tracker also allows you to quickly and efficiently change priorities as well as re-assign issues and tasks – key elements that make all the difference in a crisis.

Finally, having searchable archives of this information allows new project members, or those returning after a period of absence, to learn what transpired and why things have been done in a certain way. Without self-service archives, new participants would have to talk to everybody to find out about the project’s history, past decisions, conventions, etc., which is neither efficient nor scalable. Most of our archives are automatically built as project activities progress: mailing lists are archived, source code control history is kept forever, and issue trackers write the full history of the project’s micro-decisions.

Combined with Apache’s principles of meritocracy and consensus-based decision making, these four collaboration drivers allow our project teams to work very efficiently, and, in many cases, even more so than structured teams that do not establish those central hubs of information exchange.

Does your project team foster a common vision and provide tools for real-time status updates, real-time help requests and self-service archives to its members? If yes, congratulations: you’re on a good track to becoming as effective as an Apache project!

Many thanks to Sally Khudairi for reviewing and copy editing.

ApacheCon US 2009 picture by
Ted Leung / Creative Commons License (CC BY-NC-SA 2.0).

See also my Open Source Tools are Good For You presentation, which discusses the tools that Apache projects use to implement this.


What does Apache provide that other code repositories don’t?

November 16, 2009

People thinking about creating an open source project might rightly consider hosting on one of the various hosting services available: Google Code, SourceForge, kenai, bitbucket and github come to mind. Quick and easy, create a repository or request some resources and you’re in business.

Incubating a project at the Apache Software Foundation (ASF) takes a lot more effort than just requesting a hosting space on one of those services, so why would you do that? One can perfectly host code on one of those services with an Apache License, so what’s the difference?

I think the big difference lies in the governance model, and in fact calling the ASF just a code repository is very wrong. Let’s discuss some key elements of that.

The Apache voting process has been tried and tested since 1999, or even earlier. This is one of the things that projects coming through the incubator have to learn, led by their mentors. Learning is usually very easy as people quickly see the benefits of those simple no-nonsense rules.

The ASF also provides a well-defined structure for managing projects, and the foundation as a whole, in a fair and consensus-driven way. One could argue that structure gets in the way, and sometimes it does, but when things go wrong having a well-defined way of getting back on track helps tremendously. And this structure leaves a lot of freedom to the project’s management committee (PMC), there’s a lot of room for adapting a project’s way of working to its community and goals.

Creating an Apache project is certainly not required for all open source projects (and the foundation couldn’t scale to thousands of projects right now anyway), but for the critical infrastructure parts of one’s business (what’s sometimes called “open core”), having an established governance model makes all the difference.

The governance model is just one of the benefits that Apache projects get – there’s also the visibility, brand recognition, nice build services, and other tools, and, last but not least, the many friends that you make along the way! As everybody now knows, there are no jerks at Apache!