Adobe, Day and Open Source: a dream and a nightmare

July 30, 2010

What does the acquisition of Day by Adobe mean for Day’s open source activities? Some people are disappointed by the lack of comments about this in the official announcements to date.

Thankfully, Erik Larson, senior director of product management and strategy at Adobe, commented on Glyn Moody’s blog post quite early in the frenzy of tweets and blog posts that followed yesterday’s announcement.

Quoting him:

…we are very excited for Day’s considerable “open source savvy” to expand Adobe’s already significant open source efforts and expertise. That is a strategic benefit of the combination of the two companies. I have personally learned a lot from David Nuscheler and his team in the past few months as we put the deal together.

Not bad for a start, but we’re engineers right? Used to consider the worst case, to make sure we’re prepared for it.

Me, I’m an engineer but also an optimistic, and I’m used to start with the ideal, happy case when analyzing situations. It helps focus my efforts on a worthy goal.

So let’s do this and dream about the best and worst cases. This is absolutely 100% totally my own dreams, I’m not speaking for anyone here, not wearing any hat. Just dreamin’, y’know?

The Dream

This is late 2011.

The last few months have more than confirmed that Day’s acquisition by Adobe, one year ago, happened for strategic reasons: a big part of the deal was filling up gaps in Adobe’s enterprise offering, but Day’s open source know-how and network have brought a lot of value as well.

Day folks have played an important role in expanding the open development culture inside Adobe; Photoshop will probably never be fully open source, but moving more key components of the Adobe technology stack to open source, and most importantly open development, has paid off nicely. In terms of reaching out to developers and customers, in getting much better feedback at all levels, and in terms of software quality of course. It’s those eyeballs.

The Apache Software Foundation’s Incubator has been quite busy in the last few months. The new platinum sponsor enjoys a fruitful relationship with the foundation.

With JCR moving to their core, Adobe’s enterprise applications are starting to reach a new level of flexibility. Customers are enthusiastic about being able to access their data via simple and standards-based interfaces. Enterprise-level mashups, anyone?

JCR is not just that minor content repository API pushed by that small swiss software vendor anymore: being adopted by a major player has made a huge difference in terms of market recognition (I’m sure my friends at Hippo, Jahia and Sakai, among others, will love that one). The added resources have also helped improve the implementations, and people love the book!

With this, Apache Jackrabbit and Apache Sling have reached new levels of community participation and quality. Although quite a few new committers are from Adobe, a number of other companies have also pushed their developers to participate more, due to the increased market visibility of JCR.

Adobe’s additional resources, used wisely to take advantage of the Day team’s strengths, have enabled them to fully realize the CQ5 vision. Everything is content, really.

As in all fairy tales, the former Day team and Adobe live happily ever after. (Editor’s note: this is not Disney, can we strike that one please?)

The Nightmare

This is late 2011, and I can hear the programmers complaining in their bland cubicles.

Aaarrggghhhhh.

The few Day folks who still work at Adobe did try to convince their management to continue on the open source and open development track. No luck – you can’t argue with an US company making 4 billion a year, can you?

CQ5 customers are too busy converting their websites to native PDF (this is about documents, right?) to realize what’s going on. The most desperate just switched to DrooplaPress, the newest kid on the LISP-based CMSes block. That won’t help business much but at least it’s fun to work with. If you love parentheses, that is.

Adobe’s competitors who really jumped on the open source and open development train are gone for good, it is too late to catch up. You should have sold you shares a year ago.

Luckily, Apache Jackrabbit and Apache Sling are still alive, and increased involvement of the “Benelux Gang” (ex-Day folks spread over a few Benelux content management companies) in those projects means there’s still hope.

You wake up wondering why you didn’t accept that job at the local fast food. Computers are so boring.

Coda

I know life is more complicated than dreams sometimes, but I like dreams much better than nightmares, and I’m a chronic optimistic. So you can easily guess which scenario I’m going to work towards!

I’ll keep you posted about what really happens next. Once I wake up, that is.

Just dreamin’, y’know?

Related reading

Open Source at Adobe by my colleague and fellow Apache Member Jukka Zitting.

Open innovation in software means Open Source, a recent post of mine.

See also my collected links related to the announcement at http://delicious.com/bdelacretaz/adobeday.


Dear Oracle, can we have our nice javadoc URLs back?

July 21, 2010

If you support this request, please vote for it in the comments below and/or on twitter using the #E17476 hashtag!

Update (2010/07/24): it looks like the old java.sun.com URLs are back, thanks Oracle and especially @mreinhold!

Update (2010/07/27): see also Good Feedback and Happy Endings – The Ugly URLs.

Dear Oracle,

A while ago you bought Sun, and IIRC promised to do good things for Java. Or at least indicated you would. Or something like that.

Now, a bad thing happened a few days ago. Not a bad bad bad thing, just a tiny annoying change in the cool URLs that Sun used to publish the JDK’s javadocs. Not annoying annoying annoying but not nice.

Even Google remembers: today if I search for IndexOutOfBoundsException on Google it returns the following URL:

http://java.sun.com/j2se/1.5.0/docs/api/java/lang/IndexOutOfBoundsException.html

Which is a cool URL that shouldn’t change.

Now, requesting this URL today causes a redirect to:

http://download.oracle.com/docs/cd/E17476_01/javase/1.5.0/docs/api/java/lang/IndexOutOfBoundsException.html

Which is also somewhat cool, but not as much. Factor 10 down in coolness. It makes me assume that you’re serving javadocs from a CD, and that CD’s identifier is E17476_01. That’s useful info if you’re the filesystem driver who’s reading the CD, but I doubt filesystem drivers are searching for javadocs on Google. Also, I’m not looking at downloading anything. Just browsing, okay?

Cool URLs shouldn’t change.

Can we have the old one back? Ok, maybe with java.oracle.com instead of java.sun.com – you bought them anyway. But please please please, let the poor CD filesystem driver alone!

Thanks.

P.S. we’re having a little vote on Twitter about this, check it out at https://twitter.com/hashtag/e17476 (URL updated in 2018 to account for Twitter API changes)


This is how we work at Apache

July 16, 2010

I just had to (re-)explain how the Apache way of working makes a difference by enabling a continuous flow of information between developers.

No more begging for reports, no more boring meetings where you only exchange information: who could say no to that?

Here it is for your enjoyment. This is the same thing that I’ve been saying in my recent talks on this topic, reduced to the bare minimum.

  • All technical discussions and decisions on public mailing lists.
  • Speak in URLs: if you reference something (discussion, vote,
    code…anything) include its URL, which must be permanent.
  • Shared code repository, commit early, commit often (as in: daily at least, from day one)
  • Commit events sent to mailing lists and/or RSS feeds to which people
    can subscribe.
  • Shared issue tracker and “if you’re working on something it must be
    an issue in the tracker”
    so that progress reports are automatic. Also generates mail/RSS events.
  • Commits are linked to tracker issue IDs – by speaking in URLs in your commit messages, mostly.
  • Automatic archiving of all this information, for self-service access.

All this is public and centrally accessible of course, so everybody
gets the same information.

The main reluctance that I see when trying to convince people to work
in this way is the fear of exposing your mistakes and initial bad
designs in public. My answer is to just get over it: you’d find tons
of such blunders if you were to analyze my work at Apache in the last
ten years, yet I’m reasonably alive and kicking.


List all your Maven dependencies

July 8, 2010

Here’s a one-liner (well, two) that neatly lists all the Maven dependencies from your project. Useful to check their licenses, for example.

# first grab all dependencies
mvn dependency:resolve

# then list them with -o to keep noise low,
# remove extra information and duplicates
mvn -o dependency:list \
| grep ":.*:.*:.*" \
| cut -d] -f2- \
| sed 's/:[a-z]*$//g' \
| sort -u 

The output looks like this:

asm:asm:jar:1.5.3
asm:asm:jar:3.1
biz.aQute:bnd:jar:0.0.169
cglib:cglib:jar:2.1_3
classworlds:classworlds:jar:1.1
classworlds:classworlds:jar:1.1-alpha-2
...

so it’s also useful to detect multiple versions of the same dependency in a multi-module project.


Open innovation in software means open source

July 2, 2010

Here’s a “reprint” of an article that I wrote recently for the H, to introduce my talk at TransferSummit last week.

According to Henry Chesbrough[1], Open Innovation consists of using external ideas as well as internal ideas, and internal and external paths to market, to advance a company’s technology.

Software architects and developers are usually not short of ideas, but which of those ideas are the really good ones? How do you select the winning options and avoid wasting energy and money on the useless ones?

Feedback is the key to separating the wheat from the chaff. Fast and good quality feedback is required to steer any fast vehicle or sports device, and it works the same in software: without a performant feedback loop, you’re bound to fall on your face – or at least to be slower than your competitors on the road to success.

Innovation is not invention – it’s about value

In a recent blog post on the subject, Christian Verstraete, CTO at HP, rightly notes that innovation is not invention. Whereas the value of a new invention might be unknown, the goal of innovation is to produce value, often from existing ideas.

The output of our feedback loop must then be a measurement of value – and what better value for a software product than happy stakeholders? Other developers adopting your ideas, field testers happy with performance, experts suggesting internal changes which will make them feel good about your software’s structure. That kind of feedback is invaluable in steering your innovative software product in the right direction, quickly.

How fast is your feedback loop?

If you have to wait months to get that high-quality feedback, as you might in a corporate setting, your pace of innovation will be accordingly slow.

In the old world of committees, meetings and reports, things move at the speed of overstuffed schedules and overdue reports – slowly. In the new world of agile open source projects, fast and asynchronous Internet-based communication channels are your friends, helping people work at their own pace and on their own schedule, while collectively creating value quickly.

Open source organizations like the Apache Software Foundation provide standardised tools and best practices to foster efficient communications amongst project members. Shared source code repositories generate events to which project members can subscribe, to be informed immediately of any changes in modules that they’re interested in. Web-based issue trackers also use events and subscriptions to make it easy to collaborate efficiently on specific tasks, without requiring the simultaneous online presence of collaborators. Mailing lists also allow asynchronous discussions and decisions, while making all the resulting information available in self-service to new project members.

It is these shared, event-based and asynchronous communications channels that build the quick feedback loop that is key to software innovation. It is not uncommon for a software developer to receive feedback on a piece of code that they wrote, from the other end of the world, just a few minutes after committing that code to the project’s public code repository. Compared to a written problem report coming “from above” a few weeks later, when the developer has moved on to a different module, the value of that fast feedback is very high. It can feel a bit like a bunch of field experts looking over your shoulder while you’re working – scary but extremely efficient.

How good are your feedback “sensors”?

Fast feedback won’t help if it’s of low quality, and fortunately open source projects can also help a lot here. Successful projects can help bring together the best minds in the industry, to collectively solve a problem that benefits all of them. The Apache HTTP server project is one of the best examples, with many CTO-level contributors including a few that were involved in defining the protocols and the shape of today’s Web. If software developers (God forbid) were sold between companies the way soccer players are transferred between teams, we’d see millions of dollars flowing around.

Open source projects are very probably the best way to efficiently bring software experts together today. Industry associations and interest groups might fulfill that role in other industries, but developers like to express themselves in code, and open source projects are where that happens today.

You could of course hire experts to give feedback on your software inside your company, but it’s only a handful of companies who have enough money to bring in the level and number of experts that we are talking about – and that might well turn out to be much slower than the open source way of working.

What’s the right type of project?

Creating or joining an open source project that helps your business and attracts a community of experts is not that easy: the open source project space is somewhat crowded today, and those experts are busy people.

Judging from the Apache Software Foundation’s achievements in the last ten years, infrastructure projects have by far the highest success rate. If you can reduce (part of) your problem to a generalised software infrastructure that appeals to a wide range of software developers, those experts will see value in joining the project. Apache Hadoop is another very successful example of software architects and developers from different companies joining forces to solve a hard problem (large scale distributed computing) in a way that can benefit a whole industry. On a smaller scale, Apache Jackrabbit , one of the projects in which my employer is very active, brings together many experts from the content management world, to solve the problem of storing, searching and retrieving multimedia content efficiently. Those types of software modules are used as central infrastructure components in systems that share a similar architecture, while offering very different services to their end users.

Projects closer to the user interface level are often harder to manage in an open group, partly because they are often more specific to the exact problem that they solve, and also because it is often hard for people coming from different companies and cultural backgrounds to agree on the colour of the proverbial bike shed. An infrastructure software project can be well defined by an industry specification (such as JCR in Jackrabbit’s case), and/or by automated test suites. These are usually much easier to agree on than user interface mock-ups.

Where next?

I hope to have convinced you that open source projects provide the best feedback loop for innovative software. As a next step, I would recommend getting involved in open source projects that matter to you. There are many ways to contribute, from reporting bugs in a useful way, to writing tutorials, contributing new modules or extensions, or simply reporting on your use of the software in various environments.

Contributing, in any small or big way, to a successful open source project is the best way to see this high-quality feedback loop in action. You might also try to use the open source ways of working inside your company, to create or improve your own high-quality “innovation feedback loop”.

I cannot pretend to have the definitive answer to the “how do you select and execute the right ideas to innovate?” question. When it comes to software, however, the fast and high-quality feedback loop that open source projects provide is, in my opinion, the best selection tool.

[1] Chesbrough, H.W. (2003). Open Innovation: The new imperative for creating and profiting from technology. Boston: Harvard Business School Press


My new Flyer ebike: fast and fun!

July 1, 2010

flyer-t8hs.jpgI recently bought a new Flyer electric bike, got a faster T8 HS ex-demo for a good price.

I sold the previous C8+ to my nephew, happy that it’s staying in the family! In about 15’000km all year in any weather (including snow and accompanying salt on the road) over 4 1/2 years, I have had exactly zero problems with the C8, which says a lot about the build quality and maturity of those bikes. It needed just the usual bike maintenance, and an expected change of battery after about 600 charging cycles, but zero maintenance related to the electronics or motor. Not to mention only two punctures in 15’000km, thanks to the Schwalbe Marathon puncture-proof tires. Over time, those tires get full of small superficial holes which are mostly punctures that didn’t happen, very cool!

The new bike is an HS model, as in high speed: contrary to the old one which would only assist me until 25kmh (so you’d be faster on flat with a good bike), this one happily helps up to 45kmh or more, and it’s also a better bike to start with: 28″ wheels, more rigid frame and thinner higher-pressure tires. As with all so-called pedelec bikes, the Flyers don’t go anywhere if you don’t pedal, the assistance only kicks in (very naturally) when you ride normally, and gives you more power.

And this thing is fast: I just beat my record on the commute back from the office, 350m elevation over 12km, getting home in 29:30 which means 24kmh average speed on Lausanne’s steep hills. Not bad – you do have to pedal hard to reach such speeds uphill, but it’s a lot of fun and I get home almost as fast as any other transportation, considering the traffic density – and I don’t need to spend time at the gym after that, so I’m probably saving time all in all! The morning downhill ride takes about 20 minutes, unbeatable at 8AM unless you ride a helicopter.

The equipment is very good: Magura hydraulic rim brakes (almost as good as discs, I guess newer models have them), LED lights, lockable front fork and the SRAM dual drive which combines a 3-gear hub with an 8-gear rear derailleur to get 24 usable combinations (no “forbidden” ones like dual derailleurs). You cannot have a front derailleur on the Flyer due to the motor wheel which drives the chain, and the dual drive is really the best of both worlds in the city: gear hub for quick downchanging when stopping or surprised, and derailleur for fine tuning.

All in all, an excellent commuter’s bike if your ride is steep, or just for the fun of riding faster. The big plus with the ebike is that you can use less of your own energy if you’re tired or if conditions are bad, while still getting to your destination in a reasonable amount of time.

You do have to ride very carefully as sleepy car drivers and pedestrians often don’t realize how fast you ride on that thing, nor that you’re actually faster than cars in many tight or bumpy places. After years of motorcycling and cycling I’m used to being very clear about my intentions on the road, using obvious positioning in lanes, and that helps a lot! The city of Lausanne is also doing an excellent job in helping cyclists find safe space to ride, and most of my commute is on very low-traffic roads as a result.

Do I sound enthusiastic? That’s because I am – electric bikes are by far the best way of commuting in a steep city like Lausanne. They are somewhat expensive to buy, but maintenance costs almost nothing, and you save a lot on gym costs (and doctor’s fees I guess – cycling is good for your health). And if you drive a car or motorbike to work, you should really calculate how much that costs and draw the right conclusions!


Can I haz web?

June 15, 2010

How many people today still think information is only valid or “serious” when represented on an A4 piece of paper?

Way too many, if you ask me.

I’m always disappointed when people push out important content as PDF documents (or much worse…I won’t even name that format) attached to web pages or email messages, instead of just including the content in those web pages or messages, as a first-class citizen.

For some reason, people seem to think that information presented in A4 format has more value than the same information presented as a simple and clean web page. It is quite the opposite actually: web pages can be linked to, easily indexed, reformatted for efficient reading (thanks readability), etc.

Ted Nelson, the inventor of hypertext, wrote in 1999 already [1]:

We must overthrow the paper model, with its four prison walls and peephole one-way links

And also, in the same paper:

WYSIWYG generally means “What You See Is What You Get” — meaning what you get when you print it out. In other words, paper is the flat heart of most of today’s software concepts.

Granted, we haven’t fully solved the two-way links problem yet, but I hope you get the idea. Who needs paper or A4 pages? This is 2010, and this is the Web.

Please think about it next time you publish an important piece of information. Does it really need to live in the prison walls of a “document”? In what ways is that more valid than a web page or plain text email message?

Most of the time, almost always, the answer is: it’s not more valid, it’s just less usable.

Can I haz web? kthxbye.

[1] http://people.artcenter.edu/~vanallen/web_techniques/tednelson_liberate.htm