Apache: lean and mean, durable, fun!

May 19, 2017

Here’s another blog post of mine that was initially published by Computerworld UK.

My current Fiat Punto Sport is the second Diesel car that I own, and I love those engines. Very smooth yet quite powerful acceleration, good fuel savings, a discount on state taxes thanks to low pollution, and it’s very reliable and durable. And fun to drive. How often does Grandma go “wow” when you put the throttle down in your car? That happens here, and that Grandma is not usually a car freak.

Diesel engines used to be boring, but they have made incredible progress in the last few years – while staying true to their basic principles of simplicity, robustness and reliability.

The recent noise about the Apache Software Foundation (ASF) moving to Git, or not, made me think that the ASF might well be the (turbocharged, like my car) Diesel engine of open source. And that might be a good thing.

The ASF’s best practices are geared towards project sustainability, and building communities around our projects. That might not be as flashy as creating a cool new project in three days, but sometimes you need to build something durable, and you need to be able to provide your users with some reassurances that that will be the case – or that they can take over cleanly if not.

In a similar way to a high tech Diesel engine that’s built to last and operate smoothly, I think the ASF is well suited for projects that have a long term vision. We often encourage projects that want to join the ASF via its Incubator to first create a small community and release some initial code, at some other place, before joining the Foundation. That’s one way to help those projects prove that they are doing something viable, and it’s also clearly faster to get some people together and just commit some code to one of the many available code sharing services, than following the ASF’s rules for releases, voting etc.

A Japanese 4-cylinder 600cc gasoline-powered sports bike might be more exciting than my Punto on a closed track, but I don’t like driving those in day-to-day traffic or on long trips. Too brutal, requires way too much attention. There’s space for both that and my car’s high tech Diesel engine, and I like both styles actually, depending on the context.

Open Source communities are not one-size-fits-all: there’s space for different types of communities, and by exposing each community’s positive aspects, instead of trying to get them to fight each other, we might just grow the collective pie and live happily ever after (there’s a not-so-hidden message to sensationalistic bloggers in that last paragraph).

I’m very happy with the ASF being the turbocharged Diesel engine of Open Source – it does have to stay on its toes to make sure it doesn’t turn into a boring old-style Diesel, but there’s no need to rush evolution. There’s space for different styles.

Shared neurons and the Shadok’s First Law of Failure

January 30, 2017

This blog post of mine was originally published by Computerworld UK in 2010.

French-speaking fortysomethings might remember the Shadoks, a two-minute TV comics show that aired on ORTF when I was a kid.

Those silly (and funny) creatures need to get to Earth as their own planet, on the left of the sky, is falling into pieces. They spend their time pumping on bicycle-like contraptions to produce Cosmogol 999 fuel to power their rocket towards Earth, while respected Professor Shadoko keeps them motivated with his motto: “the more you fail, the more likely your are to eventually succeed”. So they keep failing, every single time, hoping that statistics will prove them right some day.

A researcher colleague recently asked me to demonstrate that the work done by Apache communities is more than the sum of the individual capabilities of its members. I think the answer is yes, and that how we cope with failure has a lot to do with it.

Demonstrating this with hard numbers would be difficult, but having been active in Apache projects for the last ten years I can think of a number of examples where what we sometimes call “shared neurons” make all the difference.

It usually starts with a random thought – an often half-baked idea, that might be totally unrealistic or impossible to implement, but is nevertheless exposed to the community as is, and often starts a fruitful discussion. People bounce off each other’s ideas, in a written brainstorm that’s slower than if you were talking face to face, but sometimes more efficient as people have more time to think about their contributions to the discussion.

Brainstorming in the same room is faster, but brainstorming with a worldwide group of specialists – that’s much more powerful than Cosmogol 999, even if it has to happen in writing. But sometimes that turns into a Shadok fuel-producing exercise that fails to produce useful results.

Some Apache (and other) projects use those random thoughts as a first-class design and architecture tool, marking such email discussions with [RT] in their subject lines. This serves as a warning that the discussion is not necessarily about something feasible, it’s more about putting people’s neurons together to shape and refine potentially useful ideas. And sometimes those ideas are just too silly or unrealistic to go anywhere – yet the corresponding discussions stay archived forever on our mailing lists.

Some people are afraid of exposing their unfinished ideas to their communities so early, and having them archived for posterity whatever the outcome. Your idea might turn out to be a stupid one once people nicely demonstrate why it won’t work, or someone might point you to that part of code that does exactly what you were dreaming about, and that you had forgotten about. Silly you.

In my opinion, being ready to accept such failures makes you a much more efficient contributor to open communities. Although you don’t need to fail as consistently as the Shadoks, being ready to fall on your face once in a while allows you to contribute your wildest ideas to the discussion. Combined with respect for other’s crazy ideas, and with the ability to explain nicely when people are wrong, we have a recipe for efficient sharing of neurons, where the sum is definitely greater than its parts.

I’m seeing this neuron sharing in action on our Apache mailing lists all the time, and I think accepting the risk of being wrong when exposing one’s ideas on our public lists makes a big difference. We’re not Shadoks after all – our failure stats are way too low.

Continous Deployment with Apache Sling

September 2, 2014

Today I had the pleasure to attend the Master’s thesis defense of our intern Artyom Stetsenko, titled Continous Deployment of Apache Sling Applications.

Coaching Artyom for this project has been a pleasure, he did a great job and worked independently while listening very well to our advice. He got an excellent mark for his thesis and that was well deserved. Also due to an excellent no-bullets presentation!

I have uploaded Artyom’s thesis paper here, with his permission. The code is available at https://github.com/ArtyomStetsenko/sling-devops-experiments. As the name indicates that’s experimental code, but the resulting Sling-based cluster with automated atomic deployment is functional. Just push an updated crankstart file to the Git repository and the cluster is updated atomically and without downtime.

For me the main goal was to see how we can improve Apache Sling‘s support of modern operations, with continuous deployment, immutable instances etc. I’m continuing my explorations with a Docker-based Sling cluster, the main goal being to create simple clustered environments that allow us to play with these things.

Update: I forgot to mention that my Docker cluster prototype is the basis for my upcoming talk at adaptTo() 2014 on September 23rd in Berlin. The talk’s title is “Apache Sling and devops – the next frontier” and I’ll talk about how Sling can be made more operations-friendly.

Generating hard to guess content URLs in Sling

October 27, 2010

In RESTful apps, it is often useful to create hard to guess URLs, as a simple privacy device.

Here’s a self-explaining example (with hardcoded parameters) of how to do that in Sling.

After installing this component, an HTTP POST to a node named ‘foo’ creates a child node with a somewhat long hex string as its name, instead of the usual simple names generated by Sling.

package foo;

import java.util.Random;

import org.apache.felix.scr.annotations.Component;
import org.apache.felix.scr.annotations.Service;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.servlets.post.NodeNameGenerator;

/** Example that generates hard-to-guess node names in Sling,
 * for nodes added under nodes named 'foo' 
 * To test, build and install a bundle that includes this component,
 * and run
 * <pre>
 *   curl -X MKCOL http://admin:admin@localhost:4502/foo
 *   curl -F title=bar http://admin:admin@localhost:4502/foo/
 * </pre>
 * The output of the second curl call should return something like
 * <pre>
 *   Content created /foo/dd712dd234637bb9a9a3b3a10221eb1f
 * </pre>
 * Which is the path of the created node. 
public class FooNodeNameGenerator implements NodeNameGenerator {
    private static final Random random = new Random(System.currentTimeMillis());
    /** @inheritDoc */
    public String getNodeName(
            SlingHttpServletRequest request, 
            String parentPath, 
            boolean requirePrefix, 
            NodeNameGenerator defaultNng) 
        if(parentPath.endsWith("/foo")) {
            final StringBuilder name = new StringBuilder();
            for(int i=0; i < 2; i++) {
            return name.toString();
        return null;

My best (and worst) ApacheCon so far – thanks and random thoughts

November 8, 2008

US 2008 in New Orleans has been my best ApacheCon so far! Brain dump ahead.

Great socializing and networking, which should lead to new exciting new projects in the near future.

My two talks were well received as far as I can tell.

Nice emerging buzz about Sling. Free polo shirts helped – well done Carsten! We have to do something about our docs and examples so as to not scare people away, in the meantime I’m sure the Sling folks will be happy to offer guidance on how best to structure things if you want to try it.

Big thanks to everybody involved in this very successful event – it is impossible to mention everybody here, so I’ll just pick and choose: Shane Curcuru who did a great job as the conference lead. The Stone Circle team for making everything happen in a flawless way.

Thanks to President elect Obama for allowing me to spend a historical day in the US.

Thanks to Glen Daniels for finding the jam session places – the first one was not too hot from the musical point of view (drumming on a beer keg only goes so far), but interesting to visit – ever seen a laundromat in a bar?

Thanks to the folks (sponsors IIUC) who organized the parade on Thursday evening. Walking in the streets following the great Rebirth Brass Band with cops opening the way on their nice Harleys was awesome.

Thanks to the New Orleans people for being so nice and friendly, even though I sometimes had a hard time understanding them. I’ll work on that.

For some reason, swimming in the outdoor pool at 7:30AM on a foggy day made us suspect in the eyes of the security guards. Normal hotel customers don’t do that I guess, but who said Swiss Apache folk are normal hotel guests?

Thanks to the local musicians for playing so well.

On Tuesday a few of us escaped from the barcamp (which was great, but one cannot do everything I guess) for a swamp tour with Cajun encounters. Definitely recommended, our guide was a genuine Cajun, born in the area and with lots of stories to tell. I didn’t bring a camera, but photos should be available on Flickr.

That’s for the “best ApacheCon” part – the “worst ApacheCon” is about jetlag taking almost a week to go away, and leaving me in a miserable state on Tuesday when I had a lot of work to do to finish preparing my talks and for the Big Release.

For some reason, many of us had a hard time with jetlag this week. That might be related to the food (deep fried everything) or the lack of fresh air and sunlight. Not that it wasn’t sunny, but sunset around 6PM means one does not see much natural light if following the conference sessions.

See you in Amsterdam in March for ApacheCon EU 2009!

(long post hey – that’s what you get for locking me in planes for about 10 hours ;-)

No LIFT – but thanks nouvo for the video

February 8, 2008

No LIFT for me once again this year, due to a timing collision with The Important Phase of That Project.

But thanks to nouvo.ch many videos are available to catch up. It’s not the same thing though, there are many people who I’d have loved to meet there.

Google Groups, could you talk to your Gmail cousin a bit more?

February 8, 2008

Even though Gmail bothers us with its Sender header, Google Groups’s set of virtual gray matter is apparently too small to figure out that

From: "Bertrand Delacretaz" <myself@somewhere.ch>
Sender: myself@gmail.com

comes from myself@gmail.com.

Can’t those guys talk together?