Shared neurons and the Shadok’s First Law of Failure

January 30, 2017

This blog post of mine was originally published by Computerworld UK in 2010.

French-speaking fortysomethings might remember the Shadoks, a two-minute TV comics show that aired on ORTF when I was a kid.

Those silly (and funny) creatures need to get to Earth as their own planet, on the left of the sky, is falling into pieces. They spend their time pumping on bicycle-like contraptions to produce Cosmogol 999 fuel to power their rocket towards Earth, while respected Professor Shadoko keeps them motivated with his motto: “the more you fail, the more likely your are to eventually succeed”. So they keep failing, every single time, hoping that statistics will prove them right some day.

A researcher colleague recently asked me to demonstrate that the work done by Apache communities is more than the sum of the individual capabilities of its members. I think the answer is yes, and that how we cope with failure has a lot to do with it.

Demonstrating this with hard numbers would be difficult, but having been active in Apache projects for the last ten years I can think of a number of examples where what we sometimes call “shared neurons” make all the difference.

It usually starts with a random thought – an often half-baked idea, that might be totally unrealistic or impossible to implement, but is nevertheless exposed to the community as is, and often starts a fruitful discussion. People bounce off each other’s ideas, in a written brainstorm that’s slower than if you were talking face to face, but sometimes more efficient as people have more time to think about their contributions to the discussion.

Brainstorming in the same room is faster, but brainstorming with a worldwide group of specialists – that’s much more powerful than Cosmogol 999, even if it has to happen in writing. But sometimes that turns into a Shadok fuel-producing exercise that fails to produce useful results.

Some Apache (and other) projects use those random thoughts as a first-class design and architecture tool, marking such email discussions with [RT] in their subject lines. This serves as a warning that the discussion is not necessarily about something feasible, it’s more about putting people’s neurons together to shape and refine potentially useful ideas. And sometimes those ideas are just too silly or unrealistic to go anywhere – yet the corresponding discussions stay archived forever on our mailing lists.

Some people are afraid of exposing their unfinished ideas to their communities so early, and having them archived for posterity whatever the outcome. Your idea might turn out to be a stupid one once people nicely demonstrate why it won’t work, or someone might point you to that part of code that does exactly what you were dreaming about, and that you had forgotten about. Silly you.

In my opinion, being ready to accept such failures makes you a much more efficient contributor to open communities. Although you don’t need to fail as consistently as the Shadoks, being ready to fall on your face once in a while allows you to contribute your wildest ideas to the discussion. Combined with respect for other’s crazy ideas, and with the ability to explain nicely when people are wrong, we have a recipe for efficient sharing of neurons, where the sum is definitely greater than its parts.

I’m seeing this neuron sharing in action on our Apache mailing lists all the time, and I think accepting the risk of being wrong when exposing one’s ideas on our public lists makes a big difference. We’re not Shadoks after all – our failure stats are way too low.


Microservices? Nah…Federated Services!

March 17, 2016

As much as I like the microservices concept (*) the name does not really make sense to me.

Why would decoupled services necessarily need to be micro?

Focusing on a service’s size is the wrong crusade for me, I’m happy as long as a service does one thing and one thing well – but that thing might be too large for a micro service.

We had good discussions about this with my colleagues and on Twitter recently, and after some virtual brainstorming it’s my good Apache friend Santi Gala who came up with the Federated Services term (worth a drink-of-your-choice Santi of course, next time we meet!).

I think this describes what we are after much better. Rightsizing makes much more sense than necessarily making things very small.

So here we go, with a Twitter-compatible less-than-140-characters definition:

Federated Services are independently deployable, scalable and swarm-friendly software components with language agnostic interfaces.

That’s it. I don’t think we need more than that to define those services.

WDYT?

(*) I’m working on a blog post about some elements the history of service oriented systems with another good Apache Friend. Stay tuned!


Redeploy OSGi bundles automatically with fizzed-watcher-maven-plugin

September 17, 2015

Being able to redeploy your OSGi bundles automatically when you make changes to their source code is very useful when developing Apache Sling applications, for example.

Today I tried the fizzed-watcher-maven-plugin on a Sling sample bundle, and it seems to work quite well. I just had to add the following to my POM:


<plugin>
  <groupId>com.fizzed</groupId>
  <artifactId>fizzed-watcher-maven-plugin</artifactId>
  <version>1.0.6</version>
  <configuration>
    <watches>
      <watch>
        <directory>src/main</directory>
      </watch>
    </watches>
    <goals>
      <param>clean</param>
      <param>install</param>
    </goals>
    <profiles>
      <param>autoInstallBundle</param>
    </profiles>
  </configuration>
</plugin>

And changing any file under src/main causes the bundle to be rebuilt and (via Sling’s default autoInstallBundle profile) reinstalled in my test Sling instance.

To start the plugin with that setup I used


mvn fizzed-watcher:run -Dsling.url=http://localhost:8080/system/console

See https://github.com/fizzed/maven-plugins for more info.

Filed under: very useful.


Merry Christmas avec la Soupe a la Bière a Roger!

December 25, 2014

Recyclez votre stock de bière à deux balles pour une bonne cause!

La soupe est bonne même avec de la bière à deux balles.

While waiting for the soup to be ready and for the guests to arrive, let me wish all of my readership (yes, both of you guys) a Merry Christmas!

And if you allow me un post en français pour une fois, voici la recette de la dite soupe! Si vous connaissez Roger vous savez qu’on peut lui faire confiance pour ce genre de trucs.

L’alcool s’évapore à la cuisson bien sûr, pas de problème avec les enfants, sauf peut-être que c’est assez corsé si vous faites tout juste.

Ingrédients
Compter 1 litre de soupe pour 4 personnes.

Pour 8 litres: 2 lt bouillon de boeuf, 3 lt potage bâlois (“Basler Mehlsuppe”, en sachets pour ne pas compliquer), 3 lt bière blonde.

4 gros oignons, 150 g beurre.

2 yoghourts nature, 4 oeufs, 1/2 lt crème à fouetter, 150 g fromage râpé.

Ciboulette et persil.

Préparation

Couper fin les oignons et faire revenir dans le beurre, sans roussir.

Ajouter la bière, cuire puissamment durant 15 minutes et 12 secondes.

Préparer séparément le bouillon de boeuf et le potage bâlois.

Verser le bouillon dans la bière, cuire au taquet pendant 15 minutes et 18 secondes.

Ajouter le potage bâlois, cuire à feu moyen. Faire fondre le fromage râpé en remuant.

Battre le yoghourt et les herbes, ajouter les jaunes d’oeufs battus et incorporer en battant à donf.

Cuire doucement pendant au moins une heure et 38 secondes.

Gonfler avec la crème fouettée, au dernier moment.

La soupe se garde bien, elle est même souvent meilleure le lendemain!


Continous Deployment with Apache Sling

September 2, 2014

Today I had the pleasure to attend the Master’s thesis defense of our intern Artyom Stetsenko, titled Continous Deployment of Apache Sling Applications.

Coaching Artyom for this project has been a pleasure, he did a great job and worked independently while listening very well to our advice. He got an excellent mark for his thesis and that was well deserved. Also due to an excellent no-bullets presentation!

I have uploaded Artyom’s thesis paper here, with his permission. The code is available at https://github.com/ArtyomStetsenko/sling-devops-experiments. As the name indicates that’s experimental code, but the resulting Sling-based cluster with automated atomic deployment is functional. Just push an updated crankstart file to the Git repository and the cluster is updated atomically and without downtime.

For me the main goal was to see how we can improve Apache Sling‘s support of modern operations, with continuous deployment, immutable instances etc. I’m continuing my explorations with a Docker-based Sling cluster, the main goal being to create simple clustered environments that allow us to play with these things.

Update: I forgot to mention that my Docker cluster prototype is the basis for my upcoming talk at adaptTo() 2014 on September 23rd in Berlin. The talk’s title is “Apache Sling and devops – the next frontier” and I’ll talk about how Sling can be made more operations-friendly.


So you want to talk at this conference?

August 21, 2014

I regularly review talk submissions for tech conferences, and here’s a list of what I’m mostly looking for when deciding to accept or reject a talk.

Other reviewers might be looking for different things – this is just my own criteria.

My first question is always are you going to get people interested in your stuff. Are you a dynamic speaker who keeps people on their toes, or the kind of person that delivers their talk seated at a desk. I’ve seen the latter happen, and it’s not pretty! For me, a brilliant speaker gets a slot almost every time, also because they usually know which topics will raise people’s interest.

Unless you’re famous already, the best way to convince me that you’re a good speaker is to point to a video of one of your talks. And I also need to know why you think you’re qualified to deliver this talk.

Then, I’m looking for a topic that will add value to the conference. Promoting your product or company might not add much value, whereas a talk that will open people’s minds and maybe save them hours of work in their practice is a guaranteed winner. Signs of a value-adding topic are pointers to concrete achievements using the techniques presented in the talk.

The quality of the submission comes next, especially if I don’t know the speaker. Someone who’s unable to present their ideas clearly in a talk submission is unlikely to present them clearly at the conference. Or maybe they’re a misunderstood genius, you should also look for those but they are rare. A concise submission that packs lots of useful information about what’s going to be delivered at the talk is a good promise of success.

Last but not least, original and inspiring ideas get lots of bonus points from me. Being able to predict the abstract’s contents from the title is usually a bad sign, except if it’s a talk for beginners. We don’t need conferences to exchange information today, that’s supposed to happen on the Web. Talks should be inspiring, maybe teasers to convince people to look at your value-adding stuff, but not rehash information that’s found elsewhere.

Update: I forgot to mention the movie trailer thing: a talk abstract is a lot like a movie trailer, if you feel you’ve seen all the good parts of the movie after watching the trailer, it’s not a good sign. Similarly, a good talk abstract leaves me with the impression that there’s much more to discover in the talk, compared to what’s mentioned in the abstract.


It’s just a Web server – a plea for simplicity

June 16, 2014

I’m currently working on my keynote for next week’s Connect – Web Experience 2014 conference in Basel and very much looking forward to it! Last year’s conference was excellent, and this year’s schedule looks very exciting.

My keynote is about the value of simplicity in software – including a few tales from the trenches.

We like to think of what we build with AEM as large enterprise systems, with complex requirements. Intricate workflows. Rocket science.

However, when you think about it, our systems are “just” HTTP request processors, that manipulate atomic pieces of content in a content repository.

What if you wanted to manage the Whole World Wide Web with a single system? The architecture of that 4WCMS might be quite similar to what Apache Sling provides for AEM: mostly independent dynamic HTTP request processors, selected by path and resource type, that render and/or process resources from a huge tree of content.

If our architecture works for that 4WCMS, the systems that we are actually working on are just peanuts compared to that. Managing a single site, or just a small federation of a hundred thousand sites? Easy. Yes I’m being provocative – it’s a keynote!

The inherently scalable architecture of the Web, combined with the natural decoupling that HTTP and REST (and OSGi) provide, should allow us to keep our systems simple, transparent, robust and adaptable. Yet, much too often, we fall into the “entrprisey” trap and start designing complex machinery where something simple would do – if only someone had the guts to challenge the status quo.

I have a few examples in mind, from past projects, where simplicity provided huge wins, even though it required convincing customers that had different, usually more complex ideas. My goal is to demonstrate how valuable simplicity is, and how expensive it can be to create initially. Like the story of those 28 lines of code that took three months to create, in 1999, and still live happily in production with Zero Bugs.

We shouldn’t give up – creating simple things is a lot of work, but the rewards are huge.

To paraphrase Blaise Pascal, if your system is complicated it usually means you didn’t work hard enough to make it simpler. Or maybe you have a really complex problem, but that’s not very common. And maybe that complex problem is the wrong one to solve anyway.

I hope to see you next week in Basel and until then – keep things simple!