Continous Deployment with Apache Sling

September 2, 2014

Today I had the pleasure to attend the Master’s thesis defense of our intern Artyom Stetsenko, titled Continous Deployment of Apache Sling Applications.

Coaching Artyom for this project has been a pleasure, he did a great job and worked independently while listening very well to our advice. He got an excellent mark for his thesis and that was well deserved. Also due to an excellent no-bullets presentation!

I have uploaded Artyom’s thesis paper here, with his permission. The code is available at https://github.com/ArtyomStetsenko/sling-devops-experiments. As the name indicates that’s experimental code, but the resulting Sling-based cluster with automated atomic deployment is functional. Just push an updated crankstart file to the Git repository and the cluster is updated atomically and without downtime.

For me the main goal was to see how we can improve Apache Sling‘s support of modern operations, with continuous deployment, immutable instances etc. I’m continuing my explorations with a Docker-based Sling cluster, the main goal being to create simple clustered environments that allow us to play with these things.

Update: I forgot to mention that my Docker cluster prototype is the basis for my upcoming talk at adaptTo() 2014 on September 23rd in Berlin. The talk’s title is “Apache Sling and devops – the next frontier” and I’ll talk about how Sling can be made more operations-friendly.


So you want to talk at this conference?

August 21, 2014

I regularly review talk submissions for tech conferences, and here’s a list of what I’m mostly looking for when deciding to accept or reject a talk.

Other reviewers might be looking for different things – this is just my own criteria.

My first question is always are you going to get people interested in your stuff. Are you a dynamic speaker who keeps people on their toes, or the kind of person that delivers their talk seated at a desk. I’ve seen the latter happen, and it’s not pretty! For me, a brilliant speaker gets a slot almost every time, also because they usually know which topics will raise people’s interest.

Unless you’re famous already, the best way to convince me that you’re a good speaker is to point to a video of one of your talks. And I also need to know why you think you’re qualified to deliver this talk.

Then, I’m looking for a topic that will add value to the conference. Promoting your product or company might not add much value, whereas a talk that will open people’s minds and maybe save them hours of work in their practice is a guaranteed winner. Signs of a value-adding topic are pointers to concrete achievements using the techniques presented in the talk.

The quality of the submission comes next, especially if I don’t know the speaker. Someone who’s unable to present their ideas clearly in a talk submission is unlikely to present them clearly at the conference. Or maybe they’re a misunderstood genius, you should also look for those but they are rare. A concise submission that packs lots of useful information about what’s going to be delivered at the talk is a good promise of success.

Last but not least, original and inspiring ideas get lots of bonus points from me. Being able to predict the abstract’s contents from the title is usually a bad sign, except if it’s a talk for beginners. We don’t need conferences to exchange information today, that’s supposed to happen on the Web. Talks should be inspiring, maybe teasers to convince people to look at your value-adding stuff, but not rehash information that’s found elsewhere.

Update: I forgot to mention the movie trailer thing: a talk abstract is a lot like a movie trailer, if you feel you’ve seen all the good parts of the movie after watching the trailer, it’s not a good sign. Similarly, a good talk abstract leaves me with the impression that there’s much more to discover in the talk, compared to what’s mentioned in the abstract.


It’s just a Web server – a plea for simplicity

June 16, 2014

I’m currently working on my keynote for next week’s Connect – Web Experience 2014 conference in Basel and very much looking forward to it! Last year’s conference was excellent, and this year’s schedule looks very exciting.

My keynote is about the value of simplicity in software – including a few tales from the trenches.

We like to think of what we build with AEM as large enterprise systems, with complex requirements. Intricate workflows. Rocket science.

However, when you think about it, our systems are “just” HTTP request processors, that manipulate atomic pieces of content in a content repository.

What if you wanted to manage the Whole World Wide Web with a single system? The architecture of that 4WCMS might be quite similar to what Apache Sling provides for AEM: mostly independent dynamic HTTP request processors, selected by path and resource type, that render and/or process resources from a huge tree of content.

If our architecture works for that 4WCMS, the systems that we are actually working on are just peanuts compared to that. Managing a single site, or just a small federation of a hundred thousand sites? Easy. Yes I’m being provocative – it’s a keynote!

The inherently scalable architecture of the Web, combined with the natural decoupling that HTTP and REST (and OSGi) provide, should allow us to keep our systems simple, transparent, robust and adaptable. Yet, much too often, we fall into the “entrprisey” trap and start designing complex machinery where something simple would do – if only someone had the guts to challenge the status quo.

I have a few examples in mind, from past projects, where simplicity provided huge wins, even though it required convincing customers that had different, usually more complex ideas. My goal is to demonstrate how valuable simplicity is, and how expensive it can be to create initially. Like the story of those 28 lines of code that took three months to create, in 1999, and still live happily in production with Zero Bugs.

We shouldn’t give up – creating simple things is a lot of work, but the rewards are huge.

To paraphrase Blaise Pascal, if your system is complicated it usually means you didn’t work hard enough to make it simpler. Or maybe you have a really complex problem, but that’s not very common. And maybe that complex problem is the wrong one to solve anyway.

I hope to see you next week in Basel and until then – keep things simple!


Sony Bridge for Mac, a perfect example of second-system effect #fail!

April 3, 2014

I’m sorry to be ranting here instead of writing about more creative things, but here’s a perfect example of the “second-system effect” – replacing simple software that works perfectly well with something supposedly more powerful…that doesn’t work.

My Sony Xperia V phone used to upgrade over the air. Get a notification, wait until you have a good Internet connection, start the upgrade, wait a few minutes and you’re done. Who needs more than that?

Sony apparently thinks that’s too simple. Or maybe they have other reasons to start forcing users to use their brand new shiny Sony Bridge desktop software for updates. Reminds me of my first Samsung Galaxy, years ago…I hope Samsung’s over that by now – but I digress.

There doesn’t seem to be another option to upgrade my phone this time, so I play nice, downdload and install the latest Sony Bridge V3.7.1(3710) software tool, connect my phone to my computer and start the bridge tool. Oh, by the way: how do you upgrade if you don’t have a computer? What’s with “mobile first”? But I digress.

And here’s the first “fun” result: the tool refuses to upgrade my phone now because the battery charge is less than 80%. Oh my.

IT IS PLUGGED IN, STUPID! How could you talk to it via USB otherwise? Fail. Sony Bridge, I’m starting to hate you. No, really.

Ok, I’m a good boy. I wait until my battery reaches 80% and try again. Hurray! I can get to the upgrade screen this time.

The tool says “Initializing” and “Talking to Update Engine”.

And it stays there.

And time passes.

Fourty-seven minutes now. 47 MINUTES and exactly nothing else happens. No feedback. No progress. No error messages. Did you guys take UX 101 in school? I guess not.

Sony Bridge #fail screenshot

Forget the Cancel button…it doesn’t work.

I tried twice. Had to delete some secret cache files in between both attemps, as on the second time the tool was telling me that my phone is up to date, but it’s not, as the phone itself tells me.

The result of this brilliant software evolution? I am unable to upgrade my phone. That will make it useless soon, I guess, and I didn’t pay a sizeable amount of money to have a non-upgradeable phone.

So here we have a perfect example of what software folks should NEVER do: replace a perfectly working simple system (over-the-air updates) which a new shiny thing that DOESN’T WORK and makes me YELL here which is quite unusual. Ok, maybe you don’t care about me yelling, but if you’re Sony I would expect you to be more clever than that.

So, Sony, I guess I’ll ask for a refund for this now useless phone. What do you think?

But of course, the best by far is just to re-enable over-the-air updates. That used to work very well.

Update: I was finally able to upgrade my phone by running the Sony Bridge train wre^H^H^H^H^H^H^H^H^H software on an old macbook. But still…why?


Simple startup profiling of OSGi applications

October 18, 2013

Prompted by a colleague’s question about profiling the startup time of Apache Sling systems, I vaguely remembered writing something a while ago, but couldn’t find it immediately.

Funny how one’s memory works – it took me a while to find it again, but I did indeed write an OSGi events recorder + timeline utility in SLING-1109.

That utility has since moved to the Apache Felix webconsole events plugin bundle, which is included in the Sling launchpad and thus was right here under my eyes.

Here’s how you can use that webconsole plugin to get a simple timeline of an OSGi system’s startup:

  1. Install the org.apache.felix.webconsole.plugins.event bundle in an OSGi app where the Apache Felix Webconsole is active.
  2. Set that bundle to a low start level, say 1 or 2, so that it’s activated early and captures as much startup events as possible.
  3. Configure the events recorder at /system/console/configMgr/org.apache.felix.webconsole.plugins.event.internal.PluginServlet, setting the number of events to capture high enough to record your app’s startup.
  4. Start your app.
  5. Look at the startup timeline at /system/console/events

This provides a simple graphical timeline, as shown on the screenshot below, that’s especially useful in detecting outlier bundles or services that take a long time to start up.

Webconsole Events plugin screenshot


jq : sed, grep and awk for json

September 26, 2013

From the very useful tools department: today I stumbled on jq, via Jeroen Janssen’s 7 command-line tools for data science blog post.

As the tagline says, jq is like sed, grep and awk for json: a command-line filter that lets you format, select and output JSON data.

As an example, here’s how you can list all the OSGi bundles from your Sling instance together with their state. The raw bundles.json input looks like this:

{
  "data": [
    {
      "category": "",
      "symbolicName": "org.apache.felix.framework",
      "version": "4.2.0",
      "state": "Active",
      "stateRaw": 32,
      "fragment": false,
      "name": "System Bundle",
      "id": 0
    },
    {
      "category": "",
      "symbolicName": "org.apache.aries.jmx.api",
      "version": "0.3.0",
…

And here’s the curl + jq command:

$ curl -s -u admin:admin http://localhost:8080/system/console/bundles.json | \
jq '.data | .[] | .symbolicName + " " + .state ' | sort

"derby Active"
"groovy-all Active"
"jcl.over.slf4j Active"
"log4j.over.slf4j Active"
"org.apache.aries.jmx.api Active"
"org.apache.aries.jmx.core Active"
...

Neat, isn’t it?

See jq’s tutorial and manual for more details.


The Five Wisdoms of Open Development

February 15, 2013

Preparing my Open Development in the Enterprise talk for ApacheCon NA 2013 in Portland the week after, I’ve tried to capture the basic wisdoms of day-to-day work in an Open Development environment:

If it didn’t happen on the dev list, it didn’t happen.

Whatever you’re working on, it must be backed by an issue in the tracker.

If it’s not in the source code control system, it doesn’t exist.

If it’s important, it needs a permanent URL.

What happened while you were away? Check the activity stream and archives.

As usual, following recipes without understanding what they’re about won’t help – I use those wisdoms just as a reminder of how things are supposed to happen, or more precisely how people are supposed to communicate.


Follow

Get every new post delivered to your Inbox.

Join 29 other followers