Ten years of blogging!

November 9, 2012

Although I had marked the date in my calendar, a great ApacheCon kept me busy and I missed Tuesday’s date for announcing that it’s been ten years now since I started blogging.

Not that you’d care…though it’s fun to see that my interests and professional activities haven’t changed that much in ten years. Less blogging of course, as over time this blog has morphed into an outlet for more permanent/longer content, which is less frequent of course.

The first few posts at http://grep.codeconsult.ch/2002/11/ are not earth-shattering – that was a start though.

See you on Twitter for now, as that’s where my noise has mostly moved, and we’ll see if this blog is still around in ten years!


Open Source Communites are like gardens, says Leslie Hawthorn

June 4, 2012

Great keynote by Leslie Hawthorn at Berlin Buzzwords today – here are my quick notes.

As with a garden, you need to cultivate your community, it doesn’t just happen.

You need to define the landscape – where you want the garden to grow. For a community this means clearly defining goals and non goals.

The landscape needs to be regularly assessed – are we welcoming to newbies? Can new plants grow here?

Clear the paths. Leslie mentions the example of rejecting code patches based on the coding style – if you haven’t published a style guide, that’s not the best way to welcome new contributors. Make sure all paths to success are clear for your community members.

A garden needs various types of plants – invite various types of people. Not just vague calls for help – specific, individual calls based on their skills usually work much better.

Pluck out the weeds, early. Do not let poisonous people take over.

Nurture your seedlings. Take care of newbies, help them grow.

Like companion planting, pairing contributors with the right mentors can make a big difference.

Know when to prune. Old-timers leaving a project is not necessarily a problem, things change.

Leslie cites OpenMRS as an example of successful community management – I’ll have to have a look but I already like the “how to be a casual committer” paragraph in their different types of OpenMRS developers page.

All in all, very interesting parallels. Being quite clueless about gardening, I never thought of looking at our communities from this angle, but it makes perfect sense. Broadening one’s view – that’s what a keynote should be about!


RESTful services as intelligent websites

March 27, 2012

Thinking of RESTful services as intelligent websites helps drive our architecture decisions towards simplicity, discoverability and maximum use of the HTTP protocol. That should help us design services that are easier to use, debug, evolve and maintain.

In this post I’ll try to address a few important aspects of RESTful services, considering them as intelligent websites. We might need more formal specifications in some cases, but this is hopefully a good start.

Discoverability and service homepage

Having a descriptive homepage for your service helps search engines and people discover it, and the base service URL should “never” change. Service-specific subdomains come to mind.

The service homepage includes all the necessary information such as service owner’s contact information, links to source code, etc.

News about service updates live right here on the service homepage, ideally as full content for the most recent news, but at least as links.

The key idea is that I shouldn’t have to tell you more than the service homepage’s URL for you to be able to use the service.

Even if your service is a company-internal one that’s not meant to become public, having a decent-looking homepage, or at least one that’s well organized and well readable, won’t hurt.

HATEOAS

In my pragmatic view Hypermedia as the Engine of Application State basically means links tell you where to go next.

In a website meant for humans, the meaning of a link is often expressed by logical groups: navigation links at the top left, “more info” links in a smaller font at the bottom of a page, etc.

For machines, adding rel attributes to <link> elements (in HTML, or the equivalents in other formats) tells us what we can do next. A client should be able to first try a link with non-destructive results, and get a response the supplies details about how the specified interaction is supposed to work. If those details are too involved to be expressed in a simple HTTP request/response (which should be a “maybe this is too complex” warning), links to explanations can be provided in the HTML content of our intelligent website.

Self-explaining services

The service documentation, if any is needed, is also provided on the service homepage, or linked from it.

Service interactions are designed to be as obvious as possible to minimize the need for documentation, and readable automated tests (hosted on the service website of course, or linked from it) can help document the details.

HTML forms describe service parameters

HTML forms are best way to document the service parameters: provide a form that humans can use to play with the service, with enough information such as lists and ranges of values so that users can figure out by themselves what the service expects.

The idea is that a human user will play with your service from the HTML form, then move on to implementing a client in the language of their choice.

The action attribute of <form> elements also contributes to HATEOAS – intelligent clients can discover that if needed, and it’s obvious for human users.

And of course, speaking of parameters, be liberal in what you accept, and conservative in what you send.

Speaking in URLs

Like humans, RESTful services need to be precise when they speak about something important.

If your service response says invalid input format for example, it’s not hard to include in that response an URL that points to a description of the correct input format. That makes all the difference between a frustrating and a useful error message, and is part of HATEOAS as well.

Web-friendly responses

Readable response formats will help people make sense of your service. The HTTP protocol does provide standard ways of compressing responses to avoid using more bandwidth than needed, so optimizing the response for wire efficiency does not make much sense unless you’re really expecting measuring huge traffic. And even if you need an optimized binary response format, there’s probably a way to make that optional.

HTTP status codes

Thou shalt not abuse HTTP status codes – or you might be suddenly transformed into a teapot.

Returning a 200 OK status code with content that describes an error is a no-no: if something went wrong, the HTTP status code needs to express that.

Security

Website authentication and authorization mechanisms and secure connections work for machine clients as well, no need to reinvent that wheel.

HTTP sessions are a bad idea in a RESTful context of course, state is driven by hypertext as discussed above.

Character encodings

The issues are the same for human users as for machine clients: you need to play by the HTTP protocol rules when it comes to character encodings, and using UTF-8 as the default is usually the best option.

Stable service URLs

As with good websites, once a service has been published at a given URL, it should continue working in the same way “forever”.

A substantially different new version of the service should get its own different URL – at least a different path containing a version number, or maybe a new subdomain name.

Long-running jobs

Regardless of human or machine clients, you usually don’t want HTTP requests to last more than a few seconds. Long-running jobs should initially create a new resource that describes the job’s progress and lets the client know when the output is available. We’ve had an interesting discussion about this in Apache Stanbol recently, about long-running jobs for semantic reasoners.

Service metadata

Non-trivial services often have interesting metadata to provide, which can have both static and dynamic parts, like configuration values and usage statistics for example.

Here again, the website view helps: that metadata is just made accessible from the service homepage via links, ideally with both human (a href) and machine (link rel) variants.

Coda

Designing your RESTful service as an intelligent website should help people make good use of it, and will probably also help you make its interactions simpler and clearer.

If your service can’t be presented as a website, it might mean that your interaction design is not optimal, or not really RESTful.

I’m happy to see these opinions challenged, of course, if someone has any counter-examples.

Update: I should have mentioned that this post, and especially the “intelligent websites” concept, was inspired by conversations with Roy Fielding, in the early days of Apache Sling. I haven’t asked him to review it, so this doesn’t mean he would endorse the whole thing…I’m giving credit for inspiration but still liable for any mistakes ;-)

Update #2: See also my REST reading list, a regularly updated collection of links to articles that I’ve found useful in understanding and explaining REST..


Stefano’s Mazzocchi’s Busy List Pattern

December 6, 2011

Since being part of a larger company, I’m hearing people saying “we need a new mailing list for that” way too often.

People are afraid of busy mailing lists sometimes, but in terms of fostering open collaboration and communities a busy list is excellent.

Another problem is people writing 1-to-1 emails instead of writing to the list, as an attempt to avoid making the list even noisier. If your message is on-topic with a well written subject line, and as concise as possible, it’s not noise and definitely belongs on the list.

Stefano’s Mazzocchi briefly explains this in his ApacheCon 2006 slides titled all you wanted to know about Open Development community building but didn’t know who to ask. I haven’t found that pattern described in a more accessible way than among Stefano’s 303 PDF slides so far, so here’s a summary that I hope is faithful to the original idea.

The Busy List Pattern

Here’s my summary of that part of Stefano’s slides (starting at page 200 in his PDF file), with some additional comments of mine.

The pattern starts with somebody suggesting that the mailing list is too noisy and should be split in multiple ones.

Restaurants and night clubs, however, know that packed rooms help marketing…what’s more boring than being alone in a restaurant?

But we’re not a bar…we can’t go on getting 200 messages on that list every day, can we?

Actually we can…if everybody who posts to the list is extra careful about how they post, a list with 200 or more messages per day is perfectly manageable – but only if the subject lines are very carefully chosen and evolved (see below), and only if people are very careful about what they post, to maximize clarity and avoid wasting other people’s time.

We should strive to keep our lists as packed as possible. It’s hard to set a limit, but before splitting a list you might ask if people are using it efficiently. First try to improve the signal to noise ratio, and if that really fails you might consider splitting the list.

If you really need to split the list, do it by audience and not by topic – a consistent audience will lead to an interesting list, whereas scattering topics all around makes cross-topic discussions painful.

Careful with those subject lines

The subject line of messages makes all the difference between a noisy list an a useful one.

Choose sensible subject lines that allow people do decide if they want to read your message or not.

Reread your subject line before sending – does it really express what your message is about, and does it contain any relevant call to action?

Change those subject lines when the thread changes topics.

Address one topic only per thread.

Use [MARKERS] in subject lines to tag messages.

Simple rules like this will boost your list’s efficiency tremendously – there’s more good stuff like this in Stefano’s slides, make sure to have a look!


MANIFEST.MF must be the first resource in a jar file – here’s how to fix broken jars

November 15, 2011

Some tools, like the Sling OSGi Installer, require the MANIFEST.MF to be the first file in a jar file, or they won’t find it.

This happens when using the java.util.jar.JarInputStream class to read a jar’s manifest, for example.

The manifest is where OSGi bundle headers are found, for example, so not having it in the right place makes the jar unusuable as a bundle.

I won’t discuss here whether this requirement is part of the jar file spec (it would make sense, as this makes sure you can read it quickly even from a huge jar file), but anyway there are many cases where this is required.

To fix a jar where this is not the case, you need to unpack the jar and recreate it, as in this example, starting from a broken.jar in the current directory:

$ mkdir foo
$ cd foo
$ jar xvf ../broken.jar
$ mv META-INF/MANIFEST.MF /tmp/mymanifest
$ jar cvfm fixed.jar /tmp/mymanifest .

That’s it – creating the new jar with the jar utility puts the MANIFEST.MF in the right place.


Quick notes from Mark O’Neill’s Transfer Summit 2011 talk

September 8, 2011

Here’s my quick on-the-spot notes of Mark O’Neill’s (@marxculture) excellent presentation, on how a government’s IT can innovate.

Citing Andrew Savory, never thought I’d see govt. spokesperson be entertaining and informative. Reminds me of my excellent experiences working for the Swiss Parliament services in the late nineties.

Here’s my unedited notes – slides should be available soon. URLs and emphasis added by myself.

UK gov spends 20 billion pounds a year on IT, with 20 top suppliers.

Mark’s got a 20MB mailbox to manage this.

Different velocities: technology, business society.

His role: track and align to those velocities.

Velocity of change in his government IT world is “very different” from what it is on the outside. The gap is increasing.

The more you diverge from the velocity of the market, the higher your costs are.

The options are: do the same, do nothing or ask a different question.

The challenge: no money – although they’re spending 20 billion pounds a year…

The government is currently driven by the policy cycle: problem -> draw up a policy -> interpret and implement policy -> monitor and evaluate. The customer is not part of this picture. Need an innovation model.

He shows a picture of a washing machine for dogs: there is an e-petition about this, the e-petition site (http://epetitions.direct.gov.uk/) got 16k such petitions in 4 weeks, 3.8 million visitors, 20 million page views, 1.5 million signatures. The e-petition site was built in 6 weeks with 6 people, cost 80 thousand pounds including security costs. The system will be released as open source shortly.

The future: small projects are developed by his office, up to 3 months with contracted companies, if more than 3 months need to ask different questions.
20 billion pounds a year should be one of the most dynamic, diverse and entrepreneurial market in the world. What needs to be done to achieve that?

Leaner procurement mechanisms. Use open standards and open source. Look at the processes, check how much paperwork and overhead is actually necessary.
Challenge to new projects includes two questions: what is it that you want to do, and why do you want to do it that way.

Decompose the project in smaller units of work – also gives more chance for SMEs to jump in.

Three layers: infrastructure, app, support. Procurement is currently often based on a complete silo – moving to smaller units can help reuse and sharing.

Rethink the approach to make it possible to buy an excellent application form an SME that does not provide infrastructure or services.

Working on G-Cloud, private onshore cloud for UK government services. (http://www.cloudbook.net/directories/gov-clouds/uk-government-cio-council).

Talking of “agile” “cloud” is a bit like talking about “magic” these days ;-)

Conversation always trumps process.

Two key differences between agile and waterfall.

First difference: with waterfall you talk to developers, they go away for six months, come back with something that you don’t want. Agile: you talk all the time.

Second difference: with waterfall, you might not be around anymore when problems arise. With agile, you could be fired after three months…

Tools are hard. Discuss, share, build, learn. Cannot deliver success without using efficient tools, take example from open development teams in the outside world.

Need to build mechanisms to learn, share and discuss what the team is doing.

Success factors:

1) Be the most dynamic, diverse and entrepreneurial market in the world.

2) IT should just work.

3) Reuse, dialogue, agility, ownership should be part of the day-to-day business.


Turning 42, and why I love my job

August 24, 2011

I’ll be turning 42 in a few months (counting in base 12 of course) and it feels like a good time to reflect on what it is that makes my job enjoyable. My father was a carpenter, and both my brothers started with that as their first job, so I’m kind of the disruptive element of the family (I didn’t say black sheep, ok?).

So, why did I choose to work with cold electronics (my first degree) and computers instead of working with a noble and beautiful living thing like wood?

After some thinking I came up with four key elements.

The first key element is creating cool things. Note that I don’t say creating cool software: I realized that for me the creative process is more important than what exactly is being created. Coolness is obviously a subjective measurement, so it’s hard to define precisely. Lean and maintainable software that people find useful definitely falls into that category for me.

Next is working with bright and fun people. Being active in the Apache Software Foundation, and joining Day in 2007 made me realize how stimulating it is to work with people that impress you every day with their technical and other skills. People who are fun to work with help keep some distance with the Big Problems At Work. Technical and other problems are bound to happen in any job, and that’s when your colleagues’ attitudes make all the difference. Software and work are not always the most important things in life.

Using efficient and fun tools comes next – in my previous life as an independent software developer and architect I sometimes had to put up with lame environments and tools at customer sites, and that can be depressing when you’re aiming for quality and efficiency. My first grade math teacher kept saying that good craftsmen use good tools, and she was right!

The fourth element is keeping a good work-life balance. I tend to engage myself 100% in my work, but for that to happen I need to be able to engage myself 100% in other things at regular intervals. This often means disconnected holidays “away from the grid”. I also decided long ago to never work on Sundays, unless there’s really no other way, which is rare. This has helped me keep my sanity during those phases when the rest of the week is totally crazy.

The fun thing is that those four elements would totally apply to being a carpenter…and I actually did enjoy helping at my father’s shop during school holidays when I was a kid. I’m not planning on going back though – now that my son learned carpentry as well, he’s making fun of me every time I try!


Follow

Get every new post delivered to your Inbox.

Join 31 other followers