jq : sed, grep and awk for json

September 26, 2013

From the very useful tools department: today I stumbled on jq, via Jeroen Janssen’s 7 command-line tools for data science blog post.

As the tagline says, jq is like sed, grep and awk for json: a command-line filter that lets you format, select and output JSON data.

As an example, here’s how you can list all the OSGi bundles from your Sling instance together with their state. The raw bundles.json input looks like this:

{
  "data": [
    {
      "category": "",
      "symbolicName": "org.apache.felix.framework",
      "version": "4.2.0",
      "state": "Active",
      "stateRaw": 32,
      "fragment": false,
      "name": "System Bundle",
      "id": 0
    },
    {
      "category": "",
      "symbolicName": "org.apache.aries.jmx.api",
      "version": "0.3.0",
…

And here’s the curl + jq command:

$ curl -s -u admin:admin http://localhost:8080/system/console/bundles.json | \
jq '.data | .[] | .symbolicName + " " + .state ' | sort

"derby Active"
"groovy-all Active"
"jcl.over.slf4j Active"
"log4j.over.slf4j Active"
"org.apache.aries.jmx.api Active"
"org.apache.aries.jmx.core Active"
...

Neat, isn’t it?

See jq’s tutorial and manual for more details.


List all your Maven dependencies

July 8, 2010

Here’s a one-liner (well, two) that neatly lists all the Maven dependencies from your project. Useful to check their licenses, for example.

# first grab all dependencies
mvn dependency:resolve

# then list them with -o to keep noise low,
# remove extra information and duplicates
mvn -o dependency:list \
| grep ":.*:.*:.*" \
| cut -d] -f2- \
| sed 's/:[a-z]*$//g' \
| sort -u 

The output looks like this:

asm:asm:jar:1.5.3
asm:asm:jar:3.1
biz.aQute:bnd:jar:0.0.169
cglib:cglib:jar:2.1_3
classworlds:classworlds:jar:1.1
classworlds:classworlds:jar:1.1-alpha-2
...

so it’s also useful to detect multiple versions of the same dependency in a multi-module project.


Faster testing with the Maven CLI plugin

May 5, 2009

Although it’s not that new, I discovered Don Brown’s Maven CLI plugin only this morning, and played with mojavelinux‘ s enhanced version which supports -D parameters and profiles, among other things.

The great thing is to be able to run a simple test or test -D MyTest command quickly. You first start Maven with mvn cli:execute-phase, which gives you a maven2> command prompt to start Maven lifecycle phases. As Maven is already started, phases run much quicker than when starting from scratch.

In my experiments, the test command ran about five times faster than using mvn -o test, but the difference depends how fast your tests are, of course.

To setup the plugin, I’m adding the following to my settings.xml, so as to not interfere with project’s POMs, as the CLI is more an environment feature than a project thing:

<!-- 
  mvn settings.xml that enable the CLI plugin described at

http://tinyurl.com/maven-cli-plugin

  (For example "mvn cli:execute-phase")
-->
<settings>
  <pluginGroups>
    <pluginGroup>org.twdata.maven</pluginGroup>
  </pluginGroups>
  <profiles>
    <profile>
      <id>cli-plugin</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <pluginRepositories>
        <pluginRepository>
          <id>repository.jboss.org</id>
          <name>JBoss Repository</name>
          <url>http://repository.jboss.org/maven2</url>
        </pluginRepository>
      </pluginRepositories>
    </profile>
  </profiles>
</settings>

Find more info on the mojavelinux page.

Great tool – thanks Don Brown and mojavelinux!


svn one-liner: when did bob commit stuff here?

January 14, 2009

From the blogging this to remember next time department:

find src -type f \
| grep -v svn \
| while read f
do 
  echo "FILE:$f" 
  svn log -q
done \
| egrep 'FILE|bob'

Additional svn log options like -r can be used to restrict the range.

Update: Julian mentions a way to avoid the grep, using find src -name .svn -prune -o -type f -print


The girl effect

November 28, 2008

As a fan of well delivered presentations (who cannot afford forklifts when speaking unfortunately) I find the girl effect‘s use of kinetic typography incredibly efficient. Full-screen viewing recommended.

Delivering a 50-minutes presentation like this would require paramedics on site I guess, but it looks so efficient for short and powerful messages…maybe I should try that in a future talk ;-)


What svn revision was this branch created in?

November 26, 2008

Let’s talk about non-controversial subjects for a change: how do I find out in which svn revision a branch was created?

The answer is svn log --stop-on-copy . which stops at the revision that created the branch.

Courtesy of Jukka, filed in the notes to self category.


Wicket In Action Review – not!

October 3, 2008

This is not a review: apologies to the Wicket In Action people for not reviewing the book yet, my review copy has been sitting on my desk for weeks, just read a few pages here and there and I like what I see! I reviewed an early version of the book almost two years ago, and it was good already, so I’m very confident.

In the meantime, Sylvain has a review – and I’ll meet him tomorrow on our way to our Canal du Midi cycling trip, so please bombard his site until then ;-)

The wicketinaction.com website is the place for more info about the book.

I intend to review the book based on a concrete use case that I’d like to implement with Wicket – I’ll put the book to the test and see if it allows me to do what I want. Stay tuned – but don’t hold your breath as the next weeks look quite packed!

Cycling trip next week means I’ll be offline, so don’t expect anything from me next week.


Follow

Get every new post delivered to your Inbox.

Join 30 other followers