We just got back from a two-week trip to Japan, which was simply amazing. I’ve seen more temples and Torii gates than I care to remember; I’ve seen the spring festival of Takayama and the Peace Museum of Hiroshima. We went on a pilgrimage to Akihabara, shopped at Super Potato, and had drinks at 8-bit Cafe.

As most of my Japan knowledge comes from exclusively from playing the Yakuza games, one of my personal highlights was visiting the real-life “Kamurocho” and seeing the iconic Kabukicho gate.

Side note: Tokyo is about just about the only city on earth where I’d feel comfortable walking around the red light district at night snapping pictures with my big touristy camera.

This is frankly the best damn session report I’ve read in my entire life. At 36 pages long, it is an superbly well-written essay about the psychology of gaming, friendship, and the sheer ridiculousness of a certain monster wargame.

“Troy,” I said, “the reason I want so much to play this game is because nobody wants me to play this game. There is virtually no person you could present this to without getting a stunned, pitying look in return, and that ticks me off. It ticks me off because what so freaks people out is the time involved in this endeavor, time. Everything in adult life is designed to steal it away from us, and my God, look how we go along with the scheme so willingly.

“‘I have no time for such things anymore,’ we say, and then we spend our afternoons making our lawns pretty and shopping for junky Ikea furniture and ferrying kids to soccer games and gawking at cable TV, and above all, working at jobs we never really wanted. ‘Sure,’ we say, ‘when I was young and didn’t have all these responsibilities, I could spend hours doing this kind of thing. But that was then, and this is now.’

“Well, Troy,” I went on, “I want to be the guy who suddenly, at age 42, does spend hours doing this kind of thing, if only to feel what it’s like to take back a little piece of the soul I’ve sold to the company I slave for, to the obligatory evenings with people I’m not sure I even like, to daily errands, the lines at the DMV, to tax forms, to tedious family visits. This game is a slap in the face to all thinking creatures who live in such dire fear of the sands sifting through the hourglass. Playing a monster war game on this scale is ridiculous, a waste of energy, a waste of time, and so I want to do it. Let spite rule the day, Troy. Let’s learn and play A World at War!”

So, I was curious about elnode. And I wanted to play more with my Raspberry Pi. It only seemed logical to combine the two.

You know, because I can.

(I promised @nicferrier a writeup on this project, so here goes.)

Installing Emacs 24

The Emacs that comes with the Raspian distribution is version 23.4. Unfortunately, elnode requires Emacs 24, so I had to compile my own. I dreaded doing this, but it actually turned out pretty straightforward.

You need to apt-get install some build dependencies like libncurses-dev (and probably build-essentials if you don’t already have them), then you’re ready to go:

1
2
3
4
5
6

 > wget http://ftp.gnu.org/pub/gnu/emacs/emacs-24.2.tar.gz
 > tar xvfz emacs-24.2.tar.gz
 > cd emacs-24.2
 > ./configure
 > make 
 > sudo make install

This would be a good opportunity to make a sandwich, or learn some sanskrit, as building everything will take quite a while.

Running Emacs

By default the elnode webserver runs on Emacs startup, so the idea is to fire up emacs and just leave it. One way of doing that would be to ssh into the Pi and run emacs inside a tmux session (I tend to use tmux for all remote access anyway). This makes the emacs process persistent, and when I next log in I can just resume the session.

However, with this approach I’d still need to login in and start manually every time the server restarts, so the setup is kind of fragile. Especially since I have builders running round the house cutting the main power every now and then.

So instead I use a simple init script to run emacs in daemon mode (using the aptly named --daemon switch). This way I can still connect to the running emacs process using emacsclient, but I don’t have to bother with starting it manually anymore.

The only caveat is that since I want start-stop-daemon to fork off a new process to run emacs as the pi user (running a web-connected emacs as root seems like a really bad idea), I can’t rely on it to generate a correct PID file for me. Instead, we’ll have to do that from inside emacs (see next section).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

#!/bin/sh

set -e

# Must be a valid filename
NAME=elnode
PIDFILE=/home/pi/elnode/elnode.pid
UID=1000
#This is the command to be run, give the full pathname
DAEMON=/usr/local/bin/emacs
DAEMON_OPTS="--daemon --load /home/pi/.emacs.d/init.el --load /home/pi/elnode/web.el"

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin"

case "$1" in
    start)
        echo -n "Starting daemon: "$NAME
        start-stop-daemon --start --quiet --pidfile $PIDFILE --chuid $UID --exec $DAEMON -- $DAEMON_OPTS
        echo "."
      ;;
    stop)
        echo -n "Stopping daemon: "$NAME
        start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
        echo "."
      ;;
    restart)
        echo -n "Restarting daemon: "$NAME
        start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE
        start-stop-daemon --start --quiet --pidfile $PIDFILE --chuid $UID --exec $DAEMON -- $DAEMON_OPTS
      echo "."
      ;;

    *)
      echo "Usage: "$1" {start|stop|restart}"
      exit 1
esac

exit 0

When I start the deamon I get some weird error messages from elnode, but as far as I can tell it actually runs fine.

1
2
3
4
5
6
7
8

pi@raspberrypi ~ $ sudo /etc/init.d/elnode start                                                                                                                  
Restarting daemon: elnode("/usr/local/bin/emacs" "--load" "/home/pi/.emacs.d/init.el" "--load" "/home/pi/elnode/web.el")
Starting Emacs daemon.
deleting server process
elnode-error: elnode--sentinel 'deleted.' for process  *elnode-webserver-proc* with buffer *elnode-webserver*
found the server process - NOT deleting
elnode-error: Elnode server stopped
.

Running elnode

Elnode is available from Marmalade, which makes it very easy to install from inside emacs. I also grapped the xmlgen library to help with HTML generation.

This is my little Hello World-script. There’s nothing remarkable about it, really. The first section generates a PID file to be used by the init script to check if the process is running. I stop the default server, as it binds to “localhost”, and I want to bind to the actual server name (raspberrypi.local in my case) in order to serve remote requests. You can see it in action here.

Did I say there was nothing remarkable about this script? Yeah, apart from the fact that it’s running a webserver inside a 25 year old text editor on a computer that’s about as large as a deck of cards.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

;;; -*- lexical-binding: t -*-

;; write the pid file
(with-temp-file "/home/pi/elnode/elnode.pid" (insert (format "%d" (emacs-pid))))

(require 'cl)

;; stop the default server
(elnode-stop 8000)

(defvar request-count 0)

(defun hello-handler (httpconn)
  "Hello, world"
  (elnode-http-start httpconn 200 '("Content-Type" . "text/html"))
  (elnode-http-return httpconn
            (xmlgen
             `(html
           (head
            (title "Hello, elnode!")
            (style "body { margin:100px auto auto auto; width: 400px; }"))
           (body
            (p "Hello, I am a small "
               (a :href "https://github.com/nicferrier/elnode" "webserver")
               " running inside Emacs 24 on a "
               (a :href "http://www.raspberrypi.org" "Raspberry Pi")
               ".")
            (p "The local time is "
               ,(format-time-string "%H:%M")
               " and I have served a total of "
               ,(incf request-count)
               " requests."))))))

(defun root-handler (httpconn)
  (elnode-hostpath-dispatcher httpconn '((".*/favicon.ico" . elnode-send-404)
                   (".*" . hello-handler))))

(elnode-start 'root-handler :port 8000 :host "raspberrypi.local")

I do Java for a living. Apart from a two-year hiatus back in 2003-05 I’ve been a Java guy ever since I left university in 1997 (Man, sixteen years now! I didn’t realize.).

Java is… well, it’s Java. For all it warts, it’s actually a pretty decent environment to get things done in. Yeah, it’s verbose as hell. Yeah, most projects involve heaps of boilerplate XML. And yeah, Maven is still the best we’ve got. But still, when you know your way around it’s not bad. The 3rd party library support is outstanding, and most of it is Open Source. Performance is generally good. And the cross-platform compatibility is at this point something we just take for granted.

There have been other languages. I was heavily into Common Lisp for a while. I would prefer it over java were it not for the lack of standard libraries. Clojure seemed like a logical progression, but for some reason I couldn’t get into it. I love the premise: “Functional programming with full access to the Java ecosystem,” but for some reason I never stuck around.

I was on the Scala wagon, I know enough Ruby to be dangerous, and I’ve flirted with Go, Haskell, Python, and CoffeeScript. All excellent languages in their own right. None of them really stuck. I’ve read the books, the blogs, the twitters. I know these are the languages the cool kids use nowadays. Maybe it was me, maybe I wasn’t hipster enough.

But there is one language that has grown on me. That hasn’t gone away or been replaced. It’s a ugly little thing, full of warts and cruft that has accumulated over many years. It is a Lisp, but it’s awkward and eccentric even by Lisp standards. It lacks basic necessities like namespaces and, for the longest time, lexical scoping. The standard library is confused and naming conventions are all over the place. Many Common Lisp staples (like loop, map, and reduce) are missing. There is a Common Lisp compatibility package, but using it was controversial and for a long time actively discouraged.

I am, of course, talking about emacs lisp. I took stock of my github projects and realized that apart from the Java stuff, almost all of my recent activities are elisp. This was not something I had planned, it’s just that Emacs is such an awesome elisp IDE. I’m more productive in elisp than in any other environment. When I make code changes I’m not just connecting to a REPL, I’m actually changing the very environment I’m currently coding in. This is incredibly powerful.

So, Java and elisp. Not very cool, I know. But do not underestimate the power of knowing your tools and getting shit done.

The main focus at the JFokus conference last week seemed to be the Raspberry Pi. Sure, Oracle was pushing Java 8, JEE 7 and whatnot, but everybody was talking about the Little PC That Could. There were raffles where you could win them, and according to rumors they were given away to the audience at the embedded systems keynote.

I happened to attend a technical session, mostly because there was nothing better on. Gerrit Grunwald showed off a Pi-based system he had set up to display the temperature in his home, and I was sold immediatly. As I left the room I flipped open my laptop, found a reseller and ordered myself a starter kit. A few hours later I got a notice saying it had shipped and by early afternoon the next day it had been delivered, a full two days before I got home myself.

These are my impressions after playing with it over the weekend:

• It really is small. Really, really small. Of course, I seem to have a thing for small computers.
• Video output is strictly HDMI. It hooked up to the TV and booted nicely, but when I tried connecting it to my old 17” monitor using a HDMI-DVI converter I couldn’t get any signal at all. It doesn’t really matter all that much, as getting a SSH server up and running was trivial, and I kind of prefer accessing it through the network anyway.
• I had the notion that Go would be a good language to do some systems programming in, but getting the infrastructure in place was not trivial. Essentially you have to apply a small patch and rebuild the debian packages before installing, and remember to put export GOARM=5 in your .bashrc.
• The CPU isn’t great. Building Go from scratch took me well over 30 minutes. I realize I may have been spoiled by my MacBook.

And now I have a trivial webserver running:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

package main

import (
  "fmt"
  "net/http"
  "html"
  "log"
  )

var port = 8001

var s = &http.Server {
  Addr: fmt.Sprintf(":%d", port),
}

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
      fmt.Fprintf(w, "Hullo, %q", html.EscapeString(r.URL.Path))
  })
  log.Printf("starting web server on port %d", port)
  log.Fatal(s.ListenAndServe())
}

I’ve been juggling glassfish servers a lot lately, and found the asadmin CLI admin tool pretty nice. There was one thing lacking though: ZSH tab completion. So, being a hacker, I thought to myself: “How hard can it be?”

Pretty hard, as it turns out. The ZSH completion system is ridiculously complex and poorly documented. There is a pretty good ebook, but it is from 2003 and I’m sure a lot of things have changed since then. Other than that, then only tutorial on writing your own completion functions was an old Linux Mag article.

After a lot of trial and error, borrowing code from various oh-my-zsh plugins, and nearly giving up in frustration, I’ve now produced a plugin of my own. It’s pretty large (1147 lines), but has completions for all the 265 subcommands that I know of. I quickly realized that maintaining the argument completions for all these subcommands would be quite impractical, so I did what any hacker would do: I wrote quick-and-dirty program that goes through all the glassfish jar files on the classpath, looking for classes that implement the asadmin commands. It then looks at annotations and uses a bit of introspection to determine what arguments are available for each command, and what completions (if any) should be used to complete these. The end result is a ZSH script fragment which I then insert manually into my completion script. Overall, this has worked out pretty well. And it sure beats doing it by hand.

Anyway, if you work with asadmin and would like some hot completion action going, check out my oh-my-zsh branch on github.

After hacking on and off on emacs-eclim for the better part of three years, it’s finally time to make an actual, honest to God, release! With a version number and everything!

Even better, I made two releases, both at the same time. Thing is, the eclim guys switched the format of the response messages to JSON with eclim 1.7.3. That was a rather good change, but it meant we had to do quite a bit of work on our end to get everything working properly. And that’s why it finally made sense to start talking about emacs-eclim releases. If you are stuck on an older version of eclim, you need emacs-eclim version 0.1; but for eclim 1.7.3 or later you must run emacs-eclim 0.2, or HEAD.

So, my love affair with Inform continues. Whatever love I housed for the in-built editor however, has withered and died. The keybindings all felt wrong, I kept accidentally narrowing the scope of the code, and had to use the mouse way too much for it to be a pleasant writing experience. So I thought, Hey, why not write an emacs mode for Inform? I’ve been meaning to learn about major modes and syntax highlighting anyway, and this is as good an excuse as any. After a bit of research, it turned out that creating a new major mode was pretty straightforward.

First, since we want headings in the source code to stand out, we’ll define a new font face, based on font-lock-preprocessor-face. It could be inherited from any face, really, but let’s pick one that’s likely to have been included in most colour themes.

1
2
3
4

(defface inform7-heading-face
  '((t (:inherit font-lock-preprocessor-face :weight bold :height 1.2)))
  "Face for Inform 7 headings"
  :group 'font-lock-highlighting-faces)

Then some keybindings for our mode. There’s not much for us to do here, but let’s make RET automatically jump to the correct indentation level; that’s always nice:

1
2
3
4
5
6

(defvar inform7-mode-map
  (let ((map (make-sparse-keymap)))
    (define-key map (kbd "RET") 'newline-and-indent)
    (define-key map "\C-j" 'newline-and-indent)
    map)
  "Keymap for inform7 major mode")

The easiest way to add syntax highlighting for our mode is to use regexp-based matchers. Inform 7 hasn’t got that much of a formal syntax, but it would be nice to highlight keywords like if, then and otherwise and for section headings in the text. We should also find some way of supporting indexed text (i.e. anthing inside a string surrounded by []).

The last one is a bit tricky, as the same delimiters are also used for comments (see below), and it’s not actually possible to distinguishing between them using only regexps. The solution here, prepending a ‘.’ to the matcher, is a workaround that’s good enough for most cases though.

One thing to watch out for is that that the names of the faces must be symbols. For the built-in faces there are variables defined for each of these, but we must be careful to quote the name of the face we defined above.

1
2
3
4
5
6

(defconst inform7-font-lock-keywords
  `(( ,(regexp-opt '("Include" "Use" "let" "say" "if" "otherwise") 'words) . font-lock-keyword-face)
    ("^\\(\\(?:Book\\|Chapter\\|Part\\|Section\\|Volume\\) - .*\\)" . 'inform7-heading-face)
    (".\\(\\[.*?\\]\\)." 0 font-lock-variable-name-face t)
    )
  "Highlighting expressions for inform7-mode")

We now have everything we need to define our new mode. Current wisdom holds that it’s a good idea to extend an existing mode using define-derived-mode, and have someone else do all the heavy lifting. In our case, we want to extend sws-mode, which handles significant whitespace and provides functions for intelligent indentation of lines and blocks of code. It is part of jade-mode.

Defining our major mode is deceptively simple; we just pass in a mode name, the name of the mode we’re deriving from, a printable name that will be displayed in the status line, and a body of code that will be executed when the mode is initialized. define-derived-mode will take care of the rest and set up a keymap, syntax table, mode hooks and ensure that the parent mode and the hooks are called on init. In our case, we’ve already created the keymap manually, so define-derived-mode will use that one instead.

1
2
3
4

(define-derived-mode inform7-mode sws-mode "Inform7"
  "Major mode for editing inform 7 story files."
  (visual-line-mode)
  (set (make-local-variable 'font-lock-defaults) '(inform7-font-lock-keywords nil t)))

Two things remain, though. First, the line indent function does always increases the indentation of new/empty lines by one step. For inform code, we only want that if the preceding line ends with a ‘:’, otherwise we’d like the new line to have the same indentation level as the previous one.

It turns out, that can be accomplished by adding some advice around sws-do-indent-line, which is the function in sws-mode that actually carries out the indentation:

1
2
3
4
5
6
7
8
9

(defadvice sws-do-indent-line (around inform7-indent-blank () activate)
  "Ensure proper indentation of blank lines according to Inform7 conventions."
  (if (and (eq major-mode 'inform7-mode) (sws-empty-line-p))
    (if (save-excursion
          (previous-line)
          (looking-at "^.*:[ \t]*$"))
        (indent-to (sws-max-indent))
      (indent-to (sws-previous-indentation)))
    ad-do-it))

The other thing we need to fix is highlighting of comments in the text. We don’t want to do this with regexps, as it’s tricky to get this working reliably for multi-line comments. Instead, we’ll take advantage of the built-in facilities for syntactic highlighting. Incidentally, you may have noticed that we didn’t have to do anything to get highlighting of strings in the code; the syntactic parser gives us that functionality for free.

These two lines modifies the syntax table, telling the parser to treat [ and ] as comment delimiters.

1
2

(modify-syntax-entry ?\[ "<]" inform7-mode-syntax-table)
(modify-syntax-entry ?\] ">[" inform7-mode-syntax-table)

So there we are, 41 lines of code all in all, not bad for a start.