Showkr - an in-browser application#

Some time ago I made quite simple, but (I hope) useful web application called Showkr. And now I decided to tell a story about why and how I did that - I think that could be useful considering growing popularity of in-browser applications.

Why

Flickr has quite heavy pages, and when you’re watching full set of photos, loading time of every next one is quite annoying (I’ve just tried, and it takes up to a second on a fast connection).

And with Showkr you open a page, wait for a reply from Flickr’s API and then relax watching photos, they are all on the same page. Especially given that everything is tailored for such activity - just photos, comments and every meaningful hotkey pair is mapped - j/k, up/down, space/shift+space. Welcome!

How

The first thing, which saved quite a bit of my time, is Twitter Bootstrap. There is nothing to tell, if you haven’t heard about it yet - now you did. Gives pretty look to your pages in almost no time.

Make

Another one is GNU Make. I never properly knew how to write Makefiles

• was scared in childhood by artifacts generated by autoconf/automake. But some time ago I’ve started to use make as they use rake - for some small tasks. In other words, like a simple organizer for shell commands.

And after doing this for years I felt that time has come. CoffeeScript wants to became JavaScript, templates want to became JavaScript, index.html wants to be different for development and production, and they don’t really want to do it at improper times (like generating JS in runtime).

So pretty and understandable Makefile has risen. And I’m going to cite some moments so that not only I, but others can also be educated by classics.

Basics

So, I have a directory with CoffeeScript files, and I’d like to convert them in JavaScript: nobody wants to include CS on client-side - that just makes your site slower. Let’s start with definition that we have those files:

SOURCE = $(wildcard app/*.coffee)

And a rule to compile them:

build/%.js: app/%.coffee
    @mkdir -p $(@D)
    coffee -pc $< > $@

This can look quite nasty, but I’ll explain syntax and it’s not impossible to live with it - it’s useful and dense DSL (though you probably could have something prettier). Makefile has:

• variables
• functions
• rules

Everything else doesn’t bother use. Here we have variable SOURCE, which contains a result of execution of function wildcard. Both variables and functions are retrieved (or executed) with $(...) construction (excluding single-letter variables, then you can simply write $x). Functions, naturally, desire some arguments.

Thing with a colon and indented body is a rule. Tells us that file with .js extension placed in build/ directory, depends on a file with exactly some name, but in directory app/ and extension .coffee.

This rule has two instructions or, in make-speak, recipes. Those recipes are calls to usual shell commands, though every is executed in its own shell instance (if you set variables here, they won’t be saved). Every recipe is printed to a standard output unless it starts with an @ - in this case it’s hidden.

Also, make provides you with a number of variables with strange-looking names. $@ - file-target (which we want to get), $< - first (and the only here) dependency. $(@D) - directory part of target file name. I decided to stop worrying about directories being created before they are necessary, and jsut started to create them everywhere I write something to a file.

And a final (intermediate final) chord: rule which will make this work:

all: $(patsubst app/%.coffee, build/%.js, $(SOURCE))

This rule is the first so that running just make will start it, it tells us that rule all depends on some files and we have already defined a rule for building those files. Which exactly files - everything in $(SOURCE), but with app/ replaced with build/, and .coffee with .js. That’s understandable, our compilation depends on having JS files in build directory. And every file depends on corresponding CoffeeScript file, which we defined earlier.

Running make in directory will compile every source file to JS. More than that, if you will run make again, it will only compile files which were changed - make looks for a file change date and makes no unnecessary moves.

It seems that nobody will need this, given that coffee -bco build/ app/ makes the same. Well, first of all, it’s not the same - it doesn’t track file modification time, but compiles everything (and you could have a lot of everything), and after that, it’s not only about CoffeeScript! But let’s not get ahead of ourselves.

Clean up

We have first incarnation of make file:

SOURCE = $(wildcard app/*.coffee)

all: $(patsubst app/%.coffee, build/%.js, $(SOURCE))

build/%.js: app/%.coffee
    @mkdir -p $(@D)
    coffee -pc $< > $@

What can we clean up? Well, we don’t need a list of source files, only results, so let’s change beginning of file this way:

SOURCE = $(patsubst app/%.coffee, build/%.js, $(wildcard app/*.coffee))

all: $(SOURCE)

More

It’s easier to understand now what main rule needs - targets. What do we need now? We have to put all our generated stuff to index.html.

Small digression: I don’t use `require.js`_ here, since I’m too lazy to make it work together with ender, so modules want to be loaded serially directly from the index page. In the other case, I wouldn’t have this part and index.html would be a static page, but I believe right now it’s a good excuse to write more rules.

So, our main rule wants index.html:

all: $(SOURCE) build/index.html

How do we generate it? I decided to do the right thing and take awk to process it. Now index look like that:

...
<head>
...
<!-- js-deps -->
</head>
...

And then I have an awk script, which takes DEPS environment variable and puts it into html:

/<!-- js-deps -->/ {
    split(ENVIRON["DEPS"], DEPS)
    # this way it goes from 1 to 9 instead of random ordering
    for (i = 1; DEPS[i]; i++)
        printf("<script type=\"text/javascript\" src=\"%s\"></script>\n", DEPS[i])
    next
}

1 # print everything else

I don’t want to make this article long with an explanation of make, but you can read some manual or google for more documentation.

Rule for building index looks herewith:

build/index.html: index.html $(SOURCE)
    @mkdir -p $(@D)
    DEPS="$(SOURCE:build/%=%)" awk -f build.awk $< > $@

What’s new here? We remove directory name, referring to a variable with substitution (similarly to $(patsubst ...), which we used earlier). It seems that’s all: mkdir created a directory, awk read a script, changed a file, redirected result to our target ($@ == build/index.html). Quite elegant.

Now make will compile (if necessary) CoffeeScript and then index.html.

Production version

Now we have to compile version for deployment - single JavaScript file. Okay:

prod: all prod/app.js prod/index.html

prod/index.html: index.html
    @mkdir -p $(@D)
    DEPS="app.js" awk -f build.awk $< > $@

prod/app.js: $(SOURCE:build/%=prod/%)
    @mkdir -p $(@D)
    cat $^ | uglifyjs > $@

Here make prod will take all dependencies of prod/app.js ($^ is another automatic variable and contains all dependencies of the rule) and minify them in target location. And will compile index.html.

I should say that all those directory name replacements in variable names annoy me a bit, so I will clean them up. That’s what I got in the end:

SOURCE = $(patsubst app/%.coffee, %.js, $(wildcard app/*.coffee))

all: $(addprefix build/, $(SOURCE) index.html)

build/%.js: app/%.coffee
    @mkdir -p $(@D)
    coffee -pc $< > $@

build/index.html: index.html $(addprefix build/, $(SOURCE))
    @mkdir -p $(@D)
    DEPS="$(SOURCE:build/%=%)" awk -f build.awk $< > $@

prod: all $(addprefix prod/, app.js index.html)

prod/index.html: index.html
    @mkdir -p $(@D)
    DEPS="app.js" awk -f build.awk $< > $@

prod/app.js: $(addprefix prod/, $(SOURCE))
    @mkdir -p $(@D)
    cat $^ | uglifyjs > $@

Another else?

We have a nice Makefile already. Let’s add templates! I have some in app/templates directory with .eco extension and I’d like to see them with extension .eco.js (to be differentiable from just .js).

TEMPLATES = $(patsubst app/%, %.js, $(wildcard app/templates/*.eco))

all: $(addprefix build/, $(TEMPLATES) $(SOURCE) index.html)

build/templates/%.js: app/templates/%
    @mkdir -p $(@D)
    ./eco.js $< $(<:app/%=%) > $@

prod/app.js: $(addprefix prod/, $(TEMPLATES) $(SOURCE))
    @mkdir -p $(@D)
    cat $^ | uglifyjs > $@

Here ./eco.js is a custom script to run Eco templates compiler, which applies necessary wrapper to results (ender module boilerplate). It takes two parameters - path to a file and module name (templates/something.eco). Templates will be minified in a single file with the application.

Important details

JS file order is important to me (because Ender modules are synchronous), so I just set them directly:

SOURCE = $(patsubst %,%.js,util api models viewing browsing showkr)

Article describes a situation when it’s ok to have default sorting (f.e. for AMD).

Function wildcard can’t find files recursively, so when I have subdirectories in structure, I use $(shell find ...) - your usual find.

That’s all, I hope, that I covered basics well enough. You can find full contents of Makefile (this link points to a version which existed in time when article was written) in the repository.

Architecture

Let’s get back to actual application. It’s built on top of `Backbone.js`_, which is probably most popular library for writing MVC applications on JavaScript. That’s not by randomness; Backbone doesn’t try to hide implementation details (as opposed to Ember, which I’ve tried and was unhappy), but organizes everything very well.

Core

Core part of application is a Router Showkr. Application is started by constructing this router’s object.

Main function, besides routing (calling functions by address in location hash), is view management. Router can create View by an unique identifier and switch between existing ones. Once created, views are not destroyed, so that there is no need to wait for same data for Flickr again.

The rest

The rest is simple - views initialize the models and inner views, models fetch data from Flickr (using redefined sync and parse methods).

Most of the models have some nested collection, so I’ve got a hierarchy User -> SetList -> Set -> PhotoList -> Photo -> CommentList -> Comment. Nested collections are initialized when model is initialized, but fetch is started where it makes sense - photos are fetched right after fetching set information, but comments only when photo is rendered.

To be honest, I have no desire to write detailed tutorial for Backbone - there is a lot of them already. But if you’re interested it’s worth it to look at source.

Epilogue

I had some thoughts to add Picasa support, but I’m a bit reluctant - I don’t use it myself, but API is vastly different, so it’s a lot of work. I would gladly accept patches though.

This article was written to guide those who need that to employ better techniques and tools for build in-browser applications. I hope that reading this article was interesting to you. And if you have any questions which were not covered, just send a mail to me.