Dimagog Blog

Clojure All The Way - Deep Knockout

2013-07-31T00:00:00+00:00

This is a fourth and final article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).

Building up on the [previous post][] we will achieve much deeper integration of Clojure data structures with Knockout.js. But it comes with a price …

Our Goal

By the end of the [previous post][] we’ve implemented observable-ref that completely hides Clojure-to-JavaScript conversion call to clj->js function, but we know it’s still there.

Maybe it’s silly to worry about it, but I still do. So what if we set our mind to get rid of it completely, and have “turtles all the way down”? That’s my goal for this post and it’s up to the reader to decide if it is achieved or not :-).

Know the Enemy

Let’s have a detailed look at what our Clojure data structures are and what our “enemy” (for this post) clj->js does for us.

The data we are rendering (headers of HTTP request) is a Clojure vector of 2-element vectors of strings. In type-safe language like C# or Java it would be Vector<Vector<string>>.

The clj->js function recursively converts our Clojure vector of vectors into JavaScript Array of Arrays. I.e. not only outer vector is converted, but each element of it (inner vector of strings) is also converted to Array. The recursion stops there because string representation is the same across ClojureScript and JavaScript and no conversion is needed.

The Easy part - Inner vectors

Later in this post it will become clear why this is the easy part, but let’s start with inner vectors.

Note: Code samples include only interesting parts, full source code is available below.

Shallow outer vector conversion

We want to stop converting inner vectors into Arrays. I.e. we don’t need a deep recursive conversion of outer vector that clj->js provides. Instead we want to convert outer vector to Array and leave its elements intact. The result of this step should become Array of vectors of strings.

This is easy, we can use into-array function in place of clj->js in observable-ref function:

1
2
3
4
(defn observable-ref [r]
  (let [state (ko/observable (into-array @r))]
    (add-watch r state (fn [obs _ _ new] (obs (into-array new))))
    state))

The only changes to this function compared to the [previous post][] are into-array calls in lines 2 and 3 where clj->js calls used to be.

Of cause if we try to render HTML now it won’t work because our KO bindings look like this:

<td data-bind="text: $data[0]"></td>
<td data-bind="text: $data[1]"></td>

and $data[0] access syntax does not work for Clojure vectors.

Accessing PersistentVector elements from JavaScript code

If we examine vector’s “class” PersistentVector from pure JavaScript perspective we’ll see the nth method that we want to call to get vector’s elements. But it looks like this:

cljs$core$IIndexed$_nth$arity$2: function (coll, n)

We certainly can call it from JavaScript, and it works:

$data.cljs$core$IIndexed$_nth$arity$2($data, 0)

but it’s not very user-friendly to say the least.

Fortunately JavaScript is a very flexible and dynamic language. We can easily extend PersistentVector with helper get method that we need:

1
2
3
4
5
(aset (aget [] "__proto__")
      "get"
      (fn [index]
        (this-as this
                 (nth this index))))

We use empty vector [] in line 1 to get to PersistentVector’s prototype. And add a new method get that takes an index and returns element at that index for this. this-as in line 4 gives us access to this from ClojureScript.

Now we can change our KO bindings in HTML file to use our new get method:

<td data-bind="text: $data.get(0)"></td>
<td data-bind="text: $data.get(1)"></td>

Not too bad, and rather straight-forward. At this point our Client works again and we can see the table of HTTP headers rendered correctly.

The Hard part - Outer vector

The only piece of data conversion logic we have left is into-array calls that convert our outer vector into JavaScript Array. If we get rid of it - we’ve reached our goal.

Why it’s hard

But this is where it becomes tricky. The problem is that we use KO’s foreach binding to iterate over our collection of headers:

<tbody data-bind="foreach: $root">

And foreach expects it to be JavaScript Array. Period. No kidding.

I’ve tried to teach KO to iterate over custom collection, and I’ve asked KO group. It’s not possible for now.

Should we give up?

At this point we still have achieved a tangible improvement: the elements of our collection are not converted anymore, so even when we create a copy of outer vector it is a shallow copy: only the “skeleton” is copied, but elements are reused. And they potentially have a bigger memory footprint.

However there is a saying:

If the mountain will not come to Muhammad, then Muhammad will go to the mountain.

In our case: if foreach would not take anything but Array then vector should become an Array. Or at least pretend to be one.

Now how do we pretend? The access pattern from KO is to read length property first and then read fields [0], [1] ... [length-1]. We can easily add all these fields to our vector but it defeats the purpose: we would copy all vector elements into fields 0, 1, etc. It’s no better than into-array call.

If only there was a way to intercept field access calls in JavaScript …

WARNING: Danger Zone

To the best of my knowledge there is no portable way to intercept field access in JavaScript. So we’ll have to use JavaScript “Experimental Features” that are already implemented by some modern browsers (Firefox and Chrome support what we need), but are not standard yet and may not even be enabled by default.

This is why the rest of this post is not practical [for now] but hopefully still entertaining.

If you are willing to proceed and use Chrome, you must navigate to chrome://flags URL (enter it manually in address bar, Chrome blocks navigation to special pages from random web sites :-), find “Enable Experimental JavaScript” feature and enable it.

ECMAScript 6 Proxy API

ECMAScript 6 draft specifies Direct Proxies - a mechanism to create a proxies that:

… enable ES programmers to represent virtualized objects (proxies). In particular, to enable writing generic abstractions that can intercept property access on ES objects.

Exactly what we need!

Wrapping outer vector in a Proxy

A new helper function vector-as-array makes vector sort-of look like Array in a narrow sense required by KO. This is by no means a full proxy to emulate Arrays, just a bare minimum required for our purposes.

KO only needs 2 things from Array: length field and 0, 1, etc. fields. Well, we can give it what it wants:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(defn vector-as-array
  "Creates JS Proxy around Clojure vector to make it look like JS array,
  without copying data"
  [v]
  (.create js/Proxy
           (js-obj
             "get" 
               (fn [_ prop]
                 (case prop
                   "length" (count v)
                   (get v prop)))
             "getPropertyDescriptor"
               (fn [obj prop]
                 (.getOwnPropertyDescriptor js/Object js/Array prop)))))

The get function (lines 7-11) checks if accessed property name is length and returns count of vector v if it is (line 10). Otherwise (line 11) it simply delegates to vector’s get function to fetch individual element.

The getPropertyDescriptor (lines 12-14) is required for when KO does “reflection” on our “array” (like checking if “length” property exists) and simply delegates calls to js/Array.

The Finish Line

The last change is to modify observable-ref once again, this time replacing into-array calls with our newly-created vector-as-array function:

(defn observable-ref [r]
  (let [state (ko/observable (vector-as-array @r))]
    (add-watch r state (fn [obs _ _ new] (obs (vector-as-array new))))
    state))

And it’s hard to believe, but IT WORKS!

Conclusion

Here is a quick summary of our accomplishments:

We’ve marshaled Clojure data structure created on the Server all the way to the Client using edn encoding which is a natural textual representation for Clojure data.
We’ve received this data on the client using core.async to make our code look sequential
Then we’ve edn-decoded response and called regular Clojure functions on received data to extract and sort the headers
We’ve animated our HTML page by manipulating only Clojure data structures
Finally we’ve created KO bindings directly to Clojure data without EVER converting it to JavaScript.

Granted, the final step required sacrifices (using experimental JavaScript features), and this is why I’ve mentioned above that it depends on reader’s point of view if final goal was achieved or not.

I’ve certainly achieved my goal of learning new interesting technologies and I had lots of fun along the way!

And if anyone ever reads this, I hope you did too :-).

Source code

Full source code can be found on GitHub.

If you want to build and run it locally, execute:

git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay4 --single-branch ClojureAllTheWay4
cd ClojureAllTheWay4
lein ring server

Clojure All The Way - Knockout

2013-07-22T00:00:00+00:00

This is a third article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).

This time we’ll continue building on [previous post][] and will integrate ClojureScript with Knockout.js.

Where we are

We start where we left off in the [previous post][]: we have a Server that implements echo HTTP API and calling it returns edn-encoded data.

And we have a Client capable of calling this API, receiving data, and edn-decoding it. But it displays the data (headers of our HTTP request) in a very primitive way by simply setting text of DOM element.

Our Goal = ClojureScript + Knockout

We’d like to nicely format received headers as HTML table. We can of cause write ClojureScript code to build HTML for it using Hiccup-like library. But where is fun in that!

It would be more entertaining to use some JavaScript data-binding library. First, because I like MVC-based approach. But more importantly because it will allow us to explore ClojureScript integration with existing JavaScript framework.

We’ll use Knockout.js (abbreviated KO for the rest of this post) as it happens to be my favorite MVVM framework for JavaScript. And also KO claims that it is very smart and highly optimized in minimizing DOM modifications, so I expect it to be efficient as well.

There is an excellent 20-min Knockout.js demo video on KO web site. If you completely new to KO I’d recommend to watch it first. It will make material below easier to understand, plus it’s a truly masterfully-done demo.

Naive Knockout integration

Let’s see what is the minimal effort required to integrate our code with KO, and then we’ll improve on it.

Preparing input data

First we need to massage our data: headers are represented as a map but KO requires it to be array’ish collection if we want to use its foreach binding. Easy enough: map converted to seq is actually a sequence of vectors, each vector being a key-value tuple. While we at it let’s sort it by header name as well:

  (let [headers (->> "/api/echo" get-edn <! :headers (sort-by first))]

Calling sort-by is the only addition to our previous code. It converts map to seq and also sorts it by first element of each tuple, i.e. by header name.

The last step is converting Clojure data to plain JavaScript data that KO understands. We do it by using clj->js function that will recursively convert our Clojure vector of vectors into JavaScript Array of Arrays:

(clj->js headers)

I know the whole premise of the series is to avoid such explicit conversion. Please bear with me, we’ll deal with it in following sections.

Adding Knockout bindings to HTML

Note: Code samples include only interesting parts, full source code is available below.

We need to change our default.html as follows (only relevant parts are shown):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<head>
  <script src="http://ajax.aspnetcdn.com/ajax/knockout/knockout-2.2.1.js"></script>
  <script defer src="app.js"></script>
</head>
<body>
  <div id="content">
    <table>
      <thead><tr><th>Header</th><th>Value</th></tr></thead>
      <tbody data-bind="foreach: $root">
        <tr>
          <td data-bind="text: $data[0]"></td>
          <td data-bind="text: $data[1]"></td>
        </tr>
      </tbody>
...

First we add reference to KO itself in line 2. Then we create a table (line 7) with static headers (line 8) and KO-bound body (line 9). The foreach in line 9 iterates over $root binding (not explained yet) expecting it to be an array and replicates its HTML body for each element while binding $data to it.

In our case each element is a key-value tuple for each header represented as a 2-element array. So tds in lines 11-12 simply extract first element for the header name and second for the value.

Applying root Knockout binding

Now to connect our ClojureScript data to KO bindings in HTML we just need to call ko/applyBindings. And our final code for this step looks like this:

1
2
3
(go
  (let [headers (->> "/api/echo" get-edn <! :headers (sort-by first))]
    (ko/applyBindings (clj->js headers))))

We obtain the data and massage it in line 2. Then convert Clojure data to JavaScript using clj->js and pass it as a root binding to KO using ko/applyBindings in line 3.

And viola! Our headers are shown in a nice-looking table (with little CSS help):

Now readers familiar with KO are probably shaking their head in disgust: no observables, and applyBindings should be called only once (not from some potentially reusable go-routine).

And let’s not forget about explicit clj->js call.

Fear not! we’ll take care of it right away!

Using computed observables

My first (but not final) improvement attempt is to use KO’s [computed observable] to do automatic clj->js translation.

KO documentation says that extenders could be a better fit, but I’m more familiar with computed observables.

First we create a helper function to build our [computed observable][]:

1
2
3
4
5
6
(defn observable [val]
  (ko/computed
    (let [state (ko/observable (clj->js val))]
      (js-obj
        "read"  (fn [] (state))
        "write" (fn [new] (state (clj->js new)))))))

ko/computed observable (line 2) takes a JavaScript object created in line 4 that specifies read and write functions. The actual state is kept in regular ko/observable created in line 3.

KO observables are functions. Calling it with no arguments returns current value and calling it with one arguments sets its value. So (state) in line 5 gets the current state and (state (clj->js new)) in line 6 sets it (after converting new to JavaScript representation).

Now our main code looks like this:

1
2
3
4
5
6
(def view-model (observable []))
(ko/applyBindings view-model)
 
(go
  (let [headers (->> "/api/echo" get-edn <! :headers (sort-by first))]
    (view-model headers)))

We create our view-model in line 1 using our observable helper, and set it to empty vector first. Then we bind it in line 2, this needs to be done only once. And when we obtain our headers in line 5 we set view-model observalble in line 6 (remember - observables are functions). This in turn triggers DOM update.

Note: using global vars like view-model is not recommended in good Clojure code, but will do for our sample code.

Great! clj->js conversion is nowhere to be seen in our main code and we adhere to good KO practices, but …

What’s wrong with it?

First, getting/setting view-model by calling it as a function is not very Clojury.

Second, and more importantly, if we read (view-model) we’ll get JavaScript-converted objects, not the original. The original is “lost in translation”. Which means that if we want to modify our view model in ClojureScript we need to keep it separately somewhere.

And where do we usually keep changing data in Clojure? In refs of cause! At this point a light bulb should begin hovering above your head: the refs in Clojure have watchers, so they can notify whoever is interested when they change.

refs sound suspiciously familiar to ko/observables: both track changing state and both send notifications when change happens. Can we somehow bridge the two? We certainly can!

Using Observable refs

Let’s create a helper function to create a ko/observable tracking Clojure ref:

1
2
3
4
(defn observable-ref [r]
  (let [state (ko/observable (clj->js @r))]
    (add-watch r state (fn [obs _ _ new] (obs (clj->js new))))
    state))

observable-ref takes a ref as parameter r (the ref itself, not its value!). We create a ko\observable named state in line 2 with initial value of r converted to JavaScript. Then in line 3 we add a watcher for the r.

The watcher fn takes 4 parameters, but we only care about 2: obs is our observable state and new is the new value of the ref r. Then we simply convert new value to JavaScript representation and put it into obs.

And finally we return state observable from the function so that it can be passed to KO.

Our main code now changes to this:

1
2
3
4
5
6
(def view-model (atom []))
(ko/applyBindings (observable-ref view-model))

(go
  (let [headers (->> "/api/echo" get-edn <! :headers (sort-by first))]
    (reset! view-model headers)))

Notice that view-model created in line 1 is now a Clojure ref (in this case an atom). And we don’t even save the observable returned by our observable-ref helper, we just pass it directly to ko/applyBindings in line 2.

And modifying our view-model (line 6) is as simple as modifying any other Clojure atom, yet DOM immediately reflects the changes!

Isn’t it nice! Let’s have some fun with it …

Animating our table

Just for fun, let’s pretend that our Internet connection is very slow and headers are received one by one, slooowly. We can emulate this by conjing headers to view-model one at a time with delay:

(go
  (let [headers (->> "/api/echo" get-edn <! :headers (sort-by first))]
    (doseq [h headers]
      (<! (timeout 500))
      (swap! view-model conj h)))

Then we’ll delay for a second and pretend that terrible computer virus eats our headers one at a time:

    (<! (timeout 1000))
    (while (seq @view-model)
      (swap! view-model rest)
      (<! (timeout 500)))

Now delay for another second to keep the suspense, and finally restore our headers table to calm down the user:

    (<! (timeout 1000))
    (reset! view-model headers)

Pure Clojure data manipulation code, yet KO faithfully reflects all our changes in the DOM!

Mission accomplished!

Or is it? I have tentative plans for another post in the series, but it requires a little more research. I’ll keep you posted.

UPDATE: Keeping you posted, here is the next post.

One last thing: optimized builds

Unlike Steve Job’s famous “one last things” this one is boring, but important. If we try to build our ClojureScript code with :advanced optimization it will not work. This is because Google Closure compiler minifies all names in produced .js file including names like ko and observable. Certainly not what we want.

The solution is to add so-called [externs file] and feed it to Google Closure compiler, so it would know which names it is not supposed to touch.

Conceptually the [externs file][] is similar to C/C++ .h headers files, or .asmmeta files for C#. Ours is called ko.externs.js and looks like this:

var ko = {};
ko.applyBindings = function() {};
ko.observable = function() {};
ko.computed = function() {};

Just declarations, no code. And after adding it to project.clj:

:externs ["cljs/ko.externs.js"]

… we can build our optimized code, and it will work correctly:

lein cljsbuild once opt

If you are wondering how big is produced optimized app.js file (I sure was), it is 118'871 bytes.

Source code

Full source code can be found on GitHub.

If you want to build and run it locally, execute:

git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay3 --single-branch ClojureAllTheWay3
cd ClojureAllTheWay3
lein ring server

Clojure All The Way - The Client

2013-07-18T00:00:00+00:00

This is a second article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).

The previous post dealt with the Server part, now let’s look at the Client side.

Starting Point

We start where we left off in the previous post: we have a Server that implements echo HTTP API and calling it returns edn-encoded data.

And we have a Client capable of calling this API and receiving data, but it still treats it as text, and not as structured Clojure data. The client was actually explained not in the last post, but in post before that.

First, small improvement to GET function

Note: Code samples include only interesting parts, full source code is available below.

The GET function in previous client post was implemented using (>! ch ...) inside go-routine (line 6):

1
2
3
4
5
6
7
8
(defn GET [url]
  (let [ch (chan 1)]
    (xhr/send url
              (fn [event]
                (let [res (-> event .-target .getResponseText)]
                  (go (>! ch res)
                      (close! ch)))))
    ch))

In comments to that post Alexander Solovyov suggested using put! instead. It’s a good idea (thanks Alexander!) since we don’t care when channel write completes. Here is the change:

1
2
3
4
5
6
7
(defn GET [url]
  (let [ch (chan 1)]
    (xhr/send url
              (fn [event]
                  (put! ch (-> event .-target .getResponseText))
                  (close! ch)))
    ch))

the inner let and go blocks are gone and we use put! in line 5 instead.

Now back to our main topic …

Getting Clojure data from Server response

Let’s add a new function get-edn that uses GET function above to get data from the Server, and then converts it to Clojure data structures, i.e. edn-decodes it. It is actually very simple, all we need to do is call read-string function from cljs.reader namespace.

And in addition to being useful for our purposes it will also show how async functions compose.

1
2
3
4
5
(defn get-edn [url]
  (go 
    (-> (GET url)
        <!
        read-string)))

Let’s look at it from inside out. In line 3 we call our GET that returns a channel that will eventually contain result (text). In line 4 we read the value from this channel potentially “parking” this activity, but not blocking. This is why we need this code wrapped in go-routine. And then in line 5 we pass returned text to read-string that edn-decodes it and returns Clojure data structure (let’s call this value result).

But what happens next? How is result propagated to the caller of get-edn? To answer that we need to understand what get-edn returns. Following normal Clojure rules get-edn returns the value returned by go form. This is a channel, now who writes to this channel? The go form does. Here is how go works:

first it creates a channel and promptly returns it to the caller
eventually it evaluates all statement inside, potentially “parking” this activity when <! and >! forms are encountered
it writes the value of the last statement to the channel, close!es the channel and completes.

In our case the last and only statement inside go is our “conveyor” producing result defined two paragraphs ago, i.e. decoded response. The go form writes result to the channel it has created and this is how it gets to get-edn caller.

And this is how async functions compose! Notice that we didn’t even have to create a channel explicitly in get-edn!

The reason we had to create channel in GET function is because we were converting callback-based API to asynchronous one. The caller of our fn in there was not async-aware, and actually ignored the return value. So we had to propagate it to GET caller thru manually-created channel ch.

Calling get-edn function

To prove to ourselves that get-edn returns Clojure map let’s call some function on it before displaying it, for example let’s extract only :headers:

1
2
3
4
5
(go
  (dom/set-text! (sel1 :#log)
                 (-> (get-edn "/api/echo")
                     <!
                     :headers))

The whole block is wrapped in go routine because we use <! (read channel) form. We call get-edn in line 3 that returns a channel, we read from it in line 4, potentially “parking” (but again, not blocking) this activity. Then we call :headers in line 5 to get only headers portion of respose proving that result is actually a Clojure map.

And if you run the sample code, you’ll see the headers displayed.

Now this is nice, but the way they are displayed is kind of lame. We’ll do something about it in the next post, while staying on topic of using Clojure data structures as much as possible.

Source code

Full source code can be found on GitHub.

If you want to build and run it locally, execute:

git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay2 --single-branch ClojureAllTheWay2
cd ClojureAllTheWay2
lein ring server

Clojure All The Way - The Server

2013-07-14T00:00:00+00:00

This is a first article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).

In this post we’ll develop a Web server in Clojure (using Ring and Compojure) that exposes simple API to serve Clojure data structures in their natural text representation (fancy term for this is edn-encoded).

The Basics are not covered

The basics of creating Ring+Compojure Web server are covered very well on Interwebs. In this post I’ll only focus on creating API returning edn-encoded Clojure data.

Serving the site itself (static files)

Note: Code samples include only interesting parts, full source code is available below.

Let’s start with vanilla web server that simply serves static files:

1
2
3
4
5
(defroutes app
  (GET "/" [] (file-response "default.html" {:root "html"}))
  (files "" {:root "html"})
  (not-found "<h1>404 Page not found</h1>")
)

line 2 returns (not redirects to) default.html when server root is accessed
line 3 serves static files from “html” directory
line 4 is only hit if static file is not found and returns 404

Adding Echo API

Now let’s add an echo API that simply echoes back the client’s HTTP request.

To be precise: upon receiving HTTP GET /api/echo request the server will respond with 200 OK and body containing edn-encoded Clojure representation of request (a map of maps, strings and numbers).

Of cause we are not going to do heavy-lifting ourselves - that’s the whole point of Ring. To paraphrase Apple’s commercials “there is a middleware for that”. And it’s called ring.middleware.format-response. Let’s use it:

1
2
3
4
5
6
7
8
9
(defroutes handler
  (GET "/" ...)
  (GET "/api/echo" request
       {:status 200
        :body request})
  ...)

(def app (-> handler
             wrap-clojure-response))

The app was renamed to handler (line 1) and new app wraps this handler into wrap-clojure-response (lines 8-9).

The new Compojure route (lines 3-5) simply takes a request (a Clojure map) and builds a 200-response from it setting :body to request (still a Clojure map).

And the wrap-clojure-response middleware is the one converting Clojure map :body returned by handler to edn-encoded textual representation.

There is one problem with this code: :body of request that we are sending back is not a Clojure data structure. It’s a Java object of HttpInput org.eclipse.jetty.server.HttpInput type. That’s not good: it cannot be edn-encoded and Client won’t be able to decode it anyway. A small tweak removes it from the response we are sending (call to dissoc in line 3):

1
2
3
  (GET "/api/echo" request
       {:status 200
        :body (dissoc request :body)})

Testing

To test our Server we can use a slightly modified client from my earlier post. And sure enough it shows client’s request returned back to us.

But ClojureScript Client still treats response as text (not structured Clojure data). We’ll deal with it in the next post.

Source code

Full source code can be found on GitHub.

If you want to build and run it locally, execute:

git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay1 --single-branch ClojureAllTheWay1
cd ClojureAllTheWay1
lein ring server

Making HTTP requests from ClojureScript with core.async

2013-07-12T23:40:00+00:00

I’ve always been a big fan of async/await feature in C# (long before it was publicly available). So, naturally, I was very excited to read the announcement about Clojure core.async library. Even more so after I’ve learned that it works for ClojureScript as well.

Here is an example of using core.async to convert callback-based goog.net.XhrIo send API to sequential-looking one.

Leningen config (boring stuff)

Note: Code samples include only interesting parts, full source code is available below.

(defproject ...
  :repositories {
    "sonatype-oss-public" "https://oss.sonatype.org/content/groups/public/"
  }
  :dependencies [
    [org.clojure/clojure "1.5.1"]
    [org.clojure/clojurescript "0.0-1820"] 
    [org.clojure/core.async "0.1.0-SNAPSHOT"]
  ...

We need Sonatype because it is hosting SNAPSHOTs, and there is no release version of core.async available yet. Also I’m not using the latest version of ClojureScript because Light Table does not like it.

Defining GET function (cool stuff)

First the ns header:

(ns app
  (:require
    [goog.net.XhrIo :as xhr]
    [cljs.core.async :as async :refer [chan close!]])
  (:require-macros
    [cljs.core.async.macros :refer [go alt!]]))

It’s not idiomatic to have all uppercase function names in Clojure, but since this function mimics http GET method we’ll make an exception for it.

The idea is to return a channel from GET function that will have a result of send call when it completes. The callback simply extracts the result and writes it to the channel:

(defn GET [url]
  (let [ch (chan 1)]
    (xhr/send url
              (fn [event]
                (let [res (-> event .-target .getResponseText)]
                  (go (>! ch res)
                      (close! ch)))))
    ch))

We create channel ch of size 1 (default is 0) because there is no reason to block callback until reader arrives. Instead it writes the response once it becomes available and disappears.

You may think that callback could be simplified as:

; WARNING: Broken code
              (fn [event]
                (go (>! ch (-> event .-target .getResponseText))
                    (close! ch)))))

but that does not work. This version tries to extract result from event inside go routine which will execute eventually. By the time it runs event does not have the result anymore.

Also it’s a good idea to close! the channel ch after we write the result res, so if anyone tries to read from ch again by mistake, they will get nil returned immediately.

Using GET function (even cooler stuff)

Let’s define a helper log function first:

(defn log [s]
  (.log js/console (str s))))

And now we can call our GET function and print the result:

(go
  (log (<! (GET "http://dimagog.github.io/sitemap.txt"))))

We have sequential-looking code that looks like it’s blocking, but it’s not. It is fully asyncronous!

Source code

Full source code can be found on GitHub.

If you want to build and run it locally, execute:

git clone https://github.com/Dimagog/AsyncGET.git
cd AsyncGET
lein ring server

About Me

2013-07-12T22:40:00+00:00

I work for Microsoft on Windows Azure team where I (mostly) program in C#. And in my spare time I love to play with new and interesting languages and technologies inside and outside of “Microsoft stack” such as Clojure, Haskell, Erlang, F#, Scala, Riak, Datomic, Storm, Akka, Hierarchical State Machines, ANTLR, etc.

I tend to forget a lot of what I learn, so I’ve decided to start this Blog where I can keep and share my notes as well as practice technical writing.

Hopefully others will find something useful here as well. Comments and suggestions are certainly welcome!

I share links that I find interesting on Google+ and on Facebook.
My Twitter is @DimaPhone (not dimagog), and here is my LinkedIn page.

- Dmitry Kakurin (dimagog)