[EDIT: Dec, 2012] I once again revisited this problem with a recent project and rewrote the code below into a (much more advanced) requireJS module, adding things like onSuccess and onError event handling for the Queue. You can find the code here. I’ll be using it in the Data Generator 3.0.0 rewrite [https://github.com/benkeen/generatedata].

https://gist.github.com/4242808

I’ve lost count of the number of times this problem has caused me headaches. So this last time I decided to devise a generic solution – a javascript execution queue – that should work for all similar cases. Read below for an discussion of the problem and solution. I found a number of other solutions on the web, but none that fit well enough for my case. So here’s my entry to the genre…!

The Problem

When you use innerHTML to insert some content into a webpage, you cannot instantly access the content via javascript. You have to wait several milliseconds – only then can the elements be accessed.

If several hundred forum posts are anything to go by, people generally rely on the trusty setTimeout function to get around this problem, but every time I did this it really stuck in my craw. This was for two reasons:

  1. Using a timeout involves guesswork: are you sure that timeout’s long enough to ensure the code can now safely execute? How about those cases where people’s browsers are busy doing other things, or are just plain slow?
  2. I didn’t understand why it was even necessary. It was as though this phenomenon broke the sequential nature of javascript: surely a line of code would only get executed after the previous line had finished executing?

Well, to answer #2 for similarly bewildered people: statements DO only get executed after the previous statement has been completed. The confusion lies in the fact that all modern browsers implement innerHTML by converting the raw string into DOM nodes after it’s been inserted – and this takes time. So while the initial statement of setting the innerHTML has been completed, the content is still not accessible for a short period of time.

I recently encountered a situation that no judicial use of timeouts could solve. The problem was with the load mechanism for my Data Generator script. When a user is logged in and wishes to load a saved form, it uses Ajax to pull from the server the entire form content stored in a JSON object, then populate the form, row by row. This can potentially take large amounts of processing time – even several seconds, depending on the size of the form. In this situation, there were so many page elements being shifted around, it was simply too complex to get right with multiple setTimeouts.

An aside about innerHTML vs. DOM / A disclaimer for why I used innerHTML

Building and manipulation DOM nodes is elegant, controlled and standardized. Reason enough to take that approach, you’d think. But I do believe some situations call for innerHTML. Here’s a few things to keep in mind:

  • innerHTML is faster than DOM manipulation, both for processing time and development time,
  • innerHTML is simpler to use and therefore less prone to errors than DOM manipulation,
  • using innerHTML over the DOM can result in (potentially significantly) less JS code to be downloaded by the client browser,
  • innerHTML contents after insertion are, on all major browsers, converted to DOM nodes and therefore normally accessible through JS (eventually…),
  • innerHTML, even though non-standard, is implemented on all major browsers (albeit with minor, frustrating differences).

In the case of the Data Generator, I had large chunks of HTML for each data type available to be duplicated for any particular data row at any time. For example, if the user chose “State / Province”, the HTML that needed to be inserted into the row totalled 50 lines and maybe 100 DOM nodes (not to mention attributes). Now imagine building that with the DOM…! Instead of 50 lines of HTML and one innerHTML call, we have 200 lines of dense javascript. In this case innerHTML was a simpler, faster, solution with less javascript to be downloaded by site visitors.

The Solution

Okay, back on topic.

So to solve this, I implemented a separate javascript queue layer, which would enforce the sequential execution of statements – even for this situation with innerHTMLs timing problems. Instead of accepting that a statement is complete when it normally finishes it’s execution, I’d define my own custom boolean test to determine this status. (For anyone that’s done much actionscript, this technique is probably quite familiar for queuing your events). Here’s the code: as you can see, it’s quite simple.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/*
  [0] : code to execute - (function)
  [1] : boolean test to determine completion - (function)
  [2] : interval ID (managed internally by script) - (integer)  
*/
var g_queue = new Array();
function process_queue()
{
  if (!g_queue.length)
    return;

  // if this code hasn't begun being executed, start 'er up
  if (!g_queue[0][2])
  {
    // run the code
    g_queue[0][0]();
    timeout_id = window.setInterval("check_queue_item_complete()", 50);
    g_queue[0][2] = timeout_id;
  }
}

function check_queue_item_complete()
{
  if (g_queue[0][1]())
  {
    window.clearInterval(g_queue[0][2]);
    g_queue.shift();
    process_queue();
  }  
}

There’s a single global array: g_queue which stores the actions to queue. The first two elements are function objects. I chose that particular data structure because it provided an elegant way to pass in groups of statements, not just a single line of code, which wasn’t sufficient in my case.

The overall process is pretty straightforward: it executes the content of the first index ONCE, then puts the boolean test (the second index) in a setInterval call, to be executed however often you want (here, it’s set to 50 milliseconds). Only when the boolean test returns true does it remove the entry from the queue and start processing the next code in line. If there’s nothing in the queue, it just returns.

Here’s how you’d add an item to the queue.

1
2
3
4
5
6
7
8
9
g_queue.push( [
  function() {
    // here's the code that takes TIME (e.g. innerHTML statements). These statements only get executed once.
  },
  function() {
    // this is our boolean test (e.g. if ID inside the innerHTMLed content exists).
    // It would return TRUE when the code in the previous function has been fully executed.
  }
    ] );

Then, once you’ve added one or more items, just call the process_queue() function to sequentially execute the statements. And bingo! “True” sequential processing even for innerHTML calls.

A final note on javascript closures: I believe using closures like in the anonymous function() definitions above don’t cause memory leaks in of themselves, since there are no circular references and the browser should be able to garbage collect the useless function after they’ve been executed. You can push entries into the queue in any execution scope. Please correct me if I’m wrong about this, though.

I very much hope this solution comes in handy!