JavaScript does not have "pass by reference"

About every 17 minutes, somebody on Stackoverflow answers a question or posts a comment that implies that passing an object reference to a JavaScript function is an example of "pass by reference". Those people are all wrong, so I am typing in this post so that I can link to it and save myself some time.

These wrong people are not bad people, and they're almost always trying to answer a question and be helpful. The problem here is that "pass by reference" has a very clear meaning when discussing programming language semantics. What it doesn't mean is that it's possible to pass a reference to a modifiable object into a function. Yes, you can do that in JavaScript (and Java and lots of other languages), but that's not what "pass by reference" means. The key here is the super-important word by in that phrase.

Here's the simple test to see if a language supports "pass by reference":

Can you write a function "swap" in Language X that looks and does something like:


variable x, y;
x = 1;
y = 2;
swap(x, y);
print(x + ", " + y); // prints "2, 1"


If the answer is "yes", then congratulations, your language supports "pass by reference". C++ is one of those. You'd write "swap" something like this^*1:

void swap(int &x, int &y) {
  int tmp;
  tmp = x;
  x = y;
  y = tmp;
}

The "&" thing in C++ tells the compiler that instead of passing in the value of parameters when calling the function, it should pass a reference to the parameters. Now, because (in C++, at least as far as I remember) you can only get a reference to something like a variable, and not something like the constant "14", that means you get an error if you do

swap(14, y + 1);


Note also that this works in C++ because the compiler knows about the nature of the called function; it has to be able to see a declaration somewhere for things to work.

Languages like JavaScript have no such luxury. JavaScript has no idea whatsoever what any function actually looks like. References to functions are determined during the process of expression evaluation, and all the runtime knows (modulo whatever sorts of smarty-man optimizations are going on) is that it's got a reference to a function and a list of argument expressions to evaluate. There's no such thing (except in the perverse case of the "arguments" monster in functions, which is a whole 'nuther thing) as a reference to a variable (or a computed value, of course) in JavaScript.

JavaScript is always and only a "pass by value" language. When you pass a reference to a mutable object, you're passing a value — the reference to the object. You can't write "swap" in JavaScript.

Algol 60^*2 had a weird parameter passing variant called "pass by name". The idea was that an expression in the calling environment would be treated as something to do whenever the formal parameter in the function was referenced in an expression. A JavaScript version of that would be that you'd pass in a function, and the called function would invoke that passed-in function whenever it wanted the value:

function passByName( paramFunction ) {
  if (paramFunction() == 2) {
    somethingSomething( paramFunction() );
  }
  else {
    whatever( paramFunction() - 1 );
  }
}

So in Algol you'd write something like:

  passByName( foo + 17 );

and the compiler would do something more-or-less like wrapping that expression up in a function, and in the called function references to the parameter name would result in calls to that generated function. Those things were called "thunks" as far as I know.

So: yes, it's true that in JavaScript it's possible to use an object with properties to make it possible for functions to change stuff in the calling environment other than via a simple return value. That's not "pass by reference".

*1 - I barely remember C++. It's something like that.
*2 - I don't know anything about Algol 60 other than what some dude told me in college 30 years ago. The "pass by name" concept is cool anyway and it would probably work something like that. It's kind-of like Haskell lazy evaluation in some ways, I think. Maybe.

Togglefrog

Mostly because it's fun to think up silly names for plugins, but also thanks to experience on Stackoverflow seeing and answering lots of questions, I've been working on another jQuery plugin.

The Togglefrog plugin attempts to handle the (usually) simple problem of forms with sometimes-relevant, sometimes-not sections. Consider a reservation or booking form for a service that allows for some number of discounts based on organizational affiliations. Each separate type of affiliation might call for different messaging, authentication inputs, and even further choices. In the form, radio buttons or checkboxes select between options, and it's nice to make it clear that some parts of the form are only interesting in certain situations.

For a simpler, goofy example, consider a lunch menu with "Vegan", "Vegetarian", and "Omnivore" sections. To clarify the separation between them, the user is asked which general menu they're interested in, and then the form will show the separate choices of that type.

Vegan Vegetarian Omnivore

Below those would be the choices, but ideally we'd like for only the right choices to show up; we don't want vegans to panic at being offered a steak.

What Togglefrog does is allow the markup itself to indicate how the conditional sections of the form — either individual elements or sections grouped by container elements — to indicate that they're to be controlled by a toggler. A toggler can be a radio button, a checkbox, or an option in a <select> element.

To do that, the sections of the form (and note that really forms have nothing directly to do with this, it's just part of this example) is given one or more "data-" attributes: <div style='padding: 10px 0px 10px 5px'> <input type='radio' name='diet' id='vegan' value='vegan'> Vegan <input type='radio' name='diet' id='veggie' value='veggie'> Vegetarian <input type='radio' name='diet' id='omni' value='omni'> Omnivore </div> <div class='togglefrog menu' data-toggler='vegan'> <input type='radio' name='meal' value='sprouts'> Sprout salad with chickpea chips<br> <input type='radio' name='meal' value='tofu'> Toasted pressed tofu with soy-garlic gravy with sweet/sour green beans<br> <input type='radio' name='meal' value='beanburger'> Black bean burger on spelt-quinoa bun with homemade ketchup and sweet potato fries </div> <div class='togglefrog menu' data-toggler='veggie'> <input type='radio' name='meal' value='grill'> Grilled cheddar with tomato on spelt-quinoa bread with sweet potato fries<br> <input type='radio' name='meal' value='taco'> Seitan and pepper stir-fry and aged Monterrey Jack in whole-wheat tortillas with tomato-parsley salad<br> <input type='radio' name='meal' value='soup'> Creamy broccoli cheddar soup with pumpkin seed crackers </div> <div class='togglefrog menu' data-toggler='omni'> <input type='radio' name='meal' value='meatburger'> Angus beef burger on spelt-quinoa bun with homemade ketchup and sweet potato fries<br> <input type='radio' name='meal' value='salmon'> Alaskan salmon steak with chipotle glaze and aromatic brown rice<br> <input type='radio' name='meal' value='chicken'> Chardonnay-poached chicken breast with buttered peas and aromatic brown rice </div> First you see the markup for the three radio buttons, then the form sections. The "data-toggler" attributes give the "id" values the plugin looks for to find the togglers.

If that markup were put on a page that imported jQuery and Togglefrog, and the <body> tag also had class "togglefrog", then there's not even any code to write. The plugin wires up everything marked with that class (except the body element, of course) guided by the "data-" attributes. Oh, and we'll need some CSS too: div.menu { position: absolute; display: none; } div.menu.toggled { display: block; } The "toggled" class will be added by the plugin to each toggled element when its toggler is "on" (checked for a checkbox or radio button, and selected for an option). You can see this particular example in action at this jsfiddle page. Go there now and play with it. Note that initially, none of the menu types are selected; normally I don't like that, but it's helpful with this example. As soon as you click a menu button, you'll see the corresponding items show up.

You may be saying, "Gee, big deal - a whole plugin just for something as simple as that?" Well, two points. First, if you're working with a web application that's got lots of this stuff, it's tedious and error-prone to re-code the same stuff over and over again for every particular form. Besides, the plugin isn't that big.

The second point is that it's not really so simple. Note that the menu item radio buttons for the different meals are all named "meal". That way, the back-end form handler only has to worry about one parameter — "meal". The plugin makes sure that any toggled section that's toggled "off" (that is, its toggler radio button is unchecked) then all the inputs are marked as disabled. The only ones that'll be submitted will be those corresponding to the menu choice.

Note also that if you click "Vegan" and select something, then click "Omnivore" and select one of those, and then go back to "Vegan", your original choice is preserved. The plugin handles the juggling of those radio buttons so that the currently-visible members of a like-named group keep any former selection they may have had.

In addition to just hiding/revealing form sections, the plugin can be used to do other sorts of style changes. For example, it's pretty easy to implement your own "fancy" checkbox mechanism with it. You'd do something like this:

• Put a checkbox on the page and make it hidden. (You can't just make it "display: none" because then it won't work in IE; make it transparent or position it way off to the left of the page.)
• Create a <label> element and style it with a sprite background for your fancy checkbox (or radio button or whatever). Make the label be toggled by the checkbox, and also "for" the checkbox.
• In your CSS, use the "toggled" class (which you can override by the way) to move the sprite between the untoggled and toggled states.

The plugin is available from github with a short README and a minimal test file. I'll write up a real documentation page at some point. I also need a cute frog logo.

On Javascript Dependencies

I've been reading some posts about large-scale, or "enterprise", applications involving Javascript. As a brief aside I'll note that to me, to call software "enterprise" is to pronounce a terrible condemnation upon it, to evoke my memories of awful intranet applications, sprawling old distributed client-server systems, horrible user interface designs obsolete at their debut, and tremendous, sometimes astronomical costs of purchase and deployment. Though I'm still basically doing "enterprise software development" I never say those words to myself. But that's just a personal problem. It's funny how words get poisoned.

Anyway, here's an interesting article I just read this morning. I'm really impressed at some of the apparent capabilities of those tools the article discusses. However, the "standard folder structure" described therein evinced a distinctly different approach to the management of client-side code (well, client-side everything, by extension) than what I've come to think of as the One True Way. It's always interesting to read something like that and have one's own notions illuminated by glaring contrast with someone else's. I'm not going to say my ideas are better of course, but I think they're fundamentally different. It's going to be a little hard to describe why, because I don't think I'm equipped with the terminology necessary to discuss the subject; I'm not even sure I know exactly what the subject is. Nevertheless, as I've been preparing to do a quick presentation about the little wallflower jQuery plugin that I wrote a few weeks ago, I've been forced to think not only about the way I actually go about designing and arranging and managing my application as it grows, but also about the way I think I should be doing it.

Back a really long time ago, in the early '90s, one of the trendy must-read business books of the time was Peter Senge's The Fifth Discipline. I guess it says something about how business/management improvement trends go that you can pick up copies of that book now for the cost of shipping. In other words, it's almost literally worthless. But it's a pretty good book, as things like that go. Senge's an interesting guy, an engineer by training who got all fired up about the topic of "organizational learning", and that's what he's been doing at MIT's Sloane School of Management for a couple decades. The book is basically about that: what it means for an organization to learn.

Much of the material in The Fifth Discipline is sourced from the work of two other Sloane School researchers, Chris Argyris and Donald Schön. Since the 1950's, those two have been doing some really interesting work on the theory and mechanics of learning, both individual and organizational. Check out Dr. Argyris's Wikipedia page for a brief taste of what he's written about. One of the lasting concepts I took away from one of his books (which, you'll note, is not considered worthless) is an idea that Senge also latched onto, that of recognizing and contemplating the distinction between theory in use and espoused theory. An espoused theory is something you type into a blog post about how everyone should architect their enterprise application. A theory in use is how a development team actually does it. (Seriously, that book is one of those eye-opening, mind-opening books that you just can't forget, providing enormous insight into all the stupid things that go on in organizations large and small. If everybody read and believed in the stuff Argyris and Schön have written about, the world would be a better place.)

Now I'm not going to pontificate about how you should architect your web applications. My point in bringing up those books and those ideas is that writing about how one does things can be pretty humbling. I know that the way I actually work isn't as close as I'd like it to be to the grandiose fantasies I entertain about The Perfect Application. More than that, however, what I learned from those books is that I should not only be aware of the ways my actual work doesn't live up to my ideals, but also to be aware of the reasons I think those ideals are ideal. What do I want from my application design, and why do I want it? If I were able to fully realize my ambitions for perfect design, would I perform some sort of test to see if I actually was getting what I wanted? That's something that's generally left out of most process improvement activities — and I mean process improvement activities as actually done, not as they're described in books and blog posts. How many teams have you worked on that, after taking up some trendy activity like stand-up meetings, have actually applied an even vaguely scientific testing method to see whether the new activity was helping? How many really analyze the goals for starting their process improvement in the first place, and subsequently test to see if achieving the goals helps out their business bottom line? I think the same rigor should apply to technical processes like application construction and code management, but in my experience that's not so common.

One of the difficulties I'm having with this subject is that I don't know what to call the central activity. That activity is the construction of the application code behind an application, and the design for how that construction should proceed. I use the term "construction" because I've been driving through a bit of it for several months now on my way into and out of town. Civil engineering projects involve an outward architecture: somebody designs what a bridge should end up looking like, the shape of the arch, the look of embankments, the decorations on railings and lane dividers, and all those other things that'll be mostly ignored by the public when the project is complete. However, I suspect (knowing essentially nothing at all about what civil engineers do) that a whole lot of work goes into the design of the construction effort itself. I'm sure that there's a tremendous amalgamation of experience that instructs thousands of planning decisions for a big roadway project. Those must be planned with a rigor that I've never seen even distantly approached in any software project I've experienced. Maybe I've just been unlucky, but I don't really think so.

What do I call, then, the design and planning of software implementation, and by extension the theory of that design? The term "application design" seems to be understood as being more about what an interface looks like and what application users should experience. "Coding practices" to me means "tabs or spaces" and details like that. I'm going to use the term "application implementation design" because I can't think of anything else. I'm absolutely not an expert, of course, as should be clear by this admission that I literally don't know what it is that I'm talking about, or at least what to call it.

Now back to Javascript. What is it that I want from the way I type in Javascript code? To put it another way, what is it that I want to be true at some point in the future when the surrounding application is, if not "finished", something large and functional and under heavy use? Of course it's clear that I want it to work and to work well. That means it doesn't break in small browser windows, it doesn't slow down pages, it doesn't annoy users, and so on, satisfying all those little de rigueur stipulations we twitter about constantly. But I also think there are aspects to the application that are of great importance to the developer experience. What should it be like to add a new feature? To revise an old feature? To fix bugs? To cope with off-the-shelf framework and tool upgrades? And once I've answered those questions, why do I think those are good goals? In other words, why would it be good for the process of fixing a bug to play out in some particular way? Thought of that way, it's a pretty big topic, and though I've been doing it for almost 30 years now it's challenging to tease apart good experiences I recall from the bad ones.

Please Either Shut Up Or Talk About Javascript

Here's the point I'm trying to make. My own goals for the shape of my application code — and not just where the files are in folders, and what the folders look like, and how they're arranged, but the underlying ways in which the code all fits together — are mostly summed up like this:

I want to minimize the number of dimensions of code expansion that result from a given increment in functionality.

In other words, I want it to be the case that when new functionality is called for, a developer should have to type in as little new code as possible. OK well that's pretty obvious; that's what everybody wants. The tough part, however, is being sensitive about what sort of new code we consider to be "matter of course" new code — code that we think of as having to be written due to laws of nature. That's tough because it's easy to grow numb to some things that really aren't laws of nature, but consequences of application implementation choices. For example, in my current application, when I need a new database table or changes to an existing one, I type in a little SQL. To a lot of people nowadays, that seems horrible (or at least that's an impression I've formed). The framework should do that! Well, maybe; I'm aware of how much that part of my world deviates from some current fashion, but in that particular case I actually prefer it that way. In other words, I'm aware of the fact that I've chosen a scheme that requires me to type in some SQL manually, but I know what my reasons are and I'm OK with that.

In a former position, I worked on a really large web application: a huge pile of Java on the server side, almost a thousand separate .jsp files, and lots and lots of Javascript. The application implementation design in that situation was such that implementing new pages meant, more-or-less as a matter of course, writing Javascript code for all the widgets on each page. Though there was some re-use of some components, like confirmation dialogs for example, the Javascript required to integrate a confirmation dialog with a particular new page was pretty much unique to the page. With relatively little experience dealing with an application of that magnitude (and at the time the development effort was really ramping up, back in 2003 or so, there probably weren't a lot of people around anywhere who had much perspective on what a huge AJAX-heavy application would be like for the developers involved), all of that per-page plumbing seemed pretty natural to me. I got pretty good at sort-of automatically typing in the stuff I needed, knowing how to think ahead for what a particular sort of page was likely to need. Lots of the script code was directly in the pages because it was often reliant on bits of information that'd be dropped into the code by the server-side template engine (that is, via JSP JSTL/EL expressions).

How else could such code possibly be written? Well I didn't actually ponder that question very often, at least at first. Probably sometime around the hundredth new page, however, I started to get this nagging feeling that something was wrong. At that point, I started to see the value (now so glaringly apparent) in common modular Javascript components. And as I started writing those, I started to also see the need for some sort of mechanical way that the chains of component inter-dependencies could be dealt with automatically. If there were some way that the component dependencies could be declared, that'd factor out a big headache from page construction and maintenance. We never got there, but it sure seemed like one of those "ideals".

When I started work on my current application, I spent a lot of time looking around at various dependency management tools and modularization techniques. Many of them looked great, but I didn't really dive in because I was also immersed in the problems of getting an application concept started from scratch. I'd never really done that, so I had to spend a lot of time thinking both about the outward-facing design as well as the application implementation design; the first problem generally got the lion's share of my anxiety. I was also spending a large amount of time thinking about server-side application implementation, so the Javascript module problem didn't get a lot of attention.

All this time, another "ideal" banging around in my head was a desire to really get away from in-page Javascript, and to get much closer to a universally unobtrusive way of creating pages. My server-side investigations had led me to the Stripes framework which seemed like some magical heaven-sent gift written by people who shared most of my opinions about that problem space, and were a million times smarter than me too. One of the nice things Stripes provides on the page construction side of things is a simple but powerful layout templating system. Once I started using that, it became clear that if we were careful the pages we'd be writing for application features could be really succinct. The layout templates could handle just about all the boilerplate page code that, in my previous position, had definitely seemed like a law of nature.

Well stuffing all the boilerplate into layout templates brought an implication to the domain of managing the Javascript code that supported fancy page behaviors. If I were going to have an implementation design based on the idea that every page would somehow grab the components it needs, then dependency management or not there'd have to be some communication of those needs between pure page code and the layout templates; otherwise, how would the Javascript end up at the browser? For all their convenience, the mechanisms available for pages to "talk to" the Stripes layout template code are a little crude. On top of that, of course, the templates themselves are JSP code, and I really don't like writing code that has to "think" very much in JSP; it's hideous. (Is it a law of nature that such page-template communication should be hideous? Probably not ...)

This situation drove me down the path I'm on now. The more I tinkered with it, and the more that some early deadlines loomed in getting the application put together, the more it looked like the simplest thing to do would be to just stick a common block of script tags in the layout template, and have every single page just pull in essentially all the Javascript I had. I had concerns about performance of course, naturally imagining the worst about what that could lead to in the future. However, I had to get some pages working and make it possible for other people on the team to write pages too, so it was the simplest place to start.

The implementation of my unobtrusive behaviors proceeded, generally on an as-needed basis. A new page would need something, and I'd cobble together some way for the HTML to announce that need (a new class name, or grouping under a particular element id, or whatever) and write the Javascript to deal with it. The code evolved along the lines of little stanzas like this: $where.find('.dialogLink').each(function() { var $link = $(this); // ... }); Each page feature, in other words, had a chunk of code that would look for the tell-tale mark in the HTML indicating the need for some activity. The "$where" variable was the DOM root under which I needed the work done, and was generally either the page body as a whole, or else a piece of the page into which new content had just been dropped. I was lumping all this code into a single routine that would be applied upon page load, and also when page fragments were loaded dynamically. As you can imagine, this function became ridiculously big after only a short while. I knew it was awful, but it did work after all and I had other things to worry about.

To address the problem of the insane size of my "do all the behaviors" script, I eventually broke up the code into lots of separate files, basically organized around the page features they supported. That helped a lot with keeping the code base manageable and maintainable; fixing a behavior meant a change only to a small file, not touching any other Javascript in the repository. However, I couldn't possibly stand the idea of extending my layout template to include thirty or forty separate script tags, even if they were small.

To solve the script tag problem, I leveraged the fact that as a Java application I had a build system anyway. I added an Ant task to the build that leveraged another tool we were also incidentally using, the Freemarker template engine. The Ant task processes a small Freemarker template file that is a Javascript skeleton. Freemarker directives in the template call out to routines in the Ant task that find those little script files in the source tree and supply them for inclusion in the Freemarker output. The result is a horrible-looking yet syntactically correct monolithic Javascript file that gets cranked through YUI compressor (giving the unanticipated but welcome benefit of a Javascript syntax check at build time) and dropped into the final .war file. I even concatenate the jQuery and jQueryUI libraries into my script, so ultimately what resulted was something that I would have considered completely insane just a few months previous: every single one of the pages of this application import exactly one script, that being the sum total of all the Javascript I've got.

That's obviously a disaster, right? One huge (about 250K at present) script applied to every single page? All those jQuery selector searches eating up time when 90% of them aren't going to actually do anything? Well, yes, early on we did have some performance problems. Part of that was foolish ignorance, mostly around my failure to use smart event handling (bubble-based handlers with jQuery "live" and "delegate"). Part of it was badly-written selectors, like that ".dialogLink" above. So we did spend time worrying about that and tuning the script, and I experienced a lot of anxiety at the prospect of having to break it all up. Eventually, however, we crossed a line due to good ideas from my co-workers and good ideas from blogs and forums, and I realized that my gigantic monolith of code really wasn't posing a performance problem after all, at least not simply due to its bulk. I've now got good selectors for everything, and dealing with all that Javascript takes only a very short time as my pages are painted by the browser. One of the benefits of having just one file, of course, is that it's the only script a client browser has to cache.

The mechanism supporting all my unobtrusive page features was still a mess, however, and that's what led me to type in the wallflower code. I wanted to see what a more organized approach might look like. If and when I actually try and convert my real code to use wallflower, I'll probably stick with the idea of the build-time "assembly" step, so that my individual wallflower features will live on as separate source files. But it's been a really long time since I've spent any time at all thinking about a component dependency manager. Why should I? I've "solved" the dependency problem (without explicitly planning to do so, and in fact without really being aware that I was doing it) by having exactly one common dependency for every page on the site. That's just hard-coded into the template, and I'm done. New page with new needs? Sure, I'll just add a new little behavior script file, and it works. Given how fast things currently run (and while it's truly breathtaking how fast modern Javascript interpreters are, this stuff works fine even in IE7), I doubt I'd see problems if I had 3 times as many behavioral features to support. I doubt I'll get there, of course, mostly because a site with that many whiz-bang page features would be terribly annoying (I'm looking at you, mint.com).

Another unanticipated benefit — at least, I think it's a benefit — of all this is that it's led me to having a much more disciplined approach to adding fancy behaviors to the site. In my previous job, because the way code was written was almost certain to involve so much specific new code to be created for everything, having unique behaviors on individual pages for no really good reason was pretty common. There was nothing in the process standing in the way of that, after all. Now, when thinking of new application features, I tend to think in terms of things I can already make pages do. Yes, there are two or three "special" parts of my app that do things no other pages do, and I confess to a few pages with on-page Javascript (I'm working on it, really), but keeping things under control I think really helps with maintaining consistency, and that's good not just for the development team but also for the end user.

So that brings me to this morning. I've been typing in this ridiculously long ramble for a while, but not until I saw a couple of posts about "enterprise" components did a unifying theme really click in my head. I'm delighted that people are finding success with smart dependency management schemes, and I'm sure I've got a lot to learn about large-scale web applications from other people approaching the problem in more well-informed ways than I did. (That'd probably be just about anybody doing it.) However, there's a lot I like about the world I've made for myself. What are the "laws of nature" that I'm assuming? Where are all my unnoticed wastes of time? Would one giant script file really work out in an application of the scale that my previous position involved? I'm not really sure at the moment. I also don't know whether what I'm doing is something that lots of people do, that's been written about elsewhere, possibly extensively, and possibly with well-reasoned heavy criticism. If you know, and you're not too exhausted after reading all this stuff, leave me a comment or two.

Wallflower - Unobtrusive jQuery

Unobtrusive Javascript — the page implementation technique of keeping as much raw Javascript code as possible off the pages, and applying it by implication from external script files — is a generally-accepted best practice and has been extensively written about in every web design blog that exists. I therefore won't expound upon the topic itself other than to say in summary that one generally marks page elements with classes or possibly attributes (especially in HTML5 pages), and then writes Javascript code that "knows" what classes and attributes to look for and what those annotations mean and thus knows what the elements want done to them.

That's all easy enough, but once you get rolling on a site of even modest extent, the variety of page markers starts to grow, as does the code that understands them. When interesting, more involved page behaviors arise out of design progress or updates, affected elements must communicate more details about the behavior to the code. As soon as there are two such interesting behaviors, the need arises to organize and "standardize" the page-to-code communication scheme. (Ideally, it's preferrable not to have to maintain separate unobtrusive Javascript files, instead keeping a central "do smart things to the page" script that can handle any of the "behavioral vocabulary" on the site. It's not mandatory of course, but growing a forest of little page-specific script files reaps its own problems.)

As the number of features grows, so too grows the code. Feature by feature, the code for each behavior is likely to differ more than the markup annotation, so there's even less organization. In my experience, I ended up with a surprisingly large function, hundreds of lines of innocuous little stanzas that, combined, were a monster. That's in an application that comprises less than 100 separate main pages.

Yet another challenge arises on sites that involve dynamic content — content fetched after initial page load time to populate tables, dialogs, status views, or whatever. Quite often, those fragments of content will also require unobtrusive intervention by the Javascript monster. Now that mess has to be wrangled into some sort of re-invocable tool.

To this situation, I now ask that you contemplate the introduction of another developer working on pages. And another one. They've been hired or redirected because some new functionality has to be built out in a hurry. That's not making you feel comfortable, right? What's the source base going to look like when the smoke clears?

On the application site that currently occupies my time, I've cobbled together a mechanism to handle this problem, and it's mostly under control. However, it's got a wild organic nature to it that defies explanation, and I really wouldn't want to create something like that again.

Wallflower

Because I couldn't find anything out there that claimed to deal with this problem, or even anybody else's description of the problem, I decided to write a simple jQuery plugin to act as a focal point. The plugin, called "wallflower" (wallflowers are unobtrusive), doesn't actually do any of the unobtrusive Javascript work for you. What it does provide, however, is a framework for organizing it all.

Wallflower use is based on the identification of "features" — the elements of what I called your site's "behavioral vocabulary". This aspect of the plugin is good for you and good for your users. When every page of your site is a unique little snowflake with its own behavior and its own quirks, you've set yourself up for a maintenance nightmare (again something I know from experience), and you're making your users have to learn the unique ways of each page. With organized unobtrusive Javascript, you'll naturally gravitate towards consistency across your site, because you'll grow a toolkit for page personality that will naturally become familiar to your users too.

A wallflower feature primarily consists of a "handler" function. The handler function is what "knows" what a page element wants done when the element has been made to wave the special little flag corresponding to the feature. The feature must be introduced to wallflower with a registration call before the plugin can know to apply it to page elements. The introduction consists of a simple call to the plugin to tell it the feature's name, how to identify elements that want to be affected, default parameters, and how to find per-element parameters. Most of that can be defaulted, even the handler function itself — wallflower will assume that the handler function is an existing jQuery instance method ($.fn.whatever) if the handler isn't supplied. That's particularly useful with things like the jQuery UI widgets. For example, the following initialization call: $.wallflower('datepicker'); tells wallflower that it should look for elements marked with the class "datepicker", look for parameters embedded in the class and in the default wallflower "data-wf" attribute, and then for each such element invoke the jQuery UI "datepicker" function with those parameters. Features don't have to be jQuery plugins of course; the handlers can be functions written just for wallflower. Another simple example: $.wallflower('timestamp', function() { $(this).text('Loaded at ' + new Date()); }); would pair up with page markup like this: <div id='timestamp-footer' class='timestamp'> </div> to put a timestamp onto the page when wallflower is applied.

Invoking wallflower would generally be done at page "load" or "ready" time. Wallflower, when invoked on a jQuery object, applies the features it's been told about to the selected elements. In other words, $(function() { $('body').wallflower(); }); would treat the entire page. For dynamically-loaded content, one would apply the plugin only to the container into which content had been loaded: $('#dialog-container').load(url, function() { $(this).wallflower(); }); It's possible to selectively deactivate particular features, which might be handy when initializing dynamic content and you'd like to skip the expense of initializing page-global content like the nav or page sidebars: $('#dynamic-table').load(url, function() { $(this).wallflower({'sitenav': false}); }); That would apply every feature except the one named "sitenav".

Details

There are two wallflower APIs: a "global" function on the jQuery object and an ordinary jQuery instance method. The global function is for registration of handlers. The instance method is for applying behavior to the page.

There's a general form for plugin registration: $.wallflower({ 'feature-name': { 'handler': function(params) { ... }, 'selector': 'selector', // defaults to '.feature-name' 'attribute': 'attr name', // defaults to 'data-wf' 'evalAttr': true, // boolean 'depends', ['name','name', ...], // defaults to [] 'defaults': { 'param': value, ... } // defaults to {} }, 'feature-name': { ... }, ... }); With that form, you can register as many features as you like. Specifics of the initialization block are:

handler

function to be called when wallflower is initializing a portion of the DOM. The function is called in the context of one element wrapped in a jQuery instance. The function is passed a parameter object built from the default parameters supplied at registration time, overlaid by parameters found on the element, and then finally overlaid again by parameters supplied to the invocation of wallflower at application time.

selector

the jQuery selector string to be used when searching for page elements to be affected by the feature. By default, the selector is formed by treating the feature name as a class name.

attribute

the name of the element attribute that should be inspected for feature parameters. Wallflower uses the jQuery "metadata" plugin to extract parameters from element class values. By default, it also looks in the "data-wf" attribute, if present, and also with the metadata plugin. That plugin expects what amounts to JSON notation in the class (surrounded by curly braces) or in the attribute. When providing element parameters in that form, the attribute values should look like a JSON object with parameter names and values. In the (probably rare) event of an element requiring the application of two or more features, the parameter JSON can include a preliminary layer of names and values, where the names are feature names and the values are themselves the parameter name/value objects. (See also the "evalAttr" configuration option.)

evalAttr

boolean flag indicating whether the configuration attribute value should be interpreted as a JSON name/value expression, or as a simple value. For some features, using a different attribute as a source for parameter information because the attribute naturally conveys to the feature handling what the element wants to do. An example might be a feature that used the value of an "input" tag's "maxlength" attribute to guide behavior. In that case, the markup should involve a simple number for "maxlength". Setting "evalAttr" to "false" for that feature would result in a parameter being included in the block passed to the handler named "maxlength" and with the value given in the element HTML.

depends

a list of feature names that, upon feature application, be applied before this feature. This mechanism allows for the (probably rare) case of one form of behavior application being required before another makes sense. An example might be the establishment of some sorts of event handlers that a subsequent feature will trigger. Features that claims no dependencies are applied in unspecified order.

defaults

parameters for the feature that should be used in the absence of parameters encoded in the HTML with affected elements, or passed in to wallflower at feature application time.

The following shortcut registration forms are recognized: $.wallflower('feature-name'); // same as: // $.wallflower({ // 'feature-name': { // 'handler': $.fn['feature-name'], // 'selector': '.feature-name' // } // }); $.wallflower('feature-name', function() { ... }); // same as: // $.wallflower({ // 'feature-name': { // 'handler': function() { ... }, // 'selector': '.feature-name' // } // }); $.wallflower('feature-name', { param: value, ... }); // same as: // $.wallflower({ // 'feature-name': { // 'handler': $.fn['feature-name'], // 'selector': '.feature-name', // 'defaults': { param: value, ... } // } // }); $.wallflower('feature-name', { param: value, ... }, function() { ... }); // same as: // $.wallflower({ // 'feature-name': // 'handler': function() { ... }, // 'selector': '.feature-name', // 'defaults': { param: value, ... } // } // ");

Invocation

When calling wallflower to make stuff happen to the page, you use the jQuery instance form: $(whatever).wallflower(parameters); The parameters passed to wallflower, if any, should be an object of a form similar to the generalized registration object: $(whatever).wallflower({ datepicker: { minDate: 1 }, dialog: { modal: true } }); In other words, the parameter object's keys represent feature names, and the values are used to override values from elements and from the defaults from registration.

In addition, features can be selectively disabled by including them by name in the parameter object but setting the value to the constant "false": $(whatever).wallflower({dialog: false}); If you need to work from the other direction, preferring to disable all features and then selectively enable a few, you can do that too: $(whatever).wallflower({ '*': false, // disable everything: 'datepicker': true }); That will ensure that only "datepicker" elements are affected. It would also work to give the "datepicker" entry a block of override parameters.

Markup

Element markup should be easy, but part of the responsibility for that is yours. Whenever possible, features should be registered with selectors that don't require the page code to do anything at all in order to be located and affected by features. That all depends of course on the nature of the feature: there's probably nothing at all intrinsic about an "input" tag that needs to have the date picker hooked onto it.

Parameters can be supplied in two ways (well, really three but the third one's a secret). First, they can be embedded in the "class": <input class='datepicker {minDate: 1}' type='text' name='the-date' > Everything between curly braces in the class will be handed over to every feature. In the unlikely event that an element requires intervention from more than one feature, you can do this: <input class='datepicker zap {datepicker: {minDate: 1}, zap: {zap: "full"}}' type='text' name='the-date' > That will allow each feature to get its own set of parameters.

If you don't like messing up your "class" strings, the next option is to use attributes. HTML5 promises that attributes whose names start with "data" will validate. Thus, by default, wallflower looks for parameters in "data-wf": <input class='datepicker' data-wf='minDate: 1' type='text' name='the-date' > For attributes, the parameter block need not be surrounded with curly braces. The same convention of labeling sub-objects by feature name applies to attribute values, in the case of collision between features.

Internals

There's not really a lot of code involved with wallflower. The registration process is pretty simple, and most of the code is involved with the piddly details like handling the shortcut registration invocations and dealing with the "depends" system. The registry of handlers is just an array sorted by the "ranking" computed from the partial ordering imposed by the dependencies. (In lots of cases, there'll be no dependencies anyway.)

At the feature application end, the plugin simply works through the list of registered plugins using the feature selectors to find elements to be operated upon. The code treats each matched element separately because each one has to be examined for feature parameters. Those are pulled from the element and then passed to the feature handler, laid over the original defaults from registration and overridden by any parameters for the feature found in the call to the plugin.

Performance-wise, things could get out of hand if all your features did nothing but rely on class-based element selection, and if your pages are big. Thus, initializing features with efficiently restrictive selectors is important when there are lots of features to contend with. For example, instead of allowing the "datepicker" feature to just look around for anything with "datepicker" in the class string, it'd be better to set it up with a more efficient selector: $.wallflower('datepicker', { selector: 'input.datepicker' }); Now jQuery will only have to look at "input" elements. If some features only apply to portions of pages contained within an element that's got a predictable "id" value, like for example a "help" section: $.wallflower('help-toggle', {selector: '#help-section li.help-toggle'}); could really cut down on DOM work by the jQuery selection engine.

My experience (with a system that does basically what wallflower does but without a nice organizer like wallflower) is that so long as the selectors are efficient, applying even a lot of features to whole pages does not result in a significant page load penalty. Obviously it costs something, but on my current site with around thirty or forty separate features, my application pages are snappy and there's no apparent lag. It does, however, depend on selectors: my site has been slow when I've added a feature with a sloppy, careless selector.

Getting It

The github repository for wallflower is here. I barely understand github and git so I really can't provide help or commentary or anything; as far as I can tell the files are there for the plugin itself and a modest demo page. This is an initial release, and it will probably poison your pets if you actually use it.

The plugin relies (for now) on the "metadata" plugin from John Resig, Yehuda Katz, Jörn Zaefferer, and Paul McLanahan. The plugin (version 2.1) is built into the current source and is installed if wallflower doesn't see "$.metadata". If you're using an older version, wallflower should still work fine; it doesn't (yet) use the "html5" metadata stuff.

IE7 is why I drink so much

Cool bug in IE7; I think I ran across it years ago in IE6, but back then I thought everything any browser did was essentially magic so I didn't worry too much. Make a table. Put rows in your table, rows filled with glorious content. Give some columns — header and data cells alike — a class, like maybe "hideMe". Put a control somewhere on your page, preferably a checkbox. Wire that up to a Javascript handler that adds/removes a class from the enclosing table. Maybe your class can be called "hide". It is not important what these things are called. Please don't lose sight of the forest by becoming preoccupied with the color of the lichen on the trees. Now. Add some CSS such that a table cell (or header cell) with class "hideMe" inside a table with class "hide" is not displayed.

   table.hide th.hideMe, table.hide td.hideMe {
     display: none;
   }
  

OK now what have we got? We've got a table with some columns and a control that you can click to make some columns go away. It's like a "peek a boo" game. Sometimes dogs really freak out over stuff like this. Now, let's say you want to use something like this for real. Let's say you want a table for something actually useful, and you want a "condensed" mode and a "full" mode. Thus your "condensed" mode will involve the outer table tag having that "hide" class, while the "full" mode will not. Possibly you'd like your page to start off in condensed mode, because the "full" mode has columns that only dumb people want to look at. This brings us to IE7. Your page loads, and you set things up so that the table has the "hide" class, and sure enough the "hideMe" columns are not shown. Great! That's exactly what we wanted! However we have to put that checkbox on there so that the dumb people can toggle the display of "hideMe" columns. So we do. In browsers developed outside of Mordor, everything works exactly like you'd expect. Not in IE7. IE7, perhaps like its Balrog predecessor IE6, has something up it's sleeve. When the page first loads, the hidden columns are hidden. Click that checkbox. Do you see the columns? No? Well, that's because IE7 has decided that table sub-elements hidden when the table is first rendered will never ever be visible ever, even if Elrond and Gandalf throw Frodo into the volcano and jump in after him. Note that if you load your page configured for dumb people, with the "hideMe" columns visible, then everything works. Things are just problematic when parts of tables (and just tables) are hidden upon initial DOM rendering. (Yes, this applies to stuff dynamically added to the DOM later — the Dark Lord is pretty thorough.) I don't know of any good way around this. A bad way around it is to make sure that things are set up in "everything visible" mode upon initial DOM rendering, and then have something come back in a few milliseconds and make things look the way you actually want them to look. I suspect that IE7 users have so many bruises on their brains that the page wiggle thusly induced might actually seem comforting and familiar, like the smell of burned waffles at your grandparents' house.

Safari 5 Has Some Printing Problems

A bug report was logged against my application today concerning the new Safari 5 release. On the Mac it apparently displays some layout issues that I don't see on the Windows version, so I can't say much about those (though they seem pretty fishy; button elements styled with borders rendered such that the borders aren't completely drawn doesn't seem like a CSS issue to me ...). Also reported however was a problem with printing. Some of the application (the "help" pages) are printable, and have a "print" stylesheet that arranges for some differences with the screen version. Some elements are hidden, links are printed a different way, etc. The print version works fine in IE7 & 8, Chrome 4 and 5, Firefox 3.5 and 3.6 (and probably 3.0/3.1 too), and Safari 4. In Safari 5, however, it's completely broken — not a little wrong, but obviously messed up and totally unreadable. I messed around with the stylesheet randomly with no luck, and so I started on a much simpler test page (here). That page is also broken when printed from Safari 5, in a very similar way too. Hmm. I decided to start with the part of the stylesheet that was most relevant to the paged media context: the @page rules. The stylesheet had:

@page { margin: 1in 1.75in 1.5in 1.5in; } @page :left { margin: 2in 1.75in 1.5in 1.5in; }

Those do pretty much nothing at all in all the browsers I know of, but sure enough when I commented those two lines out, printing started working. Searching around the Apple documentation, I could find nothing to suggest that those rules should cause a freak out. Nor could I find any mention of a change in semantics from Safari 4 to Safari 5; in fact I could find no mention of @page on the Apple site at all, even though they do list some unsupported CSS stuff. (Note that @page is a CSS2 thing, though again none of the browsers seem to pay attention to it.) It finally dawned on me what was going on in Safari 5's little mind: those @page rules, formerly ignored completely, were being treated as if they were "*" rules. That is, the margin was being applied to everything on the page, and every block-level element. Well no wonder things looked funny. The moral of this story is to keep @page rules out of your print stylesheets, as most everybody probably does anyway.

Safari focus() and popup alerts

Generally, browsers allow Javascript code to force a particular text field (or various other things) to be in focus. For a text field, that means that the cursor will appear in the text field and blink (or do whatever it does), and keys typed will cause text to appear there. In Safari, a text field glows blue when it's in focus, which I think is a security measure, and a fairly irksome one for designers at that. (What if my page is green? Ugh.)

Well on top of that, while I'm bitching about Safari, there's another limitation with forcing focus. In general, Safari refuses to allow itself to grab focus overall; that is, to grab focus on a desktop application basis. That's done by calling the window.focus()method. IE is generally delighted to push itself in front of other desktop windows. Firefox will do it if the user's configured it to do so. The WebKit browsers, however, are uncooperative. That in and of itself doesn't really bother me; there are few cases where I really feel that's justified, and windows that raise themselves more often than not annoy me to the point of a killin' rage.

However, this reluctance to impose itself causes a weird effect with Safari with respect to Javascript alert() pop-ups. I think that because the window generated by an alert() steals "application focus" from the browser itself, then after that point attempts to force focus on a form field fail. This situation is not really a contrived one: consider a form submission event handler that wants to point out that a particular text field must be filled out. The handler launches a pop-up alert()to convey the message, and then when the dialog is dismissed it tries to set focus. Well, that works in other browsers, but not Safari. Safari seems convinced that its application window has lost desktop focus, and therefore it ignores attempts to give focus to form fields in the page.

This sample illustrates the problem. (It's not very interesting in browsers other than Safari, of course.) If you visit that page and click the "Click me" button, you'll get a pop-up. Dismiss it and then do nothing. Nothing will happen when the page posts a log message indicating that it has tried to force focus. Now, click the "Click me" button again, and again dismiss the pop-up. This time, right after the pop-up goes away, click anywhere in the page (not the input field of course). That click should bring Safari to the foreground, so to speak, and then after a couple seconds the attempt to force focus will work, and the input field will get that irritating fuzzy blue border.