Archive for the ‘Javascript’ category

WebGUI Carousel Wobject

April 6th, 2009

Carousel Wobject

There’s a new wobject coming in the latest WebGUI beta (version 7.7.3), called Carousel. On the surface, it’s a nice little integration between WebGUI and YUI, but below the surface it’s a great multi-purpose tool for rendering content dynamically in Javascript or Flash.

It’s a common requirement to want to display rich text via Javascript or Flash. For example you might have a clickable diagram that shows different text depending on what part of the diagram a user clicks on.

Giving content managers access to the dynamic text can be rather cumbersome. In the worst case scenario the text is hard-coded in the Flash or Javascript source, out of reach of the content manager. Better solutions are where you have a bunch of editable Articles or Snippets that the Flash or Javascript reads from.

None of these solutions are very satisfactory though, because as far as the content manager is concerned the dynamic object is a single asset, and they really want a simple way to edit all of the text elements from the same place.

We designed the new Wobject specifically for this scenario. The Edit Screen of the wobject allows content managers to add as many rich text editor (RTE) boxes as they like (dynamically, a la FilePile). These RTEs are displayed one after the other on a single screen, allowing the content manager to edit all of the text in a single place. For each RTE, the content manager can also specify a unique ID.

carousel-edit

By default, the template for the Carousel wobject renders your RTE items into an attractive YUI Carousel widget:

carousel-view

The Wobject template does this by looping over all of the RTEs and generating markup expected by the YUI Carousel widget. This has the nice side-effect of allowing your widget to gracefully degrade in non-js browsers.

Taking a step back, we have a WebGUI wobject that lets us manipulate a collection of RTEs. The only part of the wobject specific to the YUI Carousel is the template. Let’s have a look at it:

<div class="yui-skin-sam">

...

<tmpl_if item_loop>
<div id="<tmpl_var assetId>">
<ol>
 <tmpl_loop item_loop>
	<li class="item" id="<tmpl_var itemId>">
 <tmpl_var text></li>
</tmpl_loop></ol>
</div>
<script>
 YAHOO.util.Event.onDOMReady(function (ev) {
 var carousel = new YAHOO.widget.Carousel("<tmpl_var assetId>", {
 isCircular: true,
 numVisible: 1,
 animation: { speed: 0.5 }
 });
 carousel.render(); // get ready for rendering the widget
 carousel.show();   // display the widget
 });
 </script>

</tmpl_if></div>
</script>

We can just as easily create our own template to do something else with the data. Returning to our original example, say you want to use the RTE text in a Flash diagram. All you have to do is change the template to output the contents of each RTE into a textarea dom element, styled to be hidden from view, with the id attribute set to the RTE unique id. Then, your Flash (or Javascript for that matter) can easily read in the rich text via DOM methods (e.g. document.getElementById(RTE_unique_id).value) and display it dynamically and/or do whatever it likes with it.

For example, we might use a template such as:


<tmpl_loop item_loop>
 <textarea id="<tmpl_var itemId>" style="visibility: hidden;"><tmpl_var text></textarea>
 </tmpl_loop>

The only coordination required is that the person deploying the wobject would need to know how many text blocks the flash/javascript expects, and what unique ids to use.  For some flash/javascript widgets you could ignore unique ids altogether by adding a special class attribute to the generated textareas and then using a class selector to dynamically select all matching textareas on the page.

This should make it easier to use dynamic flash/javascript widgets in WebGUI whilst still retaining the power and ease of use of a content management system.

The curse of console.log

April 3rd, 2009

Here’s a common scenario, you’re on-site demonstrating some fancy new Javascript interface you’ve written, only to discover that some part of it that was working so beautifully on your dev box inexplicably breaks. hmm you think, must be a bloody IE bug. You try it in Firefox.. it still doesn’t work, what the?!

Later that day you get home and on a hunch run a quick search through your source tree:

> ack  ‘console.log’
www/extras/wobject/Survey/dd.js
96:console.log(destEl);

The culprit is Firebug’s console.log() function. The damn thing is just too handy. So handy that a few years ago the collective web development community abandoned alert() as the #1 debugging weapon of choice. Now we all use console.log(). Forget the firebug debugger, nobody can be bothered bringing up their js source in the script tab and clicking the break button or setting a watch variable, that’s like, 4 clicks too many man! Whack in a of console.log(a, b, c)s and it’s all there, plain as day in the firebug console, ready to be clicked on and examined further  in the DOM tab.

Until you forget that not everyone in the world runs firebug. Hard to believe, I know. But actually, the problem occurs for anyone who hasn’t got the firebug console turned on for YOUR SITE. The problem being of course that you probably always have the firebug console turned on for your site, but no-one else in the world does. So the likelihood of being bitten are pretty high.

Of course, there are lots of workarounds. One approach is to just define console.log as an empty function (and while you’re at it, you may as well define all the other Firebug functions too):


if (!("console" in window) || !("firebug" in console)) {
 var names = ["log", "debug", "info", "warn", "error", "assert", "dir", "dirxml", "group", "groupEnd", "time", "timeEnd", "count", "trace", "profile", "profileEnd"];
 window.console = {};
 for (var i = 0, len = names.length; i < len; ++i) {
 window.console[names[i]] = function(){};
 }
}

In my case the code was part of a drag and drop method used in WebGUI. And like all sane web projects, WebGUI oursources its cross-browser Javascript concerns to a library (in our case YUI). And it turns out YUI has a perfectly good logger of its own, guaranteed to be available on any page that YUI is loaded onto. Hello YAHOO.log();

Now, “YAHOO.log” is 1 letter shorter than “console.log”, so why does everyone end up using console.log() all the same? Maybe they don’t like typing in upper-case. More probably they noticed that when they use YAHOO.log(‘blah’); nothing appears in the firebug console. What gives? Well, you need to tell YAHOO to pass on log  messages to the console, that’s what. Type the following line first, and everything will work out just fine:


YAHOO.widget.Logger.enableBrowserConsole();

Of course, that’s a lot more letters than just typing console.log(), and what’s more you need to use YAHOO.lang.dump() to serialise objects passed to log() (no nice firebug object examination), which admittedly is a lot less fun. But it does mean a little less broken code in production.

UTF8, JSON, Perl, and MySQL (guess where this is heading..)

January 27th, 2009

Today I learned a little bit about UTF8 in the context of JSON, Perl and MySQL.

Recently one of my clients who was using the Survey wobject in WebGUI noticed their modperl server grinding to a halt. Funny how one little decision can trigger such a catastrophic failure. Funny in the sense that it takes you a few days to repair the damage and track down the root cause of the problem.

The reason modperl was grinding to a halt was easy enough to find – child processes chewing up as much memory as they could whenever a web request touched a Survey instance. The cause of that was also quickly apparent.. one of the text fields in a Survey instance was filled with ~150Mb of rubbish data. The smoking gun looked like this:

In contrast to obsessions, your worries don\x{c3}\x{83}\x{c2}\x{83}\x{c3}\x{82}\x{c2}…

An apostrophe in the text and then BAM! everything goes haywire (that \x{blah} unicode text goes on for 150Mb..)

Turns out the user who triggered the problem was cutting and pasting text from Microsoft Word. As anyone who has worked with people who cut and paste text from Word into content management systems knows, Word uses non-ASCII characters for “smart quotes”, apostrophes, dashes, triple periods, etc.. Those non-ASCII characters usually show up in HTML as garbage characters. In this case it looked like those characters were causing the Survey internals to break.

On closer inspection Survey was breaking as follows: any non-ASCII text was converted to a few garbage characters. On the next view/edit, each of these garbage characters were turned into a few more garbage characters, and so on.. which meant that in no time at all the web server had ground to a halt and the Survey table was weighing in at hundreds of Mb. Fun!

Now, I knew that JSON was supposed to be UTF8 encoded, and Survey serialises its data to JSON, so somewhere along the way the JSON string/Perl String/MySQL string encoding was breaking.

Best practices in WebGUI dictate that you work with JSON as non-encoded, non-binary, Perl UTF8 strings and then let wg modules such as WebGUI::Session::Form do the encoding for you (via the Encode module). Most of the time you don’t need to care if a string in Perl is UTF8 or not, and doing things this way means that most of the time you can remain blissfully unaware of what’s happening under the hood.

In accordance with this, Survey uses JSON::to_json/from_json everywhere. On a hunch I switched all the from_json calls to decode_json, which expects a binary, UTF8-encoded string instead of a non-binary string, and voila, the problem disappeared. So somewhere along the way our JSON-encoded string was being converted to a binary string. The question was: where?

The plot thickened when I noticed that calling Encode::decode_utf8() immediately after reading the string from mysql also removed the issue (decode_utf8 turns a binary UTF8 string into a non-binary utf8 string).

With the help of perlmonkey2 and perlDreamer on #webgui, we found the root cause.. the json-encoded string was being stored in a mysql field of type longblob as opposed to a non-binary type (such as text, mediumtext, etc..). Something was transparently converting it to a binary UTF-encoded string (perhaps DBD::mysql?), and when  Survey then pulled this binary string out the database, it was trying to use from_json() to decode it (which expects a non-binary string).

A quick change to the field type:

alter table Survey modify surveyJSON long

and all was well again with the universe, but if you’d asked me 4 hours ago what mysql field type should be used for json my first thought would probably have been a binary type, because all I knew was the JSON is UTF8 encoded. A little bit of knowledge can be a dangerous thing..

The halting problem in Javascript

September 10th, 2008

I’ve been getting a lot out of The Little Schemer recently. It’s an amazingly little book. One topic covered is the Halting Problem which I first came across back when I was studying Theory of Computation at university. The authors use Scheme throughout the book, but I thought it’d be useful to translate the discussion into Javascript since Javascript is quite accessible for newcomers to functional programming.

Some functions always return a value. For example, the following simple function always returns the value of PI:

function pi(){
    return Math.PI;
}

Some functions go into infinite loops and never halt. For example, the following function enters a while loop and never comes out:

function eternity(){
    while(true) {
        // goodbye, cruel world
    }
}

Wouldn’t it be nice to be able to tell ahead of time whether a function is going to halt or go into an infinite loop? (cue evil laughter from the computer scientists in the back row)

Let’s write a function that tells us whether another function halts. (It’s easy to write functions that take other functions as arguments in Javascript because it’s a functional programming language). What would such a function look like?

function halts(fn){ // takes one argument, fn (a function)
    // code that determines if fn halts goes here
}

We’re not sure yet what code to put into the function body, but let’s start by looking at what we know about this function. It has to return True or False. True if the argument fn halts and returns a value, False if not. Simple right?

Let’s try out halts() on some functions we’ve already talked about to get a feel for how it’s supposed to work.

halts(pi); // returns true because pi() halts
halts(eternity); // returns false because eternity() doesn't halt

So far so good. Ok, let’s create one last function just for fun and see how halts() goes.

function maybe() {
    if (halts(maybe)) {
        eternity(); // never ends..
    } else {
        return "ok I halted!";
    }
}

The question is, does maybe() halt? Well let’s see..

  • Let’s assume that maybe() halts. Therefore the if clause is true and so eternity() gets called.. meaning that maybe() never halts. Wait! That can’t be right! We just contradicted ourselves. We must have guessed wrong.
  • Ok, in that case, let’s say that maybe() doesn’t halt. Therefore the else clause gets called instead, and we return a value. Meaning that we halted. But that’s a contradiction too!

What this means is that it’s impossible to write a function like halts() that determines whether another arbitrary function halts or not, because the very definition of such a function leads to a contradiction. Sucks to be a programmer huh?

Thanks Alan M. Turing (1912-1954), and Kurt Gödel (1906-1978).