Handling The Return of jQuery AJAX form POSTs

2010 February 3
by kevin

The discussion on stackoverflow about handling different dataType returns from AJAX calls really got me thinking about the best pattern for POSTing form data to the server.

This really ties in nicely to my previous blog post about Patterns of Organization when mixing MVC frameworks with javascript. If you are using your own RESTful API when creating a new object in your domain (a.k.a. POSTing some form to create a new foobar) what is the best pattern to follow?

Let’s first assume that you are POSTing your form via AJAX because you just popped up a lightbox that contained the form rather than doing a whole round trip to render a whole new page with the form. Therefore you have some code that looks like this:

    $("form#foobar").submit(function(){
        $.ajax({type: 'POST',
                   url: '/foobar',
                   data: $(this).serializeArray(),
             success: function(data){
                       // do something with data;
                   }
        });

Under the old MVC pattern you probably re-rendered the form if the user gave you bad data or forgot to fill out a field and then on success, you either re-directed them to the new page showing them their new data or immediately rendered that new page in the POST controller. However, in the javascript example above it becomes quite cumbersome to expect multiple possible returns. We do not want to be parsing the returned HTML to figure out if it is a re-rendered form because of a validation failure or if it is the HTML of our new item. The obvious solution is to return JSON which is easy to inspect. But I really like the idea of sending back an error HTTP response code if validation fails.

Firstly, if I am within my own application and retrieving data I do not want to write code to turn JSON into HTML. I know how I want my data to look on the page so why not just return it straight away into HTML. This allows me to keep my markup in templates where it should be. By returning validation failures as, well, failures, it allows me to instantly handle two different return types - JSON for an error code return, and HTML for a new item return. If JSON is returned I can easily pick out the error message and update the DOM of the form to display the erred fields. But if my POST was successful, I then kill the form and slide in my new HTML to update my existing list of data all nicely formatted.

Now my javascript looks something like this:

    $("form#foobar").submit(function(){
        $.ajax({type: 'POST',
                   url: '/foobar',
                   data: $(this).serializeArray(),
                   dataType: 'html',
             success: function(html){
                       $("div#data").append(html);
                   },
                   error: function(XMLHttpRequest, textStatus, errorThrown){
                       var errorObject = JSON.parse(XMLHttpRequest.responseText);
                       // update form with stuff in errorObject
                   }
        });

JQuery, AJAX, JSON, HTML and Code; Patterns of Organization

2010 February 3
by kevin

I’ve been thinking a lot lately about the emerging patterns in web development in the post Web 2.0 world. AJAX and javascript are mainstream now and no longer just parlor tricks. It is de rigueur to implement RESTful web services and have an API. But with all these AJAX calls whizzing about and JSON, XML, HTML, and whatnot being passed to and fro, it is easy to end up with a tangled mess. The one thing about mixing your web framework with javascript is that you now have a lot of code going on that leaks outside of your nice, neat Model-View-Controller (MVC) pattern.

I have seen javascript used extremely sparingly in an application all the way up to extremely heavy usage with javascript controlling not only logic but the entire layout of the page. But to keep a nice separation of responsibilities I believe the server side code should be responsible for laying out the structure of a page. We can then componentize the dynamic data bits and create a nice structure whereby javascript widgets are responsible for making AJAX calls to fill in those components.

For example, you have a page which is rendered by your MVC framework. But on that page is a div that contains a list of dynamic data. When the page is rendered there is just a placeholder for the list. Using JQuery event handlers you then make an AJAX call to fetch the data from your own RESTful API and populate the div.

What I like about this pattern is:

  • you still have a strong MVC pattern
  • you are eating your own dog food in terms of your API
  • initial page loads are quick
  • you can constantly poll for new data to give your application a real-time feel

An important point to note in this pattern is that you can still have your MVC framework responsible for rendering the data into HTML. Your RESTful API should accept a format parameter to set whether the return type is JSON, XML, or HTML. Internally, your own application can use the HTML return data type so that you do not have to write a bunch of javascript to turn the JSON into HTML. This way you can keep all your markup in templates and not scattered in javascript code.

Mobile Telecommunication Companies Fail At Math

2009 August 27
by kevin

Let’s start with a very fair premise - within the modern context a cellular telephone network is just a network that pushes data around. That data can be voice data, video data, text data, whatever data. In terms of data, voice data is actually one of the most challenging because in order to have real-time voice communications you have to have a certain level of quality of service. The same could be said about video if it was in the context of video calling. Therefore, you would expect to see some sort of premium on voice and video calling services over other data services. As for other data like text messaging, email, web traffic, etc. this is easy data to push around, especially plain text data like email and text messaging.

So, let’s take a look at how Rogers Communications in Canada charges for their Fido mobile services relative to each other. For those of you not from Canada I should forewarn you that Canada is well known for its lack of mobile competition and ridiculous fees.

Fido charges 3¢/KB for raw data. With the launch of the iPhone they reluctantly offered a special plan giving 6GB of data for $30/mo.

If you used 6GB of data outside of that plan you would receive a charge of a wopping $188,743.68 ! (6GB is 6,291,456KB x .03 = 188,743.68)

WOAH! Does that mean that Rogers is giving away $188 thousand dollars in service for only $30 ?!?  What a deal!!  If that is true their bancrupcy should be immenent.  Or is it that their 3¢/KB charge outside of a plan is grossly over-inflated?

So let’s take a look at their voice communication charges, which by our earlier premise should be more expensive.

Outside of a plan or on overage minutes, Fido charges 35¢/min.  A quick google tells us that 1 minute of voice roughly takes up 367 KB of data.  Based on Fido’s data rates a voice call should cost $10.95/min. (367 x .03 = 10.95)  Therefore, a typical monthly plan that provides 250 minutes of talk time should be costing $2,737.50/mo.  Really?!?  Something is definitely amiss in their pricing structure.

Finally, let’s look at the ubiquitous text messaging.  An SMS outside of a plan or on overage costs 15¢ per message (yes, really, for those reading this from civilized countries like Cambodia.)  An SMS is 140 bytes of data, therefore based on the grossly over-inflated data rate of 3¢/KB, an SMS should cost 0.4¢ (140 bytes is 0.1367188KB x 0.3 = 0.004) Wait a second!  SMS should only cost 0.4¢ per message even at the grossly over-inflated data rates?!?

As you can see, there is no rhyme or reason to the pricing structure of SMS or data.  The way free-market capitalism is suppose to work is that, in the presence of competition, the cost of goods and services should be driven towards their actual costs.  Obviously this is not happening because either Rogers should be going bankrupt for giving away $188,000 of service for $30, or they should be driven out of business for such ridiculous pricing structures by another company that more transparently and fairly charges for the same services.  But no one ever said we have free-market capitalism in Canada.  That is why we have the CRTC, to ensure fair pricing amongst our government sanctioned monopolies of Bell, Telus, and Rogers!  So CRTC, what say you?

Here is a nice breakdown of what each service would cost if charged based on the pricing structure of each of the other services:

Services Voice SMS Data
Voice $0.35/min $0.04/msg $0.98/MB
SMS $402.70/min $0.15/msg $1,123.60/MB
Data $10.95/min $0.0004/msg $3,072/MB

Notes:

  • 1 minute of voice requires the transfer of 367 KB of data
  • 1 SMS message is 140 bytes of data
  • 1 MB is 1024 KB

UPDATE: I have been involved with a company called Roam Mobility over the last couple of months and they are addressing some of the egregious roaming charges from the Canadian telcos. You can check out their roaming rates at www.roammobility.com

Pylons AuthKit forward and RecursiveMiddleware’s ForwardRequestException

2009 July 26
by kevin

If you are running Pylons 0.9.7 with the latest “stable” AuthKit 0.4.4 you may run into some troubles using the forward AuthKit method. If you are like me, you followed all the smatterings of examples scattered about the Pylons and AuthKit site only to keep receiving a ForwardRequestException anytime you hit an authorize protected controller.
read more…

Python Local Variable Scope Bug Within An Inner Function Definition

2009 June 25
by kevin

In Python, you can declare a function within another function and the scope of local variables in the outer function carry through to the inner function. However, you have to be careful with the re-assignment of the variable inside of the inner function due to some funkiness that may or may not be a bug (I say its a bug but who am I?)

read more…

How to Make JQuery UI Tabs Linkable and Bookmarkable

2009 April 30
by kevin

With JQuery UI Tabs v.3 there isn’t an option to make the tabs bookmarkable. In other words when you click on the tabs the URL does not change therefore you can’t link or bookmark anything other than the default tab.

Here is how I solved the problem… [UPDATED!]

read more…

MySQL data files .MYD, .MYI, and .frm

2009 March 20
by kevin

A MySQL database creates a number of different data files in the mysql data directory (typically /var/lib/mysql). The following are the differences between the data files created:

File Extension Purpose
.frm table definition
.MYD table data
.MYI table indices

I believe these files are created for both MyISAM and InnoDB table types because the MySQL documentation references these types with respect to MyISAM and I definitely see them on my InnoDB tables.

I couldn’t find this information directly in the MySQL documentation and google wasn’t much help either so hopefully this post will save someone else some search effort.

Doing UNIX Shell For Loops The Right Way

2009 March 16
tags: ,
by kevin

Typically for loops in UNIX shell look like this:

for file in `ls *.c`
do
    cmd $file
done

However, the above has a serious problem in that it does not handle spaces in file names and will actually split the filename in two.

The following is a solution in bash:

files=(*.c)
for f in “${files[@]}”
do
    cmd "$f"
done

But my favourite way to perform a loop and probably the most elegant solution is to use file globbing:

for f in *.c
do
    cmd "$f"
done

How To Submit A Sitemap file to Yahoo

2009 March 13
tags: ,
by kevin

Yahoo appears to want to make it extremely difficult to figure out how to submit a sitemap.xml file to Yahoo Site Explorer.  You  would think it would work similarly to Google where there is a page that allows you to upload a sitemap.xml file either directly or providing the URL.  Indeed Yahoo supports the sitemap format as declared by this article on the Yahoo Search Blog but nowhere on Site Explorer can you find any mention of the sitemap format. read more…

Deleting Massive Amounts of Comments in Wordpress

2009 February 21
by kevin

If you are like me and neglected your wordpress blog for a ridiculous amount of time you probably have a ridiculous amount of comments awaiting moderation. Unfortunately, wordpress only displays 20 comments per page so processing 10,000+ spam comments will take forever! And clicking on “Check For Spam” doesn’t seem to be able to handle massive amounts of pending comments. Therefore, after spending a couple of hours deleting comments 20 at a time I decided to find a better way. read more…