AJAX in CL-HTTP

AJAX is the name coined for the combined use of several Web technologies to create interactive Web pages that change their content without needing to fetch an entire new page.

A typical application is to update a list of matches on a Web page as the user types characters into a text box:

ajax.png

You can download the complete listing here:

ajax-test.lisp

Implementation

This is implimented as follows:

  • When the user types a search string into the text input box, an onkeyup event calls a JavaScript routine with the search string as a parameter. 
  • The JavaScript routine uses the JavaScript function XMLHttpRequest() to make a GET request to a search URL on the server, with the search text as a parameter.
  • The search URL responds with data which is used to modify the Web page with DOM commands.

The data returned to the Web page was traditionally in XML format, so that JavaScript routines at the browser could parse the data and act on it appropriately. However there's no reason why the data shouldn't be plain text, HTML, or even Lisp data.

In this example the data is returned as plain text, but with HTML formatting commands included.

The JavaScript scripts

The application first defines a JavaScript function called submitRequest() in the head section of the page. This has the following format:

function submitRequest(url, target)

where url is the URL of the page to be called, and target is the id of a div section in the document, where the returned text will be inserted.

The application then calls this function with appropriate parameters in the onkeyup parameter of the text input field:

onkeyup="submitForm('/data?' + this.value, 'zone');"

where /data? is the search URL we are going to define to return the data, this.value gives the contents of the text input field, and zone is the id of the div where we want to put the data.

Using JavaScript with CL-HTTP

CL-HTTP includes several procedures to help with incorporating JavaScript scripts, and scripts in other languages, into Web pages:

define-script allows you to specify a script, with the JavaScript to be emitted in the head of the page to define the script, and the JavaScript to be emitted in the body of the page to call the script. In each case you can specify parameters, to pass values from Lisp to the JavaScript being emitted.

intern-script is used to register the script.

declare-script emits the definition of the script to the stream.

with-event-handlers associates a script with a particular event, so it can be specified in the :events parameter of an HTML field.

Defining the submitRequest() script

In our application the submitRequest() script is defined by the define-script procedure. This has the format:

(define-script name (:JavaScript):script script :caller caller )

where name is a name used to refer to the script, script is the definition of the JavaScript routine, and caller is the JavaScript to call the JavaScript routine. Here's the full definition:

(define-script ajax (:Java-Script)
               :caller "\"submitRequest('/data?' + this.value, 'zone');\""
               :script "
function submitRequest(url, target)
{ 
        var req = null; 

        document.getElementById(target).innerHTML = 'Started...';

        if (window.XMLHttpRequest)
        {
                req = new XMLHttpRequest();
                if (req.overrideMimeType) 
                {
                        req.overrideMimeType('text/plain');
                }
        } 
        else if (window.ActiveXObject) 
        {
                try {
                        req = new ActiveXObject('Msxml2.XMLHTTP');
                } catch (e)
                {
                        try {
                                req = new ActiveXObject('Microsoft.XMLHTTP');
                        } catch (e) {}
                }
                }

        req.onreadystatechange = function()
        { 
                document.getElementById(target).innerHTML = 'Wait server...';
                if(req.readyState == 4)
                {
                        if(req.status == 200)
                        {
                                document.getElementById(target).innerHTML  = req.responseText;	
                        }	
                        else	
                        {
                                document.getElementById(target).innerHTML='Error: returned status code '
                                + req.status + ' ' + req.statusText;
                        }	
                } 
        }; 
        req.open('GET', url, true); 
        req.send(null); 
	}")

Note that I've been careful to use single quotes for strings in the JavaScript to avoid the need to escape double-quotes in the Lisp strings.

Defining the page

Here's the definition of the demo page. It uses intern-script to intern the script defined by the define-script procedure, declare-script to write out the definition of the script, and with-event-handlers to associate the call to the script with a :key-up (onkeyup) event:

(defun show-ajax (url stream)
  (declare (ignore url))
  (let ((ajax-script (intern-script :ajax :java-script))
        (title "AJAX Demo"))
    (with-successful-response (stream :html)
      (with-html-document (:stream stream)
        (with-document-preamble (:stream stream)
          (declare-title title :stream stream)
          (declare-script ajax-script stream))
        (with-event-handlers
            (ajax-event
             (:java-script :key-up (event-caller ajax-script)))
          (with-document-body (:stream stream)
            (with-section-heading (title :stream stream)
              (accept-input 'string "text" :events ajax-event :label "Text:" :default nil :stream stream)
              (with-division (:id "zone" :stream stream)))))))))

Note that the accept-input procedure is not inside a form, as we are using JavaScript to request the result, rather than posting a form. We will intern this URL with:

(export-url "http://localhost:8000/ajax.html"
            :computed
            :response-function 'show-ajax)

Defining the response function

Finally, we define the function to return the result to the JavaScript call. It uses some test data consisting of the textual version of the first 10000 integers:

(defparameter *things*
   (let (things)
     (dotimes (x 10000 (reverse things)) 
       (push 
        (format nil "~r" x)
        things))))

Here's the procedure that responds to the search url /data?. For example, calling:

/data?ten

will return:

<p>ten<br>one hundred and ten<br>two hundred and ten<br>three hundred and ten<br>
four hundred and ten<br>five hundred and ten<br>six hundred and ten<br>
seven hundred and ten<br>eight hundred and ten<br>nine hundred and ten<br></p>

It uses these routines to filter the list of integers and return the first 10:

(defun first-n (n list)
  (if (> (length list) n)
    (subseq list 0 n)
    list))
(defun filter (text things)
  (remove-if-not
             #'(lambda (item)
                  (if text (search text item) t))
             things))
Here's the routine:
(defun emit-data (url stream)
  (with-slots (search-keys) url
    (let ((text (first url:search-keys)))
      (with-successful-response (stream :text)
        (with-paragraph (:stream stream)
          (dolist (item (first-n 10 (filter text *things*)))
            (write-string item stream)
            (break-line :stream stream)))))))

It's interned with this definition:

(export-url "http://localhost:8000/data?"
            :search
            :response-function 'emit-data)

Loading the JavaScript from a file

If you prefer you can save the submitRequest() routine to a file, and make it available with:

(export-url "http://localhost:8000/submitRequest.js"
            :java-script-file
            :pathname "submitRequest.js")

You can then specify that the page should load it from a file by changing the define-script routine to:

(define-script ajax (:Java-Script)
               :caller "\"submitRequest('/data?' + this.value, 'zone');\""
               :location "http://localhost:8000/submitRequest.js")

References

For more information about AJAX see:

Ajax (programming), a Wikipedia article that explains the technology.

Ajax Tutorial (Asynchronous Javascript + XML), an excellent tutorial on which the JavaScript in this application was based.


blog comments powered by Disqus