A CL-HTTP Primer

Tips for High Volume

This page gives some tips for configuring a CL-HTTP server for a high volume of requests.

Configuration file settings

The following settings should be made in the CL-HTTP configuration file, usually configuration.lisp.

Report bugs by email:

(setq http::*bug-report-destination* :mail)

Don't report idle connection errors:

(setq http::*report-idle-connection-scavenges* nil)

(setq *log-resolve-ip-addresses* nil)

(set-maximum-number-of-connections 640.)

Note that increasing this will increase the number of open files that the server needs; see Increasing the number of open files below.

Keep the log file open, for efficiency:

(log-file-stream-stays-open t)

(setq http::*connection-backlog* 128)

Don't allow errors in the server to cause a Lisp break:

(setq http::*debug-server* nil)

(http::log-notifications-on (http::multiport-access-logs) nil)

Increasing the number of open files

On most versions of Mac OS the maximum number of open files per application is set to 256. With the maximum number of connections set to more than the default of 30 you can sometimes exceed this limit.

If you get the error "Too many open files" check the maximum number of files with:

launchctl limit

Change the limit by editing the config file:

sudo pico /etc/launchd.conf

A recommended setting is three times the number of connections, so for example insert the single line:

limit maxfiles 1920

Type Ctrl-O to save the file and Ctrl-X to exit. After restarting the new limit will take effect.

Monitoring the number of connections

The following LispWorks application displays an activity gauge, showing the number of connected users:

The gauge automatically rescales to the maximum number of users so far, which is the figure displayed to the right of the scale.

The value of *maximum-number-of-connections*, set by set-maximum-number-of-connections described above, is displayed in the title bar of the gauge.

(defun make-activity-gauge ()
    (let* (process
           (max 10)
           (gauge
            (capi:contain
             (make-instance 'capi:slider :start 0 :end max :orientation :horizontal 
                            :title (princ-to-string max) :title-position :right)
             :title (format nil "Activity (~a)" http::*maximum-number-of-connections*)
             :destroy-callback #'(lambda (interface)
                                   (declare (ignore interface)) (mp:process-kill process)))))
        ;;
        (flet ((update-gauge ()
           (loop
            (let ((activity (length (http:all-servers))))
              (setf (capi:range-slug-start gauge) activity)
              (when (> activity max)
                (setq max activity)
                (setf (capi:titled-object-title gauge) (princ-to-string max))
                (setf (capi:range-end gauge) max))
              (sleep 2)))))
      ;;
      (setq process (mp:process-run-function "Server Activity" nil #'update-gauge)))))