Customize and build a local lighttpd test server

While fiddling around writing a simple CGI-based web application, I found myself needing a web server for testing purposes.1 A key requirement for that server (lighttpd) being that it needs to be built and installed without root permissions and run locally from my home directory, the setup-process is a bit more involved than would be a default installation. In this post, I therefore describe how to install, configure and run lighttpd given the above conditions.

The post consists of three sections. In the first section, I discuss the various build and installation steps by means of a makefile. The second section shows how to interact with the server. (Readers eager to run lighttpd right away should download the makefile, put it in a clean working directory, and directly jump to section two.) Section three concludes.

Build and configure lighttpd

To install in and run lighttpd from a non-default directory, I need to build the server from source and carry out some additional configuration steps. More specifically, setting up lighttpd entails

The setup process is not very complicated as of itself, the most demanding task being to create additional directories (which we would not need when installing to the default location) and the main configuration file. Yet, to make the customized build, install and configuration process repeatable, I wrote a makefile2 which automates the steps mentioned above. 

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160

# This Makefile automates configuration, build, and deployment of lighttpd
# such that no root permissions are required whatsoever.

# =============================================================================
# Main make variables
# -------------------
# - VERSION: defines version of lighttpd that should be downloaded and built.
#            Defaults to version 1.4.59
# - PREFIX:  defines installation path. Defaults to $HOME/opt/lighttpd
#
# If you wish to customize the version or the install path of lighttpd, pass
# the appropriate value to 'make' as follows:
# make VERSION=1.4.59 PREFIX=path/to/install/dir
# =============================================================================
VERSION     = 1.4.59
PREFIX      = $(shell echo $$HOME)/opt/lighttpd

TARBALL     = lighttpd-$(VERSION).tar.gz
EXTRACT_DIR = lighttpd-$(VERSION)

# =============================================================================
# Configure server variables
# For customization, see "Main make variables"
# =============================================================================
SERVER_ROOT = $(PREFIX)
SERVER_PORT = 1024

# =============================================================================
# Targets
# =============================================================================
.PHONY: all test \
	configure build \
	install install_lighttpd \
	uninstall uninstall_lighttpd \
	clean \
	config_files \
	$(PREFIX)/config/lighttpd.conf \
	required_dirs required_files \
	start_lighttpd.sh \
	stop_lighttpd.sh

default: build

# =============================================================================
# Download and extract source tarball
# =============================================================================
$(TARBALL):
	if command -v curl; then \
		curl -O "https://download.lighttpd.net/lighttpd/releases-1.4.x/$@"; \
	elif command -v wget; then \
		wget "https://download.lighttpd.net/lighttpd/releases-1.4.x/$@"; \
	else \
		echo "Neither curl nor wget installed."; \
	fi

$(EXTRACT_DIR): $(TARBALL)
	tar -xf $<

# =============================================================================
# Standard build process (wrappers around ./configure, make, and make install)
# =============================================================================
configure: $(EXTRACT_DIR)
	$(info $(PREFIX))
	cd $(EXTRACT_DIR) && \
	./configure --prefix=$(PREFIX)

build: configure
	cd $(EXTRACT_DIR) && \
	$(MAKE)

install: install_lighttpd config_files scripts

install_lighttpd:
	cd $(EXTRACT_DIR) && \
	$(MAKE) install

uninstall: uninstall_lighttpd
	rm -rf $(PREFIX) \
	rm -rf start_lighttpd.sh stop_lighttpd.sh

uninstall_lighttpd:
	cd $(EXTRACT_DIR) && \
	$(MAKE) uninstall

clean:
	cd $(EXTRACT_DIR) && \
	$(MAKE) clean

# =============================================================================
# Set up lighttpd configuration:
# - create config directory in $PREFIX/lighttpd/
# - create lighttpd.conf
# =============================================================================
config_files: \
	$(PREFIX)/config \
	$(PREFIX)/config/lighttpd.conf \
	required_dirs \
	required_files

$(PREFIX)/config:
	cp -R $(EXTRACT_DIR)/doc/config $(PREFIX)/

$(PREFIX)/config/lighttpd.conf:
	printf '# main variables and directories\n' > $@
	printf 'var.server_root = "$(SERVER_ROOT)"\n' >> $@
	printf 'var.log_root      = server_root + "/log"\n' >> $@
	printf 'var.state_dir     = server_root + "/run"\n' >> $@
	printf 'var.conf_dir      = server_root + "/config"\n' >> $@
	printf 'var.document_root = server_root + "/htdocs"\n' >> $@
	printf 'server.document-root = document_root\n' >> $@
	printf 'server.errorlog = log_root + "/error.log"\n' >> $@
	printf 'server.pid-file = state_dir + "/lighttpd.pid"\n' >> $@
	printf '# general settings\n' >> $@
	printf 'server.port = $(SERVER_PORT)\n' >> $@
	printf 'server.use-ipv6 = "disable"\n' >> $@
	printf 'server.follow-symlink  = "disable"\n' >> $@
	printf '# load modules\n' >> $@
	printf 'server.modules = (\n' >> $@
	printf '  "mod_access",\n  "mod_alias",\n' >> $@
	printf '  "mod_redirect",\n  "mod_cgi"\n' >> $@
	printf ')\n' >> $@
	printf 'index-file.names += ( "index.html" )\n' >> $@
	printf 'include "conf.d/mime.conf"\n' >> $@
	printf 'include "conf.d/access_log.conf"\n' >> $@
	printf '# configure cgi\n' >> $@
	printf 'alias.url += ( "/cgi-bin" => document_root + "/cgi-bin" )\n' >> $@
	printf '$$HTTP["url"] =~ "^/cgi-bin" {\n' >> $@
	printf '  cgi.assign = ( "" => "")\n' >> $@
	printf '}\n' >> $@
	printf '# allow dirlisting\n' >> $@
	printf 'dir-listing.activate = "enable"' >> $@

required_dirs:
	cd $(PREFIX) && \
	mkdir log run htdocs && \
	mkdir htdocs/cgi-bin

required_files:
	cd $(PREFIX) && \
	echo "Welcome. This is lighttpd." > htdocs/index.html && \
	touch run/lighttpd.pid

# =============================================================================
# Create start and stop scripts
# =============================================================================
scripts: start_lighttpd.sh stop_lighttpd.sh

start_lighttpd.sh:
	echo "#!/bin/bash" > $@
	echo "$(PREFIX)/sbin/lighttpd -f $(PREFIX)/config/lighttpd.conf" >> $@

stop_lighttpd.sh:
	echo "#!/bin/bash" > $@
	echo 'PID_LIGHTTPD=$$(cat $(PREFIX)/run/lighttpd.pid)' >> $@
	echo 'if [ ! -z "$$PID_LIGHTTPD" ]; then' >> $@
	echo '  printf "Stopping lighttpd server (pid $$PID_LIGHTTPD).\\n"' >> $@
	echo '  kill $$PID_LIGHTTPD' >> $@
	echo "else" >> $@
	echo '  printf "No running instance of lighttpd found.\\n"' >> $@
	echo "fi" >> $@

The makefile consists of seven blocks. In the first block (lines 4-23), I define several default variables that govern the build and installation process, the most important being PREFIX, which sets Lighttpd’s installation path (in this case to $HOME/opt/lighttpd). I leave the details about the remaining variables to the interested reader.

The second block (lines 24-27) defines Lighttpd’s root directory (which is set to depend on PREFIX) and the port to which to listen (port 1024).

The third block (lines 28-43) consists of make-related housekeeping (the details of which are left again to the interested reader).

Block four (lines 44-58) deals with downloading the source tarball and extracting the files needed to build lighttpd. This step requires a working internet connection. To skip the download, point make to the tarball’s location by overriding the default value of the variable TARBALL when invoking make (i.e. make TARBALL=path/to/tarball).

The fifth block is the core step for automatically building the lighttpd test-server (lines 59-88). Automatic customization is achieved by the makefile calling lighttpd’s make targets with variables modified for the present use case. As defined in line 71, install depends on the targets install_lighttpd (which wraps lighttpd’s default install target) and config_files (consisting of configuration tasks required for lighttpd to run). config_files and its dependencies are shown in the next block.

The sixth block describes the configuration tasks needed for running lighttpd with CGI support (lines 89-142). The tasks involve

  1. copying the module configuration files (located in $(EXTRACT_DIR)/doc/config/) to the directory where lighttpd has been installed (this location being $HOME/opt/lighttpd/ in my case)
  2. creation of the main configuration file lighttpd.conf (lines 103-131)
  3. within $HOME/opt/lighttpd, creating a run directory (needed to store the server’s process ID), as well as log, htdocs, and htdocs/cgi-bin, which will contain the server logs, index file, and CGI applications respectively (lines 133-141).

Finally, block seven will create a start and stop script (see lines 143-160). Recall that the lighttpd.conf created by the makefile sets the port number to 1024. Lighttpd blocking this port while running, attempts to start it twice or more will not work unless a different port number (and hence different configuration file) is specified for each instance.

Run locally

Assuming a clean working directory containing the above makefile, download, build, and install lighttpd ($HOME/opt/lighttpd) in a shell: 

# download, configure and build
make
# install
make install

To start the server, run: 

bash start_lighttpd.sh
# alternatively, assuming the start and stop scripts have execution
# permissions
./start_lighttpd.sh

To check whether lighttpd is actually serving content, navigate to 127.0.0.1:1024. The server should return “Welcome. This is lighttpd”.3 

curl 127.0.0.1:1024

Welcome. This is lighttpd.

Stop lighttpd: 

bash stop_lighttpd.sh

The server can be uninstalled by calling the uninstall target. This will remove the lighttpd directory in $HOME/opt/ as well as the start and stop scripts. 

make uninstall

Conclusion

Developing a CGI application requires a suitable web server. For my use case, the server should run locally and be installed in $HOME (instead of e.g. /usr/local/bin). As quite prominently display in the main code listing, makefiles provide an ideal way to structure and run (customized) build processes. In fact, make && make install takes care of (almost) everything.

  1. Developing a CGI web application may at first seem rather uncommon in light of the plethora of current technologies and web frameworks (e.g. JavaScript; Ruby on Rails, Spring, Django). My application however consists only of a remote procedure on the server which is called via a URL and returns a json-formatted response. Given this use-case and my limited knowledge about web frameworks, I preferred to stick to the simplicity of CGI. ↩︎

  2. Incidentally discovering the numerous marvels of make (for more on this, see the documentation e.g. here). ↩︎

  3. For simplicity, the index.html created automatically contains no markup. ↩︎