Update 15 Jan 2007: I have updated the post as I’ve found a better implementation of this script. Indeed, in my previous solution I was running the server as root. Now I’ve simplified the script, eliminated the wrapper and I’m running my servers as a different user (www-data in my case).
In the constant search for the perfect deployment, I’ve decided to have a look at the Django FastCGI option. The idea is to launch several FastCGI servers with Nginx as a proxy redirecting the requests.
The deployment proposed implies that “in most cases you’ll be starting the FastCGI process on your own”. And, of course, we will be stopping it on our own as well. For this they propose this script:
#!/bin/bash
# Replace these three settings.
PROJDIR="/home/user/myproject"
PIDFILE="$PROJDIR/mysite.pid"
SOCKET="$PROJDIR/mysite.sock"
cd $PROJDIR
if [ -f $PIDFILE ]; then
kill `cat -- $PIDFILE`
rm -f -- $PIDFILE
fi
exec /usr/bin/env - \
PYTHONPATH="../python:.." \
./manage.py runfcgi socket=$SOCKET pidfile=$PIDFILE
It’s a short script, and it works just well. But I was looking for a bit more flexibility. So I decided to develop my own init.d script.
The init.d script is called fastcgi,
an admittedly unimaginative name 
The main requirement is that it
should integrate without problems with the init.d scripts, and that it
should allow to transparently and uniformly start, stop and restart
FastCGI processes for several Django sites with minimum configuration.
#! /bin/sh
### BEGIN INIT INFO
# Provides: FastCGI servers for Django
# Required-Start: networking
# Required-Stop: networking
# Default-Start: 2 3 4 5
# Default-Stop: S 0 1 6
# Short-Description: Start FastCGI servers with Django.
# Description: Django, in order to operate with FastCGI, must be started
# in a very specific way with manage.py. This must be done
# for each DJango web server that has to run.
### END INIT INFO
#
# Author: Guillermo Fernandez Castellanos
# .
#
# Version: @(#)fastcgi 0.1 11-Jan-2007 guillermo.fernandez.castellanos AT gmail.com
#
#### SERVER SPECIFIC CONFIGURATION
DJANGO_SITES="site1 site2 site3"
SITES_PATH=/path/to/sites
RUNFILES_PATH=$SITES_PATH/run
HOST=127.0.0.1
PORT_START=3000
RUN_AS=www-data
#### DO NOT CHANGE ANYTHING AFTER THIS LINE!
set -e
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="FastCGI servers"
NAME=$0
SCRIPTNAME=/etc/init.d/$NAME
#
# Function that starts the daemon/service.
#
d_start()
{
# Starting all Django FastCGI processes
PORT=$PORT_START
for SITE in $DJANGO_SITES
do
echo -n ", $SITE"
if [ -f $RUNFILES_PATH/$SITE.pid ]; then
echo -n " already running"
else
start-stop-daemon --start --quiet \
--pidfile $RUNFILES_PATH/$SITE.pid \
--chuid $RUN_AS --exec /usr/bin/env -- python \
$SITES_PATH/$SITE/manage.py runfcgi \
host=$HOST port=$PORT pidfile=$RUNFILES_PATH/$SITE.pid
chmod 400 $RUNFILES_PATH/$SITE.pid
fi
let "PORT = $PORT + 1"
done
}
#
# Function that stops the daemon/service.
#
d_stop() {
# Killing all Django FastCGI processes running
for SITE in $DJANGO_SITES
do
echo -n ", $SITE"
start-stop-daemon --stop --quiet --pidfile $RUNFILES_PATH/$SITE.pid \
|| echo -n " not running"
if [ -f $RUNFILES_PATH/$SITE.pid ]; then
rm $RUNFILES_PATH/$SITE.pid
fi
done
}
ACTION="$1"
case "$ACTION" in
start)
echo -n "Starting $DESC: $NAME"
d_start
echo "."
;;
stop)
echo -n "Stopping $DESC: $NAME"
d_stop
echo "."
;;
restart|force-reload)
echo -n "Restarting $DESC: $NAME"
d_stop
sleep 1
d_start
echo "."
;;
*)
echo "Usage: $NAME {start|stop|restart|force-reload}" >&2
exit 3
;;
esac
exit 0
The variables of the script are:
- SITES_PATH: The folder where all the sites are stored, a site per folder. The folder structure i supposed to be a common folder ($SITES_PATH, in
our case) where each site has a known folder ($SITES_PATH/www_guindilla_eu/,
$SITES_PATH/www_haruki_eu/, $SITES_PATH/complu_haruki_eu/) where all configuration files
are stored (manage.py, settings.py,…). More info about this here.
- DJANGO_SITES: The names of the folders where the different sites are.
- RUNFILES_PATH: The path to the folder where .pid and .socket files (if using Unix sockets) are stored.
- HOST: The IP address of the host.
- PORT_START: Each server will be running in a different port. The first server will run on PORT_START, the second on PORT_START+1, etc.
- RUN_AS: We want the FastCGI servers to run under a
different user than root. RUN_AS is this user. You can put the username
or the UID of the user.
In order to use, configure the initial variables, put the fastcgi file in the /etc/init.d/ directory and execute update-rc.d /etc/init.d/fastcgi defaults in a terminal. This way the daemon will be automagically be started at each system start-up.
The script is generally pretty easy to understand. Some sensitive parts are:
- #HOST=`/sbin/ifconfig | /bin/sed -n -e ‘s/\(.*\)inet addr:\(.*\)B.*$/\2/p’`
In my script I used the IP address 127.0.0.1. Sometimes we simply want
to use the IP of the server. This line finds this address automatically.
- d_start() start-stop-daemon can store the pid file of the daemon it starts in $RUNFILES_PATH. But this requires root access, and by using the option –chuid $RUN_AS we loose the ability to write root-only folders. So we delegate the pid file writting to manage.py. Another problem is, the stored pid would correspond to the FastCGI process while start-stop-daemon will consider the env pid. Thus, what will happen when doing two consecutive fastcgi start is
that a new FastCGI process will not be spawned, but the pid file will
be overwritten with a more recent pid, thus making any further fastcgi stop
command break. The solution is to manually check for already existent
process (by checking that their pid files exist) and launching the
FastCGI processes otherwise.
- The d_start() function will increase the port used by one for each different server, starting from the $PORT_START onwards.
- d_stop() start-stop-daemon
is used here, although there is no real need. The pid files are removed
once the FastCGI process stopped in order to identify which sites are
running.
This script uses TCP sockets for communication, but it is really simple to modify it in order to use Unix domain sockets:
- The sockets will be stored with the .pid files, and have the same name but with the.socket extension.
- Replate the argument runfcgi socket=$RUNFILES_PATH/$SITE.socket pidfile=$RUNFILES_PATH/$SITE.pid by runfcgi host=$HOST port=$PORT pidfile=$RUNFILES_PATH/$SITE.pid in all the script. Use this last value whenever you need to identify the socket.
- Remove the HOST and PORT variables from the fastcgi file.
And that’s it. Your script will work with Unix domain sockets.
Of course, the script is far from perfect:
- It is the first of the kind I do and my sh mastery is pretty low.
- It does not have a finely grained site management. All are treated alike.
- The port management is done a bit ad-hoc and could cause some configuration synchronization issues if not treated with care.
- The FastCGI server does not work under root, but still has some flaws.
For example, the .pid files can be read by www-data and can not be
written in /var/run/. This is due to the fact that Django’s manage.py
does not seem to have an option to downscale to a lower-permission user
as www-data.
- The script is known to be working in Debian and Ubuntu based systems, but will not work on Gentoo due to it’s particular init.d system. It should work at is in other Redhat based systems (Mandriva, Suse,…), although this has not been confirmed.
Still, I think it is a script that can be
useful and that will save me an important amount of time. And it’s like
a kid, no matter how ugly it is, it’s mine and I love it
Any feedback would be greatly appreciated!
**** OLD POST FOLLOWS FOR REFERENCE ****
In the constant search for the perfect deployment, I’ve decided to have a look at the Django FastCGI option. The idea is to launch several FastCGI servers with Nginx as a proxy redirecting the requests.
The deployment proposed implies that “in most cases you’ll be starting
the FastCGI process on your own”. And, of course, we will be stopping it on our own as well. For this they propose this script:
#!/bin/bash
# Replace these three settings.
PROJDIR="/home/user/myproject"
PIDFILE="$PROJDIR/mysite.pid"
SOCKET="$PROJDIR/mysite.sock"
cd $PROJDIR
if [ -f $PIDFILE ]; then
kill `cat -- $PIDFILE`
rm -f -- $PIDFILE
fi
exec /usr/bin/env - \
PYTHONPATH="../python:.." \
./manage.py runfcgi socket=$SOCKET pidfile=$PIDFILE
It’s a short script, and it works just well. But I was looking for a bit more flexibility. So I decided to develop my own init.d script.
The first thing I did was to write a wrapper around the python manage.py command. I saved it in /www/run/django_fcgi :
#! /bin/sh
[ $# = 4 ] || echo -n "Usage: $0 django_site ip port pid_file"
DJANGO_SITE=$1
IP=$2
PORT=$3
PID_FILE=$4
/usr/bin/env python $DJANGO_SITE/manage.py runfcgi \
host=$HOST port=$PORT pidfile=$PID_FILE
This script will make the use of the start-stop-daemon easier. Of course, this wrapper is not strictly necessary, as the command line can be directly executed in the init.d script, but this wrapper makes it more convenient.
Then comes the proper init.d script. I called it fastcgi, an admittedly unimaginative name The main requirement is that it should integrate without problems with the init.d scripts, and that it should allow to transparently and uniformly start, stop and restart FastCGI processes for several Django sites with minimum configuration.
#! /bin/sh
### BEGIN INIT INFO
# Provides: FastCGI servers for Django
# Required-Start: networking
# Required-Stop: networking
# Default-Start: 2 3 4 5
# Default-Stop: S 0 1 6
# Short-Description: Start FastCGI servers with Django.
# Description: Django, in order to operate with FastCGI, must be started
# in a very specific way with manage.py. This must be done
# for each DJango web server that has to run.
### END INIT INFO
#
# Author: Guillermo Fernandez Castellanos
# .
#
# Version: @(#)fastcgi 0.1 11-Jan-2007 guillermo.fernandez.castellanos AT gmail.com
#
#### SERVER SPECIFIC CONFIGURATION
#HOST=`/sbin/ifconfig | /bin/sed -n -e 's/\(.*\)inet addr:\(.*\)B.*$/\2/p'`
DJANGO_SITES="complu_haruki_eu www_haruki_eu www_guindilla_eu"
SITES_PATH=/www
DAEMON=/www/run/django_fcgi
RUNFILES_PATH=$SITES_PATH/run
HOST=127.0.0.1
PORT_START=3000
#### DO NOT CHANGE ANYTHING AFTER THIS LINE!
set -e
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="FastCGI servers"
NAME=$0
SCRIPTNAME=/etc/init.d/$NAME
# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0
#
# Function that starts the daemon/service.
#
d_start()
{
# Starting all Django FastCGI processes
PORT=$PORT_START
for SITE in $DJANGO_SITES
do
echo -n ", $SITE"
# As $DAEMON is a shell script, the process runs with the name "bash",
# not "$DAEMON".
# Thus I can not use the start-stop-daemon method here.
# start-stop-daemon --start --quiet --pidfile $RUNFILES_PATH/$SITE.pid \
# --exec $DAEMON -- $SITES_PATH/$SITE $HOST $PORT $RUNFILES_PATH/$SITE.pid \
if [ -f $RUNFILES_PATH/$SITE.pid ]; then
echo -n " already running"
else
$DAEMON $SITES_PATH/$SITE $HOST $PORT $RUNFILES_PATH/$SITE.pid
fi
let "PORT = $PORT + 1"
done
}
#
# Function that stops the daemon/service.
#
d_stop() {
# Killing all Django FastCGI processes running
for SITE in $DJANGO_SITES
do
echo -n ", $SITE"
start-stop-daemon --stop --quiet --pidfile $RUNFILES_PATH/$SITE.pid \
|| echo -n " not running"
if [ -f $RUNFILES_PATH/$SITE.pid ]; then
rm $RUNFILES_PATH/$SITE.pid
fi
done
}
ACTION="$1"
case "$ACTION" in
start)
echo -n "Starting $DESC: $NAME"
d_start
echo "."
;;
stop)
echo -n "Stopping $DESC: $NAME"
d_stop
echo "."
;;
restart|force-reload)
echo -n "Restarting $DESC: $NAME"
d_stop
sleep 1
d_start
echo "."
;;
*)
echo "Usage: $NAME {start|stop|restart|force-reload}" >&2
exit 3
;;
esac
exit 0
The variables of the script are:
- SITES_PATH: The folder where all the sites are stored, a site per folder.
- DJANGO_SITES: The names of the folders where the different sites are.
- DAEMON: The path to our wrapper.
- RUNFILES_PATH: The path to the folder where .pid and .socket files (if using Unix sockets) are stored.
- HOST: The IP address of the host.
- PORT_START: Each server will be running in a different port. The first server will run on PORT_START, the second on PORT_START+1, etc.
In order to use, configure the initial variables, put the fastcgi file in the /etc/init.d/ directory and execute update-rc.d /etc/init.d/fastcgi defaults in a terminal. This way the daemon will be automagically be started at each system start-up.
The script is generally pretty easy to understand. Some sensitive parts are:
- #HOST=`/sbin/ifconfig | /bin/sed -n -e ‘s/\(.*\)inet addr:\(.*\)B.*$/\2/p’` In my script I used the IP address 127.0.0.1. Sometimes we simply want to use the IP of the server. This line finds this address automatically.
- d_start() start-stop-daemon stores the pid file of the daemon it starts in $RUNFILES_PATH. The problem is, this stored pid correspond to the FastCGI process while start-stop-daemon will consider the bash wrapper pid. Thus, what will happen when doing two consecutive fastcgi start is that a new FastCGI process will not be spawned, but the pid file will be overwritten with a more
recent pid, thus making any further fastcgi stop command break. The solution is to manually check for already existent process (by checking that their pid files exist) and launching the FastCGI processes otherwise.
- The d_start() function will increase the port used by one for each different server, starting from the $PORT_START onwards.
- d_stop() start-stop-daemon is used here, although there is no real need. The pid files are removed once the FastCGI process stopped in order to identify which sites are running.
This script uses TCP sockets for communication, but it is really simple to modify it in order to use Unix domain sockets:
- The sockets will be stored with the .pid files, and have the same name but with the.socket extension.
- Modify the wrapper so it uses: /usr/bin/env python $DJANGO_SITE/manage.py runfcgi socket=$SOCKET pidfile=$PIDFILE
- Replace the argument $HOST $PORT by $RUNFILES_PATH/$SITE.socket in all the script. Use this last value whenever you need to identify the socket.
- Remove the HOST and PORT variables from the fastcgi file.
And that’s it. Your script will work with Unix domain sockets.
Of course, the script is far from perfect:
- It is the first of the kind I do and my sh mastery is pretty low.
- It does not have a finely grained site management. All are treated alike.
- The port management is done a bit ad-hoc and could cause some configuration synchronization issues if not treated with care.
- The script is known to be working in Debian and Ubuntu based systems, but will not work on Gentoo. I could not test it in other systems (RH, Fedora,…).
Still, I think it is a script that can be useful and that will save me an important amount of time. And it’s like a kid, no matter how ugly it is, it’s mine and I love it
Any feedback would be greatly appreciated!
Entries
Dried tomatoes are one of the typical Italian ingredients. They are delicious with a large array of dishes: pasta sauce, salads, sandwiches,…
Me gusta todo tipo de cocinas, pero entre todas he de destacar las cocinas asiáticas. Probablemente el hecho de que mi pareja sea japonesa es una de las razones, pero incluso antes de conocerla los aromas y delicias asiáticas me apasionaban. De hecho, una de mis respuestas preferidas a “¿Te gusta el 
Estamos en la época de los fresones. En estos momentos, estas frutas de sabor intenso y precio variable están en lo mejor de su forma. En efecto, al menos en Madrid, el precio puede variar entre menos de 2€ a más de 5€ el kilo, prácticamente de un día para otro. Así que hay que aprovechar los días de precios bajos para crear reservas. Si posible, reservas que duren mucho mucho tiempo.
In order to start the year in a positive note, I screwed my network. Indeed, it has been down for a week. The downtime was due to a router failure and my holidays at 2000km from home. It was very bad luck indeed!
Recent Comments