I’ve been exploring using Nginx to front our Apache websites. I found a fair amount of documentation online but most of it was for nginx on top of a backend application running on the same host – so, lots of examples of load balancing among various ports on 127.0.0.1. In my case I have name-based Apache virtual hosts running at different IP addresses. Also my goal is not to set up automatic load balancing – our session-based application will not work well with it – but rather I want to be able to rapidly and manually switch a web address to a different machine so I can perform maintenance on or updates to the offline systems.

To summarise, I want the publically accessible crashingdaily.com website to be a proxy server to our internal, Apache name-based virtual hosts w1 or w2. I want to be able to choose which of w1 or w2 services the client’s request.

A diagram of my setup.


                                                216.52.184.15
                                            ----------------------
                216.52.184.243       ----- [ w1.crashingdaily.com ]
 ------       -------------------  /        ----------------------
[client]-----[ crashingdaily.com ]
 ------       -------------------  \        ----------------------
                                     ----- [ w2.crashingdaily.com ]
                                            ----------------------
                                                216.52.184.30

Not illustrated here is the possibility for authorized client browsers (restricted by IP address and/or authentication) to directly access w1 and w2. This is important because it allows our internal developers and testing team to verify a working host before we put it into service.

I present here my installation and setup notes. The hostnames and IP addresses used here are artificial.

Download and upack nginx source code.

$ curl -O http://sysoev.ru/nginx/nginx-0.6.32.tar.gz
$ tar zxf nginx-0.6.32.tar.gz

I am running Red Hat Enterprise Linux 4 and I have the openssl and openssl-devel packages installed but I was unable to use them for compiling nginx’s SSL module. The compile would fail with “undefined reference to 'krb5_free_data_contents'“. So, I download and untar the source for openssl. I’m not compiling or installing it, just providing the source to the nginx compiler. I don’t really need nginx with SSL support at this time, I’m including it for later testing.

$ curl -O http://www.openssl.org/source/openssl-0.9.8i.tar.gz
$ tar zxf openssl-0.9.8i.tar.gz

Create a user to own the nginx processes.

$ sudo /usr/sbin/useradd -c "Nginx User" -M -u 105 -s /sbin/nologin nginx

Now I configure and compile the nginx source code and install into /usr/local/nginx (the default location). Consult ./configure --help and documentation for other compile options.

$ cd nginx-0.6.32
$ ./configure \
   --user=nginx \
   --group=nginx \
   --with-http_ssl_module \
   --with-http_realip_module \
   --with-http_stub_status_module \
   --with-openssl=$HOME/openssl-0.9.8i \
   --with-md5=auto/lib/md5 \
   --with-sha1=auto/lib/sha1

$ make
$ make install

$ dir /usr/local/nginx/
conf  html  logs  sbin

Edit /usr/local/nginx/conf/nginx.conf. Here is an example, minimal configuration.

events { worker_connections  1024; }

http {

    upstream  crashingdaily.com {
        server   216.52.184.15 down;
        server   216.52.184.30;
    }

    server {
        listen       80;
        server_name crashingdaily.com; 

        location / {
            proxy_pass http://crashingdaily.com$request_uri;
        }
        
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

I start the nginx server and I’m in business.

$ sudo /usr/local/nginx/sbin/nginx

Let’s take a look at the round-trip chain of communications through the servers.

Client to Proxy:
crashingdaily.com resolves in DNS to 216.52.184.243, so the general public is routed to that machine. An Nginx virtual server there answers requests for crashingdaily.com and proxies the request to one of the IP addresses defined in the ‘upstream’ block. It does not use host names and so does not need DNS. This was a part that threw me at first. I kept trying to put backend host names, ‘w1.crashingdaily.com’ and ‘w2.crashingdaily.com’ into the upstream block. That can be made to work but it isn’t pretty and, if you configure Apache appropriately (see below), it is unnecessary. As I said earlier, I’m not doing load balancing at this time so all but one of the IP addresses are marked ‘down’. I can easily switch which IP address serves client requests by adjusting this flag.

Proxy to Backend:
When Nginx makes its requests to the backend Apache HTTP servers, 216.52.184.15 or 216.52.184.30, it sets the Host attribute in the HTTP headers to ‘crashingdaily.com’ (defined with the nginx configuration directive ‘proxy_set_header Host $host‘). The backend Apache servers use this Host header attribute of the incoming request to determine which name-based VirtualHost will handle the request. The Apache virtual host has ServerName set to w1.crashingdaily.com (similarly for w2) and also has a ServerAlias set to ‘crashingdaily.com’. So, when nginx connects to 216.52.184.15 with the ‘Host: crashingdaily.com’ HTTP header, Apache knows which virtual host to use. Defining a ‘ServerAlias’ was the missing piece I needed so I could use IP addresses in nginx’s upstream block.

The w1 and w2 ServerName configuration is optional but allows us to connect directly to the individual backend servers for testing. That is, we can privately work on the ‘down’ server via the w1/w2 host name while public interacts, by proxy, with the up server.

Backend to Proxy:
The HTTP header of the Apache response from 216.52.184.15 should contain ‘Location: http://crashingdaily.com/’ because that is the host that was requested (Recall from above that Nginx sends “Host: crashingdaily.com” in its HTTP headers).

Proxy to Client:
The client receives the proxied response from the backend server. It should have no idea that it’s been proxied.

Logging:
Normally Apache on 216.52.184.15 will log the incoming request as coming from 216.52.184.243 (the nginx server), but here the X-Forwarded-For header in Nginx’s request passes along the client’s IP address to the mod_rpaf module I have installed in Apache and allows for logging the client’s IP.

Related:

Nginx wiki
Apache Virtual Hosts
Nginx tag in Delicious

About these ads