nginx_upstream_multiplexing_keepalive

Posted on Edited on In ,

Overview

Http Keep alive(multiplexing) means after a TCP connection serves a request, it’s not closed by server, so that subsequent request can reuse the same connection to avoid create new. hence can improve http performance.
enable it from client side(make sure sever does not disable it)

  • Http 1.0: send request(explicit enable) Connection: keep-alive
  • Http 1.1 default is keep-alive(explicit disable Connection: close)

Note: even connection is keep-alive, it's not kept for ever if it's idle(most server has a timeout for idle connection).

Upstream keep-alive

Here we say how to enable upstream keep-alive, as by default nginx uses http1.0 for upstream, keep-alive is disabled, to enable upstream keep-alive and make it work, it needs

  • nginx(now as client) to cache the idle connection(keep it)
  • nginx(now as client) not use http1.0 as by default http1.0 upstream server will close the connection after served.
  • server(upstream) must not disable keep-alive, out of nginx scope.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
http {
    upstream ups {
        server 1.1.1.1;
        server 2.2.2.2;
        # cache 6 idle connection for this upstream, all servers shares the 6 free connection
        keepalive 6;
        keepalive_requests 100;
        keepalive_timeout 60s
    }
    server {
        listen 10.10.10.10:80;
        location / {
            # use http 1.1 to tell server not close the connection, hence we can cache it for a while
            # clear Connection header to avoid miss send Connection: close
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_pass: http://ups;
        }
    }
}

As mentioned, for idle connection, it’s not kept for ever, it may be closed by client(nginx) Or server due to config

  • client(nginx) closes it due to timed out(on client side) or serve a number of request
  • server(upstream) closes it due to timed out on server side

keep-alive cache

Here is how nginx cache the idle connection to make sure we only cache a number of idle connection.

upstream keepalive

Data Structure

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
/* each upstream has one */
typedef struct {
    /* max kept idle connections for the whole pool(upstream) not each peer */
    ngx_uint_t                         max_cached;
    /* close the idle keep-alive connection if it serves number of requests or timed out */
    ngx_uint_t                         requests;
    ngx_msec_t                         timeout;
    /* cache keep-alive idle connection(tcp established) */
    ngx_queue_t                        cache;
    /* free cache entry that can be used to keep idle connection, queue size is max_cached at initialization */
    ngx_queue_t                        free;
} ngx_http_upstream_keepalive_srv_conf_t;

typedef struct {
    /* for each keep-alive idle connection we have a such entry insert to conf->cache at head */
    ngx_http_upstream_keepalive_srv_conf_t  *conf;
    ngx_queue_t                        queue;
    /* idle peer connection(tcp established, not in use for request) */
    ngx_connection_t                  *connection;
    /* peer info */
    socklen_t                          socklen;
    ngx_sockaddr_t                     sockaddr;
} ngx_http_upstream_keepalive_cache_t;

API

1
2
3
4
5
6
/* check idle cache, if found, use existing connection, otherwise create a new by event framework */
static ngx_int_t ngx_http_upstream_get_keepalive_peer(ngx_peer_connection_t *pc, void *data);
/* save idle connection to cache */
static void ngx_http_upstream_free_keepalive_peer(ngx_peer_connection_t *pc, void *data, ngx_uint_t state);
/* close peer connection if timed out or closed by upstream */
static void ngx_http_upstream_keepalive_close_handler(ngx_event_t *ev);