HTTP keep-alive

HTTP keep-alive is also called HTTP persistent connection. It reduces the overhead of creating / closing multiple TCP connections by reusing one TCP connection to send / receive multiple HTTP requests.

What is keep-alive?

Keep-alive is a convention between the client and the server. If keep-alive is enabled, the server does not close the TCP connection after returning the response. Similarly, after receiving the response message, the client does not close the connection and sends the next The connection is reused when an HTTP request is made.

In the HTTP / 1.0 protocol, if the request header contains:

Connection: keep-alive
It means that keep-alive is turned on, and the return header of the server will also contain the same content.

In the HTTP / 1.1 protocol, keep-alive is turned on by default, unless it is explicitly turned off:

Connection: close

The purpose of the keep-alive technology is to reuse the same TCP connection between multiple HTTPs, thereby reducing the overhead of creating / closing multiple TCP connections (including response time, CPU resources, reducing congestion, etc.)

However, there is no free lunch in the world. If the client has not closed the connection after receiving all the information, the corresponding resources of the server are still occupied (though it is useless). For example, in Tomcat's BIO implementation, an unclosed connection will occupy the corresponding processing thread. If a long connection is actually processed, but the closed timeout period has not expired, the thread will always be occupied. problem).

Obviously, if the client and server do need to communicate multiple times, enabling keep-alive is a better choice. For example, in a microservice architecture, the user and provider of a microservice usually have long-term communication. Okay, keep-alive.

In some REST services with high TPS / QPS, if a short connection is used (that is, keep-alive is not enabled), it is likely that the client port is full. This is because a large number of TCP connections will be created in a short period of time, and after the four wave of TCP, the client's port will be in TIME_WAIT for a period of time (2 * MSL), during which the port will not be released, resulting in the port being full. In this case it is best to use long connections.

How does the client start?

Almost all the tools we use now have long connections turned on by default:

For browsers, almost all browsers you use today (including IE6) use keep-alive by default.

The Java8 HttpURLConnectionenabled by default long connection, but just leave the default connection pool five long connection 1 , if at the same time more than five threads in use, it will create a new connection, after the end of more than five parts will be active client shut down.

HttpClientBy default, Apache reserves 2 long connections for each address, and a maximum of 20 connections are reserved in the connection pool2 .

Python requests using sessions will enable long connections by default.

Here are some code notes:

Feign uses HttpClient connection pool example

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager (); 

connectionManager.setMaxTotal (maxConnections); 

connectionManager.setDefaultMaxPerRoute (maxConnectionsPerRoute);

CloseableHttpClient httpClient = HttpClients 

    .custom () 

    .setConnectionManager (connectionManager) 

    .build ();

return Feign.builder () 

        .client ( new new ApacheHttpClient (httpClient)) 

        .options ( new new the Options (connectTimeoutMills, readTimeoutMills)) 

        .retryer ( new new the Default (Retryperiod, retryMaxPeriod, retryMaxAttempts)) 

        .encoder ( new new JacksonEncoder (JsonUtil.getObjectMapper () )) 

        .decoder ( new JacksonDecoder (JsonUtil.getObjectMapper ())) 

        .decode404 () 

        .target (PredictorFeignService . class , endpoint ) ;

How to implement the server

Different servers implement keep-alive in different ways. Even in different working modes of tomcat, the processing methods are different. Here is roughly the processing logic in the NIO mode (tomcat 9.0.22):

In NioEndpoint#SocketProcessorclass, just off the internal state CLOSEDof the port:

if (state == SocketState.CLOSED) { 

    poller.cancelledKey (key, this .socketWrapper); 

}

In the Http11Processor#servicemethod, if the connection keep-alive, the final internal state would beOPEN

} else  if ( this .openSocket) { return this .readComplete? SocketState.OPEN: SocketState.LONG; } else {

     

Reserved connection, after a timeout period, will NioEndpoint#Poller#timeoutbe shut down process:

} else  if (! NioEndpoint. this .processSocket (socketWrapper, SocketEvent.ERROR, true )) { this .cancelledKey (key, socketWrapper); }

    

Further, if the spring boot, you can server.connection-timeoutbe adjusted retention time of the keep-alive connections CI, if the server itself is not provided, compared with each default configuration, Tomcat default 60s .