Subversion Repositories javautils

Rev

Blame | Last modification | View Log | RSS feed

  1. <html><head>
  2.       <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
  3.    <title>Chapter&nbsp;2.&nbsp;Connection management</title><link rel="stylesheet" href="css/hc-tutorial.css" type="text/css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.73.2"><link rel="start" href="index.html" title="HttpClient Tutorial"><link rel="up" href="index.html" title="HttpClient Tutorial"><link rel="prev" href="fundamentals.html" title="Chapter&nbsp;1.&nbsp;Fundamentals"><link rel="next" href="statemgmt.html" title="Chapter&nbsp;3.&nbsp;HTTP state management"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div xmlns:fo="http://www.w3.org/1999/XSL/Format" class="banner"><a class="bannerLeft" href="http://www.apache.org/" title="Apache Software Foundation"><img style="border:none;" src="images/asf_logo_wide.gif"></a><a class="bannerRight" href="http://hc.apache.org/httpcomponents-core/" title="Apache HttpComponents Core"><img style="border:none;" src="images/hc_logo.png"></a><div class="clear"></div></div><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter&nbsp;2.&nbsp;Connection management</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="fundamentals.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="statemgmt.html">Next</a></td></tr></table><hr></div><div class="chapter" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="connmgmt"></a>Chapter&nbsp;2.&nbsp;Connection management</h2></div></div></div>
  4.    
  5.     <p>HttpClient has a complete control over the process of connection initialization and
  6.         termination as well as I/O operations on active connections. However various aspects of
  7.         connection operations can be controlled using a number of parameters.</p>
  8.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e391"></a>2.1.&nbsp;Connection parameters</h2></div></div></div>
  9.        
  10.         <p>These are parameters that can influence connection operations:</p>
  11.         <div class="itemizedlist"><ul type="disc"><li>
  12.                 <p>
  13.                     <b>'http.socket.timeout':&nbsp;</b>
  14.                     defines the socket timeout (<code class="literal">SO_TIMEOUT</code>) in
  15.                         milliseconds, which is the timeout for waiting for data or, put differently,
  16.                         a maximum period inactivity between two consecutive data packets). A timeout
  17.                         value of zero is interpreted as an infinite timeout. This parameter expects
  18.                         a value of type <code class="classname">java.lang.Integer</code>. If this parameter
  19.                         is not set read operations will not time out (infinite timeout).
  20.                 </p>
  21.                 <p>
  22.                     <b>'http.tcp.nodelay':&nbsp;</b>
  23.                     determines whether Nagle's algorithm is to be used. The Nagle's algorithm
  24.                         tries to conserve bandwidth by minimizing the number of segments that are
  25.                         sent. When applications wish to decrease network latency and increase
  26.                         performance, they can disable Nagle's algorithm (that is enable
  27.                            <code class="literal">TCP_NODELAY</code>. Data will be sent earlier, at the cost
  28.                        of an increase in bandwidth consumption. This parameter expects a value of
  29.                        type <code class="classname">java.lang.Boolean</code>. If this parameter is not,
  30.                            <code class="literal">TCP_NODELAY</code> will be enabled (no delay).
  31.                </p>
  32.                <p>
  33.                    <b>'http.socket.buffer-size':&nbsp;</b>
  34.                    determines the size of the internal socket buffer used to buffer data
  35.                        while receiving / transmitting HTTP messages. This parameter expects a value
  36.                        of type <code class="classname">java.lang.Integer</code>. If this parameter is not
  37.                        set HttpClient will allocate 8192 byte socket buffers.
  38.                </p>
  39.                <p>
  40.                    <b>'http.socket.linger':&nbsp;</b>
  41.                    sets <code class="literal">SO_LINGER</code> with the specified linger time in
  42.                        seconds. The maximum timeout value is platform specific. Value 0 implies
  43.                        that the option is disabled. Value -1 implies that the JRE default is used.
  44.                        The setting only affects the socket close operation. If this parameter is
  45.                        not set value -1 (JRE default) will be assumed.
  46.                </p>
  47.                <p>
  48.                    <b>'http.connection.timeout':&nbsp;</b>
  49.                    determines the timeout in milliseconds until a connection is established.
  50.                        A timeout value of zero is interpreted as an infinite timeout. This
  51.                        parameter expects a value of type <code class="classname">java.lang.Integer</code>.
  52.                        If this parameter is not set connect operations will not time out (infinite
  53.                        timeout).
  54.                </p>
  55.                <p>
  56.                    <b>'http.connection.stalecheck':&nbsp;</b>
  57.                    determines whether stale connection check is to be used. Disabling stale
  58.                        connection check may result in a noticeable performance improvement (the
  59.                        check can cause up to 30 millisecond overhead per request) at the risk of
  60.                        getting an I/O error when executing a request over a connection that has
  61.                        been closed at the server side. This parameter expects a value of type
  62.                            <code class="classname">java.lang.Boolean</code>. For performance critical
  63.                        operations the check should be disabled. If this parameter is not set the
  64.                        stale connection will be performed before each request execution.
  65.                </p>
  66.                <p>
  67.                    <b>'http.connection.max-line-length':&nbsp;</b>
  68.                    determines the maximum line length limit. If set to a positive value, any
  69.                        HTTP line exceeding this limit will cause an
  70.                            <code class="exceptionname">java.io.IOException</code>. A negative or zero
  71.                        value will effectively disable the check. This parameter expects a value of
  72.                        type <code class="classname">java.lang.Integer</code>. If this parameter is not set,
  73.                        no limit will be enforced.
  74.                </p>
  75.                <p>
  76.                    <b>'http.connection.max-header-count':&nbsp;</b>
  77.                    determines the maximum HTTP header count allowed. If set to a positive
  78.                        value, the number of HTTP headers received from the data stream exceeding
  79.                        this limit will cause an <code class="exceptionname">java.io.IOException</code>.
  80.                        A negative or zero value will effectively disable the check. This parameter
  81.                        expects a value of type <code class="classname">java.lang.Integer</code>. If this
  82.                        parameter is not set, no limit will be enforced.
  83.                </p>
  84.                <p>
  85.                    <b>'http.connection.max-status-line-garbage':&nbsp;</b>
  86.                    defines the maximum number of ignorable lines before we expect a HTTP
  87.                        response's status line. With HTTP/1.1 persistent connections, the problem
  88.                         arises that broken scripts could return a wrong
  89.                             <code class="literal">Content-Length</code> (there are more bytes sent than
  90.                         specified). Unfortunately, in some cases, this cannot be detected after the
  91.                         bad response, but only before the next one. So HttpClient must be able to
  92.                         skip those surplus lines this way. This parameter expects a value of type
  93.                         java.lang.Integer. 0 disallows all garbage/empty lines before the status
  94.                         line. Use <code class="constant">java.lang.Integer#MAX_VALUE</code> for unlimited
  95.                         number. If this parameter is not set unlimited number will be
  96.                         assumed.
  97.                 </p>
  98.             </li></ul></div>
  99.     </div>
  100.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e438"></a>2.2.&nbsp;Connection persistence</h2></div></div></div>
  101.        
  102.         <p>The process of establishing a connection from one host to another is quite complex and
  103.             involves multiple packet exchanges between two endpoints, which can be quite time
  104.             consuming. The overhead of connection handshaking can be significant, especially for
  105.             small HTTP messages. One can achieve a much higher data throughput if open connections
  106.             can be re-used to execute multiple requests.</p>
  107.         <p>HTTP/1.1 states that HTTP connections can be re-used for multiple requests per
  108.             default. HTTP/1.0 compliant endpoints can also use similar mechanism to explicitly
  109.             communicate their preference to keep connection alive and use it for multiple requests.
  110.             HTTP agents can also keep idle connections alive for a certain period time in case a
  111.             connection to the same target host may be needed for subsequent requests. The ability to
  112.             keep connections alive is usually refered to as connection persistence. HttpClient fully
  113.             supports connection persistence.</p>
  114.     </div>
  115.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e442"></a>2.3.&nbsp;HTTP connection routing</h2></div></div></div>
  116.        
  117.         <p>HttpClient is capable of establishing connections to the target host either directly
  118.             or via a route that may involve multiple intermediate connections also referred to as
  119.             hops. HttpClient differentiates connections of a route into plain, tunneled and layered.
  120.             The use of multiple intermediate proxies to tunnel connections to the target host is
  121.             referred to as proxy chaining.</p>
  122.         <p>Plain routes are established by connecting to the target or the first and only proxy.
  123.             Tunnelled routes are established by connecting to the first and tunnelling through a
  124.             chain of proxies to the target. Routes without a proxy cannot be tunnelled. Layered
  125.             routes are established by layering a protocol over an existing connection. Protocols can
  126.             only be layered over a tunnel to the target, or over a direct connection without
  127.             proxies.</p>
  128.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e446"></a>2.3.1.&nbsp;Route computation</h3></div></div></div>
  129.            
  130.             <p><code class="interfacename">RouteInfo</code> interface represents information about a
  131.                 definitive route to a target host involving one or more intermediate steps or hops.
  132.                     <code class="classname">HttpRoute</code> is a concrete implementation of
  133.                     <code class="interfacename">RouteInfo</code>, which cannot be changed (is
  134.                 immutable). <code class="classname">HttpTracker</code> is a mutable
  135.                     <code class="interfacename">RouteInfo</code> implementation used internally by
  136.                 HttpClient to track the remaining hops to the ultimate route target.
  137.                     <code class="classname">HttpTracker</code> can be updated after a successful execution
  138.                 of the next hop towards the route target. <code class="classname">HttpRouteDirector</code>
  139.                 is a helper class that can be used to compute the next step in a route. This class
  140.                 is used internally by HttpClient.</p>
  141.             <p><code class="interfacename">HttpRoutePlanner</code> is an interface representing a
  142.                 strategy to compute a complete route to a given target based on the execution
  143.                 context. HttpClient ships with two default
  144.                     <code class="interfacename">HttpRoutePlanner</code> implementation.
  145.                     <code class="classname">ProxySelectorRoutePlanner</code> is based on
  146.                     <code class="classname">java.net.ProxySelector</code>. By default, it will pick up the
  147.                 proxy settings of the JVM, either from system properties or from the browser running
  148.                 the application. <code class="classname">DefaultHttpRoutePlanner</code> implementation does
  149.                 not make use of any Java system properties, nor of system or browser proxy settings.
  150.                 It computes routes based exclusively on HTTP parameters described below.</p>
  151.         </div>
  152.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e462"></a>2.3.2.&nbsp;Secure HTTP connections</h3></div></div></div>
  153.            
  154.             <p>HTTP connections can be considered secure if information transmitted between two
  155.                 connection endpoints cannot be read or tampered with by an unauthorized third party.
  156.                 The SSL/TLS protocol is the most widely used technique to ensure HTTP transport
  157.                 security. However, other encryption techniques could be employed as well. Usually,
  158.                 HTTP transport is layered over the SSL/TLS encrypted connection.</p>
  159.         </div>
  160.     </div>
  161.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e465"></a>2.4.&nbsp;HTTP route parameters</h2></div></div></div>
  162.        
  163.         <p>These are parameters that can influence route computation:</p>
  164.         <div class="itemizedlist"><ul type="disc"><li>
  165.                 <p>
  166.                     <b>'http.route.default-proxy':&nbsp;</b>
  167.                     defines a proxy host to be used by default route planners that do not make
  168.                         use of JRE settings. This parameter expects a value of type
  169.                             <code class="classname">HttpHost</code>. If this parameter is not set direct
  170.                         connections to the target will be attempted.
  171.                 </p>
  172.             </li><li>
  173.                 <p>
  174.                     <b>'http.route.local-address':&nbsp;</b>
  175.                     defines a local address to be used by all default route planner. On
  176.                         machines with multiple network interfaces, this parameter can be used to
  177.                         select the network interface from which the connection originates. This
  178.                         parameter expects a value of type
  179.                             <code class="classname">java.net.InetAddress</code>. If this parameter is not
  180.                         set a default local address will be used automatically.
  181.                 </p>
  182.             </li><li>
  183.                 <p>
  184.                     <b>'http.route.forced-route':&nbsp;</b>
  185.                     defines an forced route to be used by all default route planner. Instead
  186.                         of computing a route, the given forced route will be returned, even if it
  187.                         points to a completely different target host. This parameter expects a value
  188.                         of type <code class="classname">HttpRoute</code>.
  189.                 </p>
  190.             </li></ul></div>
  191.     </div>
  192.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e484"></a>2.5.&nbsp;Socket factories</h2></div></div></div>
  193.        
  194.         <p>HTTP connections make use of a <code class="classname">java.net.Socket</code> object
  195.             internally to handle transmission of data across the wire. They, however, rely on
  196.                 <code class="interfacename">SocketFactory</code> interface to create, initialize and
  197.             connect sockets. This enables the users of HttpClient to provide application specific
  198.             socket initialization code at runtime. <code class="classname">PlainSocketFactory</code> is the
  199.             default factory for creating and initializing plain (unencrypted) sockets.</p>
  200.         <p>The process of creating a socket and that of connecting it to a host are decoupled, so
  201.             that the socket could be closed while being blocked in the connect operation.</p>
  202.         <pre class="programlisting">
  203. PlainSocketFactory sf = PlainSocketFactory.getSocketFactory();
  204. Socket socket = sf.createSocket();
  205.  
  206. HttpParams params = new BasicHttpParams();
  207. params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 1000L);
  208. sf.connectSocket(socket, "locahost", 8080, null, -1, params);
  209. </pre>
  210.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e492"></a>2.5.1.&nbsp;Secure socket layering</h3></div></div></div>
  211.            
  212.             <p><code class="interfacename">LayeredSocketFactory</code> is an extension of
  213.                     <code class="interfacename">SocketFactory</code> interface. Layered socket factories
  214.                 are capable of creating sockets that are layered over an existing plain socket.
  215.                 Socket layering is used primarily for creating secure sockets through proxies.
  216.                 HttpClient ships with SSLSocketFactory that implements SSL/TLS layering. Please note
  217.                 HttpClient does not use any custom encryption functionality. It is fully reliant on
  218.                 standard Java Cryptography (JCE) and Secure Sockets (JSEE) extensions.</p>
  219.         </div>
  220.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e497"></a>2.5.2.&nbsp;SSL/TLS customization</h3></div></div></div>
  221.            
  222.             <p>HttpClient makes use of SSLSocketFactory to create SSL connections.
  223.                     <code class="classname">SSLSocketFactory</code> allows for a high degree of
  224.                 customization. It can take an instance of
  225.                     <code class="interfacename">javax.net.ssl.SSLContext</code> as a parameter and use
  226.                 it to create custom configured SSL connections.</p>
  227.             <pre class="programlisting">
  228. TrustManager easyTrustManager = new X509TrustManager() {
  229.  
  230.     @Override
  231.     public void checkClientTrusted(
  232.             X509Certificate[] chain,
  233.             String authType) throws CertificateException {
  234.         // Oh, I am easy!
  235.     }
  236.  
  237.     @Override
  238.     public void checkServerTrusted(
  239.             X509Certificate[] chain,
  240.             String authType) throws CertificateException {
  241.         // Oh, I am easy!
  242.     }
  243.  
  244.     @Override
  245.     public X509Certificate[] getAcceptedIssuers() {
  246.         return null;
  247.     }
  248.    
  249. };
  250.  
  251. SSLContext sslcontext = SSLContext.getInstance("TLS");
  252. sslcontext.init(null, new TrustManager[] { easyTrustManager }, null);
  253.  
  254. SSLSocketFactory sf = new SSLSocketFactory(sslcontext);
  255. SSLSocket socket = (SSLSocket) sf.createSocket();
  256. socket.setEnabledCipherSuites(new String[] { "SSL_RSA_WITH_RC4_128_MD5" });
  257.  
  258. HttpParams params = new BasicHttpParams();
  259. params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 1000L);
  260. sf.connectSocket(socket, "locahost", 443, null, -1, params);
  261. </pre>
  262.             <p>Customization of SSLSocketFactory implies a certain degree of familiarity with the
  263.                 concepts of the SSL/TLS protocol, a detailed explanation of which is out of scope
  264.                 for this document. Please refer to the <a class="ulink" href="http://java.sun.com/j2se/1.5.0/docs/guide/security/jsse/JSSERefGuide.html" target="_top">Java Secure Socket Extension</a> for a detailed description of
  265.                     <code class="interfacename">javax.net.ssl.SSLContext</code> and related
  266.                 tools.</p>
  267.         </div>
  268.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e506"></a>2.5.3.&nbsp;Hostname verification</h3></div></div></div>
  269.            
  270.             <p>In addition to the trust verification and the client authentication performed on
  271.                 the SSL/TLS protocol level, HttpClient can optionally verify whether the target
  272.                 hostname matches the names stored inside the server's X.509 certificate, once the
  273.                connection has been established. This verification can provide additional guarantees
  274.                of authenticity of the server trust material. X509HostnameVerifier interface
  275.                represents a strategy for hostname verification. HttpClient ships with three
  276.                X509HostnameVerifier. Important: hostname verification should not be confused with
  277.                SSL trust verification.</p>
  278.            <div class="itemizedlist"><ul type="disc"><li>
  279.                    <p>
  280.                        <b><code class="classname">StrictHostnameVerifier</code>:&nbsp;</b>
  281.                        The strict hostname verifier works the same way as Sun Java 1.4, Sun
  282.                            Java 5, Sun Java 6. It's also pretty close to IE6. This implementation
  283.                             appears to be compliant with RFC 2818 for dealing with wildcards. The
  284.                             hostname must match either the first CN, or any of the subject-alts. A
  285.                             wildcard can occur in the CN, and in any of the subject-alts.
  286.                     </p>
  287.                 </li><li>
  288.                     <p>
  289.                         <b><code class="classname">BrowserCompatHostnameVerifier</code>:&nbsp;</b>
  290.                         The hostname verifier that works the same way as Curl and Firefox. The
  291.                             hostname must match either the first CN, or any of the subject-alts. A
  292.                             wildcard can occur in the CN, and in any of the subject-alts. The only
  293.                             difference between <code class="classname">BrowserCompatHostnameVerifier</code>
  294.                             and <code class="classname">StrictHostnameVerifier</code> is that a wildcard
  295.                             (such as "*.foo.com") with
  296.                                 <code class="classname">BrowserCompatHostnameVerifier</code> matches all
  297.                             subdomains, including "a.b.foo.com".
  298.                     </p>
  299.                 </li><li>
  300.                     <p>
  301.                         <b><code class="classname">AllowAllHostnameVerifier</code>:&nbsp;</b>
  302.                         This hostname verifier essentially turns hostname verification off.
  303.                             This implementation is a no-op, and never throws the
  304.                                 <code class="exceptionname">javax.net.ssl.SSLException</code>.
  305.                     </p>
  306.                 </li></ul></div>
  307.             <p>Per default HttpClient uses <code class="classname">BrowserCompatHostnameVerifier</code>
  308.                 implementation. One can specify a different hostname verifier implementation if
  309.                 desired</p>
  310.             <pre class="programlisting">
  311. SSLSocketFactory sf = new SSLSocketFactory(SSLContext.getInstance("TLS"));
  312. sf.setHostnameVerifier(SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
  313. </pre>
  314.         </div>
  315.     </div>
  316.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e532"></a>2.6.&nbsp;Protocol schemes</h2></div></div></div>
  317.        
  318.         <p><code class="classname">Scheme</code> class represents a protocol scheme such as "http" or
  319.             "https" and contains a number of protocol properties such as the default port and the
  320.             socket factory to be used to creating <code class="classname">java.net.Socket</code> instances
  321.             for the given protocol. <code class="classname">SchemeRegistry</code> class is used to maintain
  322.             a set of <code class="classname">Scheme</code>s HttpClient can choose from when trying to
  323.             establish a connection by a request URI:</p>
  324.         <pre class="programlisting">
  325. Scheme http = new Scheme("http", PlainSocketFactory.getSocketFactory(), 80);
  326.  
  327. SSLSocketFactory sf = new SSLSocketFactory(SSLContext.getInstance("TLS"));
  328. sf.setHostnameVerifier(SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
  329. Scheme https = new Scheme("https", sf, 443);
  330.  
  331. SchemeRegistry sr = new SchemeRegistry();
  332. sr.register(http);
  333. sr.register(https);
  334. </pre>
  335.     </div>
  336.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e540"></a>2.7.&nbsp;HttpClient proxy configuration</h2></div></div></div>
  337.        
  338.         <p>Even though HttpClient is aware of complex routing scemes and proxy chaining, it
  339.             supports only simple direct or one hop proxy connections out of the box.</p>
  340.         <p>The simplest way to tell HttpClient to connect to the target host via a proxy is by
  341.             setting the default proxy parameter:</p>
  342.         <pre class="programlisting">
  343. DefaultHttpClient httpclient = new DefaultHttpClient();
  344.  
  345. HttpHost proxy = new HttpHost("someproxy", 8080);
  346. httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
  347. </pre>
  348.         <p>One can also instruct HttpClient to use standard JRE proxy selector to obtain proxy
  349.             information:</p>
  350.         <pre class="programlisting">
  351. DefaultHttpClient httpclient = new DefaultHttpClient();
  352.  
  353. ProxySelectorRoutePlanner routePlanner = new ProxySelectorRoutePlanner(
  354.         httpclient.getConnectionManager().getSchemeRegistry(),
  355.         ProxySelector.getDefault());  
  356. httpclient.setRoutePlanner(routePlanner);
  357. </pre>
  358.         <p>Alternatively, one can provide a custom <code class="interfacename">RoutePlanner</code>
  359.             implementation in order to have a complete control over the process of HTTP route
  360.             computation:</p>
  361.         <pre class="programlisting">
  362. DefaultHttpClient httpclient = new DefaultHttpClient();
  363. httpclient.setRoutePlanner(new HttpRoutePlanner() {
  364.  
  365.     public HttpRoute determineRoute(
  366.             HttpHost target,
  367.             HttpRequest request,
  368.             HttpContext context) throws HttpException {
  369.         return new HttpRoute(target, null,  new HttpHost("someproxy", 8080),
  370.                 "https".equalsIgnoreCase(target.getSchemeName()));
  371.     }
  372.    
  373. });
  374. </pre>
  375.     </div>
  376.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e550"></a>2.8.&nbsp;HTTP connection managers</h2></div></div></div>
  377.        
  378.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e552"></a>2.8.1.&nbsp;Connection operators</h3></div></div></div>
  379.            
  380.             <p>Operated connections are client side connections whose underlying socket or its
  381.                 state can be manipulated by an external entity, usually referred to as a connection
  382.                 operator. <code class="interfacename">OperatedClientConnection</code> interface extends
  383.                     <code class="interfacename">HttpClientConnection</code> interface and define
  384.                 additional methods to manage connection socket. The
  385.                     <code class="interfacename">ClientConnectionOperator</code> interface represents a
  386.                 strategy for creating <code class="interfacename">OperatedClientConnection</code>
  387.                 instances and updating the underlying socket of those objects. Implementations will
  388.                 most likely make use <code class="interfacename">SocketFactory</code>s to create
  389.                     <code class="classname">java.net.Socket</code> instances. The
  390.                     <code class="interfacename">ClientConnectionOperator</code> interface enables the
  391.                 users of HttpClient to provide a custom strategy for connection operators as well as
  392.                 an ability to provide alternative implementation of the
  393.                     <code class="interfacename">OperatedClientConnection</code> interface.</p>
  394.         </div>
  395.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e563"></a>2.8.2.&nbsp;Managed connections and connection managers</h3></div></div></div>
  396.            
  397.             <p>HTTP connections are complex, stateful, thread-unsafe objects which need to be
  398.                 properly managed to function correctly. HTTP connections can only be used by one
  399.                 execution thread at a time. HttpClient employs a special entity to manage access to
  400.                 HTTP connections called HTTP connection manager and represented by the
  401.                     <code class="interfacename">ClientConnectionManager</code> interface. The purpose of
  402.                 an HTTP connection manager is to serve as a factory for new HTTP connections, manage
  403.                 persistent connections and synchronize access to persistent connections making sure
  404.                 that only one thread can have access to a connection at a time.</p>
  405.             <p>Internally HTTP connection managers work with instances of
  406.                     <code class="interfacename">OperatedClientConnection</code>, but they hands out
  407.                 instances of <code class="interfacename">ManagedClientConnection</code> to the service
  408.                 consumers. <code class="interfacename">ManagedClientConnection</code> acts as a wrapper
  409.                 for a <code class="interfacename">OperatedClientConnection</code> instance that manages
  410.                 its state and controls all I/O operations on that connection. It also abstracts away
  411.                 socket operations and provides convenience methods for opening and updating sockets
  412.                 in order to establish a route.
  413.                     <code class="interfacename">ManagedClientConnection</code> instances are aware of
  414.                 their link to the connection manager that spawned them and of the fact that they
  415.                 must be returned back to the manager when no longer in use.
  416.                     <code class="interfacename">ManagedClientConnection</code> classes also implement
  417.                     <code class="interfacename">ConnectionReleaseTrigger</code> interface that can be
  418.                 used to trigger the release of the connection back to the manager. Once the
  419.                 connection release has been triggered the wrapped connection gets detached from the
  420.                     <code class="interfacename">ManagedClientConnection</code> wrapper and the
  421.                     <code class="interfacename">OperatedClientConnection</code> instance is returned
  422.                 back to the manager. Even though the service consumer still holds a reference to the
  423.                     <code class="interfacename">ManagedClientConnection</code> instance, it is no longer
  424.                 able to execute any I/O operation or change the state of the
  425.                     <code class="interfacename">OperatedClientConnection</code> either intentionally or
  426.                 unintentionally.</p>
  427.             <p>This is an example of acquiring a connection from a connection manager:</p>
  428.             <pre class="programlisting">
  429. HttpParams params = new BasicHttpParams();
  430. Scheme http = new Scheme("http", PlainSocketFactory.getSocketFactory(), 80);
  431. SchemeRegistry sr = new SchemeRegistry();
  432. sr.register(http);
  433.  
  434. ClientConnectionManager connMrg = new SingleClientConnManager(params, sr);
  435.  
  436. // Request new connection. This can be a long process
  437. ClientConnectionRequest connRequest = connMrg.requestConnection(
  438.         new HttpRoute(new HttpHost("localhost", 80)), null);
  439.  
  440. // Wait for connection up to 10 sec
  441. ManagedClientConnection conn = connRequest.getConnection(10, TimeUnit.SECONDS);
  442. try {
  443.     // Do useful things with the connection.
  444.     // Release it when done.
  445.     conn.releaseConnection();
  446. } catch (IOException ex) {
  447.     // Abort connection upon an I/O error.
  448.     conn.abortConnection();
  449.     throw ex;
  450. }
  451. </pre>
  452.             <p>The connection request can be terminated prematurely by calling
  453.                     <code class="methodname">ClientConnectionRequest#abortRequest()</code> if necessary.
  454.                 This will unblock the thread blocked in the
  455.                     <code class="methodname">ClientConnectionRequest#getConnection()</code> method.</p>
  456.             <p><code class="classname">BasicManagedEntity</code> wrapper class can be used to ensure
  457.                 automatic release of the underlying connection once the response content has been
  458.                 fully consumed. HttpClient uses this mechanism internally to achieve transparent
  459.                 connection release for all responses obtained from
  460.                     <code class="methodname">HttpClient#execute()</code> methods:</p>
  461.             <pre class="programlisting">
  462. ClientConnectionRequest connRequest = connMrg.requestConnection(
  463.         new HttpRoute(new HttpHost("localhost", 80)), null);
  464. ManagedClientConnection conn = connRequest.getConnection(10, TimeUnit.SECONDS);
  465. try {
  466.     BasicHttpRequest request = new BasicHttpRequest("GET", "/");
  467.     conn.sendRequestHeader(request);
  468.     HttpResponse response = conn.receiveResponseHeader();
  469.     conn.receiveResponseEntity(response);
  470.     HttpEntity entity = response.getEntity();
  471.     if (entity != null) {
  472.         BasicManagedEntity managedEntity = new BasicManagedEntity(entity, conn, true);
  473.         // Replace entity
  474.         response.setEntity(managedEntity);
  475.     }
  476.     // Do something useful with the response
  477.     // The connection will be released automatically
  478.     // as soon as the response content has been consumed
  479. } catch (IOException ex) {
  480.     // Abort connection upon an I/O error.
  481.     conn.abortConnection();
  482.     throw ex;
  483. }
  484. </pre>
  485.         </div>
  486.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e588"></a>2.8.3.&nbsp;Simple connection manager</h3></div></div></div>
  487.            
  488.             <p><code class="classname">SingleClientConnManager</code> is a simple connection manager that
  489.                 maintains only one connection at a time. Even though this class is thread-safe it
  490.                 ought to be used by one execution thread only.
  491.                     <code class="classname">SingleClientConnManager</code> will make an effort to reuse the
  492.                 connection for subsequent requests with the same route. It will, however, close the
  493.                 existing connection and open it for the given route, if the route of the persistent
  494.                 connection does not match that of the connection request. If the connection has been
  495.                 already been allocated
  496.                     <code class="exceptionname">java.lang.IllegalStateException</code> is thrown.</p>
  497.             <p><code class="classname">SingleClientConnManager</code> is used by HttpClient per
  498.                 default.</p>
  499.         </div>
  500.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e596"></a>2.8.4.&nbsp;Pooling connection manager</h3></div></div></div>
  501.            
  502.             <p><code class="classname">ThreadSafeClientConnManager</code> is a more complex
  503.                 implementation that manages a pool of client connections and is able to service
  504.                 connection requests from multiple execution threads. Connections are pooled on a per
  505.                 route basis. A request for a route which already the manager has persistent
  506.                 connections for available in the pool will be services by leasing a connection from
  507.                 the pool rather than creating a brand new connection.</p>
  508.             <p><code class="classname">ThreadSafeClientConnManager</code> maintains a maximum limit of
  509.                 connection on a per route basis and in total. Per default this implementation will
  510.                 create no more than than 2 concurrent connections per given route and no more 20
  511.                 connections in total. For many real-world applications these limits may prove too
  512.                 constraining, especially if they use HTTP as a transport protocol for their
  513.                 services. Connection limits, however, can be adjusted using HTTP parameters.</p>
  514.             <p>This example shows how the connection pool parameters can be adjusted:</p>
  515.             <pre class="programlisting">
  516. HttpParams params = new BasicHttpParams();
  517. // Increase max total connection to 200
  518. ConnManagerParams.setMaxTotalConnections(params, 200);
  519. // Increase default max connection per route to 20
  520. ConnPerRouteBean connPerRoute = new ConnPerRouteBean(20);
  521. // Increase max connections for localhost:80 to 50
  522. HttpHost localhost = new HttpHost("locahost", 80);
  523. connPerRoute.setMaxForRoute(new HttpRoute(localhost), 50);
  524. ConnManagerParams.setMaxConnectionsPerRoute(params, connPerRoute);
  525.  
  526. SchemeRegistry schemeRegistry = new SchemeRegistry();
  527. schemeRegistry.register(
  528.         new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
  529. schemeRegistry.register(
  530.         new Scheme("https", SSLSocketFactory.getSocketFactory(), 443));
  531.  
  532. ClientConnectionManager cm = new ThreadSafeClientConnManager(params, schemeRegistry);
  533. HttpClient httpClient = new DefaultHttpClient(cm, params);
  534. </pre>
  535.         </div>
  536.         <div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="d4e604"></a>2.8.5.&nbsp;Connection manager shutdown</h3></div></div></div>
  537.            
  538.             <p>When an HttpClient instance is no longer needed and is about to go out of scope it
  539.                 is important to shut down its connection manager to ensure that all connections kept
  540.                 alive by the manager get closed and system resources allocated by those connections
  541.                 are released.</p>
  542.             <pre class="programlisting">
  543. DefaultHttpClient httpclient = new DefaultHttpClient();
  544. HttpGet httpget = new HttpGet("http://www.google.com/");
  545. HttpResponse response = httpclient.execute(httpget);
  546. HttpEntity entity = response.getEntity();
  547. System.out.println(response.getStatusLine());
  548. if (entity != null) {
  549.     entity.consumeContent();
  550. }
  551. httpclient.getConnectionManager().shutdown();
  552. </pre>
  553.         </div>
  554.     </div>
  555.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e608"></a>2.9.&nbsp;Connection management parameters</h2></div></div></div>
  556.        
  557.         <p>These are parameters that be used to customize standard HTTP connection manager
  558.             implementations:</p>
  559.         <div class="itemizedlist"><ul type="disc"><li>
  560.                 <p>
  561.                     <b>'http.conn-manager.timeout':&nbsp;</b>
  562.                     defines the timeout in milliseconds used when retrieving an instance of
  563.                             <code class="interfacename">ManagedClientConnection</code> from the
  564.                             <code class="interfacename">ClientConnectionManager</code> This parameter
  565.                         expects a value of type <code class="classname">java.lang.Long</code>. If this
  566.                         parameter is not set connection requests will not time out (infinite
  567.                         timeout).
  568.                 </p>
  569.             </li><li>
  570.                 <p>
  571.                     <b>'http.conn-manager.max-per-route':&nbsp;</b>
  572.                     defines the maximum number of connections per route. This limit is
  573.                         interpreted by client connection managers and applies to individual manager
  574.                         instances. This parameter expects a value of type
  575.                             <code class="interfacename">ConnPerRoute</code>.
  576.                 </p>
  577.             </li><li>
  578.                 <p>
  579.                     <b>'http.conn-manager.max-total':&nbsp;</b>
  580.                     defines the maximum number of connections in total. This limit is
  581.                         interpreted by client connection managers and applies to individual manager
  582.                         instances. This parameter expects a value of type
  583.                             <code class="classname">java.lang.Integer</code>.
  584.                 </p>
  585.             </li></ul></div>
  586.     </div>
  587.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e629"></a>2.10.&nbsp;Multithreaded request execution</h2></div></div></div>
  588.        
  589.         <p>When equipped with a pooling connection manager such as ThreadSafeClientConnManager
  590.             HttpClient can be used to execute multiple requests simultaneously using multiple
  591.             threads of execution.</p>
  592.         <p><code class="classname">ThreadSafeClientConnManager</code> will allocate connections based on
  593.             its configuration. If all connections for a given route have already been leased, a
  594.             request for connection will block until a connection is released back to the pool. One
  595.             can ensure the connection manager does not block indefinitely in the connection request
  596.             operation by setting <code class="literal">'http.conn-manager.timeout'</code> to a positive value.
  597.             If the connection request cannot be serviced within the given time period
  598.                 <code class="exceptionname">ConnectionPoolTimeoutException</code> will be thrown.</p>
  599.         <pre class="programlisting">
  600. HttpParams params = new BasicHttpParams();
  601. SchemeRegistry schemeRegistry = new SchemeRegistry();
  602. schemeRegistry.register(
  603.         new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
  604.  
  605. ClientConnectionManager cm = new ThreadSafeClientConnManager(params, schemeRegistry);
  606. HttpClient httpClient = new DefaultHttpClient(cm, params);
  607.  
  608. // URIs to perform GETs on
  609. String[] urisToGet = {
  610.     "http://www.domain1.com/",
  611.     "http://www.domain2.com/",
  612.     "http://www.domain3.com/",
  613.     "http://www.domain4.com/"
  614. };
  615.  
  616. // create a thread for each URI
  617. GetThread[] threads = new GetThread[urisToGet.length];
  618. for (int i = 0; i &lt; threads.length; i++) {
  619.     HttpGet httpget = new HttpGet(urisToGet[i]);
  620.     threads[i] = new GetThread(httpClient, httpget);
  621. }
  622.  
  623. // start the threads
  624. for (int j = 0; j &lt; threads.length; j++) {
  625.     threads[j].start();
  626. }
  627.  
  628. // join the threads
  629. for (int j = 0; j &lt; threads.length; j++) {
  630.     threads[j].join();
  631. }
  632.  
  633. </pre>
  634.         <pre class="programlisting">
  635. static class GetThread extends Thread {
  636.    
  637.     private final HttpClient httpClient;
  638.     private final HttpContext context;
  639.     private final HttpGet httpget;
  640.    
  641.     public GetThread(HttpClient httpClient, HttpGet httpget) {
  642.         this.httpClient = httpClient;
  643.         this.context = new BasicHttpContext();
  644.         this.httpget = httpget;
  645.     }
  646.    
  647.     @Override
  648.     public void run() {
  649.         try {
  650.             HttpResponse response = this.httpClient.execute(this.httpget, this.context);
  651.             HttpEntity entity = response.getEntity();
  652.             if (entity != null) {
  653.                 // do something useful with the entity
  654.                 // ...
  655.                 // ensure the connection gets released to the manager
  656.                 entity.consumeContent();
  657.             }
  658.         } catch (Exception ex) {
  659.             this.httpget.abort();
  660.         }
  661.     }
  662.    
  663. }
  664. </pre>
  665.     </div>
  666.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e638"></a>2.11.&nbsp;Connection eviction policy</h2></div></div></div>
  667.        
  668.         <p>One of the major shortcoming of the classic blocking I/O model is that the network
  669.             socket can react to I/O events only when blocked in an I/O operation. When a connection
  670.             is released back to the manager, it can be kept alive however it is unable to monitor
  671.             the status of the socket and react to any I/O events. If the connection gets closed on
  672.             the server side, the client side connection is unable to detect the change in the
  673.             connection state and react appropriately by closing the socket on its end.</p>
  674.         <p>HttpClient tries to mitigate the problem by testing whether the connection is 'stale',
  675.             that is no longer valid because it was closed on the server side, prior to using the
  676.             connection for executing an HTTP request. The stale connection check is not 100%
  677.             reliable and adds 10 to 30 ms overhead to each request execution. The only feasible
  678.             solution that does not involve a one thread per socket model for idle connections is a
  679.             dedicated monitor thread used to evict connections that are considered expired due to a
  680.             long period of inactivity. The monitor thread can periodically call
  681.                 <code class="methodname">ClientConnectionManager#closeExpiredConnections()</code> method to
  682.             close all expired connections and evict closed connections from the pool. It can also
  683.             optionally call <code class="methodname">ClientConnectionManager#closeIdleConnections()</code>
  684.             method to close all connections that have been idle over a given period of time.</p>
  685.         <pre class="programlisting">
  686. public static class IdleConnectionMonitorThread extends Thread {
  687.    
  688.     private final ClientConnectionManager connMgr;
  689.     private volatile boolean shutdown;
  690.    
  691.     public IdleConnectionMonitorThread(ClientConnectionManager connMgr) {
  692.         super();
  693.         this.connMgr = connMgr;
  694.     }
  695.  
  696.     @Override
  697.     public void run() {
  698.         try {
  699.             while (!shutdown) {
  700.                 synchronized (this) {
  701.                     wait(5000);
  702.                     // Close expired connections
  703.                     connMgr.closeExpiredConnections();
  704.                     // Optionally, close connections
  705.                     // that have been idle longer than 30 sec
  706.                     connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
  707.                 }
  708.             }
  709.         } catch (InterruptedException ex) {
  710.             // terminate
  711.         }
  712.     }
  713.    
  714.     public void shutdown() {
  715.         shutdown = true;
  716.         synchronized (this) {
  717.             notifyAll();
  718.         }
  719.     }
  720.    
  721. }
  722. </pre>
  723.     </div>
  724.     <div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d4e645"></a>2.12.&nbsp;Connection keep alive strategy</h2></div></div></div>
  725.        
  726.         <p>The HTTP specification does not specify how long a persistent connection may be and
  727.             should be kept alive. Some HTTP servers use non-standard <code class="literal">Keep-Alive</code>
  728.             header to communicate to the client the period of time in seconds they intend to keep
  729.             the connection alive on the server side. HttpClient makes use of this information if
  730.             available. If the <code class="literal">Keep-Alive</code> header is not present in the response,
  731.             HttpClient assumes the connection can be kept alive indefinitely. However, many HTTP
  732.             servers out there are configured to drop persistent connections after a certain period
  733.             of inactivity in order to conserve system resources, quite often without informing the
  734.             client. In case the default strategy turns out to be too optimistic, one may want to
  735.             provide a custom keep-alive strategy.</p>
  736.         <pre class="programlisting">
  737. DefaultHttpClient httpclient = new DefaultHttpClient();
  738. httpclient.setKeepAliveStrategy(new ConnectionKeepAliveStrategy() {
  739.  
  740.     public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
  741.         // Honor 'keep-alive' header
  742.         HeaderElementIterator it = new BasicHeaderElementIterator(
  743.                 response.headerIterator(HTTP.CONN_KEEP_ALIVE));
  744.         while (it.hasNext()) {
  745.             HeaderElement he = it.nextElement();
  746.             String param = he.getName();
  747.             String value = he.getValue();
  748.             if (value != null &amp;&amp; param.equalsIgnoreCase("timeout")) {
  749.                 try {
  750.                     return Long.parseLong(value) * 1000;
  751.                 } catch(NumberFormatException ignore) {
  752.                 }
  753.             }
  754.         }
  755.         HttpHost target = (HttpHost) context.getAttribute(
  756.                 ExecutionContext.HTTP_TARGET_HOST);
  757.         if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {
  758.             // Keep alive for 5 seconds only
  759.             return 5 * 1000;
  760.         } else {
  761.             // otherwise keep alive for 30 seconds
  762.             return 30 * 1000;
  763.         }
  764.     }
  765.    
  766. });
  767. </pre>
  768.     </div>
  769. </div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="fundamentals.html">Prev</a>&nbsp;</td><td width="20%" align="center">&nbsp;</td><td width="40%" align="right">&nbsp;<a accesskey="n" href="statemgmt.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Chapter&nbsp;1.&nbsp;Fundamentals&nbsp;</td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top">&nbsp;Chapter&nbsp;3.&nbsp;HTTP state management</td></tr></table></div></body></html>