• Cloud-Native Threading & Logging
• HTTP Representation (HTX)
• End-to-End HTTP/2
• gRPC
• Layer 7 Retries
• Data Plane API
• Process Manager
• Polyglot Extensibility
• Traffic Shadowing
• Kubernetes Ingress Controller
• Prometheus Exporter
• Peers & Stick Tables Improvements
• Power of Two Random Choices Algorithm
• Log Distribution & Sampling
• Built-in Automatic Profiling
• Enhanced TCP Fast Open
• New Request Actions
• New Converters
• New Fetches
• Miscellaneous Improvements
• LTS Support for 1.9 Features

Cloud-Native Threading & Logging

Tuning HAProxy for optimal performance is now even easier. Since version 1.8, you’ve been able to set the nbthread directive to instruct HAProxy to operate across multiple threads, which allows you to make better use of multi-core processor machines. HAProxy now automatically configures this for you. It will, out of the box, set the number of worker threads to match the machine’s number of available CPU cores. That means that HAProxy can scale to accommodate any environment with less manual configuration.

You can still configure this yourself with the nbthread directive, but this makes the task simpler. It also removes the burden of tuning this in cloud environments where machine instance sizes may be heterogeneous. On systems where HAProxy cannot retrieve the CPU affinity information it will default to a single thread.

This also simplifies the bind line as it no longer requires you to specify a processsetting. Connections will be distributed to threads with the fewest active connections. Also, two new build parameters have been added: MAX_THREADS and MAX_PROCS, which avoid needlessly allocating huge structs. This can be very helpful on embedded devices that do not need to support MAX_THREADS=64.

Logging is now easier to adapt for containerized environments. You can log directly to stdout and stderr, or to a file descriptor. Use the following syntax:

HTTP Representation (HTX)

The Native HTTP Representation (HTX) was introduced with HAProxy 1.9 and it laid the foundation that will allow HAProxy to continue to provide best-in-class performance while accelerating cutting-edge feature delivery for modern environments. Many of the latest features, such as end-to-end HTTP/2, gRPC, and Layer 7 retries, are powered by HTX.

HTX creates an internal, native representation of the HTTP protocol(s). It creates strongly typed, well-delineated header fields and allows for gaps and out-of-order fields. Modifying headers now consists simply of marking the old one as deleted and appending the new one to the end. This provides easy manipulation of any representation of the HTTP protocol, allows HAProxy to maintain consistent semantics from end-to-end, and provides higher performance when translating HTTP/2 to HTTP/1.1 or vice versa.

With HTX in place, any future HTTP protocols will be easier to integrate. It has matured since its introduction and starting in 2.0 it will be enabled by default.

End-to-End HTTP/2

With HTX now being on by default, HAProxy officially supports end-to-end HTTP/2. Here’s an example of how to configure it with TLS offloading. The bind and server lines include the alpn parameter, which specifies a list of protocols that can be used in preferred order:

You can also use HTTP/2 without TLS. Remove any ssl and verify parameters from the server and/or bind lines. Then swap alpn h2 for proto h2 and HAProxy will use only the given protocol.

gRPC

HAProxy 2.0 delivers full support for the open-source RPC framework, gRPC. It allows for bidirectional streaming of data, detection of gRPC messages, and logging gRPC traffic. The gRPC protocol is a modern, high-performance RPC framework that can run in any environment. Using Protocol Buffers, it’s able to serialize messages into a binary format that’s compact and potentially more efficient than JSON.

To begin using gRPC in HAProxy, you just need to set up a standard end-to-end HTTP/2 configuration. Here, we’re using the alpn parameter to enable HTTP/2 over TLS:

Standard ACLs apply and allow for path-based matching, as shown:

Additionally, two new converters, protobuf and ungrpc, have been introduced that let you extract the raw Protocol Buffers messages.

Layer 7 Retries

Reducing downtime often involves having smart contingency mechanisms in place. HAProxy has, since its inception, supported retrying a failed TCP connection by including the option redispatch directive. With HAProxy 2.0, it can also retry from another server at Layer 7 for failed HTTP requests. The new configuration directive, retry-on, can be used in a defaults, listen, or backend section. The number of attempts at retrying can be specified using the retries directive. It is important that you know how your application behaves with Layer 7 retries enabled. Caution must be exercised when retrying requests such as POST requests. In our examples we have disabled POST requests from being retried using http-request disable-l7-retry if METH_POST

It supports a variety of error types to allow for granular control. Otherwise, you can specify all-retryable-errors, which will retry the request for any error that is considered to be retryable. The full list of retry-on options is below:

HAProxy 2.0 also introduces a new http-request action called disable-l7-retry that allows you to disable any attempt to retry the request if it fails for any reason other than a connection failure. This can be useful, for example, to make sure that POST requests aren’t retried.

Here’s an example configuration that activates Layer 7 retries:

Data Plane API

In today’s cloud-native landscape, ephemeral services are born and die quickly, deployments happen continuously, and configuration needs to be refreshed constantly. The new Data Plane API provides a modern REST API for configuring HAProxy on the fly. You can now dynamically add and remove frontends, backends, and servers. You can create ACL rules, insert HTTP routing directives, set IP and port bindings, and much more. The API updates the configuration file as needed, reloading the HAProxy process when necessary.

HAProxy has proven itself to be dynamic and extensible with its built-in Lua support and its Stream Processing Offload Engine. The new Data Plan API takes that one step forward by providing true dynamic configuration management. The API daemon runs as a sidecar process, which HAProxy can manage using the program directive in the new Process Manager. The HAProxy Data Plane API supports transactions, which allow multiple changes to be applied simultaneously. This gives you the ultimate confidence that updates are atomic.

• GitHub: https://github.com/haproxytech/dataplaneapi
• Documentation: https://www.haproxy.com/documentation/hapee/1-9r1/configuration/dataplaneapi/
• API Specification: https://www.haproxy.com/documentation/dataplaneapi/latest/

Process Manager

Several of the exciting innovations happening involve components that run as sidecar processes alongside HAProxy, such as the Data Plane API and any Stream Processing Offload Agents (SPOAs). Clearly, there’s a benefit to having central orchestration to control the lifecycle of these processes.

This release introduces support for the new Process Manager. It allows you to specify external binaries that HAProxy will start and manage directly under its master/worker mode. After enabling master/worker mode by either including the -Ws flag on the command line or adding a master-worker directive to the global section of the HAProxy configuration, you can tell HAProxy to start external programs by using the following syntax:

For example, to have HAProxy handle the start up of the Data Plane API, you would add it as a command in the program section, like this:

You can view a list of running commands by issuing a show proc command to the Runtime API:

Polyglot Extensibility

The Stream Processing Offload Engine (SPOE) and Stream Processing Offload Protocol(SPOP) were introduced in HAProxy 1.7. The goal was to create the extension points necessary to build upon HAProxy using any programming language. The initial examples were all C based. Over time, the community saw a need to show how SPOE can be extended in any language and a variety of libraries and examples were contributed. This opens the door to as many developers as possible.

In collaboration with our community, we’re excited to announce that libraries and examples are available in the following languages and platforms:

• C
• .NET Core
• Golang
• Lua
• Python

Traffic Shadowing

Traffic shadowing, or mirroring, allows you to mirror requests from one environment to another. This is helpful in instances where you would like to send a percentage of production traffic to a testing or staging environment to vet a release before it’s fully deployed. The new Traffic Shadowing daemon is written as a Stream Processing Offload Agent (SPOA) and takes advantage of HAProxy’s SPOE, which allows you to extend HAProxy using any programming language.

The Traffic Shadowing SPOA can be launched and managed using the Process Manager, as shown:

Above, we specified config mirror.cfg on the filter spoe line. Here is an example of how mirror.cfg would look:

Kubernetes Ingress Controller

Since February 2017, an HAProxy Ingress Controller for Kubernetes has been provided by community contributor, Joao Morais. HAProxy Technologies contributed features, such as adding DNS service discovery, and watched the evolution of the project. There’s a need, however, for a community-driven project that’s developed jointly by HAProxy Technologies.

The new HAProxy Kubernetes Ingress Controller provides a high-performance ingress for your Kubernetes-hosted applications. It supports TLS offloading, Layer 7 routing, rate limiting, whitelisting, and the best-in-class performance that HAProxy is renowned for. Ingresses can be configured through either ConfigMap resources or annotations and there’s support for defining secrets for storing TLS certificates.

• GitHub: https://github.com/haproxytech/kubernetes-ingress
• Documentation: https://www.haproxy.com/documentation/hapee/1-9r1/traffic-management/kubernetes-ingress-controller/

Prometheus Exporter

HAProxy now has native support for exposing metrics to Prometheus. Prometheus is an open-source systems monitoring and alerting toolkit that was originally built at SoundCloud. Its adoption has been widespread and it inspires an active community.

To begin using the Prometheus exporter you must first compile HAProxy with the component by using the EXTRA_OBJS variable. An example make command would be:

Activate the exporter within your HAProxy configuration by adding an http-request use-service directive, like so:

Read more about the Prometheus integration on the blog post HAProxy Exposes a Prometheus Metrics Endpoint.

Peers & Stick Tables Improvements

HAProxy allows propagation of stick table data to other HAProxy nodes using the Peers Protocol. HAProxy 2.0 introduces several improvements to the Peers Protocol including:

• Heartbeat
• Stick tables in peers sections
• SSL support
• Runtime API command: show peers
• New stick table counters
• New stick table data type, server_name

A node now sends a heartbeat message to its peers after a three-second period of inactivity. If there isn’t any activity within a five-second period, the peer is considered dead, the connection is closed, and reconnection is attempted.

The peers section has been expanded to allow using the bind, default-bind, server, and default-server configuration directives. It also now supports having stick tables directly within itself. This means that you no longer need to use dummy backends, as previously recommended, when dealing with many different stick tables.

In the following example, we define a stick table directly inside a peers section and encrypt traffic between nodes using SSL:

Within a frontend you would then specify the following:

Using the Runtime API, you can now get information about the various peers connections using show peers.

The stick table counters gpc1 and gpc1_rate are additional, general-purpose counters that can be incremented using configuration logic. A new stick table data type, server_name, was added. It functions the same as server_id except that the server’s name is exchanged over the wire in addition of its ID. To learn how to take advantage of stick tables, check out our blog post Introduction to HAProxy Stick Tables.

Power of Two Random Choices Algorithm

In the 1.9 release, a new load-balancing algorithm was added called random. It chooses a random number as the key for the consistent hashing function. Random load balancing can be useful with large farms or when servers are frequently added or removed, as it may avoid the hammering effect that could result from roundrobin or leastconn in this situation. It also respects server weights and dynamic weight changes and server additions take effect immediately.

The hash-balance-factor directive can be used to further improve the fairness of the load balancing, especially in situations where servers show highly variable response times.

When setting balance to random, the argument <draws> indicates that HAProxy should draw that many random servers and then select the one that is least loaded. Drawing two servers even has a name: the Power of Two Random Choices algorithm.

Specify Power of Two load balancing within your backend as follows:

You can read more about this in our blog post Test Driving “Power of Two Random Choices” Load Balancing.

Log Distribution & Sampling

When dealing with a high volume of logs, sampling can be extremely beneficial, giving you a random insight into the traffic. Typically, this sampling would need to be performed by a syslog server such as rsyslog. With HAProxy 2.0, you can now do sampling directly within HAProxy by using the log directive’s sample parameter. Multiple log and sampledirectives can be specified simultaneously.

To get started, configure logging as follows:

The first log line configures all local0 logs to be sent to stderr. The second log line configures logging to 127.0.0.1:10001 at a sampled rate. One out of 10 requests would be logged to this source. Sending 100 requests while incrementing the URL parameter i results in the following log entries:

The third log line configures logging to 127.0.0.2 on port 10002 at a sampled rate. For every 11 requests, it will log requests 2, 3, and 8-11. Sending 100 requests while incrementing the URL parameter i results in the following log entries:

Built-in Automatic Profiling

HAProxy now features the profiling.tasks directive, which can be specified in the global section. It takes the parameters auto, on, or off. It defaults to auto.

When set to auto, the profiling automatically switches on when the process starts to suffer from an average latency of 1000 microseconds or higher, as reported in the avg_loop_usactivity field, and automatically turns off when the latency returns below 990 microseconds. This value is an average over the last 1024 loops. So, it does not vary quickly and tends to smooth out short spikes. It may also spontaneously trigger from time to time on overloaded systems, containers, or virtual machines, or when the system swaps—which must absolutely never happen on a load balancer.

To view the activity you can use the show activity Runtime API command, as shown:

To view the status of profiling, use the show profiling Runtime API command:

Profiling exposes the following fetches, which can be captured in the HAProxy log:

To use these in the logs, you would either extend the default HTTP log-format, like so:

Or, extend the default TCP log-format:

Enhanced TCP Fast Open

HAProxy now has end-to-end support for TCP Fast Open (TFO), enabling clients to send a request and receive a response during the TCP three-way handshake. The benefit of this is that you save one round-trip after the first connection.

HAProxy has supported TFO on the frontend since version 1.5. Version 2.0 enhances this by adding TFO for connections to backend servers on systems that support it. This requires Linux kernel 4.11 or newer. Add the tfo parameter to a server line.

Be sure to enable retries with the retry-on directive or the request won’t be retried on failure.

New Request Actions

As a part of this release, several new http-request and tcp-request actions were introduced. Here is a breakdown of these new actions with their descriptions.

http-request replace-uri <match-regex> <replace-fmt>This matches the regular expression in the URI part of the request according to <match-regex> and replaces it with the <replace-fmt> argument.

The http-request do-resolve and tcp-request do-resolve warrant further explanation. They allow you to resolve a DNS hostname and store the result in an HAProxy variable. Consider the following example:

Here, we’re using http-request do-resolve to perform a DNS query on the hostname found in the Host request header. The nameserver(s) referenced in the mydns resolverssection (not shown) will return the IP address associated with that hostname and HAProxy will then store it in the variable txn.dstip. The http-request set-dst line in the be_mainbackend updates the server address with this variable.

This is beneficial in split horizon DNS environments, wherein the DNS server will return different results, such as publicly routable or internal-only addresses, depending on the client’s, which is the load balancer’s, source IP address. So, you could have Dev and Prodload balancers that receive different DNS records when they call do-resolve. This is much more dynamic than the at-runtime DNS resolution available in HAProxy (i.e. using the resolvers parameter on the server line), which is typically set to hold onto a DNS result for a period of time. As such, it’s also suitable for other scenarios involving highly dynamic environments, such as where upstream servers are ephemeral.

New Converters

Converters allow you to transform data within HAProxy and are usually followed after a fetch. The following converters have been added to HAProxy 2.0:

New Fetches

Fetches in HAProxy provide a source of information from either an internal state or from layers 4, 5, 6, and 7. New fetches that you can expect to see in this release include:

Miscellaneous Improvements

The following miscellaneous improvements have been made:

• SSL/TLS Ticket Keys
  • TLS session tickets help to speed up session resumption for clients that support them. HAProxy 2.0 adds support for AES256-bit ticket keys specified in both a file or through the Runtime API.
• Core Dump – Ease of Use
  • A new global directive set-dumpable has been added, which aids in enabling core dumps. It’s been known to be a pain to get a core dump when enabling the user and group settings, which disables the dumpable flag on Linux, when using a chroot and/or when HAProxy is started by a service management tool that requires complex operations to just raise the core dump limit. This directive makes it much easier to retrieve a core file.
• SOCKS4 Support
  • Introduces 2 new server keywords: socks4 and check-via-socks4 which can be used for communicating with servers within a backend over SOCKS4 and adds similar functionality for health checking over SOCKS4.

LTS Support for 1.9 Features

HAProxy 2.0 bring LTS support for the aforementioned features, as well as the following features that were introduced or improved upon during the 1.9 release:

• Small Object Cache with an increased caching size up to 2GB, set with the max-object-size directive. The total-max-size setting determines the total size of the cache and can be increased up to 4095MB.
• New fetches that report either an internal state or from layer 4, 5, 6, and 7.
• New converters that allow you to transform data within HAProxy.
• HTTP 103 (Early Hints), which asks the browser to preload resources.
• Server Queue Priority Control, which lets you prioritize some queued connections over others.
• Connection pooling to backend servers.
• The resolvers section supports using resolv.conf by specifying parse-resolv-conf.
• The busy-polling directive allows the reduction of request processing latency by 30-100 microseconds on machines using frequency scaling or supporting deep idle states.
• Lua
  • The Server class gained the ability to change a server’s maxconn value.
  • The TXN class gained the ability to adjust a connection’s priority within the server queue.
  • There is a new StickTable class that allows access to the content of a stick-table by key and allows dumping of content.
• Regression testing of the HAProxy code using varnishtest.

HAProxy 2.1 Preview

HAProxy 2.1 will build on the foundation that has been laid in HAProxy 1.9 and 2.0. Some of the exciting features planned are:

• UDP Support
• OpenTracing
• Dynamic SSL Certificate Updates

Conclusion

HAProxy remains at the forefront of performance and innovation because of the commitment of the open-source community and the staff at HAProxy Technologies. We’re excited to bring you this news of the 2.0 release! In addition to the features included in this version, it paves the way for many exciting updates, which, with our new release cadence, you’ll see more frequently.

It immediately brings support for end-to-end HTTP/2, gRPC, Layer 7 Retries, traffic shadowing, connection pooling on the server side, a Process Manager, the Power of Two Random Choices Algorithm, and a Prometheus Exporter. Of course, one of the most powerful additions is the new Data Plane API, which allows you to dynamically configure HAProxy using RESTful HTTP calls.

return Proxy.newProxyInstance(connection.getClass().getClassLoader(), Connection.class.getInterfaces(), new InvocationHandler() {

        public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
            Object result = null;
            if(method.getName().equals("close")){
                //获取连接池
                Set<Connection> set = DBCPMyPoolUtil.getSet();

                if(set.size()< maxIdle){ //检查连接数是否达到最大保存连接数值
                    //如果没有达到可保存数，将连接放入连接池
                    set.add(connection);
                    System.out.println("回收连接: "+connection);
                } else {
                    //如何连接数达到最大保存值，直接用close方法关闭连接，该连接不再被重复使用，更不会被保存到连接池中
                    if(connection!=null){
                        //此时method方法为close直接关闭method
                        result = method.invoke(connection,args);
                    }
                    System.out.println("关闭conn: "+connection);
                }
                int active = DBCPMyPoolUtil.getActive();
                active--;
                DBCPMyPoolUtil.setActive(active);

            } else {
               result = method.invoke(connection,args);
            }
            return result;
        }
    });

报错：java.lang.ClassCastException: com.sun.proxy.$Proxy4 cannot be cast to java.sql.Connection

在使用动态代理增强Connection连接对象的close方法时，我碰到了如题所示的异常。通过搜索我发现这个异常出现的原因在于我使用的mysql数据库驱动的问题，由于数据库驱动不同，Connection.class.getInterfaces()返回的结果也不同，它返回的是一个Class[]数组，然而此数组的第一个元素必须是Connection才能把创建的代理类转为Connection对象，否则就会报错。

解决方案：
　　所以这里我们可以采取一个替代方式替换Connection.class.getInterfaces()，即new Class[] { Connection.class }，这样无论数据库驱动是什么版本的驱动，都能保证这个类型转换不出错。

return Proxy.newProxyInstance(connection.getClass().getClassLoader(), new Class[] { Connection.class }, new InvocationHandler() {
...
}