Exchange 2010 HAProxy Virtual Load Balancer
An extra added cost to Exchange 2010 deployments is often a hardware load balancer, or even virtual load balancer appliances. These start at over £1000 for some of the cheaper ones and can cost tens of thousands, however there’s open source software out there that can do the same thing, just as well.
HAProxy is a widely used, reliable and stable Load Balancer for Linux and a few weeks ago I began looking at it as an alternative for Exchange 2010 load balancing and whilst having a look to see if anyone had done it before, I found a good article here. The one problem for many Exchange administrators is that they don’t have time to learn about Linux just to try it out, so I began to think that it would be great if someone made a virtual appliance (just like many of the HLB vendors sell) with an easy to use management interface aimed squarely at Exchange 2010 environments.
The Exchange 2010 HAProxy VLB Appliance is a free Layer 4 based virtual load balancer that runs in VMware or Hyper-V environments. It doesn’t require Linux knowledge to get up and running and is managed using a simple, easy to use, web-based management interface (screenshots below). For the initial release it’s not aimed at your production environment yet but as more people test it out and help refine it, future versions will be (and will be free, naturally).
Prerequisites
Before you start, you need to have an understanding of how the load balancer fits in your environment. Typically clients will connect to it for web services, like OWA, and also through Outlook via MAPI, using a Client Access Array as illustrated by this simplified diagram:
In addition to planning your environment, you’ll also need some other information for the load balancer setup:
- IP address for management
- IP address for the virtual load balancer interface, in the same subnet
- DNS and NTP server addresses
- Client Access Server IP addresses
- Network Access from the virtual load balancer interface to the following TCP ports on the Client Access Servers:
- 80 and 443 for HTTP/HTTPS
- 135,139,6001-6004,60000 and 60001 for RPC Client Access
Note that if you’re testing this in a perimeter network, you only need ports 80 and 443 open for external access to Exchange servers.
Installation
The process for installation is fairly straightforward:
- Download and import the appliance
- Boot it up and set the management IP address via the console
- Visit the web-based management interface and set a password, load balanced virtual IP address, set a few details like time zone and DNS server and finally add your first client access server.
- Log in and add your other client access servers and follow instructions within the management interface as to how to set up static RPC TCP/IP ports on your client access servers.
The following videos show the installation and initial setup procedure both for VMware and Hyper-V environments:
Management
The management interface is intended to be fairly simple. After initial setup and login, you should (after the settings have taken effect) basic statistics for the underlying HAProxy load balancer, showing the number of sessions and state of the Client Access Servers:
The initial version uses Layer 4 load balancing, and uses the client source IP address for client affinity and doesn’t have intelligent application-level monitoring and SSL offload (yet). Therefore you just need to correctly configure static RPC ports (RPC Client Access, port 60000 and Address Book Service, port 60001) on the client access servers and add the IP addresses of each client access server to load balance:
On the remaining tabs, you have access to change the management and load balancer IP addressing, set the time zone and NTP servers, update the management interface password and check the logs for the management interface and underlying software that propagates changes made through the UI:
Should you wish, you can also log in at the console using the root password credentials you’ll set on initial startup. From here, it’s a standard minimal Ubuntu installation though any changes to settings that are usually managed through the UI will be overwritten the next time they are changed through the UI:
To actually send traffic to the load balancer, you need to consider the configuration of your environment and have setup your Client Access Array then the associated DNS names for web and RPC Client Access.
Download
Version 0.1 (initial release) is available for download here:
VMware vSphere Compatible OVF file, zipped (md5sum: b60388c5aa1012abe71f5864e79a6828)
Hyper-V compatible VHD, zipped (md5sum a9ae7f9b498f96a4d6d1bb58c4c542ee)
To check md5sum values, use Microsoft File Checksum Integrity Verifier
Notes
This is the first version, so just to repeat it’s only aimed at use in your lab environment.
It’s intended that with subsequent versions it will be production ready, as this is totally aimed at being an easy to use free alternative to paid-for hardware and virtual load balancers for Exchange 2010. It needs a few extra features but most importantly it needs your feedback and testing in the wild to ensure the management interface is good enough, and to get some ideas back from the field on what sort of load it can handle in it’s current form.
One thing I can’t guarantee is support for this – comments and reports of bugs are always appreciated, but the downside of free is it doesn’t come with a support contract. However the intention is to keep developing this and add other features to the so it can compete with the expensive equivalents.
Issues
Currently, the downloadable registry file for setup of static RPC ports does not work as expected. I would suggest following the link to the technet wiki on the setup pages for instructions on manually making the changes on each Client Access Server.



September 4th, 2011 - 08:27
Hi Stevie,
great Job, but I miss the settings for stickyness, which are so important when load balancing Exchange “Services”.
Bernd
September 4th, 2011 - 08:42
Hiya,
It does have sticky persistence, based on source-IP, which can’t be changed in this version. There will be options to configure this and switch to other methods eg cookie persistence in subsequent versions.
Steve
September 4th, 2011 - 10:27
Ok. Normally the LBs with their IP address have to be the gateway on the Exchange Servers. Is that not the case with your implementation?
As far as I understand the LB is pre configured for ports 80,443, 135 (MAPI CAS), 6000, 6001 an the non SSL POP3 and IMAP ports. How about port 25 to load balance HT Servers for incoming mail?
Bernd
September 4th, 2011 - 10:39
Hi Bernd,
At the moment, this isn’t how it’s setup. It was originally my intention but I wanted to start simple first with a “dev” version to garner feedback and add features incrementally.
I’ve not added LB for inbound SMTP as the current version will give the source IP to the Hub Transport as the LB which could have unintended consequences, like RBLs not working. I was instead intending to ship the appliance with an MTA built in that when switched on functions like a mini Edge Transport server.
Steve
September 4th, 2011 - 10:57
Regarding to LB for SMTP, what do you exactly mean with “source Ip”? Normally customers have mail hygiene appliances in their DMZ and the SMTP traffic goes from this appliances to the Exchange 2010 SMTP Receive Connectors. So, if the IP address of this appliances will get through the LB to the Receive Connectors everything would be fine ….
Bernd
September 4th, 2011 - 11:09
In that scenario, you are right. As it’s not a one-size-fits-all I wanted to consider this separately once I have feedback on what people need.
September 4th, 2011 - 11:17
For small and midsize companies I normally recommend Kemp Technologies HLBs and all installations I did so far had a virtual service for incoming SMTP and my customers love it. So I think this is an issue.
Bernd
September 4th, 2011 - 11:29
OK well I will add it as a LB service then.
September 4th, 2011 - 11:37
Great!!! Have a nice Sunday
Bernd
September 4th, 2011 - 12:44
BTW, have a look and I hope you can read german
http://tinyurl.com/3zg63fm
September 5th, 2011 - 13:52
Hi Steve,
Great work, do let me know if you need help with this project. I thank you for pointing a link to my blog
appreciate it…
This is a very good functionality. I have written a PHP script for the management of the HA proxy for Exchange but sadly it isn’t made in a VM format yet … good thinking and great job.
– Alok
September 5th, 2011 - 22:02
Hi Alok (or is it Thunder Emperor
)
Thanks for the offer and thanks for the great article, After reading it, it was is the reason I wrote this management interface and put together the VM – to bring HAproxy to people who aren’t confident with Linux. Depending on how popular this is, I may well appreciate some assistance and as the codebase grows I’ll end up putting the PHP bits into Sourceforge or similar.
I’m still learning about HAProxy, but next I want to look at the SSL offload/cookie based affinity and more intelligent service availability, along with passing through the source IP (using the LB as a gateway, like some other HLBs). I’m also thinking of using Heartbeat (little rusty with it, been 3 or 4 years since I last used it in anger) to implement the HA ability.
Steve
September 6th, 2011 - 04:49
Oh Cool .. I have written another article with Cookie Based persistence for https and http (for OWA) and SSL offload using Pound. (Since HAProxy itself doesn’t offload the SSL) . I have written an small blog (http://3-4-5-6.blogspot.com/2011/08/haproxy-load-balancer-with-ssl.html )
My HAProxy implementation is in HA Pair (using linux-ha) and it works great, we may incorp that into the design for a production level system.
BTW, the deployment I did was a production system for about 2K users
- Alok
P.S_ Thunder emperor is just my screen name
Stole it off Get Backers
September 13th, 2011 - 10:57
Hello Stevie
This is well done job
Can i ask very stupid question?
Why cant my Outlook 2007 connect to the CAS server via Load Balancer? Am I missing somethin?
Thank you
September 13th, 2011 - 22:30
Hi Marin,
What kind of error are you seeing? Is the DNS setup correctly, Client Access Array setup to send clients to the Load Balancer rather than Client Access Servers themselves?
Steve
September 14th, 2011 - 13:27
Hello again
Thanks for answering to my question.
In my company there are 2 Exchange 2010 servers, each of them are CAS servers.
And the error is:
“The connection cannot be completed. The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action!
That is it
September 14th, 2011 - 16:19
Hi Marin,
It’s best if you mail me about it so I can ask a few more questions. Email is steve@goodman.net
Steve
September 15th, 2011 - 12:57
Hi Stevie, do i something wrong? when i start the vhd in my Hyper-V Server, i can not login to the console. i tried it with root or setup and without pass, but unable to login. any idea?
September 15th, 2011 - 13:00
Try username root, password setup
Steve
September 15th, 2011 - 15:54
Thanks
awesome
for first version it looks amazing
September 16th, 2011 - 22:24
have a look at using tproxy with haproxy, you can then get around the issue of the connection appearing to be from the haproxy instead of the actual source.
September 19th, 2011 - 20:49
Arne,
If you do that, then the HA Proxy must be inline with the devices to function. If you want to keep the HA Proxy in a “One Arm” mode, then tproxy will not work
, but yes … in inline deployments (which I dont suggest all that much) it is possible.
For the restriction of SMTP for the relays and such you can apply the ACL on the HAproxy itself …
September 19th, 2011 - 21:35
Thanks both – that is a good suggestion, I had been wondering what would be the best way to give an option for a inline deployment (the way Kemp, for example, recommend setup).
Does sound like I need to provide the option to do both, though.
Steve
September 23rd, 2011 - 07:05
I would say so… Most people wanting to put in a LB would prefer a one arm mode (A lot of people don’t want the LB to be the default gateway of the CAS servers) … Unless you have 2 different NIC’s, even so only one default gateway is permitted.
In any event, lets work for the worst case if both the Load balancers crash, we should still be able to quickly change the DNS and repoint to one of the CAS servers, if we need this, we will need (One -Arm Mode)
If we want to keep inline, then we will need the tproxy for the IP address to be transparent
September 28th, 2011 - 09:27
This looks pretty awesome! You mention that you don’t recommend using this in a production environment. Is it to buggy, unreliable, missing features or just that it’s not done yet? Are you still developing this project?
September 28th, 2011 - 11:37
Hi Patric,
It’s an initial reslease, so it’s best to say it’s not done yet. As it’s not “supported” in the traditional sense and makes use of other open source products it needs a community of testing before it can be gauged as reliable enough to host production workloads. The downside with something free like this is if it goes wrong there’s no-one to call on, so to mitigate that my intention is to make it as simple as possible (it breaks, throw a new one in) and have a concensus from people who are using it in test/dev that it’s reliable.
Steve
September 28th, 2011 - 15:47
Do you have the app available without the OS, as we could not seem to import this into XenServer 5.6. It keeps dropping out to BusyBox due to no driver support for Xen. I have both Debian Squeeze, and Ubuntu Natty virtuals running that I could add this to.
September 28th, 2011 - 15:55
Hiya,
At the moment it’s kind of specific to that VM (which is Ubuntu) so it expects things a certain way.
If you want to try setting it up yourself you may be better checking out the blog that inspired this – obviously there is no web GUI but if you are comfortable with Linux it’s pretty straightforward to setup:
http://3-4-5-6.blogspot.com/2011/03/ha-proxy-for-exchange-2010-deployment.html
Steve
October 20th, 2011 - 17:19
Nice work so far!
I didn’t see an option to drain connections to a host for maintenance, etc reasons. I take it if as an alternative we delete a CAS from the server pool that it drops all active sessions immediately?
October 23rd, 2011 - 00:11
Yes that should do, but I admit it would be useful to be able to temporarily stop a node rather than delete it
Steve
October 21st, 2011 - 09:01
Whats the root password for the appliance? Iv just installed it but no dice at the console?
October 21st, 2011 - 09:07
It’s ok figured it out…..
October 23rd, 2011 - 00:04
OK, one warning – When you get to it, I would suggest adding the registry entries on CAS via the technet article as the .reg files need updating
Steve
October 25th, 2011 - 17:20
I’m very impressed with your appliance. We are looking to get some exchange load balancing appliances and I came across this project. The one thing we require over what this provides is the ability to cluster two of them together so you don’t have a single point of failure. Is this something that may be added in the future?
November 8th, 2011 - 00:32
Hiya
At the moment it’s only aimed at lab/small environments, but yes it may be added in the future. As it’s only worked on in my (very limited!) spare time, I can’t guarantee anything though.
Steve
October 26th, 2011 - 08:39
Steve noticed one very small bug. MAPI clients are prompted to authenticate during auto discover at the logon process. When clients go directly to the CAS this additional prompt does not occur. Entering the credentials does complete the autodiscover process but the appliance doesn’t seem to parse the credentials as normal during MAPI profile creation.
If you need any help with testing more than happy to help. Spent many years working with Ubuntu and am very familiar with the distribution.
October 30th, 2011 - 08:23
Shawn,
I am assuming, you have set static ports, but may be only for the MAPI connection
The only time this will happen is if you have not set the RPC ports statically for the Address book. Again, depending on the Version, there is a Reg Key that we need to change (or modify a text file). If that is done, you will not have the password prompt for the Auto discover as well.
HAProxy does very well with selected number of ports, that’s why we will have to restrict the RPC ports (in this case 60K and 60K1)
Please read
http://social.technet.microsoft.com/wiki/contents/articles/configuring-static-rpc-ports-on-an-exchange-2010-client-access-server.aspx
After you have made the reg file/text file based on the above link (and rebooting the CAS / HT)
from your exchange CAS environment go to command prompt and type
netstat -an | find “LISTENING”
and ensure the 2 ports that you opened are shown as listening . After this the LB should work flawlessly
If you still need assistance, just shout at http://3-4-5-6.blogspot.com/2011/03/ha-proxy-for-exchange-2010-deployment.html and I will try and assist ….
November 8th, 2011 - 00:20
Thanks Alok for replying – I have been away from the blog for a couple of weeks and only just seeing these.
Steve
November 8th, 2011 - 00:26
Hi Shawn,
Ubuntu is just the linux distribution.. It could be any, and it was solely chosen because it had Hyper-V kernel modules built in. I’ve not seen this issue in my testing but I would follow Alok’s advice on checking everything is setup correctly on the CAS servers. The downloadable registry key (and I will update the page to state this) doesn’t work correctly so I would follow the link to the Technet wiki to apply the correct setting.
Steve
November 3rd, 2011 - 22:03
Hi Steve,
Wanted to try this out in lab, but don’t have vsphere but the good old fashioned vmware server, would vmware server not work? I noticed I can’t use the file or convert using the converter, do I have to actually have to take a physical machine and install esxi to make this work?
November 8th, 2011 - 00:17
Alok’s tip will work, but you should also be able to install the free VMware Convertor to import the OVF and then import it to a file share as a VMDK.
If you have probs let me know and I will convert it to an earlier version suitable for use with VMware Server 2 and upload as a ZIP’ed VMDK.
Steve
November 7th, 2011 - 19:49
Damian,
Just to answer your question, you can install “ESXi” on to a VMWare Workstation, that should help
Once you have booted, you can convert the format, or export that in another format
-Alok
November 9th, 2011 - 20:11
Great job!
Do you plan to launch a version compatible with IPV6?
November 10th, 2011 - 12:43
Not at the moment, though ha-proxy does support it, so it should be straightforward to install and configure on Ubuntu if you are familiar with Linux
Steve
November 21st, 2011 - 16:59
Hi,
can you add the option to have more than one HAProxy (Load Balancer) IP Configuration?
For example:
server A, server B, server C, server D.
LB-1 –> server A, server B
LB-2 –> server C, server D.
Duckie
November 30th, 2011 - 23:28
Hiya,
With HA Proxy you could indeed configure such a setup – it’s a bit more complicated than the VM appliance can do but it’s well within the capabilities of HAProxy itself.
Steve
November 24th, 2011 - 08:12
Duckie, why would you want that … In order to do what you are asking, just deploy two of the devices. If you re looking for redundancy then thats not the way to go …
November 25th, 2011 - 10:54
Duckie, I have written a blog post http://3-4-5-6.blogspot.com/2011/11/haproxy-for-exchange-2010part-2.html .. detailing how to make a redundant solution
November 30th, 2011 - 20:33
hi,
redundant is great.
what i’m try to do is, when you have two or more CAS servers & two or more HUB servers.
so one HAProxy VIF or CAS and and second VIF for HUB.
it is also good if you have Sharepoint servers and etc….
Duckie
November 30th, 2011 - 23:44
If you’re doing seperate services then a single HAproxy (or load balanced one as per Alok’s article) can still do this. It’s just this pre-packaged LB is not aimed at more advanced scenarios, it’s hoped for those you will explore more about what HAproxy has to offer. Basically whatever you can do with a Kemp or similar can be done with HAproxy, it just needs to be done via config files.
Steve
December 1st, 2011 - 06:15
Thank Steve.
I’ll go to production with HAproxy in the next few week, and let you know the results.
Eran
December 3rd, 2011 - 19:33
Duckie … You can create more than one virtual server on HA Proxy… You should be able to add another sub-interface
Interface settings
iface eth0:1 inet static
address 10.10.10.11
netmask 255.255.255.0
network 10.10.10.0
broadcast 10.10.10.255
gateway 10.10.10.1
auto eth0:2
iface eth0:2 inet static
address 10.10.10.15
netmask 255.255.255.0
network 10.10.10.0
broadcast 10.10.10.255
gateway 10.10.10.1
In the haproxy.cfg (which should be at /etc/haproxy/haproxy,cfg)
You can add it like this
bind 10.10.10.11:110,10.10.10.11:135
bind 10.10.10.11:139,10.10.10.11:443
bind 10.10.10.11:60000,10.10.10.11:60001
bind 10.10.10.11:6001-6004
bind 10.10.10.11:993-995
mode tcp
option persist
balance roundrobin
stick-table type ip size 10240k expire 30m
stick on src
server HC-CAS1 10.10.10.20 weight 1 check port 80 inter 5000 rise 2 fall 3
server HC-CAS2 10.10.10.21 weight 1 check port 80 inter 5000 rise 2 fall 3
option redispatch
option abortonclose
maxconn 40000
listen Exchange2010-HubT 10.10.10.15:25
mode tcp
option persist
balance roundrobin
stick-table type ip size 10240k expire 30m
stick on src
server HC-CAS1 10.10.10.23 weight 1 check port 80 inter 5000 rise 2 fall 3
server HC-CAS2 10.10.10.24 weight 1 check port 80 inter 5000 rise 2 fall 3
option redispatch
option abortonclose
maxconn 40000
You can see that there are 2 Virtual servers load balancing 4 different servers
VIP : 10.10.10.11 is load balancing 10.10.10.20 and 10.10.10.21
VIP : 10.10.10.15 is load balancing 10.10.10.23 and 24
This way you can put the instances.
Let me know if you have questions
December 1st, 2011 - 09:24
Ina situation where i have 2 hub/cas role installed servers can i seperate the CAS and smtp traffic through dns or how can i use the haproxy ?
December 4th, 2011 - 02:55
What is default management port? I zipped through the setup process.. I know the ip using ifconfig but how do I find the listing port.. I tried the netstat command to see a “listening” port. I will try going into the config text file now.. Please comment or update article for management port.
December 4th, 2011 - 04:09
Hi Tong,
Off the top of my head, I think it’s port 8080
Steve
December 4th, 2011 - 04:15
Thx Steve,
It was 8080. I assume once I set the listening ip, I just have to set my exchange up with the new cas array command and set correct ports? We appreciate your work here. I was about to try to setup ubuntu and haproxy without any prior linux experience. @_@ You saved us hours.
December 7th, 2011 - 23:41
Download Links are dead? Anyone have a mirror
December 8th, 2011 - 18:15
Hiya,
Give them a go now, my hosting provider had issues.
Steve
January 9th, 2012 - 14:50
Hi Steve,
thank you for sharing.
is this software need to work in network card promisc mode?
February 1st, 2012 - 18:36
Hi Bugra,
No it doesn’t.
Steve
January 22nd, 2012 - 21:45
Hi Steve, great product, works wonderfully, thanks
is there any way i can change the ports to support OCS
I am presuming this article i found is using your product?
http://blog.loadbalancer.org/load-balancing-microsoft-office-communications-server-ocs-with-haproxy/
Thanks
February 1st, 2012 - 18:25
Hi Martin,
I’ve got your email about this – Hopefully in the next release I can make it a bit more customizable for these purposes.
Steve
February 3rd, 2012 - 16:11
Got A simple Question, why use v load balancing vs WNLB?
February 3rd, 2012 - 20:22
Simple answer David,
Windows NLB creates havoc in the network, due to the association of multicast MAC for unicast IP, and the network devices don’t like it at all, so even MS suggest that we use a 3rd Party LB instead of the native NLB.
Since 3rd party loadbalancers are expensive, we created a free solution
Hope this answers it!!!
February 3rd, 2012 - 22:22
Was about to answer with much of the same, but that sums it up pretty neatly!
Only thing to add – here’s it from the horses mouse (MS) saying why they prefer LB for Exchange over NLB:
http://www.stevieg.org/2010/11/exchange-team-no-longer-recommend-windows-nlb-for-client-access-server-load-balancing/
Steve
February 7th, 2012 - 15:40
Hmmm WNLB has been a cost effective & viable solution with reasonable limitations for my deployments over the last, While id doesnt provide Application aware Load Balancing, it has proven Effective in 3 Of my Environments… Process has kept WLNB around for sometime, the truth of the matter is; drain stopping a host after a few calls to our service desk normally results into an isolated issue with a particular Host. We may take a hit on the service desk calls, however we tend to identify problems immediately. I have yet to have the need to expand to a 8 node configuration..
However after reading this post it sparked an interest..
Thanks Alok & Steve….
Still looking for some serious incentive to buy some new toys
February 13th, 2012 - 22:05
Steve two questions: 1) how well do you believe this appliance scales to? In other words how many concurrent users do you think it can have before it falls over.
2) How long does it take to fail over MAPI clients? As when i take a server offline it doesn’t fail them over at all?
February 19th, 2012 - 01:29
Hiya,
1) It should scale with the underlying hardware (vCPUs, memory)
2) Not sure, it should be instant.
Steve