Welcome to ISAserver.org

Why use Multicast NLB?

Users viewing this topic: none

Logged in as: Guest

Printable Version

All Forums >> [ISA 2006 Firewall] >> Network Infrastructure >> Why use Multicast NLB?

Page: [1]

Message

<< Older Topic Newer Topic >>

Why use Multicast NLB? - 27.Oct.2008 1:51:11 PM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

If I have an ISA array configured to use separate NICs on a private L2 VLAN for intra-array communications, why would I need to enable multicast NLB?

Port flooding is not an issue, because the NLB interfaces for the array are connected to a dedicated L2 VLAN hosted on two Cisco 6509's (HSRP) with L3 routing implemented.

From what I've read, the main benefit of multicast is that it allows the array members to "see" each other. But, since I have intra-array comm going through a separate (non-NLB) interface, and I have HOSTS file entries for the ISA servers that identify the intra-array network address for all array members, would I ever need multicast NLB? If so, why?

Unless I'm missing something, it seems like a lot of work for minimal return (for me, anyway). On top of that, convincing our network team to do something out of the ordinary like setting static ARP entries on the 6509's might just cause them to break out in hives, or something .

Thoughts?

Mike

Post #: 1

Featured Links*

RE: Why use Multicast NLB? - 27.Oct.2008 2:02:09 PM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Hi Mike,

Not everybody can create dedicated VLANs recommended for unicast and hence multicast is a good workaround to eliminate unicast flooding. Also with multicast it is possible to introduce multicast with IGMP which provides a very nice solution (or so I am told!) if your switches support it.

Also a lot of "network people" get confused by unicast NLB as it introduces 'Microsoft non-RFC weirdness' (to coin a recent customer phrase) as both hosts share a MAC address and NLB uses a MaskSourceMac process to prevent the switch from associating the shared MAC with a particular switch port. IIRC you also get multiple replies to NLB enabled hosts if you ping them from switches which some customers moan about...

You are correct that multicast was and old way of fixing intra-node communication for NLB cluster nodes, but this can now be fixed with windows SP1 and the correct reg key http://support.microsoft.com/kb/898867 or, as you are doing, using dedicated NICs for intra-array comms (I still like this approach for traffic isolation personally).

Until MS supported multicast mode, I have always used dedicated L2 VLANs with unicast NLB to minimise the issues associated with flooding and this has worked well and I have some pretty big deployments using this approach. I think MS has added the multicast option, to just provide flexibility and to erridicate potential deployment blockers for ISA. There may be some other "a big customer wanted it" reason too

Unless you want to go multicast, I cannot see much point if you already meet the other prerequties to make unicast happy (dedicated intra-array NICs and dedicated L2 VLANs). Maybe adding static ARPs makes the network guys still feel useful and could be a good reason for multicast (ok, only kidding!)

Some other good NLB stuff here: http://technet.microsoft.com/en-us/library/cc783135.aspx

Hope this helps...

Cheers

JJ

< Message edited by Jason Jones -- 27.Oct.2008 2:04:21 PM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 2

RE: Why use Multicast NLB? - 27.Oct.2008 2:19:33 PM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

Thanks, Jason. It would be nice to not have to create so many L2 VLANs for all our NLB clusters (not just ISA - app server web farms, too!).

Guess I'm going to have to cross over to the dark side and learn a lot more about low-level networking so I can speak somewhat intelligently with our networking team about this.

For now, though, I'll stick with what I know - since it works (always a plus, right?

)

Mike

(in reply to Jason Jones)

Post #: 3

RE: Why use Multicast NLB? - 27.Oct.2008 5:31:03 PM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

quote:

ORIGINAL: mascalia

Thanks, Jason. It would be nice to not have to create so many L2 VLANs for all our NLB clusters (not just ISA - app server web farms, too!).

You should be using ISA server farm web publishing where possible to remove the need for app server NLB

Good luck, in my experience the network guys seem to have little respect for Microsoft NLB and often don't really understand it. Hence you have to work together to define the best options. However, based upon experience with customers this can be quite 'challenging' at times and ISA often sits right between the app/server guys and the networks guys.

Post back with your progress or if you find out any more info...

Cheers

JJ

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 4

RE: Why use Multicast NLB? - 27.Oct.2008 7:35:05 PM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Found these statements which may aid understanding:

quote:

To coordinate their actions, all NLB hosts communicate with each other by periodically exchanging heartbeat messages on a common subnet, which is also used to receive incoming client requests. This subnet must be physically protected from intrusion. Otherwise, unauthorized heartbeat messages could be delivered to NLB hosts and disrupt cluster operations. Note that NLB heartbeat messages use a uniquely assigned Ethertype (0x886F) which is not normally routed across subnets. However, the cluster administrator must ensure that unauthorized computers and devices which could emit invalid NLB heartbeat packets are not placed directly on the NLB subnet. The effects of these disruptions are described below in the subsection "Rogue Servers".

quote:

Rogue NLB servers on an NLB subnet could emit heartbeat packets that disrupt cluster operations. Disruptions can include impeding NLBs convergence process such that cluster hosts cannot be added and recovery for a failed host cannot be completed. In addition, disruptions can block some or all of NLBs service to clients. NLB subnets must be physically protected from intrusion by unauthorized computers and devices.

http://technet.microsoft.com/en-us/library/cc780843.aspx

Reading between the lines, this seems to be where the recommendation of putting the NLB interfaces on a different subnet (broadcast domain) than potetially malicious devices becuase the Ethertype used is not normally routed.

From this and other NLB articles, I think that the NLB heartbeat messages are passed across the network that is NLB enabled and not the intra-array network. I don't think configuring ISA to send intra-array communcaiton via the intra-array NICs has the same effect on NLB heartbeats, but I could be wrong here

If I am right, this means placing the NLB interfaces onto routed subnets is the right thing to do to ensure that the only machines that can access the Ethertype packets are the actual array members themselves and nothing else.

Isolating the intra-array traffic on to a dedicated network adds no value for the above NLB needs but is recommended to isolate "private array traffic" between array members for security reasons and allows for intra host communication for pre-SP1 server or those that haven't been configured with the post-SP1 interhost registry changes.

It may be time to get your network sniffer out and look for which interfaces are passing the Ethertype 0x886f packets to be 100% sure.

Cheers

JJ

< Message edited by Jason Jones -- 27.Oct.2008 7:38:01 PM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to Jason Jones)

Post #: 5

RE: Why use Multicast NLB? - 28.Oct.2008 9:50:23 AM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

Thanks, Jason. I think you're right. I found a couple of other supporting articles, too. While there seems to be a lot of interest out in GoogleSpace about using alternate NIC's to pass the heartbeat traffic, the general consensus (such as it is) is that NLB heartbeat traffic passes between the NLB NICs themselves.

Which, in an offhand way, makes sense. If you have three servers in an array, assume they each have two NIC's in different Networks. One Network is external, customer-facing and the other is a back-end connection to the corpnet with DC's, DNS, etc....

Further assume that there is a NLB array across the external NICs on the three servers.

Reading the documentation out there, the "heartbeat" is sent to all servers in the array once every second. If an acceptable reply is not received, convergance begins to remove the unresponsive array member.

From a "NLB for Dummies" perspective, wouldn't it make sense that that traffic would HAVE to go out over the NLB NICs? Those are the NICS used to create the array. What good would it do to have your NLB heartbeat going out over NICs that aren't in the array? The idea the heartbeat is to see if a specific NLB host (and NIC) is responsive. I my simplistic mind, that would be most effective going from NIC to NIC across the NLB array.

Now, I also found a bunch of sites claiming to have custom configs that will supposedly force the NLB heartbeat to go out over a different NIC. But, even if it works, is it wise? What about the case (common in ISA arrays) where you may have three or more NIC's in a box, and maybe have NLB built across two of them? If the NLB heartbeat goes out the NLB "back-end" of the servers, what happens if only one of the NLB configs goes belly up on a server, but the other one is working fine?

My guess is that NLB would be killed for the entire server, essentially taking it out of every NLB array - even if it's not necessary. If the NLB heartbeat for a specific array goes out over the NLB NICs for that array, however, then failure of NLB on one array wouldn't necessarily cause the server in question to withdraw from all NLB arrays (assuming that ISA server is smart enough to act that way

).

Just my thoughts after a good night's sleep. Overall, I guess you could force heartbeats out over another interface (and for MSCS, you actually want to do this). But for WLBS/NLB, do you want to? From what little I've been able to find, my current answer is NO.

What do you think?

Mike

< Message edited by mascalia -- 28.Oct.2008 9:58:39 AM >

(in reply to Jason Jones)

Post #: 6

RE: Why use Multicast NLB? - 28.Oct.2008 10:16:50 AM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

So, with all that said, back to my original thought: Why use Multicast NLB?

I have a L2 VLAN set up for the ISA interfaces used to build the NLB array, so Unicast port flooding is not an issue.
The L2 VLAN is hosted on a L3 switch with routing enabled, so the NLB heartbeat traffic moving across the NLB NICs isn't routed or otherwise available for tampering (unless someone jacks a rogue server into the VLAN ).
I have a private VLAN set up for intra-array communications, and have added HOSTS file entries on all array members so that they can "find" each other over this network (I've also moved the Intra-array VLAN NIC up to the top in the adapter binding order as well - the only "tip" I found useful from the various unofficial "How to force NLB heartbeats out a different NIC" articles).

With all that said, I don't see any reason to implement Multicast or IGMP Multicast on my current ISA array. Now, if any of those things change (which they will for arrays in other sites), then I'll have to re-evaluate.

The only consolation I'm making is to go ahead and update the CSS as outlined in the Microsoft KB so that it can support other arrays that might have to use Multicast.

My only other question is whether or not I still need to make the Registry hack to enable NLB communications over Unicast (as outlined in this KB: http://support.microsoft.com/kb/898867). Is this "fix" needed and/or compatible with ISA NLB?

Thoughts, anyone?

Thanks,
Mike

< Message edited by mascalia -- 28.Oct.2008 10:19:37 AM >

(in reply to mascalia)

Post #: 7

RE: Why use Multicast NLB? - 28.Oct.2008 10:41:56 AM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

quote:

ORIGINAL: mascalia

Thanks, Jason. I think you're right. I found a couple of other supporting articles, too. While there seems to be a lot of interest out in GoogleSpace about using alternate NIC's to pass the heartbeat traffic, the general consensus (such as it is) is that NLB heartbeat traffic passes between the NLB NICs themselves.

Which, in an offhand way, makes sense. If you have three servers in an array, assume they each have two NIC's in different Networks. One Network is external, customer-facing and the other is a back-end connection to the corpnet with DC's, DNS, etc....

Further assume that there is a NLB array across the external NICs on the three servers.

Reading the documentation out there, the "heartbeat" is sent to all servers in the array once every second. If an acceptable reply is not received, convergance begins to remove the unresponsive array member.

From a "NLB for Dummies" perspective, wouldn't it make sense that that traffic would HAVE to go out over the NLB NICs? Those are the NICS used to create the array. What good would it do to have your NLB heartbeat going out over NICs that aren't in the array? The idea the heartbeat is to see if a specific NLB host (and NIC) is responsive. I my simplistic mind, that would be most effective going from NIC to NIC across the NLB array.

Now, I also found a bunch of sites claiming to have custom configs that will supposedly force the NLB heartbeat to go out over a different NIC. But, even if it works, is it wise? What about the case (common in ISA arrays) where you may have three or more NIC's in a box, and maybe have NLB built across two of them? If the NLB heartbeat goes out the NLB "back-end" of the servers, what happens if only one of the NLB configs goes belly up on a server, but the other one is working fine?

My guess is that NLB would be killed for the entire server, essentially taking it out of every NLB array - even if it's not necessary. If the NLB heartbeat for a specific array goes out over the NLB NICs for that array, however, then failure of NLB on one array wouldn't necessarily cause the server in question to withdraw from all NLB arrays (assuming that ISA server is smart enough to act that way ).

Just my thoughts after a good night's sleep. Overall, I guess you could force heartbeats out over another interface (and for MSCS, you actually want to do this). But for WLBS/NLB, do you want to? From what little I've been able to find, my current answer is NO.

What do you think?

Mike

At this rate, I think we will need our own dedicated forum

Forcing the heartbeats down the intra-array NIC sounds like a bad idea to me and very likely to be unsupported by MS for ISA w/ integrated NLB.

By default/design, if any one of the NLB enabled NICs fails in an array member, the entire array member is removed from the cluster - this is done forcibly by stopping the firewall service on that particular node. If you think about it, this is actually the only sensible option as ISA cannot tell which NIC is important or not and it would completely break bi-directional affinity as packets may 'come in' on a good NIC but need to 'go out' on a failed NIC. Is this the right option? Not sure, but I think it seems the most cautious and sensible option if you think about it...

Hence why it is so important to ensure you have HA across the entire platform e.g. multiple switches and each ISA should only ever be conencted to a single switch to avoid a single point of failure affecting all array members at the same time. It would be nice to flag an ISA Network as a critical interface and then only get ISA to fail the node if an NLB enabled interface, which was defined as critical, failed. In the scenario, you could maybe suffer loss of a perimeter network interface without failing the entire node, but not so for the internal interface.

I am pretty confident that heartbeats occur on the NLB enabled network themselves too. Hence as the intra-array is not NLB enabled, no hearbeat traffic will ever be seen on these interfaces. However, you will see port 8080 intra array traffic between the nodes, but this is ISA and not NLB.

Cheers

JJ

< Message edited by Jason Jones -- 28.Oct.2008 10:43:36 AM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 8

RE: Why use Multicast NLB? - 28.Oct.2008 11:01:44 AM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

quote:

ORIGINAL: mascalia

So, with all that said, back to my original thought: Why use Multicast NLB?

I have a L2 VLAN set up for the ISA interfaces used to build the NLB array, so Unicast port flooding is not an issue.

The L2 VLAN is hosted on a L3 switch with routing enabled, so the NLB heartbeat traffic moving across the NLB NICs isn't routed or otherwise available for tampering (unless someone jacks a rogue server into the VLAN ).

I have a private VLAN set up for intra-array communications, and have added HOSTS file entries on all array members so that they can "find" each other over this network (I've also moved the Intra-array VLAN NIC up to the top in the adapter binding order as well - the only "tip" I found useful from the various unofficial "How to force NLB heartbeats out a different NIC" articles).

With all that said, I don't see any reason to implement Multicast or IGMP Multicast on my current ISA array. Now, if any of those things change (which they will for arrays in other sites), then I'll have to re-evaluate.

The only consolation I'm making is to go ahead and update the CSS as outlined in the Microsoft KB so that it can support other arrays that might have to use Multicast.

My only other question is whether or not I still need to make the Registry hack to enable NLB communications over Unicast (as outlined in this KB: http://support.microsoft.com/kb/898867). Is this "fix" needed and/or compatible with ISA NLB?

Thoughts, anyone?

Thanks,
Mike

I've never seen *ISA* recommendations talk about putting the intra-array NIC at the top of the bind order, quite the opposite in fact, as it is always recommended to place the Internal NIC at the top.

If you look at the "Array Servers" computer group in ISA, you should see that it amends this group to reflect the IP addresses defined for intra-array communications. Hence the system policy will only permit nodes to talk to each other using the intra-array addresses and not the internal or external addresses. IIRC you will actually see node communications on other interfaces being dropped by ISA as they are not covered by the default system policy.

Based upon your setup, I can't see any value in going multicast either.

Setting up the schema to support multicast now seems like a sensible idea. Just make sure you put the array into the right NLB mode BEFORE you enable NLB as you cannot change the mode without disabling NLB. This is painful when you have already defined lots of VIPs (especially if you have lots of NICs).

You can enable the interhost reg key if you like, but this is pointless if you already have a dedicated link between array members. The reg key was created to allow people *without* dedicated NICs to still use unicast NLB but not suffer the loss of interhost communications when using a shared MAC address. I don't think ISA really cares if this fix is applied or not, just as long as though nodes can communication for intra-array comms somehow. Does it work if you don't have dedicated NICs/ Yep.

Even if I a customer could not do the VLAN work, for the sake of an extra set of NICs, I always use them for intra-array networks. It is the best security solution and still recommended by MS even though the reg key provides a very useful workaround if people just downright refuse

quote:

Securing Intra-Array Communication

To secure intra-array communication, follow these guidelines:

Upon installation, a pair of private and public keys are created for each array member. These keys are used to transfer confidential data between array members. If you believe that the keys have been compromised, create a new key pair by uninstalling and then installing ISA Server.

We recommend that you use a dedicated network adapter in a network used only for intra-array communication.

http://technet.microsoft.com/en-gb/library/bb794718.aspx

Cheers

JJ

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 9

RE: Why use Multicast NLB? - 28.Oct.2008 12:00:08 PM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

ARRGH!!!

Just when we thought we had it figured out, I had to go and actually read a book

....

Here's a quote from the Microsoft ISA Server 2004 Administrator's Pocket Consultant:

Configuring and Securing Intra-Array Communication

Intra-array communication is the means by which array members in an NLB deployment communicate.

The Internet Protocol (IP) address assigned to the network adapter on the internal network is also assigned as the intra-array address. If you have more than one ISA server in an array, a dedicated network adapter is required for intra-array communication.

Note:

With Windows Server 2003 Service Pack 1, dedicated network adapters are not required for host-to-host communication.
(...)
By default, the intra-array IP address is set to the IP address assigned to the array members' network adapter on the internal network.

And, from another online reference from Skillport called "Working with ISA Server 2004 Enterprise":

Configuring Intra-Array Communications

For mutual communication, array members can use network adapters that are connected to the dedicated intra-array network. You can enable NLB on the internal and external adapters of the ISA Server array. To enable NLB, you must configure an array member to allow it to use a network adapter and IP address that is dedicated to intra-array communications.

In an enterprise array, after you enable NLB on internal and external adapters of a firewall, you need to enable array members to communicate with each other. Array members communicate mutually using IP addresses, which are bound to the adapters that are connected to the NLB network. This helps provide full NLB support to the array members. To force an array member to use the intra-array adapter for intra-array communications:

Open the ISA Server 2004 Enterprise Edition window.

Expand the array name and then expand the Configuration node in the scope pane of the ISA Server 2004 Enterprise Edition window.

Click the Servers node.

.... (Continues on with well-known example of how to change the intra-array communications address)

Now, is it just me, or are these references contradicting what we just came to an agreement on?

According to these references, it would appear that the Intra-Array communications link is used exclusively for NLB communications on ISA when Integrated NLB is enabled.

Again, from an outside perspective, I think I can understand that. Microsoft goes to great pains to explain that ISA Integrated NLB is NOT the same as using "native" Microsoft WLBS on ISA arrays. Maybe this is one reason why the two are different?

In MSCS (for server failover clustering), you are actually encouraged to specify and use a separate NIC for the "heartbeat", but in native WLBS this is not the case.

But ISA NLB ain't WLBS....

If true, then the only reason I can see for ever implementing Multicast (or IGMP Multicast) NLB arrays in ISA is to be compatible with whatever networking infrastructure you have in place, n'est pas?

This thread is really starting to have an Alice in Wonderland rabbit-hole feel to it.

Thoughts, anybody (assuming you're not seeing cats in trees and hookah-smoking caterpillars...)

Mike

< Message edited by mascalia -- 28.Oct.2008 12:05:34 PM >

(in reply to mascalia)

Post #: 10

RE: Why use Multicast NLB? - 28.Oct.2008 1:03:55 PM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Neither of these specifically say that the intra-array is used for NLB heartbeats (Ethertype packests) and again talk about intra-array traffic which is NOT NLB heartbeats.

The standard "old" advice was always to have a dedicated NIC as ISA only supported unicast and pre-SP1 this just wouldn't work - period. SP1 blurred this view, but other Microsoft recomemndations (as per my last post) still recommend dedicated NICs.

TBH - I would stand by what we agreed on, but the only way to confirm it 100% is to netmon all interfaces and look for the Ethertype packets.

I wish someone else would provide some input to either agree or disagree with my/your thoughts...

Cheers

JJ

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 11

RE: Why use Multicast NLB? - 28.Oct.2008 1:17:35 PM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

You read my mind. Running Netmon now on all three interfaces (internal, external, Intra-array). I'll post when I see something interesting....

Mike

(in reply to Jason Jones)

Post #: 12

RE: Why use Multicast NLB? - 28.Oct.2008 1:21:52 PM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Cool, thanks

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to mascalia)

Post #: 13

RE: Why use Multicast NLB? - 30.Oct.2008 10:20:39 AM

Jason Jones

Posts: 2265
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Hi Mike,

Right, I have had some feedback from a couple of contacts at Microsoft and thought it would be useful to this discussion and for the ISA community in general...

-----
Question: Do Microsoft support using a crossover cable between two array members when they are connected using dedicated NICs for intra-array communications? If not, why not?

Answer 1: That�s not an ISA question. I�d guess it depends on the NIC and its driver. ISA runs at a higher level than media access.

Answer 2: For intra-array it should be ok.

My Answer: As it doesn't seem to be specifically unsupported, it should be fine if you understand the limitations I have already mentioned.
-----

Question: If using dedicated NICs for the intra-array communications, does this connection also carry the NLB heartbeat traffic (Ethertype 0x886F packets) or does this still only occur between NLB enabled interfaces themselves?

Answer 1: NLB heartbeats occur only over NLB enabled interfaces. That is a NLB question, ISA has nothing to do with this.

Answer 2: AFAIK NLB does not allow exchanging heart beats through different NICs and even if it does, ISA definitely doesn�t configure NLB to utilise it.

My Answer: So, I think we were correct with our final conclusion

-----

Question: Why would you use multicast mode as opposed to the default unicast mode? Is this purely for flexibility if you cannot meet the prerequisites of unicast?

Answer 1: The reasons to use one method or another are mainly related to the switches and routers you use. From my experience I saw many cases where customers need a specific method because they already have the hardware which support it and they do not want to spend more money on upgrading hardware.

My Answer: If you cannot meet the recommended requirements of using the default unicast mode, or your switches/routers do not support it, multicast support in ISA now provides additional flexilbity to solve problems that you may encounter with unicast mode in your organisation. If unicast works for you, stick with it, if not multicast is now a valid and supported option!

-----

Question: Can you explain the following statement from Technet: �When NLB is enabled, it synchronizes array members by using pure Ethernet protocol communication. This low-level traffic is not protected by ISA Server. To help secure that traffic, we strongly recommend that you place a Layer-3 router between the Internet and the NLB-enabled array. This Layer-3 router will not allow the low-level Ethernet protocol to pass, thereby helping protect the array from potentially malicious Ethernet traffic from the Internet that could disrupt the operation of NLB."

Answer 1: NLB heartbeats and any other NLB handshake communication are low level, ISA doesn�t see it. A malicious user who can access your interfaces can interfere with this handshake. By isolating the NLB enabled interfaces with a layer-3 router you prevent malicious traffic from interfering with NLB operation. Again, nothing specific to ISA, the same recommendation is valid for a web server.

My Answer: Now that we know that NLB heartbeats occur on the actual NLB enabled interfacces themselves, this technet statement makes sense (see further clarification in my next question).
-----

Question: Does the use of the word �Internet� in the above technet statement imply *any* potentially entrusted network? E.g. should *all* NLB enabled interfaces ideally be separated from clients/servers using a layer-3 routers?

Answer 1: Yes. Any NLB enabled interface should be isolated from untrusted traffic to prevent malicious interference with NLB packets.

My Answer: I think this Technet statement should be reworded as we cannot assume that the Internet is the only source of malicious attack and potentially ALL NLB enabled interfaces should be protected accordingly. This could be achieved through the use of dedicated layer-2 VLANs for each NLB enabled network, hosted on a layer-3 switch which is then configured to route between the VLANs.
-----

Question: Do you need to use dedicated NICs for intra-array communications.

Answer 1: The recommendation for a separate intra-array NIC is to avoid bandwidth-consumption/increased-contention on the Internal network due to intra-array traffic as well as to reduce the attack surface so that �holes� in the firewall that are required for intra-array communication cannot be targeted by Internal attackers.

My Answer: If your ISA Servers are runing on a pre-SP1 Windows platform and you are using the default unicast mode, then yes you will need dedicated NICs. If you are using SP1 or plan to utilise multicast mode, you do not need dedicated NICs. However, for performance and security reasons it is still recommeded to utilise dedictaed NICs for intra-array communications if you can.
-----

Question: I understand the term �intra-array communications� and how to configure for best practice, but can you explain what sort of messages (maybe with examples) are actually exchanged between array members?

Answer 1: In the context of ISA, messages which pass between array members include:
* ADAM replication and ISA synchronization with ADAM.
* ISA level heartbeat (over http) to determine availability of array members. This is done in order to correctly handle and forward CARP requests.
* CARP traffic, where one member forward http request to another member which may cache it.

My Answer: ADAM replication and ISA synchronisation only occur when the CSS role is co-located on the actual array members themselves. This co-location of the CSS role is something I personally try to avoid.
-----

I hope this is useful...

(in reply to Jason Jones)

Post #: 14

RE: Why use Multicast NLB? - 3.Nov.2008 9:42:14 AM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

Netmon followup....

hate to say it, but my skills must be more rusty than I thought. I spent two hours on Thursday using NetMon, and couldn't find a single dadgum packet that I can ascribe to NLB or the mysterious "ISA intra-array communications".

I even downloaded the new version of NetMon (3.2), and loaded all the full parsers. No filters, no protocol exclusions. Ran it on one member of the array, on all interfaces. in promiscuous mode. Nothing. Even failed over the array several times on both servers by selectively stopping and starting the MS Firewall service. Still Nothing!

Either (a) I'm really rusty, (b) ISA is hiding these packets from NetMon, (c) I'm not using NetMon correctly.

I'm betting on (a) and (c)

Ideas, anyone?

Mike

(in reply to Jason Jones)

Post #: 15

RE: Why use Multicast NLB? - 3.Nov.2008 11:48:55 AM

mascalia

Posts: 36
Joined: 13.Feb.2008
Status: offline

Jason, re:

quote:

Answer 1: In the context of ISA, messages which pass between array members include:
* ADAM replication and ISA synchronization with ADAM.
* ISA level heartbeat (over http) to determine availability of array members. This is done in order to correctly handle and forward CARP requests.
* CARP traffic, where one member forward http request to another member which may cache it.

My Answer: ADAM replication and ISA synchronisation only occur when the CSS role is co-located on the actual array members themselves. This co-location of the CSS role is something I personally try to avoid.

I'm assuming that the ISA-level heartbeat over HTTP are the port 8080 connections I see in the ISA log?
If the ISA heartbeat is tied to CARP, and you aren't using CARP, then will there be any ISA heartbeats at all? I'm guessing not, since NLB heartbeats will handle NLB convergence (when necessary). Maybe that's why I'm not seeing any ISA intra-array comm on my Netmon capture?
Question: if I have a second NIC on my dedicated CSS, would it be better to plug it into the private VLAN used for Intra-array comm? I know this would only benefit "local" arrays that can support such physical connections, but would it be better than having ADAM connections go out through the internal network interface?

Thanks for the other info. I (and others, probabably) appreciate it.

Mike

< Message edited by mascalia -- 3.Nov.2008 11:51:13 AM >

(in reply to Jason Jones)

Post #: 16