would link aggregation benefit?

[dannieboiz 13]

About to build a dedicated Linux file server and move all the loads off my htpc. The build is going to off a Xeon X3440. Still debating on weather I should get a server motherboard with ECC ram or just a desktop board and use non-ecc ram. If I went with a server board it'll most likely have dual NIC so I'll be able to setup link aggregation. Then of course I have to get an appropriate switch for it. Since it's going to be my so called NAS/media server I think any bit of extra bandwidth would help. I just don't know if MBS would benefit from it. Especially for streaming when I'm away from home. 

 

I have a 15Mbps upload speed. When using FTP I get a transfer rate of only 2ish Mbps. So I assume that's what I get when I watch movies away from home.  Would link aggregation increase this any bit?

[Danee 57]

Your funnel would just be wider at the top, so no wouldn't help any bit when away from home.

[ebr 14913]

Your funnel would just be wider at the top, so no wouldn't help any bit when away from home.

 

Nice analogy...

[Starlionblue 80]

As mentioned above, your speed facing the Internet would be no different since your upload speed is well below the limits even for a single gigabit card. Any benefits from fiddling with the network would be at home only.

 

I have set up link aggregation at home, mostly so I could fiddle with it and learn how it works. There are three main points to remember:

1. Equipment. You need a smart switch that supports link aggregation. Most do, but you're still talking a switch that is at least five times as expensive as a plain vanilla switch. You also need network cards that support link aggregation. This is by no means all of them.
2. An aggregated link does not speed up transfers to a single endpoint. That is, a file copy from your server to your HTPC will not be any faster. However total bandwidth is higher so if you are running, say, two simultaneous file copies, one from server to HTPC and the other from server to a third computer, your aggregate copy speed will be higher. Well, it should be higher but it depends on network topology a bit and is by no means a sure thing.
3. When serving movies through MB, you're not running a full speed file copy. The movie is streamed from the server. Even serving two or three simultaneous clients with HD, the bandwidth is well below what you can get with just one gigabit card.

Aggregate links are fun to play with, but unless you are in a corporate environment with hundreds of endpoints for your file transfers you don't need them. A network expert at IBM told me that the only good reason to have multiple NICs was if the server needed to be on different subnets simultaneously. Otherwise you were just complicating things. If you wanted more speed, he said to go with 10gig ethernet.

 

In home networks, implementing Jumbo Frames (supported by most cards and switches nowadays) will give you far more bang for the buck when doing large file transfers than anything else. My testing shows that I can get about 50MB/sec with a normal link (aggregated or not), while with Jumbo Frames this goes up to around 80-85MB/sec.

Edited July 11, 2014 by Starlionblue

[The Baron 64]

There is another thing you should be aware of. One of the ports in a team will be used for management purposes, so if you have 4 ports in a team, 3 will be used to transmit/recieve data. The fourth will not.

 

Another thing, looks like you will need to use a server OS (2008/2012) if you want to use NIC Teaming (Aggregation). Seems that Windows 8, etc, does not support teaming.

[Starlionblue 80]

 

Another thing, looks like you will need to use a server OS (2008/2012) if you want to use NIC Teaming (Aggregation). Seems that Windows 8, etc, does not support teaming.

 

Yes and no. For example, if you use Intel NICs that support teaming, Intel ProSet software will allow teaming regardless of OS. I use Windows 8.1 and teaming.

 

 

There is another thing you should be aware of. One of the ports in a team will be used for management purposes, so if you have 4 ports in a team, 3 will be used to transmit/recieve data. The fourth will not.

 

 

I have not experienced this with LACP and two teamed Intel NICs.

Edited July 11, 2014 by Starlionblue

[FlyGuyAZ 5]

The funnel analogy was good!! Like Starlion said, it's really only worth it for the experience of playing with it. 

[Deathsquirrel 741]

Also worth noting that apps that bind to the network driver stack can blue screen your box when you run teamed NICs.  The app I support every day, which shall remain nameless, is an example.

[DeeMac 17]

In home networks, implementing Jumbo Frames (supported by most cards and switches nowadays) will give you far more bang for the buck when doing large file transfers than anything else. My testing shows that I can get about 50MB/sec with a normal link (aggregated or not), while with Jumbo Frames this goes up to around 80-85MB/sec.

 

Agreed on JUMBO frames.. but remember, jumbo frames is for the internal network/LAN only and wont help you "get faster transfer speeds" going out over the internet.

[jahblessed 0]

i have a few questions regarding this. i will post my htpc config first.

 

Motherboard: Gigabyte GA-X79-UP4 rev 1.1

CPU             : Intel Core i7-4820k ( Corsair Hydro H110 will be installed in a week )

Graphics      : Gigabyte R9 280x   (second will be arriving soon to use Crossfire)

Memory        : GSkill 32GB DDR3 ( Soon 64GB)

Main HDD     : 256GB Samsun 830 (adding second soon for Raid 0 VIA Apricorn Velocity X2 SSD upgrade kit)

Other HDD   : 5x2TB [10TB Total] Caviar Black ( RAID 0 ) for media storage connected VIA RocketRaid PCi-E Card (cant recall model) so speeds wont be limited by on board sata port bandwidth cap.

Internet        : Verizon Fios Quantum 150/150 (all my wiring within my home network is Cat5e)

NIC               : Intel Pro/1000PT Quad Port Gigabit NIC PCi-e card (IBM OEM Version)

OS               : Windows 7 Ultimate x64

Power Supply: 1200w modular power supply

Modem         : Fios Wireless Dual Band N Router/modem

 

 

my switch can definitely handle link aggregation for sure (forgot the model) and i was able to Team the 4port using Intels software.

 

my questions are...:

 

1). my motherboard has a feature in the bios called Network Stacking and is currently Disable by default.. should i enable this and how would i go about utilizing this feature with my NIC Card and MB3

2). should i include the onboard NIC into the Team I create with the other 4ports? cant recall chip model for it or if it can support Teaming but it is Gigabit link

3). if MB3 can see a boost in streming outside my residence through my current setup/NIC teaming, how do i go about properly configuring it to take full advantage of all that i have on it?

4). do i have to configure my Fios router in any way to further enhance my experience?

 

 

i mostly use this to stream outside my network, otherwise i would just go to my htpc and watch via HDMI link to my tv. if i can see a benefit from a quad setup, would adding another quad (8 total gig ports) show any more of a increased improvement? any sugguestions would be greatly appreciated. thank you!

Edited August 1, 2014 by jahblessed

[dannieboiz 13]

What is your Upload speed of your internet? Like you, I stream a lot outside my network and quickly learned that no matter what I did inside, the bottle neck is from the ISP. 

 

I have a Cradlepoint router that can provide multiple WAN, I'm using Cable 30/12 Mbps UL/DL plus Verizon LTE aircard that I get around 15/9 where I'm at using Load Balancing. I think it works ok but ultimately, I'd like to have a Peplink or similar router that can do LAG on the WAN combing all WAN to a single output but the price of the Peplink is way up there. 

 

The short answer to your question is your LACP won't help your external streaming. 

[Danee 57]

If Verizon Fios Quantum 150/150 means you have 150Mbit up and down, my first post in this thread applies.

A single Gbit interface is 1000Mbit, you'll be needing a WAN upload of at least 1001Mbit before such actions make sense.

[Starlionblue 80]

If Verizon Fios Quantum 150/150 means you have 150Mbit up and down, my first post in this thread applies.

A single Gbit interface is 1000Mbit, you'll be needing a WAN upload of at least 1001Mbit before such actions make sense.

 

At least 1001, but in reality much more than that since you a 1000mbit link won't give you more than about 700mbit even in ideal conditions.

 

Here in HK one company does offer gigabit fiber, but even if I had that I wouldn't consider link aggregation. Besides, I don't think you can't trunk more than one NIC to the modem anyway.

[moviefan 184]

A network expert at IBM told me that the only good reason to have multiple NICs was if the server needed to be on different subnets simultaneously. Otherwise you were just complicating things. If you wanted more speed, he said to go with 10gig ethernet.

 

 

There are other reasons to have multiple NICs in corporate environments, including physical switch, port and cable redundancy.

 

At home on a gigabit network, LAG is much less useful..  

 

It cannot increase speeds for regular TCP communications because the way LAG works (in almost all cases) is by distributing the individual TCP conversations across more than one physical link and then pinning those TCP connections to those physical links.  So any individual connection still only can receive the maximum speed that a single physical link provides.  You could use software to break up a file transfer into multiple TCP  streams and then take advantage of those multiple links but I don't imagine many people are doing that on their home networks.  Most people are using regular CIFS.

 

In corporate environments you typically have so many clients accessing the same resource that this would be beneficial because there are MANY TCP connections to be distributed effectively to increase performance and reduce potential bottlenecks.

 

The only scenario where LAG really helps on a home network is if you have a central storage device that can push more than 1Gpbs IO and you regularly have more than one client performing large file transfers simultaneously.  This is pretty uncommon I would say on most home networks.  

[moviefan 184]

There is another thing you should be aware of. One of the ports in a team will be used for management purposes, so if you have 4 ports in a team, 3 will be used to transmit/recieve data. The fourth will not.

 

 

I am not sure which method of teaming you are using but I have never seen such behavior before.

[Starlionblue 80]

I am not sure which method of teaming you are using but I have never seen such behavior before.

 

AFAIK the only reason to have one port "down" is if you use a "hot standby" port that is inactive in normal operation.

 

There are other reasons to have multiple NICs in corporate environments, including physical switch, port and cable redundancy.

Fair point.

[dannieboiz 13]

I gave up on all these LACP at home after I did some testing.

 

ReadyNAS 4200 Dual NIC setup with LACP

Server 2012 dual NIC + LACP

Netgear GS108T-+ LACP

R/W speed is roughly 107-110Mbps

 

On the other hand, between my HTPC with a single NIC and my ReadyNAS I'm also achieving the same speed. 

 

So my conclusion, unless you planning on streaming to a lot of LOCAL clients, LACP is a waste of time at home. In a few years when 10Gbe reach the consumer level, then we'll have another discussion.

[moviefan 184]

I gave up on all these LACP at home after I did some testing.

 

ReadyNAS 4200 Dual NIC setup with LACP

Server 2012 dual NIC + LACP

Netgear GS108T-+ LACP

R/W speed is roughly 107-110Mbps

 

On the other hand, between my HTPC with a single NIC and my ReadyNAS I'm also achieving the same speed. 

 

So my conclusion, unless you planning on streaming to a lot of LOCAL clients, LACP is a waste of time at home. In a few years when 10Gbe reach the consumer level, then we'll have another discussion.

 

 

Exactly proving the point.  You will not increase transfer speeds for single connections using LAG unless you combine it with software which can effectively break up and distribute the transferred data across more than one physical link.  LACP acts exactly as I said.  And with LACP, depending on the hash distribution model you choose, the same IP address pairs might be pinned onto one link anyway.  You would have to make sure your hashing algorithm utilizes not only the source and destination IP address pair as the inputs for the hash calculation but also the TCP ports being used   This is typically possible but is NOT the default.

Edited August 4, 2014 by moviefan

[jahblessed 0]

thank you guys for the explanation. i just one last question. would it be better if i dedicate the quad card for mb3 traffic and the onboard ethernet for my regular inbound/outbound traffic (websurfing / regular youtube streaming / etc)?

[Danee 57]

You will need 2 subnets for that, pulling that off requires good knowledge of networking, don't want to be rude, but if you're asking the question your knowledge is not enough.

 

Also, I don't see the benefit. Your LAN connection is 1000mbit, your WAN 150. Even if you download whatever at max speed you'll still have 850mbit available for local traffic.

[JeremyFr79 228]

As other's have said LACP is only useful in the home environment for redundancy, you will 99.9% of the time see no help in speeds.  Though it can be said if you set it up right you can load balance between the connections which can be helpful in some scenarios.  But other than that it's just something to play with.  In my personal setup I use LACP for redundancy.  My main server uses a Fibre Channel Disk Shelf which has redundant fibre connections to the Server, which is then LACP'd to my main switch, which is also LACP'd to my Secondary switch as well (which is the only point where speed can be helped since it's dual Gig links feeding off a Gig Switch into a 24 port 10/100 switch.) 

[MSattler 387]

As other's have said LACP is only useful in the home environment for redundancy, you will 99.9% of the time see no help in speeds.  Though it can be said if you set it up right you can load balance between the connections which can be helpful in some scenarios.  But other than that it's just something to play with.  In my personal setup I use LACP for redundancy.  My main server uses a Fibre Channel Disk Shelf which has redundant fibre connections to the Server, which is then LACP'd to my main switch, which is also LACP'd to my Secondary switch as well (which is the only point where speed can be helped since it's dual Gig links feeding off a Gig Switch into a 24 port 10/100 switch.) 

 

Spot on, LAG/LACP is for redundance.  I have a home built SAN write now with memory/ssd caching getting me up to 6,000Mbps writes.  Now in the end my ESXi servers are still limited to the iSCSI 1GB connections I have, and even though the ESXi servers have multiple connections it does not help one specific virtual machines.  My write speeds still max out when the 1GB ethernet connection maxes out.

 

That being said, Infiniband 20GB dual port adapters are $25 each on eBay.  Use them without a switch and direct connect to your storage, or spend $100 for a 10GB switch, or $200-$300 for a 20GB switch.   That right there will get you 10-20GB per dual port connection, and if you are maxing that out, you have bigger issues.

[MSattler 387]

Main HDD     : 256GB Samsun 830 (adding second soon for Raid 0 VIA Apricorn Velocity X2 SSD upgrade kit)

Other HDD   : 5x2TB [10TB Total] Caviar Black ( RAID 0 ) for media storage connected VIA RocketRaid PCi-E Card (cant recall model) so speeds wont be limited by on board sata port bandwidth cap.

 

 

Not to nitpick, but why are you so hell bent on Raid 0 for your OS and your Data drives?  If you lose one of those 2TB drives, you are going to lose all of your media storage.  Are you doing it for the performance gain?  If so you may be better off using some of that memory with PrimoCache <will cost you some $>, and switching over to at least RAID 5.  Yeah you are gonna lose 2TB of storage but will have some protection against a drive loss, and PrimoCache and the memory caching will most likely give you faster read/write speeds.