Example of the new gen: https://www.amazon.com/Wiitek-Transceiver-Compatible-UF-RJ45...
Old gen: https://www.amazon.com/10Gtek-SFP-10G-T-S-Compatible-10GBase...
Typically the old gen uses a Marvell AQR113C, and the new gen uses a Broadcom chip that I forget the number of off hand.
The thermal performance of the new modules is so much better that they'll make you want to throw away all of your old modules as soon as you try one. The old ones consumed a lot of power and put out a lot of heat. You can find examples of people adding active cooling to blow on them because they're so bad.
The new ones are great. They get warm but it's completely manageable. And the power draw is also closer to what the SFP+ ports on your device were probably designed to handle, so compatibility is better too.
Even with the new modules, their ethernet cable runs were always in excess of what 10GBE could handle. I remember one at ~140 meters that was constantly renegotiating.
The worst part is, they had not a single element in their network that didnt support fibre. The fibre sfp modules for their radios was the same cost as the copper.
I love that these 10GBE modules exist but please (please please) just run fibre if you can.
That said, IIRC a lot of people running fibre + power.
Using this module, I was able to get a stable 10 gig over a 75 feet long, 20 year old run of Cat 5e.
Ironically, one of my switches with the RJ45 module runs cooler than the one without because Ubiquity does not let you do any fan control on Unifi Enterprise 8 PoE.
> Built-in Newest Marvell chip, Max. Power Consumption 1.65W
Is that wrong?
Also, I'm using this:
https://www.amazon.com/dp/B0B3F5DSXJ?ref_=ppx_hzsearch_conn_...
which says
> Build in Marvell chip, Low Power Consumption (2.0W MAX @ 30m)
I have had 0 problems. One I use for my Sonic.net Ont and the other for my a 10G switch. Maybe I don't have issues because the cable lengths are so short (1-2')?
I just checked and both are VERY hot to the touch. Now I'm worried.
I have a MikroTik CRS304-4XG-IN on my office desk with three out of four ports at 10gbit and it is perhaps 20 degrees above ambient on the outside. Warm but not hot. Passively cooled design.
A normal Windows laptop runs hotter than that when idle.
I used to daily drive Gentoo on my laptop. Lovely in the winter!
The cost, power, and length issues meant that it wasn't exactly well-received by the datacenter market back in 2006(!) when it was first released: DAC was the far more attractive option for a link from server to top-of-rack, and fiber was obviously superior (if not plain required) for anything beyond a hop to the next aisle over.
This left an incredibly tiny market, so obviously beyond the initial investment very little effort was put into developing new products for it. So now the prosumer market is hitting the limit of 1Gbps, 2.5GBASE-T and 5GBASE-T (both based on the techniques of 10GBASE-T, by the way) are becoming the norm, and suddenly network vendors remember that box of ancient 10GBASE-T transceiver chips that has been collecting dust in their warehouse.
Aaand suddenly you've got people buying what they think is a brand-new technology, but which is actually designed and manufactured using technology from a decade and a half earlier, and 10GBASE-T gets a bad name for being "hot" and "power-hungry". Turns out it is actually reasonably well-behaved if you actually make use of modern technology!
I expect we'll be using it for quite a while. 25GBASE-T and 40GBASE-T are even deader: A standard from 2016, which a decade later doesn't have a single available product? Mandatory switching to Cat8 cabling - and only a 30 meter range??? And no forwards-looking compatibility? Yeeaah, no thanks.
…and a lot of consumer boards have spare x1s
But searching for specific gen4 x1 10-gig chips (like RTL8127) does produce some reasonably-inexpensive options for cards with sfp+. Give it a whirl. :)
[1]: I wrote this like it's a new problem, but looking back it may be an issue as old as the hills. I distinctly remember searching for any-brand 10-meg cards with DEC Tulip chips ~30 years ago, because that was the inexpensive PCI chip that actually worked. :)
Industry went for 25G and 40G gear is long EoS and EoL.
It's wild to me that 10gbit isn't the norm by now and tech people who should know better seem to think WiFi matches or even exceeds even 1gbit ethernet. My MBP connects to my WiFi7 setup(Ubiquiti E7) at a nominal 1.5-1.9gbit but Time Machine backups and file transfers are slower than plugging into 1gbit ethernet, probably in large part due to latency and retransmissions. Not to mention that ethernet works with near 100% reliability with dramatically less variation in speed and error rate.
Sorry, this is snarky and off topic, but I'm nostalgic for the days when Time Machine "just worked".
Before I was using one of the common Synology consumer NAS boxes that are often recommended. The NAS didn't report any errors with the drives or its own hardware, but at least once a month TM would glitch on at least one of my home laptops.
My new setup is an Asus FLASHSTOR 12 Pro Gen2 FS6812X. For a year now it's been running without a single apparent TM glitch while backing up multiple personal laptops and my work laptop. Sometimes I'm plugged in and sometimes I'm backing up over WiFi, but it's always worked.
I tried various recommended settings for the Synology and nothing helped so I strongly suspect that the Synology network protocol(SMB, AFP, etc.) implementations were either buggy themselves or at least not compatible with quirks in Apple's implementations. Synology->Asus fixed all my TM problems instantly and seemingly permanantly!
Again. I’m on a Synology, and your comment is interesting.
I've been on a lucky streak for several years now, where I haven't gotten that one on any of my devices.
"Preparing backup..." taking an unreasonable amount of time is a regular occurrence, and some edge cases around adjusting TM backup size quotas aren't handled well. But other than that, TM has been working reasonably well for me to back up 10 TB over SMB to a Synology NAS.
My gripe is much more with Apple's abysmal support for SMB and NFS, especially after deprecating AFP. I've been back and forth between them over the years and over several OS versions, and their implementations for both are just terrible.
But over time SMB, for me, proved slightly more stable and performant, with the right tweaks in smb.conf, and authentication and permissions/ownership are easier to deal with than NFS, so I stuck with that.
I also yearn for the days where TM just worked, because somehow, the alternatives are even worse:
- Arq Backup does some things quite well, which is why I use it as part of my 3-2-1. But some of its bugs and implementation decisions just scream "hobby grade" to me.
- Kopia looks interesting, but it's not mature enough yet. Failed for me with absolutely cryptic error messages during repo init both times I tried it, with versions several months apart.
- Restic, Borg / Vorta: Not turnkey enough for me.
TM heavily throttles disk I/O used for backing up in order to ensure that normal user activity isn't affected. That makes it appear that TM is dramatically slower than you would expect which greatly annoys me. This becomes obvious after you run this command which will make both the preparing and transferring phases go closer to the theoretical speed you'd expect:
sudo sysctl debug.lowpri_throttle_enabled=0
Amusingly enough Time Machine to a local drive failed completely.
I’ve used it for a long long time and never knew these glory days.
I honestly just don't need it. Part of that is my ISP options top out around 1200Mbps, certainly. But I also just don't need that kind of speed inside my home. Streaming video needs at most 20Mbps or so, and I don't do much in the way of large file transfers. And when I do large file transfers, it's usually from or to the internet, and my home network is not the bottleneck there.
10G was too big of a step up from 1G. The expense and power required made it unattractive. Only recently have the interfaces for 10G over twisted pair become reasonably low power.
2.5G and even 5G are in a much better spot. It's where I recommend most people start as a default.
As much as I wanted to "future proof" by having a 24 port 10GbE switch... why? I'll just wait and buy one when I have a use for it.
As much as I enjoy looking at those wiring cabinets where every cable is cut to exactly the right length to reach a single port on the switch, this is why I prefer to leave an amount of slack in the wiring: It's good to be able to pull different wires to different switches depending on your needs.
One small high speed switch with enough ports for the couple of devices that can use it. One gigabit switch with a lot of ports to provide connectivity everywhere else.
https://techspecs.ui.com/unifi/cloud-gateways/udm-beast?subc...
You can also get enterprise C3850 with 10G ports for ~100$, but they will be loud and power hungry.
2 things here. Upthread is the discussion about the old 10GBE modules that would constantly turn off due to overheating. Thats left a sour taste in a lot of peoples mouths.
I dont have anything in my home network that matters enough to have 10GBE anywhere. If I did, I would just get fibre. My wireless is fine for most purposes except some HD streaming, and plain old 1 gig works fine there.
However, I have multiple Macs where I simply have a USB-C laptop SSD attached for Time Machine and they have worked without issue for years. These laptop SSDs come in huge sizes nowadays, and you don't need an especially performant one, so they can be pretty cheap.
The high expense of 10gig is, in part, because it isn't widely necessary and the people buying it are willing to pay extra.
I think the price has more to do with where you live and how the market is structured than how necessary it is. In Japan where there is competition between ISPs, I pay about $40/mo for 10Gbps.
Routers have also come down in price to where they are pretty affordable for consumers. I use a Ubiquiti Cloud Gateway Fiber [1] which has three 10Gbs ports (two SPF+, one 10GbE) for $279. A TP-Link router [2] with an upstream 10GbE port and 2.5GbE LAN ports and Wi-Fi 7 is about $140. 2.5 GbE NICs have become cheap and ubiquitous and could commonly be found on $150 mini pcs (before memory and SSD prices went crazy).
Yeah it's more than more than most people need, but I definitely appreciate having the increased speed when downloading 50GB games, uploading 200GB files to YouTube, or backing up files to the cloud. I've probably never maxed out the full 10Gbps, but exceeding 1Gbps is pretty easy in relatively common use cases.
[1] https://store.ui.com/us/en/category/cloud-gateways-compact/c...
[0] <https://mikrotik.com/product/crs305_1g_4s_in>
[1] <https://mikrotik.com/products/group/switches?f[0]=s%3Ac&f[1]...>
Having 100 TB of storage in your home basement is an even more extreme minority than that. ;)
A gigabit connection is more than enough for a 500-person call center today.
Over 20 years ago, I was used to having 1g LAN for basic workstations and laptops in an office setting and probably 10-20g uplink from the building (shared by hundreds of staff). I also used 1g at home for my very small LAN between laptop, desktop, and SAN functions. But, my home ISP links were often terrible, such as 128k ADSL or even just a tethered GPRS phone at some points.
You end up with entirely different work styles when you have these different resource constraints.
During the day I need to pull large data files from the work VPN so it's nice that that can happen at full speed even when Steam and movie streaming are also at full throttle. Combine that with backups and moving various files back and forth to my NAS and I'm very happy to have 10Gb local wiring.
Nearly nobody has multigig anything in the home, a probably surprisingly large percentage of business networking is 1gig LAN or less. And most people would not notice the difference if they did.
I am glad it works for you, but everyone else most certainly doesn't need it. (Yet.)
Personally, I do try for mostly gigabit in my home, because I do selfhost, but I have a ~800 Mbps download service (200 Mbps upload, it's asymmetric) that was only 500 Mbps when I signed up. And to be honest most of my patch cables are CAT5e because I'm cheap. I do make sure to run CAT6 through walls though because I don't want to ever have to do it again.
Also, I used to have Astound, and I feel so much sympathy for Google Fiber customers, you have no idea what's coming. If you thought Google had a reputation for bad customer service... just wait!
You can hook up a few key components cheaply and 10gb pci cards are something like $20.
For 10G with copper, short runs will work fine on most anything (10m), with longer working ok on cat-5e cable (20 - 30m or so).
So, if you have existing wiring, especially in wall, just give it a try first, before gutting things. Most 10g ethernet adapters give you BER stats to see the error rate.
And, if there are issues, multi-gig may be an option, where it will drop down to 5 or 2.5 gbps, which is same as 10G, but with a reduced symbol table to handle the lower SNR.
The gotcha is both ends needs to be good radios. So a router to router bridge tends to work better than router to end user device. Also had near LoS which presumably helped a ton
a) You like small computers and these only comes with RJ45 ports and 0 PCIe slots
b) You enjoy fighting with super rigid cables, port flaps and hot modules
* 10Gbps SFP+ PCIe NICs (Mellanox CX3 and similar cards) are already dirt cheap ($15 for OCP adapted ones, $50 for native PCIe cards) because datacenters are upgrading to CX5
* 10KM SM SFP+ modules are already dirt cheap (~$6) because 4G BBU-RRU communication uses these
* Silent 10Gbps SFP+ switches start appearing (still a little pricey); 10Gbps RJ45 switches are not available anywhere
After 2024:
* 25G/100G SFP+ PCIe NICs are dirt cheap (~$80 for single port 100G NIC)
* 100G QSFP28 modules are in acceptable price; 25G SM/CWDM SFP+ modules area already dirt cheap (~$50 for 6-slot CWDM ones) because 5G BBU-RRU communication uses these
* Silent 100Gbps switches start appearing (you can get 24*25G + 4*100G under 100W; still a little pricey); 100Gbps RJ45 switches are not available anywhere
Most of my edge devices are still on 2.5GbE though, and I'm increasingly aware that for anything with plain SATA disks, the drives are the real bottleneck. Once I LAG'd 2×2.5GbE to get a 5Gbps pipe, it became obvious the network wasn't the slow part anymore in a lot of cases.
And yeah, the 10GbE SFP+ modules run hot, so hot that I would not lay my fingers on them for more than 2 seconds. I stuck 2 copper heatsinks on my module, not sure they do much but the module runs smoothly. Even so, I'm pretty happy with the overall setup: from my 10GbE-equipped Mac I can saturate multiple machines at once and I no longer think about the network most of the time, which was the goal.
I did find some Kevlar-reinforced options that are supposedly ~0.3mm, but they seem to be raw fiber without connectors, purposed for drones, and I'm not sure about global availability.
This is not quite correct.
The primary problem is cross-talk. Copper wire itself will carry the relevant frequencies up to 100m without issue but even with balanced pairs the balancing is not perfect and the "dirty paper precoding" is not perfect so some cross-talk will occur. How long you can go with Cat-5e depends on how well the wire is twisted, how many wires are bundled together, are there any loops or tight bends, and other factors. Cat-6A guarantees less cross-talk with more twists, better balancing, and a plastic separator inside the cable to make the cross-talk more regular and thus easier to cancel out.
Bottom line is: for almost any normal home or apartment any quality Cat-5e cable properly terminated will carry 10GBase-T without issue. In fact if you have problems I would first re-terminate the cable before assuming you need to run new cable. Cat-6 or 6A just isn't necessary.
As a PSA: beware of "CCA". I've noticed Amazon and eBay are absolutely flooded with cheap chinese electrical and networking cable that shows nice shiny copper in the pictures but is actually "copper clad aluminum". If they mention anything at all they code it as "CCA" cable without explaining what that means.
CCA cable cannot, by definition, be ethernet cable. I won't get into the full technical details but the standard was amended to clarify that only pure copper wires are acceptable for ethernet. Personally I would not dare use CCA for anything. It has lower performance, lower current-carrying capability for the same wire diameter (inherent in aluminum), and introduces the risk of oxidation and loosening of connections as people will treat them as copper connections when aluminum needs special installation procedures and connections to avoid them coming loose over time. For electrical connections especially this not only can but absolutely will lead to a fire over time if not treated with the appropriate care. All it takes is a little bit of mechanical action scraping off the thin copper layer and you now have an effectively aluminum wire - a time bomb ticking away.
If you want a full FW solution that can actually FW+NAT at 10G bidirectional without breaking a sweat then something like the FortiGate 90G is the cheapest thing I've found that performs really well across the board. Great QoS, great latency, amazing throughput performance (does well with even small packet sizes in a single stream), easy enough to use UI (once you get oriented), low power. If you want to enable all of the NGFW stuff (e.g. AV and IPS) then it'll dip below line rate though.
If you just want something that NATs/connection direction oriented filtering like a "normal" home router then something like the MikroTik CCR2004 can get you better than the performance they got on the VP2440 + give you 12 ports of 10G SFP+ to work with. If you were planning to do "fancy" FWing/functionality beyond a normal home NAT FW (with decent managed switching built in) then the feature set will be a bit limiting, of course.
It is nice moving/streaming large files across the network at 10 gbit. It really is ten times less waiting than with plain old gigabit.
Of course, most of the time I'm working with lots of small files and then the spinning disk array in the NAS has no chance to saturated the this giant pipe, or even a normal gigabit connection...
FWIW, Cat 5e supplanted Cat 5 25 years ago.
So I bought a reel of that even though I was only going to be using 1000-BaseT. I don't remember there being too much premium on the wire itself.
Did you check the jackets? I've got Cat5 (no e) marked cable running 10gbaseT. A lot of cable exceeds the specs on its jacket and specified wire provides enough signal to noise for the provided length in dense conduit. When you have shorter runs, without dense wiring, lesser cable can work.
People downthread has written, cat6 was already the goto thing 15+ years ago so perhaps that is what they put in when the office space was built. It is a well built office!
Man I wish I had the kind of money/hardware to get 10gbit disk io out of a nas.
Four or five 7200rpm disks in an array can also sustain it for sequential reads of big files. That costs less than a gaming laptop.
A potato can saturate a 10gbit line if the data is mostly in memory.
For lots of small files in random sequence and cold caches it gets a lot harder.
I have 1.5/900 fibre to my house, and I bring a 2.5 line from the modem to my home office where a 2.5 switch delivers it to my workstation, laptop, and unraid NAS. But those devices are all themselves just gigE I think, and I've yet to come up against a download (even a torrent) that seems like it would have really benefitted from having the entire theoretical 1.5 pipe available.
Home users don’t need more bandwidth to improve their internet experiences, they need lower latency, less congestion and less loss.
https://help.netflix.com/en/node/306
https://learn.microsoft.com/en-us/microsoftteams/prepare-net...
More and more regular people are getting network storage appliances. More and more people have laptops with SSDs that can write at 4 or 5 GB/s. Why shouldn't they get to use all of it?
> I've yet to come up against a download (even a torrent) that seems like it would have really benefitted from having the entire theoretical 1.5 pipe available.
There are many things along the way that would get in the way of a home user downloading something from the internet that would hit that 5GB/s speed. It's not that people should be "banned" from it or something, more that the investment cost isn't worth it.
What’s described in the post is the tech equivalent of supe-ing up a sports car and then driving it in rush hour traffic. It’s fun to geek out doing it, but practically in everyday use the difference will be negligible. Even with large file uploads and downloads, there’s a good chance that services won’t reach those throughputs end to end.
What’s telling is that the post shows screenshots and charts from artificial speed tests. No videos of the Dropbox client chugging away with throttled uploads.
In my experience unless you actually pay attention and get something with a dram cache it will sustain that speed for all of 5 seconds and then drop to near useless, and with the current dram shortages that is getting harder and harder to justify.
I just did a build out where ram cost as much as the GPU and both are individually 3x more expensive than the CPU and MB combined.
A decent 2TB nvme drive was also more expensive than the CPU and MB combined.
A 10GB network connection can happily transfer at that speed all day, your SSD is unlikely to maintain Sata speeds unless you actually shelled out for something decent, what came from the manufacturer in the laptop is not that.
proper 4k, like the one you watch from a blu-ray, will have peaks of 150mbps.
the 4k we see on streaming services is awfully overcompressed.
and yeah, you can see the difference, it's day and night.
Torrents (as you mention) can be equally as fast. Well seeded, private tracker torrents can easily download at above 1Gbps (depending on piece size, etc).
Welcome to Italy :D
Sorry, I’m not familiar with any other way.
100 GbE is 15 years old too depending on which standard you think of. (The most common physical layer, SR4, is 11 yrs old)
Gigabit ethernet came much faster after 100 M.
https://www.amazon.com/stores/InvisiLight/page/DFF3C042-0D2E...
high latency, high error rates, and terrifying heat output from SFPs (which the author noted for himself)
the only cat6 left in my home network is the link to verizon's ont, because in their infinite wisdom the ONLY connectivity offered was 10g-base-t
The new 10GBASE-T SFPs are actually not too bad - you can get the full 100 meters at half the wattage it takes for the old space heater generation ones to reach 30 meters. Based on the article, the author did not know there were newer cooler options for the ~the same price.
They all just sit in a drawer, if I need a USB-C ethernet adapter i just grab my trusty 1 gig one.
Meanwhile I'm sat here wishing I could justify running any ethernet in my apartment, but improving wi-fi tech means I never can...
I wonder if you could negotiate down to 1gbit until you see some level of activity, if that would help at all?
I'm still eyeing 10Gb, but if my home needs +30w for three computers, I don't feel like it's really worth it. Would love to see more details on the power consumption from folks, especially tuning for idle.
But I'll keep using a gigabit switch because I have absolutely no idea what I'd use 10G for. It's crazy that gigabit was affordable for me as a student in the early 00s and between then and now we've gone from DVDs to 4K and it's still plenty fast enough. In fact, most people are happy with WiFi (not me, though).
How about this one? https://mikrotik.com/product/crs305_1g_4s_in
You can use fiber optic SFP+ in it which should not run as hot.
Run fiber everywhere for the fun of it, instead of copper Ethernet. Here is an interesting post about it:
https://sschueller.github.io/posts/wiring-a-home-with-fiber/
You can simplify it and avoid splicing which is the most complex thing about it.
You can fix the thermal issue either by adding a small fan (Noctua is great) or by adding more radiators: https://pics.ealex.net/share/UxeSf_AWHLIuc-qzK5zl7JIgQvQDAZh...
I've been running it like this in a closed comm box for the last 3 years without any issues. SFP+ modules actually do not use that much power, it's just that it's concentrated into a small package, resulting in high temps.
"The apartment has structured cabling -- each room has one or more RJ45 sockets in the wall," ...
Which is the main problem most folks face.
wish the standard was "conduit" instead of "bake-this-years-tech-into-the-wall" which doesn't always last...
But the simple truth for all those decades is this: When there's already cat-whatever cable in the wall, it generally still works.
Decently-installed conduit (ie, actually-usable conduit) adds a ton of time and expense, which is why it is very seldom used for data circuits in residential structures.
The cable that exists is a lot better than the conduit that doesn't. And copper ethernet is bog-standard like MP3 is: It isn't the best in any technical sense at all, but everything supports it. Universal compatibility is pretty nice.
---
So the ongoing cost of copper 10gbe is electricity. Someone else here in the comments says that a copper 10GBe SFP+ module can use ~3 Watts, or that a newer one can use about 1.5 Watts.
We can be generous by using the larger figure of 3 Watts, and 8 devices..
With 4 ports, eight 10-gig endpoints @ 3 Watts each, and $0.19 per kWh [delivered]: That's $3.28 per month, or about $400 per decade.
If we assume 1.5 Watt endpoints, then that number halves.
If we subtract the power consumption of fiber SFP+ modules (or media converters or whatever) to make the number a relative comparison instead of an absolute, then that figure goes down further.
Not so bad, compared to conduit.
Fiber is much better, but in my case the house already had Cat5 Ethernet wiring (originally used just for phones!) everywhere.
Another use case for 10G Ethernet is PoE for the WiFi access points. Although you can't use SFP+ modules for that, of course.
So I ended up just buying a switch that supports 2.5Gbps over copper since it's such a huge expense to step up. Anyway, my devices are pretty much 2.5Gbps (APs, flex mini switches, etc).
I pretty much always get at least 4600Mbit/s both directions over AT&T, and generally cap it out. Spectrum typically gives me at least 800Mbit/s down. The ISPs are definitely the bottleneck because neither provides dedicated infrastructure from each house to the CO, instead you have some sort of aggregation point which has a shared backhaul and that means you compete for resources, but having the largest plan on each at least gives you traffic priority.
Realistically, I don't need any of this, I was doing just fine with normal gigabit fiber with a 2.5GbE network before I moved last year, but it's nice knowing everything is as fast as possible in my path and that eliminates congestion and local network resource contentions as causes when a problem arises, I know it's pretty much always upstream of me somewhere.