When you’re building a workstation or server intended for high reliability and solid performance, one of the chief obstacles you’ll face today is picking a processor and motherboard that can supports lots of affordable and reliable RAM. Ideally you want to be able to load up your computer with 32GB RAM or more. Popular inexpensive desktop CPUs from Intel support up to that amount of memory, but they do not allow the use of ECC (Error Correcting Code) memory. New processors in the “Zambezi” FX family released by AMD in the last few months bring ECC memory support to desktop PCs. AMD’s new Opteron 4200 and 6200 series workstation server processors compete very favorably on price vs. performance with Intel’s low-end E3 server line for applications that require a lot of RAM.
Why You Need ECC Memory
Many desktop motherboards are now supporting 32GB RAM. Many server motherboards are supporting 128GB RAM or more per processor. So for a dual to quad processor mainboard you could be looking at 256GB to 1TB of RAM. That’s hard disk drive territory in terms of capacity. As I discussed previously in Why You Should Use ECC Memory In Your Next Computer, with that much memory it’s likely on many computers a week won’t go by without a bit flipping randomly somewhere in memory. So you really must have ECC when you’re talking about that level of RAM if you intend to avoid weird problems such as random crashes from memory errors.
It used to be that 8GB or more RAM would be fantastically expensive. But today you can pick up 16GB of desktop grade non-ECC memory for less than $100. 16GB of server grade ECC memory costs somewhere around $150.
32GB of server registered SDRAM with ECC protection runs around $300 to $400. That is less than the price of many mid-range SSDs which are all the rage these days for speeding up a slow computer. But without ECC protection against stray bit errors, it’s simply quite risky to be using such large amounts of SDRAM.
What’s particularly nice about the 8GB and larger registered ECC DIMMs is that they are much less expensive than their similar capacity unbuffered SDRAM DIMM counterparts, especially the few unbuffered 8GB and larger DIMMs available with ECC. Getting 32GB of unbuffered SDRAM with ECC as four sticks of 8GB each will probably cost you upwards of $600, maybe even over $1000. The $300 or more in savings from using the registered DIMMs is often enough to pay the added cost of a better server motherboard, processor, case, and power supply versus the commodity equivalents that only use unbuffered memory.
Intel’s Poor Design Choices
Intel, the leading maker of desktop and laptop PC processors, has chosen to cripple its Core i3/i5/i7 desktop and mobile processors and E3 low-end single processor server chips with inferior memory controllers. It appears their motivation is to drive anybody who cares about computers being reliable and needs lots of RAM to pop for their E5 and higher server chips. For Intel that’s a lucrative market as such processors typically cost around $500 to $2000 each. In the case of the really high end Intel E7 server processors, it could be even more than that.
Today’s Intel desktop and mobile processor chips lack support for ECC memory entirely, meaning that any stray bit of radiation or noise in the memory can bring your whole computer crashing down without warning. If you’re just playing games on your PC or watching YouTube videos, maybe that is tolerable. But if you are using your PC for your work, it’s probably not.
Fortunately for Intel, they have a convenient scapegoat — Microsoft. They can just blame weird behaviors and crashes on Windows because so many easily believe that Windows is unstable crap. The problem with this is that Windows 7 actually is quite stable at this point. Sometimes there’s bad drivers or firmware that causes such crashes. But the odds are high that a significant number of the BSODs (Blue Screen of Deaths) that Windows users are seeing are coming from broken or unreliable hardware, particularly SDRAM memory. SDRAM DIMMs that experience random bit flips or electrical noise that corrupts a bit being read or written can trigger crashes or even corrupt data on your hard drive or SSD.
The low-end E3 server chips are very much like the Core i7 desktop processors as they use the same Sandy Bridge processor cores. In the E3 entry server chips, Intel did include support for basic ECC that can correct for single-bit memory errors. Unfortunately, they only support unbuffered memory modules. That means at the moment anything more than 16GB of RAM demands a ridiculous surcharge if you want to use ECC RAM in a server or workstation using an Intel E3 processor.
Unbuffered ECC DIMMS cost typically 50% to 300% more than their registered server grade ECC counterparts at the 8GB size plus they are very hard to find. A few months back, the best prices for such unbuffered 8GB ECC DIMMs were around $400 a crack while the registered 8GB ECC DIMMs, which won’t work with Intel E3 processors, were less than $100 from many vendors. Today the situation isn’t quite as bad, the prices are down to around $150 to $300 versus around $80 to $100 for many registered ECC DIMMs of the same 8GB capacity. Still, that is a hefty price penalty. It’s one of the major reasons why anybody expecting to need or want more than 16GB RAM should be looking at some other processor and motherboard options.
AMD’s FX and Opteron Processors Offer Good “Bang for the Buck”
So you might think you’d simply pay more for a better Intel processor such as an E5 and get a corresponding motherboard, too. You could use the savings from the registered RAM to pay for part of the cost difference.
That’s a great idea, but you might be better off buying AMD processor chips instead of Intel. As has been the case for years, AMD’s “performance for the dollar” ratio is often better than Intel’s.
AMD’s new Bulldozer processor cores introduced in the second half of 2011 have ECC support as incorporated in AMD’s desktop line of Zambezi CPU chips (FX-8120 and FX-8150). If you pick a good motherboard, you can use them with unbuffered ECC memory. There are several reports that the ASUS Crosshair V AM3+ motherboard supported unbuffered ECC just fine. But again, that’s unbuffered RAM and you are going to pay a premium for that. Figure for 32GB RAM, you will pay at least $250 to $500 more than if you could buy registered RAM of the same size.
Oddly, at the moment if you go that route then you’ll be stuck using slower 1333MHz (PC10600) memory versus the 1600MHz (PC12800) that is starting to become widely available for registered ECC DIMMs. Motherboards for AMD FX processors often support overclocked memory at 1866MHz or higher, but unbuffered ECC DIMMs right now are almost impossible to find at any speeds faster than 1333MHz.
AMD’s Opteron 4100 and 6100 series processors have supported ECC memory for the past couple of years. The cheapest processors from that group are around $100. The cheapest mainboards are around $200 to $250. These will work fine with DDR3 ECC registered DIMMs running at up to 1333MHz. For 8GB DIMMs, the price each with such specs is around $70 to $100 depending upon where you buy.
If you are buying new hardware today, I’d suggest you consider AMD’s Opteron 4200 and 6200 series processors. They support memory speeds up to 1600MHz. They often cost less than a comparable performance 4100 or 6100 series processor, often much less. Right now they only provide a small performance boost at the moment versus similar numbered older processors (e.g., Opteron 6272 versus 6172), but that will probably turn into a moderate boost after more programs are recompiled with new compilers to support the advanced instructions the chips include.
Bad Press Undeserved
AMD has recently taken a beating in the press for lower than expected performance of its new Bulldozer cores in its “Zambezi” FX desktop processors and “Valencia” Opteron 4200 and “Interlagos” Opteron 6200 server chips. Yes, they are slower than the very fastest Intel chips. That’s in part because to get top speed out of these new chips, software needs to be recompiled which will happen over time.
Even if you only get the current performance out of these chips, frankly does a 10% to 20% processor speed difference really matter that much for most uses these days? Do you want to pay $1000 for a somewhat faster Intel server CPU or $500-600 for an AMD CPU that is almost as fast? AMD seems to be winning the current “bang for the buck” contest with Intel.
Intel Has Edge on Idle Power Consumption
However, I do think it is only fair to Intel to note they have gotten really good at lowering CPU power usage so over the course of several years it is possible that Intel’s higher prices may be offset substantially by savings in electricity bills. However, that probably depends in part upon how the computer is used. If it is running at a high load all day long, the power savings may simply not be major. Intel’s power savings seem to be highest when the processor is the least loaded.
AMD’s ECC Often Better Than Intel’s
Even on chips where Intel has implemented ECC, such as the E3 line, their ECC protection often doesn’t measure up to AMD’s. Intel usually puts in just simple single bit error correction circuits. While that helps a lot, it doesn’t come close to the benefits of AMD’s “chipkill” ECC implementation found in their Opteron processors. This ECC algorithm can correct for even a flakey memory chip that has cross-bit leakage because it spreads the ECC bits around the DIMM so that even a single totally failed chip cannot crash the system.
AMD didn’t invent the “chipkill” algorithm — credit goes to IBM decades back. But AMD is due credit for putting it into relatively inexpensive server processors.
Consider Overall System Balance
Often the CPU is no longer the bottleneck in a workstation or server. Inadequate RAM or a slow disk system is far more often an obstacle to performance. It’s easy to completely fill the RAM memory even on a desktop single-user PC with 8GB RAM. Simply open up a few dozen web pages and a few other typical programs (word processor, spreadsheet, etc.) and your PC will be busily paging code and data to and from your hard disk drive. For such a PC, often the cheapest and most significant performance upgrade is to add more RAM. But that’s precisely what it is difficult and expensive to do with Intel’s current desktop CPU offerings once you pass about 16GB RAM.
If you’re lucky, you might have a SSD (Solid State Drive) which is way faster at random access virtual memory paging than a mechanical hard drive. But still no mass storage device today beats the performance boost of simply adding more RAM to a PC that doesn’t have enough. And if you make the mistake of having too little RAM and putting your virtual memory paging or swap file on your SSD, you will wear that expensive SSD out a lot faster.
So I’d argue that a lot of folks building workstations or servers today should be giving the AMD Opteron processors serious consideration for the reasons discussed above. What’s particularly neat is that virtually every one of the Opteron 4100 series mainboards will support Opteron 4200 chips with just a BIOS update. The sames goes for most mainboards that took 6100 chips — upgrade the BIOS and they work with the 6200 series.
Single-Threaded Software Favors High Clock Speeds, Newer Multithreaded Software Favors More CPU Cores
If you’re running a lot of single-threaded desktop PC software, this is where the Opteron 4200 series may have the advantage over the 6200 series. The 4200 series chips are basically half a 6200 series chip. Because of power and thermal considerations, the 4200 series with half the cores of the 6200 can be operated at higher clock speeds. So an Opteron 4238 with its 6 cores can be run at 3.3GHz whereas the fastest Opteron 6200, the 6282 SE, runs at 2.6GHz with its 16 cores.
The future is not single-threaded code, however. As quad core CPUs have become common, many software developers are now updating their programs with higher core and thread counts in mind in order to gain performance. A well multithreaded program running on a 16 core CPU at 2.1GHz will almost certainly outperform a single-threaded program running on just one core of a 3.4 GHz processor. The performance difference can be very substantial for users who are running a lot of programs at once.
I think what we’ll see happen in the next couple years is that virtually any program for which performance is a serious consideration is going to be modified to support many more processors cores. With Intel and AMD both shipping boatloads of quad core desktop processors, multicore is here to stay. And now that AMD is shipping desktop processors with 8 cores and server processors with up to 16 cores, it is clear that in the future the performance champ is going to be at least a quad core chip and that no single-threaded program is going to be able to take full advantage of it.
AMD Opteron 6100/6200 Offers Good Performance/Price Ratio and Upgrade Path
You can pick up an Opteron 6100/6200 series mainboard such as the excellent ASUS KGPE-D16 dual processor board for around $400 to $450. If you are aiming at saving money and scaling up the system over time, pop in an Opteron 6128 for about $250 and you’ll get solid performance good enough for most uses as it’s around as fast as AMD’s Phenom II six-core chip or Intel’s recent two core i5 chips. Then when you’re ready for a faster CPU in a couple years, you can yank out the cheap 6128 and replace it with a twice as fast 6272 — or two — without having to change any other hardware. It’s great to have an easy upgrade path so you don’t have to waste your time extensively overhauling a system and reinstalling everything from scratch.
If you need something faster right away, look at the Opteron 6200 “Interlagos” series. An Opteron 6272 performs about twice as fast as the 6128 for about $300 more. Plus that leaves you with a second processor socket to upgrade later if you need more CPU or RAM than what the first processor socket and its associated 8 DIMMs sockets can provide.
A helpful resource for getting a rough idea of how fast a processor can be is the PassMark high-end CPU benchmark list. For instance, it shows the Opteron 6272 (sold for around $550) with a score about the same as the cheaper (around $320) Intel i7 2600K desktop CPU and about 50% faster than the Intel E5645 server processor which also sells for around $550. Please don’t use this list as the only resource you consult. Many other sites may offer more applicable benchmarks such as Cinebench, AES encryption, server workload, and others that are more applicable to your use. But this list does do a good job of at least showing a rough ordering of performance along with an approximate price for each CPU. It’s very helpful when you’re in your initial stages of figuring out what CPU to pick for a computer build.
AnandTech’s overview of the Opteron 6200 series has some helpful performance and pricing comparisons, too. They agree that the Opteron 6200 series offers more performance for the dollar than Intel E5 line does right now:
The Opteron 6276 offers a better performance per dollar ratio. It delivers the performance of $1000 Xeon (X5650) at $800. Add to this that the G34 based servers are typically less expensive than their Intel LGA 1366 counterparts and the price bonus for the new Opteron grows. If performance/dollar is your first priority, we think the Opteron 6276 is an attractive alternative.
Opteron 4200 Options
Another nice option that is somewhat less expensive is the ASUS KCMA-D8 dual processor board for around $280 to $300, about $150 cheaper than the KGPE-D16. It supports two C32 socket chips in the AMD Opteron 4100 and 4200 “Valencia” series. You can get performance similar to one 16-core 6200 series processor on the KGPE-D16 by using two C32 socket processors. Not only is the board cheaper, but the case and power supply can be a little less expensive, too, as the KCMA-D8 works with smaller power supplies and is an ATX size board that fits in many cheap computer cases.
The much larger SSI EEB sized KGPE-D16 requires a case like the mid-priced CoolerMaster HAF 932 (around $140-160) or the inexpensive (often around $70-80 when on sale at Newegg) Rosewill RSV-L4000 server chassis. And it needs a power supply with TWO 8-pin 12 volt connectors, generally meaning a 700 watt or higher power supply, whereas the KCMA-D8 only needs one 8-pin 12 volt connector.
I’ve used the ASUS KGPE-D16 in a Rosewill RSV-L400 chassis with a PC Power & Cooling Silencer Mk II 950 watt power supply for some recent builds myself. I’ve been overall happy with this combination. (If you need a tower chassis rather than rackmount, I’d suggest the CoolerMaster HAF 932.)
This setup has a lot of room for growth for processors, RAM, and disks, enough that I expect to be easily able to scale starter systems up to relative monsters with dual 16-core CPUs, 128GB RAM, and 8TB or more hard disk storage without having to replace the SDRAM DIMMs or any parts other than maybe the cheap Opteron 6128 processor that is often good enough for starters. I suspect other people may be doing the same, so there may be a glut of used Opteron 6128 chips on the market in the next year or so as people decide to upgrade to faster CPUs as their needs grow. Already you can find some used 6128 chips on Ebay for $100 to $150.
When you’re picking memory for Opteron systems, try to get 1600MHz DIMMs that are single or dual rank. Quad rank DIMMs are OK if you plan to only populate one DIMM per memory channel, but if you add a second quad rank DIMM to a channel then the speed drops considerably. For instance, putting in two Kingston quad rank 8GB ECC DDR3 1333MHz DIMMs in one Opteron memory channel will get you 1066MHz performance. Dual rank used to be a lot more expensive, but right now it is often only $5 to $10 per DIMM more at the 8GB size. At 16GB, however, there are fewer dual rank offerings and often people are using 1066MHz DIMMs, too, as the 1333MHz ones are still quite expensive.
I’m particularly happy to see the supply of 1600MHz DDR3 registered ECC DIMMs growing quickly. A couple of months ago they were impossible to find. Now at the vendors that have them in stock, they are often priced around the same as the older 1333MHz DIMMs. For instance, Superbiiz is selling 8GB 1600MHz DDR3 ECC registered DIMMS for about $75. Hopefully Newegg and other vendors will catch up and start stocking such DIMMs soon. With such parts it is relatively cheap and reliable to pop 32GB RAM or more into a computer built around the AMD Opteron 4200 and 6200 series processors.