Upgrade odin memory to match ysera? #1105
Comments
|
When both machines are at 100% ysera is consistently generating 5% more tiles. This is under 1 metatile per second. A "new" odin or ysera would cost 2200 GBP, so 12% of the cost of an identical server for 5% of the performance gain. We wouldn't buy another odin/ysera, we'd buy something newer, and the cost effectiveness of the upgrade is even worse in comparison to that. I don't think we should do this. |
|
Have you ever considered upgrading the CPUs instead of RAM? Although @pnorman may be right that upgrading the RAM on these machines with already 256GB may not be worth the money in terms of performance gains, upgrading the CPUs might be worth it. I noticed many of the current OpenStreetMap servers use mid-range CPUs. While that is perfectly understandable to keep the cost down of a newly acquired brand new system, as the CPU cost and choice can be a very significant part of the cost of a new system, it also means there is considerable room for performance improvements once more powerful high end refurbished CPUs of the same generation hit the market, often at a cost that is only a fraction of the original new price. E.g. the current Supermicro systems run dual Intel Xeon Gold 5120 105W TDP CPUs. According to PassMark benchmarks, those have a 29566-MT and 1730-ST score in dual CPU configuration, which is well below the top of their generation. A nice replacement for those CPUs would be the Intel Xeon Gold 6148. This, not very latest series, also FCLGA3647 socket, processor with 150W TDP should be supported on the system according to the docs of Supermicro. These processors score 46668-MT and 2110-ST PassMark results in dual CPU configuration, which is about 55-60% higher in multi-threaded performance than the current processors. So essentially, replacing all four 5120 processors in odin and ysera with 6148 CPUs would potentially result in a performance equal to three of the current servers, with added bonus of about 20% higher single threaded performance as well. These processors can now be had for as little as about 200-225 euros including VAT, e.g. see these sites: So for about 800 euro's, you could extend the life of these two servers for another few years with a clear uptick in performance. Replacing CPUs is entirely doable. I replaced the dual Xeon E5-2680 v4 CPUs in my HP Z840 workstation with dual E5-2699 v4 processors, and got a similar about 45% uptick in multi-threaded performance closely matching the PassMark results. Yes, it is not as quick and easy as popping in and out a new memory bank in a slot. But reading the available maintenance documentation and carefully follow the torc adjustment instructions regarding screwing down the heat sink on the processors, should allow a successful replacement without to much hassle. Only the very latest behemoth processors and sockets seem to be quite finicky and fragile, and may need special torc-adjustable screw driver tools to safely replace a CPU from what I read, but not these older processors. |
|
I now noticed that piasa is already using that suggested Intel Xeon Gold 6148 processor in an HPE Proliant system in a similar role as tile server, so it would be nice to compare throughput on piasa when under max load with the odin and ysera systems to see if the figures of real world performance match up to expectations. |
|
@mboeringa Yes, I did put some time into looking at potential CPU upgrades. I had primarily been looking at Cascade Lake, as these are the best the system supports. I wasn't able any great value Xeon Cascade Lake CPUs on the reseller market. Best I could easily find is the Xeon Gold 6230. In pure render speed there doesn't seem to be a huge improvement from Xeon Gold 5120 and a Xeon Gold 6148. Realistic difference is only maybe only 20% / 25% improvement. The Xeon 6148 does indeed look like very good value. I've a large collection of different high-end thermal pastes and high-end thermal pads. Despite this I am not a huge fan of server CPU upgrades. It is a risk prone exercise. Our Supermicro systems are built by Supermicro in the Netherlands and pre-certified by them. |
Is this anecdotal or your own experience? Have you ever had real failure of a system after an upgrade? My track record of doing these upgrades is admittedly non-existent, as I only did this twice (swapped out a Core i5 for i7, and these Xeons in the HP Z840), but I honestly and generally found it easier than expected if done with some due care. The main challenge is spreading the thermal paste properly, so as not to create heat spots on the processor where it cannot properly dissipate its heat, as you undoubtedly know. Since the upgrade some 1,5 years ago though, I have trashed my HP Z840 for months on end at max CPU load without any noticeable (thermal) issues. The system has been rock solid ever since. Yes, the Cascade lake processors do not yet seem to have come down enough in price to be worth it, while the 6148 is nearly the same speed for a fraction of the price. Of course, if the real world experience still doesn't add up, it may need some further deliberation. Any other potential bottlenecks that may cause it, or some configuration needing a change to push the processors to their true potential? |
|
I wouldn't want remote hands to do this, so it would have to be Grant on one of his site visits. Time always feels limited on those. |
|
We have decided to upgrade, if possible. At the same time we will upgrade the RAM to make the machines the same. The Supermicro chassis supports up to 165W TDP except for 6144, 6146, 6244, 6246, 6250 and 6256. The motherboard itself supports 205W CPUs and up to 28 cores. The manual says it supports Dual Intel Xeon 81xx/61xx/51xx/41xx/31xx series or 82xx/62xx/52xx/42xx/32xx series processors. The cheap processors are
8173M is about 10% faster than 8160 During daytime peak the CPU pressure is 4% and IO pressure is < 1% so we're not currently IO limited. We limit parallelism to slightly under the capacity of the machine so it remains responsive, thus the CPU pressure would be much higher if we didn't limit. |
|
The servers has a SuperServer 1029P-WTRT chassis which says it ships with 2x SNK-P0067PSMB "1U Passive High Performance Front CPU Heat Sink" SNK-P0067PSMB specs say "Up to 165 Watts" |
|
I have found online resellers selling systems with 2x |
|
|
The 6148 still seem the best value for the money in this comparison. High MT performance, high base frequency, and low cost. Added benefit is the slightly lower TDP that doesn't reach the limits of supported TDP of the systems, leaving some headroom. |
|
I have ordered the matching memory. I am also going to go ahead with the purchase of the 4x Xeon 6148. Thank you @mboeringa for the recommendation! |
|
Order of 4x Xeon 6148 CPUs confirmed. Bargain price of £425 total. |
|
Orders arrived. CPU and RAM now in stock in Catford. |
|
Memory upgraded and confirmed working. CPU upgrade pending. |
|
CPU upgraded but running very very hot. The thermal paste takes awhile to settle in and there is a very bad airflow pattern with the switches. We can create new tickets if needed. |
ysera and odin are twins.
We recently upgraded ysera from 256GB to 384GB of RAM.
Do we also want to upgrade odin from 256GB to 384GB of RAM?
Cost: £260 inc
Remote hands is an extra fee, else upgrade can be timed with the next site visit.
The text was updated successfully, but these errors were encountered: