HPC Blog Dratted laws of physics. Cranking up frequencies is difficult due to leakages from ever smaller guard rails on the electron highways inside the processor. You have to jack up the power to make sure the instructions make it through, which leads to thermal problems.
Martin Hilgeman, HPC consultant with Dell EMC, gave a tour de force presentation at last month's HPC Advisory Council meeting in Lugano, Switzerland. He explained the conflict between Moore’s Law and Amdahl’s Law. Sure, we get more transistors, cores, and such in compliance with Moore’s Law, but it’s damn hard to actually use them all efficiently due to Amdahl’s law.
We can still get more cores on a processor, but can they all be used efficiently by software? It’s also possible to have more sockets per system as well, but that is also a challenge for power/density and cooling.
A real problem, as Hilgeman points out, is memory bandwidth per core. It’s been dropping by a significant amount over the succeeding generations of Intel processors. This problem is highlighted when you consider that some workloads, like today’s Big Data apps, frequently encounter data sets with very sparse matrices. In other words, a lot of "zero" entries. These types of data sets have very low arithmetic density and are thus memory bound.
Hilgeman goes on to discuss how to significantly increase HPC code performance when you don’t have the source code, and talks about some tools from Dell that make this easier to achieve.
You can get the slides here (PDF) or watch the video below.