godelski • 1 day ago 

  > modern CPUs are so fast that instructions are almost free.
Please don't.

These things compound. You especially need to consider typical computer usage involves using more than one application at a time. There's a tragedy of the commons issue that's often ignored. It can be if you're optimizing your code (you're minimizing your share!) but it can't be if you're not.

I guarantee you we'd have a lot of faster things if people invested even a little time (these also compound :). Two great examples might be Llama.cpp and FlashAttention. Both of these have had a huge impact of people (among a number of other works) but don't get nearly the same attention as other stuff. These are popular instances but I promise you that there's a million problems like these waiting to be solved. It's just not flashy, but hey plumbers and garbagemen are pretty critical jobs too

1
EnPissant • 23 hours ago

You haven't refuted the parent comment at all. They asserted that instructions are insignificant, and other things, such as memory accesses, dominate.

2 replies
godelski • 4 hours ago 

  >> You especially need to consider typical computer usage involves using more than one application at a time. There's a tragedy of the commons issue
These resources include:

  - disk/ssd/long term memory
  - RAM/System memory
  - Cache
  
  BUT ALSO
  - Registers
  - CPU Cores
  - Busses/Lanes/Bandwith
  - Locks
  - Network
My point is that I/O only dominates when you're actually acting efficiently. This is dominating in the case of measuring a single operating program.

You're forgetting that when multiple programs are running that there's a lot more going on. There's a lot more communication going on too. The caches are super tiny and in high competition. To handle interlacing all those instructions. Even a program's niceness can dramatically change total performance. This is especially true when we're talking about unoptimized programs because all those little things that the OS has to manage pile up.

Get out your computer architecture book and do a skim to refresh. Even Knuth's Book (s)[1] discuss much of this because to write good programs you gotta understand the environment they're running in. Otherwise I'd be like trying to build a car but not knowing if you're building it for the city, Antarctica, or even the moon. The environment is critical to the assumptions you can make.

[0] https://en.wikipedia.org/wiki/Nice_(Unix)

[1] https://www-cs-faculty.stanford.edu/~knuth/taocp.html

windward • 17 hours ago

They do, until you have a tough problem that's still too slow after it's cache efficient