Yes, it does seem small, but the BEAM often has response times in microseconds (μs). If you are used to that and something 'blows out' to milliseconds, then I can see why alarms might get triggered.
If it's millions of calls and it's 15 additional milliseconds per call, that's going to spectacularly gum up a system that's tuned in microseconds. Even if there's no additional load, the queues all just got several thousand times longer.
This exchange made me think about how I've always been a fan of using the "natural" units of a system when reporting metrics.
I do this because it helps those unfamiliar with the system more intuitively understand relative sizes, those unfamiliar with converting between units in their head to understand numbers, and even for those familiar, it can avoid confusion or misattributed units.