bluGill 3 days ago

Hundreds of hours of manual testing. I don't have to do safety certificates, but my code gets 500 hours of manual testing (I'm not allowed to give real numbers, these numbers are close enough) - they find enough critical can't ship issues where the fix is risky enough to start all over that we typically are doing 2500 hours of manual testing. on every release.

We have a large automated test suite that runs on every build and takes hours. The problem with automated tests is they only verify situations you thought of work the way you think they should, while human testers find slight variations of setup that you wouldn't think matter until they do. Human tests also find cases where the way you expect things to work don't make sense in the real world.

1
bregma 2 days ago

Wait until you find out about the cat test. It found a failure mode no human had thought of. No amount of the developer claiming a test like that was not fair was enough to invalidate the results. No actual cats were harmed but treats may have been given.

ModernMech 2 days ago

Do you have more context? I'm having trouble googling what you're referencing.

noisy_boy 2 days ago

Simulate a cat walking on the keyboard to handle weird inputs?

ModernMech 2 days ago

Isn't that just fuzzing? I thought maybe there was a specific thing called the cat test.