Is it something specific to iOS Telegram client?
On Android Telegram works with denied access to the contacts and maintains its own, completely separate, contact list (shared with desktop Telegram and other copies logged in to same account). I'm using Telegram longer than I'm using smartphone and it has completely separate contact list (as it should be).
And WhatsApp cannot be used without access to contacts: it doesn't allow to create WatsApp-only contact and complains that it has no place to store it till you grant access to Phone contact list.
To be honest, I prefer to have separate contact lists on all my communication channel, and even sharing contacts between phone app and e-mail app (GMail) bothers me.
Telegram is good in this aspect, it can use its own contact list, not synchronized or shared with anything else, and WhatsApp is not.
I’ve never allowed Telegram on iOS to access my contacts, camera, or microphone and it’s worked just fine.
Looks to me like it was a bug. Not giving access to any contacts broke the app completely but limited access works fine except for an annoying persistent in app notification.