What kind of threat model do you operate under where Signal Technology Foundation knowing (one of) your phone numbers is a large risk?
The fact that signal knows my phone number doesn't bother me.
The fact that anyone who knows my phone number can know that I use signal does bother me.
May I ask why you’re bothered by this fact?