I'm referring to the majority of users not having (or wanting to use) phone numbers.

Some of these will be willing and able to pay $1/month to Twilio for a workaround, but most probably won't.