I have never, nor will I probably ever, heard someone proclaim the # meaning pound.
For 40 years, I have only ever referred to it as either the "number sign" or the "pound sign", and give dirty, disdainful looks to anyone who refers to it as anything resembling "hash".
Then again, for 40 years, I have only ever referred to elevators, flashlights, and soda as those things, and have never heard a single person refer to them as "lifts", "torches", or "pop" (except occasionally people on TV or the Internet).
Your issue might be a regional thing, an age thing, or just because everyone you've ever known is a terrible person.
Nope, it's because America is crazy moreso than anything else. :P ---