What if someone would never have sex with a trans woman who has not had surgery? I don't see how it is transphobic to not want a penis involved. But I agree that that person may still find them attractive.Nobody has to have intimacy with a penis (or a vagina!) if they dont want that. Its important to communicate boundaries with a potential partner.
Not advocating saying super straight though.