The issue is that the 1000th best man in many sports is better than the best woman.
If you made the sport unisex that would be the same thing in practice as making those sports cease to exist for women after puberty.
Imagine if there was some race of humans out there that was literally twice as strong and 6 inches taller than the average man. Their sports teams are full of 500 pound 7 and a half foot tall giants that are bigger, stronger, faster, and have far stronger bones than regular men.
Would you want to compete against them? What if they took some pills to make them merely 400 pounds of muscle and 7.5 feet tall?
I like that you put it in such a vivid, easy-to-imagine way. Drives home the point.