I mean, he doesn't have to be a pedo to be a weirdo. That was far from the only thing he got negative attention for.
But anyway, he was one of the greatest entertainers of the 20th century, or even #1.
There's a difference between canon and not-stupid.