When I was a kid i thought some of the episodes were scary too. Most of the things they deal with involve murders with unusual circumstances. It's instinct to fear the unknown.
And yeah it was worth it. I watched the show pretty regularly when it first came out back in 93 but i stopped right when they really started getting into the whole alien conspiracy stuff. So it was worth it too see how they decided to play that out on top of being genuinely intrigued and entertained by the explanations they were able to come up with for the different x files.