The game itself was never bad, it was the old generation console releases that weren't even playable that hit the games reputation pretty hard.
Also people had expectations incredibly high due to the amount they spent on marketing the game, AND because Witcher 3 was so good.
I myself had my expectations too high because I loved Witcher 3 so much. I thought CDPR could do no wrong.