Part 4 had some of the most impactful if only because the story had time to actually mourn them unlike most of the other parts.I actually really like Giorno and part 5. Its probably the part with the least amount of downtime between major fights, but it makes the threat of the mafia and assassins feel real.
Part 5 was pretty bad and it doesn't help how terrible a MC Giorno is. Part 6s ending though I personally didnt like and killed the whole franchise for me up till then.