I would at least knock out hard diaries. Theres solid QoL from them, plus the xp from quests and diaries helps more the earlier you snag it. All the diaries came out before the push for more difficult quests so none of the hard ones are required for them.