Well what's easier for you to deal with, either feel guilty for a minute or experience uncomfortable social anxiety for a day.I'm looking at it the opposite. The uncomfortable anxiety you know lasts for a day. But the guilt you never really know until it happens.
Personally the guilt would fade pretty quickly, so I just wouldn't go.