I would imagine Vision doesn't have memories because Wanda didn't want them. Like somebody else said, Vision is dead, so the WandaVision Vision is a construct. If he knew he was an Avenger and everything that implies, I doubt he would have settled into that life without a second thought (running away from Sokovia accords, duty to help people,...). I'm also suspicious of Pietro's questioning of Wanda of how she managed everything; he seems to be the most dangerous outsider.
I really don't understand why people see Agnes as a potential villain when she feels to me as just the supporting character of this show. I could see her having a bigger role, but I doubt she's the mastermind. On the other hand, Vision did make the note of her getting lost in her hometown, implying either Westview is not her hometown or she's being affected by the magic. Aside from that, what has she done? Help Wanda with the anniversary, help with the magic show and prep her for Dottie, ...
Though honestly, I'm surprised I remember anything about this episode...Elizabeth Olsen in that costume was very...distracting.