I don't think you are meant to have milk in Earl Grey.
Once I ordered it, due to it being mentioned by Picard, and when I asked for milk I was told no and advised to order a different type of tea if I wanted milk.
You can have Earl Grey with milk or lemon. Lady Grey, on the other hand, should always be with lemon.