Who told you it doesn't matter? For top programs, GPA and GRE scores are really important. It is true that, unlike law school or some other pre-professional tracks, raw objective measures alone won't get you in, but they will keep your application from getting thrown out.
From there, it's just a matter of strong recommendations and prayer. Lots of prayer.
--
The Many Forms of Kobe:
http://i39.tinypic.com/oqx98z.jpg -- credit to KIRBEH