In real life you could start at any gym you want. I think the same is true in the games but there are usually things blocking your path to later gyms.
In real life it would be ridiculous for the gyms to literally be in view but you can't walk through some trees unless you have a Pokemon that knows Cut