gps is accurate to a few meters, which isnt accurate enough i dont think. assume a dirt road with no markings, surrounded by more dirt. how would the car stay on its side of the road, not crossing over onto the other, or leaving the road and driving in very bumpy s*** dirt that can flip the car?
I don't think you fully comprehend how accurate the sensors can be. If you "tell" the car there's a road within 3 feet of spot X, and the car's sensors scan across and register "BUMPY TERRAIN -> 15 FOOT WIDE FLAT SPACE -> BUMPY TERRAIN" it's going to be pretty easy to register the flat part as the road and drive on the right half of that flat part.
There's no need to overcomplicate the thought process here.
assume well-driven sand vs very similar sand that isnt well driven. exact same colour ---
