Precisely. You only really see paparazzi harassing people in the hotspot cities. When's the last time anyone's ever heard of a celebrity getting hounded by paparazzi in Nunavut? This is why certain celebrities like Keanu Reeves don't get seen often unless they've got a big role coming. They live in places where there's not much going on, and it's more trouble than it's worth to go fly to some quiet city to maybe get a quick snap or two of some celebrity.
Plus celebrities have more than one property they stay at, so even if they needed a residency nearby for work purposes, they could still have a house far away from the noise.
And Ye has already been in trouble with dealing with paparazzi, so it was smart of him to move out of there and be with nature. It's quite possible that this incident was the moment where he started thinking of getting the hell out of there.