Backyard Fence with shared driveway

Hi AverageJoe;

So it sounds like your neighbour was waiting for the new owners (you) to move in before raising the issue of the newly built fence....not uncommon I'm afraid.

I'm going to make some assumptions in answering your question:
1. You are in Ontario.
2. There actually is an easement on title in the location that you think it is.
3. The fence actually is encroaching on the easement.

According to the law nothing can block or obstruct an easement (which is what the shared driveway is), regardless of whether the purpose of the easement - in your case accessing garages at the rear - is no longer relevant. Therefore if the fence has been built in such a way that a portion of it is inside the easement, then your neighbour has the right to ask for it to be removed. 

Practically speaking, though, their options as to how to do this pose some challenges for them. If you refuse, their only option is to take legal action which is both time consuming and very expensive. This is not a matter that the police or city will help them enforce. 

I hope this helps.