I think it should be remembered that unlike the days of old, supporters are no longer only based in the town/city that the team are from. TV coverage has opened things up to many more people than the fathers walking down to to the ground with their sons on a saturday afternoon. Fans of teams now populate all corners of the country - world, even - so there will always be a neccessity for many to travel somewhere.
Admittedly the distance from the home teams ground for this game is substancial but it's daft to think that everyone will be travelling down from up north.
Also saying the the game should be in the Millenium Stadium for instance is no better than Wembley, surely?