The big issue with the stadia is that as part of the bid the IRB has been guaranteed a certain level of income, which is underwritten by the Govt. Had they only used only rugby stadia they wouldn't have sold the tickets or, more importantly, the hospitality to hit their income target.
Sadly it's not just football where money talks.