I am having a similar problem, I keep increasing the memory limit which gets me a bit further each time, however I am yet to complete a site map.
Is there any way of estimating the memory limit i will need based on the number of pages? It is a large site and as an esitmation i would guess there is somewhere in the region of upwards of 50,000 pages.