Hello!
I have a problem with generator. I start crawlling and everything it's fine. When it reaches 150-170.000 pages eating all memory stops and everything freezes.
I've increased the values to:
max_execution_time 4000
memory_limit 4096M
But nothing has changed
Any suggestion?