Memory Limit Problem

[MrBaseball34]

I've been modifying the Google Base module to add a lot of functionality to it.

Been making good progress, however, have run into a problem that I don't know how to handle.

One of our shops has 24,000+ products and another has 100,000+.

You can guess that in trying to get all products using this code:

$Products = Product::getProducts(intval($this->_cookie->id_lang), 0, NULL, 'id_product', 'ASC');

is going to take up a lot of memory.
I increased my memory_limit in the php.ini file in the root to 256BM but I still get a memory exhausted error.

How should I handle this? I'm thinking of doing a kind of pagination thing but am not sure how to do it.

Anyone understand how to do this and can give me some pseudo-code on the solution?

[rocky]

Yep, pagination is how I got around this problem in the osCommerce Import module. Just use store the current page starting from 1, then use it to calculate the pagination parameters for the Product::getProducts function. Store the results in an array at the beginning of the loop, then process them, then increase the page number. Since each page overwrites the previous page, you won't run out of memory. Make the loop stop whenever a full page of products isn't returned.

[MrBaseball34]

After posting that, I went through and did a pseudo-code of how I thought the flow should go.

Everything works fine if s the text version but when I try to generate an XML version if goes bonkers and starts the loop over again after it is finished.

I use the same looping mechanism for both, the only thing that is different is the output.
Any ideas what could cause that? Apache may be resetting but that wouldn't make it go into the loop again would it?

[rocky]

Here's how I would do it:

$n = 100;
$p = 1;

do
{
  $products = Product::getProducts(intval($this->_cookie->id_lang), ($p - 1) * $n, $n, 'id_product', 'ASC');

  // Process products

  $p++;
}
while (sizeof($products) == $n);

This should process all the products 100 at a time.