Jump to content

Memory Limit Problem


MrBaseball34
 Share

Recommended Posts

I've been modifying the Google Base module to add a lot of functionality to it.

Been making good progress, however, have run into a problem that I don't know how to handle.

One of our shops has 24,000+ products and another has 100,000+.

You can guess that in trying to get all products using this code:

$Products = Product::getProducts(intval($this->_cookie->id_lang), 0, NULL, 'id_product', 'ASC');


is going to take up a lot of memory.
I increased my memory_limit in the php.ini file in the root to 256BM but I still get a memory exhausted error.

How should I handle this? I'm thinking of doing a kind of pagination thing but am not sure how to do it.

Anyone understand how to do this and can give me some pseudo-code on the solution?

Share this post


Link to post
Share on other sites

Yep, pagination is how I got around this problem in the osCommerce Import module. Just use store the current page starting from 1, then use it to calculate the pagination parameters for the Product::getProducts function. Store the results in an array at the beginning of the loop, then process them, then increase the page number. Since each page overwrites the previous page, you won't run out of memory. Make the loop stop whenever a full page of products isn't returned.

Share this post


Link to post
Share on other sites

After posting that, I went through and did a pseudo-code of how I thought the flow should go.

Everything works fine if s the text version but when I try to generate an XML version if goes bonkers and starts the loop over again after it is finished.

I use the same looping mechanism for both, the only thing that is different is the output.
Any ideas what could cause that? Apache may be resetting but that wouldn't make it go into the loop again would it?

Share this post


Link to post
Share on other sites

Here's how I would do it:

$n = 100;
$p = 1;

do
{
  $products = Product::getProducts(intval($this->_cookie->id_lang), ($p - 1) * $n, $n, 'id_product', 'ASC');

  // Process products

  $p++;
}
while (sizeof($products) == $n);



This should process all the products 100 at a time.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...

Important Information

Cookies ensure the smooth running of our services. Using these, you accept the use of cookies. Learn More