Odd behavior CSV Combination Import server 500 error yet combos keep adding after error

[BTutt]

PS Ver. 1.6.0.14

 

If you get a server 500 error - don't give up. Read on...

 

I was doing a CSV import for product combinations of about 2,200 lines in a CSV file. After a few minutes I get a 500 server error. I was bummed... I took a look at my products to see what products were updated and which products failed. I noticed some had not been updated. As I continued to investigate, to my surprise, it appears the combination additions are still taking place after the 500 server error. More and more of my products are being updated AFTER the 500 error. Even 20 minutes later the additions are still happening!

 

Hopefully all the products receive the new combinations. So, if you have a 500 error when importing a CSV, check for updates after you get your 500 error. There must be a bug in the import and it is taking way too long. I am thankful the bug is in my favor.

[BTutt]

More errors and issues with 500 errors... But I solved the problem.

Symptoms: When importing a bunch of combinations in a CSV file the server was throwing a 500 Internal Server error. The cart was failing to process the combinations. Sometimes it was a complete failure, and occasionally it was only updating a few products. I looked at the web server logs and found that the issue was my request was timing out.

The logs in my case were located here:

/usr/local/apache/logs/error_log

 

Here is the solution to 500 Server errors if you have access to your server using WHM:

Via WHM C-Panel Navigate to

Home » Service Configuration » Apache Configuration » Global Configuration

Under Timeout set it to a higher rate. 300 was the default (5 minutes). I set a new timeout to 7200 (120 minutes). You may need more if you are uploading a lot of items in a CSV file.

 

Via WHM navigate to

Home » Service Configuration » PHP Configuration Editor

 

Under max_execution_time set it to a higher rate. 30 was the default. I set a new time of 7200.

I also tweaked the memory limit because I wanted to see if better performance would be key to good results.
Under memory_limit I raised that value to 1024M (1G). I have 8G of ram so I thought I would have plenty of ram to do that. You may not have that much available.
Looking at the server ram during a CSV import I noticed that the server was using about 1.3 G of memory. I do have multiple sites on the server, so some of that ram was being used elsewhere. I am guessing this helped the process, though.

I hope this information helps anyone having CSV import problems. If anyone notices an issue with my recommendations, please suggest a better way.
 

Edited August 12, 2015 by BTutt (see edit history)