Dolke Posted June 12, 2013 Share Posted June 12, 2013 (edited) Hi, This is a direct question for all prestasop developers and people behind the project: Is CSV importe functionality sabotaged on purpose so I can only import/update no more then around 70 to 100 products per one CSV file??? After numerous phone calls with the support agents in my hosting company, after weeks of work and after making hundreds of tests and changing the settings I have no other to think but that this feature is sabotaged on purpose! As I explained in this topic: http://www.prestasho...oduct-csv-file/ I can only import small chunks of my data at the time and that is pretty useless. Anyone can relate to this problem? Thank you Dean Edited June 12, 2013 by Dolke (see edit history) Link to comment Share on other sites More sharing options...
pbuild Posted June 12, 2013 Share Posted June 12, 2013 I can import a couple thousand products in one CSV, so this is perhaps something specific about your setup? Link to comment Share on other sites More sharing options...
sadlyblue Posted June 12, 2013 Share Posted June 12, 2013 I don't believe they are doing anything close to what you are implying. Long scripts usually time out, or exced memory limit, or any other limitations. We have a dedicated server, and even we get problems with very long scripts. Mainly because it's not advise to set unlimited resources to php. So eventually, the scripts "break". I prefer to "split" into smaller tasks, or if its a really tested script to run it from cli. And even then it's not 100%. I made a script to import more than 5000 to our store at once to run from cli, and it also timed out a couple of times. It was too much data. With pics and everything... Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 Guys, thank you for taking interest in my case. I spoke with the HostGator support agents (more then 10h all together on the phone and chat support) and they run several tests in order to see what exactly is causing this error to appear. They told me that it is not timeout nor CPU and memory limit but someother errors that appearing in the server log. This is what agent told me they are seeing: "premature end of script headers...." @pbuild Where are you hosting at and what plan do u use? @sadlyblue Sir, so prestashop recommends that (if that is possible in our hosting environment) set all php settings to maximum values? I cant believe that you have to do that even on the dedicated server! Forgive me, but then this whole part of prestashop is just poorly imagined. You cannot let your software to overtake hardware limitations, or if this is absolutely necessary and doesn't have an alternative, they should write some sort of statement informing users of possibility this situation happens. Link to comment Share on other sites More sharing options...
PrestaCoder.com Posted June 12, 2013 Share Posted June 12, 2013 I don't believe they are doing anything close to what you are implying. Long scripts usually time out, or exced memory limit, or any other limitations. We have a dedicated server, and even we get problems with very long scripts. Mainly because it's not advise to set unlimited resources to php. So eventually, the scripts "break". I prefer to "split" into smaller tasks, or if its a really tested script to run it from cli. And even then it's not 100%. I made a script to import more than 5000 to our store at once to run from cli, and it also timed out a couple of times. It was too much data. With pics and everything... Agree with you. @Dolke To import products in bulk with pics and everything (thousands), you will need to have a good server. If your hosting is managed you can ask your hosting provider to help you a little bit Link to comment Share on other sites More sharing options...
pbuild Posted June 12, 2013 Share Posted June 12, 2013 I'm on inmotionhosting.com - just their cheapo starter/launch plan, nothing high end at all. Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 Well that is what Im trying to do sir for over a week now. What do you mean by a "good" server? A good server is a working one, it depend of your needs right What is a good hosting sever environment for prestashop with lets say 3500 products, around 200 uniqe visits per day? I was not able to find such a requirements anywhere on the forum nor official presta blog, documentation, etc... Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 I'm on inmotionhosting.com - just their cheapo starter/launch plan, nothing high end at all. I have a real hard time now thinking that I was not able to complete simple tasks like deleting 20 products at once from the backend on the HostGator.com VPS level 5 and you say you imported CSV file with over a 10 000 products at once? Link to comment Share on other sites More sharing options...
pbuild Posted June 12, 2013 Share Posted June 12, 2013 (edited) I have a real hard time now thinking that I was not able to complete simple tasks like deleting 20 products at once from the backend on the HostGator.com VPS level 5 and you say you imported CSV file with over a 10 000 products at once? No, I said I imported a couple thousand. I've never tried 10,000. Actually, let me look... it was ~1600 products, so I guess I was rounding up. I was importing about 8 fields (reference, name, price, quantity, image, features (2 each), description, category (one each). I had "use product reference as key" set. It did take a minute or two. p.s. I also find that product deletion in the Back Office is slow. I can delete 50 at once with no problem, but I'm definitely waiting for it to happen when I click. Edited June 12, 2013 by pbuild (see edit history) Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 This is what I just cant understand. One say that even on a dedicated server he chops his CSV file and get timeouts and other problems, and the other hosts at shared hosting for 5$/mo and get to import 10000 products in single CSV file without any issues? So, Im I crazy or what? Well 1600 is a good number because I cant do more then 70 and I have 3500 Link to comment Share on other sites More sharing options...
sadlyblue Posted June 12, 2013 Share Posted June 12, 2013 This is what I just cant understand. One say that even on a dedicated server he chops his CSV file and get timeouts and other problems, and the other hosts at shared hosting for 5$/mo and get to import 10000 products in single CSV file without any issues? So, Im I crazy or what? Well 1600 is a good number because I cant do more then 70 and I have 3500 I'm wasn't importing from csv, but from a xml webservice. And importing images, to 5 shops (multi store setup), with specific prices for each one, more than one language,... Link to comment Share on other sites More sharing options...
sadlyblue Posted June 12, 2013 Share Posted June 12, 2013 @sadlyblue Sir, so prestashop recommends that (if that is possible in our hosting environment) set all php settings to maximum values? I cant believe that you have to do that even on the dedicated server! Forgive me, but then this whole part of prestashop is just poorly imagined. You cannot let your software to overtake hardware limitations, or if this is absolutely necessary and doesn't have an alternative, they should write some sort of statement informing users of possibility this situation happens. I don't set unlimited anything. And i believe noone should set unlimited resources for php. If i did that, when developing, i would run the risk of creating loops, that would then crash the server. Even on a tested script it's a risk. When i rarely do that, i usually keep my eyes on system resources to prevent overusage. Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 Sir, as we speak im just updating my products (so no images involved at all), and only 4 attributes so my CSV has 4+1 (one extra is for ID) column and it still breaks on 70+ pruducts. Link to comment Share on other sites More sharing options...
sadlyblue Posted June 12, 2013 Share Posted June 12, 2013 One thing i can tell you. I'd rather write a script to import a csv that big, that import it using prestashop function. Let me tell you why i prefer it. Prestashop has many options, that it has to take into account when importing. Multishop, several currencies, languages, stock options, combinations,... As when i import, write a small script and i know where everything goes. No need to keep checking it. So it's faster. If you ask me the functions to import csv in prestashop could be improved. I would have to agree. But it's not on purpose. It just takes a lot of hard work. There are also other things to improve in prestashop. And it will keep improving. I had a greater disapointed with the latest google sitemap module, and had to reinstall the older version... I agree it's not perfect, but don't agree it's on purpose. 1 Link to comment Share on other sites More sharing options...
Dolke Posted June 12, 2013 Author Share Posted June 12, 2013 I see, thank you for sharing your thoughts. Actually I have left with one big dilemma. Is this Prestashop behavior related that needs to be improved and fixed or this is server-side problem. Answer to that will save me a lot of nerves and time but it seems like I'm not close to an answer... Link to comment Share on other sites More sharing options...
SULTD Posted March 18, 2015 Share Posted March 18, 2015 Hello Dolke, Can you please confirm if you ever solved this issue? I'm trying to set up Prestashop with the webspace also hosted by Hostgator and I get a timeout of anything over 100/125 products. Any information you have would be appreciated. Link to comment Share on other sites More sharing options...
alexiscruz007 Posted March 18, 2015 Share Posted March 18, 2015 you need split the cvs, i have only upload 10 product at time, the problem is the prestashop upload a make about 4 imagen for every poduct. Link to comment Share on other sites More sharing options...
jgamio Posted March 18, 2015 Share Posted March 18, 2015 The standard import is too slow but work I work with 700 products and at least 8 combinations by product without problem BUT on my VPS not on a regular Hosting . When I work on my customers Hosting some times I cant use the admin because is to slow. Prestashop need indeed a specify configuration without regular Hosting restrictions I think Prestashop use to much resources and is not improve some areas. Simple task like import and translations cant be done in some Hosting not only for the configuration, I think are processes that are not improve, just work. I recommend you if you have a 3500 products store you need to start to look for a paid module or custom solution because you cant depend on these import process. 1 Link to comment Share on other sites More sharing options...
wdburgdorf Posted March 19, 2015 Share Posted March 19, 2015 (edited) Trust the manual: There is no problem. "... easily fill your product catalog when you have a very large amount of products ..." (http://doc.prestashop.com/display/PS16/CSV+Import+Parameters) It is not clearly defined what they mean by "easy" or by "very large amount", though. But seriously: apparently there are many users who suffer from import timeouts. I'm one of them, although I need to import only a bit less than 300 products. If there was a paid module, I'd consider paying for it. Otherwise, can anyone share his workflow, how he splits the csv and imports efficiently, a script perhaps? Thank you! EDIT: In the manual it says "You can import more than one file at a time" - how? Edited March 19, 2015 by wdburgdorf (see edit history) Link to comment Share on other sites More sharing options...
RogueWaveLimited Posted March 19, 2015 Share Posted March 19, 2015 I can highly recommend the "ImportFast" module. It will cost you about $100 but it will save you hours and hours of work trying to get around the standard import. Link to comment Share on other sites More sharing options...
wdburgdorf Posted March 19, 2015 Share Posted March 19, 2015 Thank you! I'm currently testing this store manager: www.prestashopmanager.com. $200 for standard, $350 for professional version. Imported my 460 products with images in 20 seconds, looks nice so far. The csv has a different format than Prestashop standard, but that's not a big obstacle. With the PS standard importer I cannot reliably import more than 25 products per file. Pretty useless ... 1 Link to comment Share on other sites More sharing options...
Luckyy Posted May 14, 2015 Share Posted May 14, 2015 I have similar issue. I can import only a few products if I'm importing product with pictures. I know that's because CPU time is limited to 30 second by my hosting company (HOME.pl) so after 30 seconds I get PHP timeout. I made php.ini with max_execution_time = 600 and it shows in PrestShop that timeout is 600s but still provider limits overwrite this value and after 30 second I get the same error. Do you have any idea how to solve to issue? I can create multiple csv files with 10 products in each file but it will be painful. Link to comment Share on other sites More sharing options...
RogueWaveLimited Posted May 14, 2015 Share Posted May 14, 2015 ImportFast! Link to comment Share on other sites More sharing options...
selectshop.at Posted May 14, 2015 Share Posted May 14, 2015 @ Luckyy - http://prestashop.dataco.dk/en/home/3-import-fast.html Link to comment Share on other sites More sharing options...
Recommended Posts