emiauto Posted September 21, 2016 Share Posted September 21, 2016 Hi! I have CSV with 96k items inside. When I try to import the file, I get an error - "Request timeout". But even after this error it continue to load products. And stopped after about 4500. Of course I can split this file to about 20 parts and import it. But somebody know how to import large CSV at once? Link to comment Share on other sites More sharing options...
rocky Posted September 23, 2016 Share Posted September 23, 2016 If you've already increased your max_execution_time as much as your host will allow, there's nothing you can do but split it into smaller files. Link to comment Share on other sites More sharing options...
emiauto Posted September 23, 2016 Author Share Posted September 23, 2016 If you've already increased your max_execution_time as much as your host will allow, there's nothing you can do but split it into smaller files. Thanks, rocky. Now I understand that for such quantity of products a good VPS should be used. Not shared resources hosting! That's all. Link to comment Share on other sites More sharing options...
system_error86 Posted October 12, 2016 Share Posted October 12, 2016 Hello guys, has anyone use the import to upload more than 6000 products?? Is anyone who use cron job to do that? How much time do the server to upload the csv file??? I would like to import every 5 till 10 minutes 6000 products. Has anyone any advice how can I do that?? I have a module for import but it takes too long to upload the csv. Thanks Link to comment Share on other sites More sharing options...
emiauto Posted October 13, 2016 Author Share Posted October 13, 2016 Hi! Sorry, I do not have such experience. In my case I can import about 30000 at once. But it is a long process. About 8 hours... Server resources and configs are very important. Why by Cron? What is a Total quantity? It is a single import? Or you want to import it every day? Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 Hello thanks for your response, we have 2 physical stores and 1 eshop we are retail store. We have over than 6000 quantites. I want to import via cron avery day per 10 minutes beacuse we have all the time changes in the stock Thanks Link to comment Share on other sites More sharing options...
emiauto Posted October 13, 2016 Author Share Posted October 13, 2016 Hello thanks for your response, we have 2 physical stores and 1 eshop we are retail store. We have over than 6000 quantites. I want to import via cron avery day per 10 minutes beacuse we have all the time changes in the stock Thanks csv is a method for manual imports. (IMHO) Try to find the programmer (freelancer for example). You need a php script/parser. It will work faster. Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 I have already a module that imports the products. If I did with prestashops import I would at least a year to import the products! What do you mean script / parser exactly? The module uses the objects of prestashop. Link to comment Share on other sites More sharing options...
emiauto Posted October 13, 2016 Author Share Posted October 13, 2016 I mean the same. Prestashop's csv import is not the best way. To resolve the similar problem some time ago with not Prestashop (another engine) my programmer wrote a parser (php) - without any csv. It work directly with DB. And it was very fast. Sorry for my English Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 what engine do you use? Link to comment Share on other sites More sharing options...
emiauto Posted October 13, 2016 Author Share Posted October 13, 2016 It was 3 years ago. And it was Webasyst. Now I am using Prestashop. But principe is the same. Import to DB directly will be faster than CSV import. In any engine, in any case. Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 Yes ok but to do this must have 2 dbs with same tables!! In my case I have an erp with diferrent db tables compared with prestashop, so the only way is the csv file. The problem is that do a much time. Link to comment Share on other sites More sharing options...
emiauto Posted October 13, 2016 Author Share Posted October 13, 2016 If you have access to both DB, any php programmer/coder can write a script which will take data from one DB (from any table) and put it to another DB (to any table). Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 Unfortunatelly I haven't access to erp db Link to comment Share on other sites More sharing options...
system_error86 Posted October 13, 2016 Share Posted October 13, 2016 sir do you have this php code ? Could you mind to give me to see the logic? Link to comment Share on other sites More sharing options...
emiauto Posted October 17, 2016 Author Share Posted October 17, 2016 Sorry. At this moment I am not using any php parser. It was on old site. And it is closed 2 years ago. Link to comment Share on other sites More sharing options...
Kogkalidis Posted July 28, 2022 Share Posted July 28, 2022 On 9/21/2016 at 11:57 AM, emiauto said: Hi! I have CSV with 96k items inside. When I try to import the file, I get an error - "Request timeout". But even after this error it continue to load products. And stopped after about 4500. Of course I can split this file to about 20 parts and import it. But somebody know how to import large CSV at once? Is there any chance of you still having somewhere this big file? I am building an import module and want to stretch-tes it. Link to comment Share on other sites More sharing options...
Ali Samie Posted July 29, 2022 Share Posted July 29, 2022 On 7/28/2022 at 1:50 PM, Kogkalidis said: Is there any chance of you still having somewhere this big file? I am building an import module and want to stretch-tes it. You can duplicate products for infinte times. If you are testing there is no need for bigfiles. You can create a file of n times of copied products 1 Link to comment Share on other sites More sharing options...
Kogkalidis Posted July 29, 2022 Share Posted July 29, 2022 1 minute ago, stifler97 said: You can duplicate products for infinte times. If you are testing there is no need for bigfiles. You can create a file of n times of copied products True story. But it's not only to test the size, but the file hierarchy as well. As any XML file is different in the way it presents the data. Regards Link to comment Share on other sites More sharing options...
Ali Samie Posted July 29, 2022 Share Posted July 29, 2022 I have done it with the inspiration of ProductImportHandler and I recommend you to start with simple products. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now