[NEVER SOLVED] image folder structure crashes inode server limits!

[papaprix]

Hello guys,

 

First of all, I have an store with 3.000 products, each one has 5 pictures. My Inode ser ver limits is 500.000 and I have reached it, my website is offline because of this...

 

There is a big problem in Prestashop and I have doing a month long research and I never have found a solution or workarround.

 

It is about Image folder structure. I have read post about cache problems concerning inodes server limits. Remember that each file or folder is an inode.

 

I will be concise and also share a long explanation.

In short: Using product's reference or SKU as folder name, and inside this folder all images related.

 

Long, but interesting explanation:

 

Indeed , save all image files in a single directory is not as convenient , especially if it is a very wide range of products .

 

But let's take the example of a number of products not over 333 . Assuming that each product has 3 images, images that are 999, bearing in mind that PrestaShop saves each image in 5 different sizes (depending on the theme), plus the original, that are 6 files for each image , making a total of 5,994 image files.

 

Given that from 500 files in one directory on disk performance suffers, it is a good idea to try to organize those 5,994 image files in subfolders, and so planned the PrestaShop development team, in the early 1.4 of the series .

 

It was decided to organize image files in this way :

 

Each image is registered in the database with a unique identifier , which is an integer . As this number is used as a guide to determine which subdirectory save 6 files corresponding to the image in question , plus an index.php file to prevent PrestaShop puts can directly visit the url for that subdirectory . Decomposed identifier digits and each digit is entering a subdirectory, creating it if it does not exist.

 

For example , the image id . 1 le shall the subdirectory 1 / , the image 11, the subdirectory 1/1 / , and image 111, the subdirectory 1/1/1 / . Or the image 352, the subdirectory 3/5/2 / . You see, for each digit there is a level of depth in the directory tree .

 

So far so good , but no one thought to do math ? Huh saw that you could create an excessive number of directories? Maybe you're wondering "Well, and where's the problem? " . I'll tell you .  Try to get off or FTP upload product images a PrestaShop store. It's frustrating, because for each directory change , you must submit a request to the server, and although not an excessive number of images ( in our example , 999 ), it is sooo slow. But this is not the main problem.

 

We have 999 images , which correlate identifiers , starting with one to 999. Inside the img / p /, imaging products PrestaShop folder, we will have 9 subfolders called 1 to 9 (no zero wallet because there is zero image identifier) ​​. Within each of these 9 folders , we have 10 folders , called 0 to 9 ( no zero folder that would match the image identifier 10, 20, ... ) , and within each of these 10 folders , other 10 folders called 0 to 9 . These are the necessary directories to store the files in those 999 pictures : 9 * 10 * 10 = 900 directories. And those directories contain in total 999 * (6 + 1) = 6,993 files ( 999 images per 7 files , which are 6 files . Jpg plus index.php ) .

 

Well , until then keep well, but will stretch a little thing , let's put 3333 instead of 333 products . So would 9999 images ( 3 results) , which will result in 9 * 10 * 10 * 10 = 9,000 directories, and 9999 * 7 = 69,993 files. This implies a consumption of 9,000 + 69,993 = 78,993 inodes on your hard disk. But what is an inode ?

 

The inodes come to be the total of directories and files on the drive . And there is a limit , first identified by the unit , but on shared web hosting and virtual servers usually set An arbitrary limit , since it is understood that excessive consumption of inodes affect disk performance. You can find limits of 100,000 , 215,000 , 430,000 , ... If your accommodation is a dedicated server, we are talking about a limit of several million inodes , but the fact is that most people who ride a PrestaShop online store do not have this capacity and face these lower limits.

 

There are stores that have thousands of products, and we are seeing inodes consumption grows exponentially with the number of images , which is very easy to achieve inodes limit imposed by the hosting provider . Do not forget also that the images of the products is not the only thing on the disc . I have seen a limit of one and a half million inodes being overwhelmed by a PrestaShop store with 40,000 products and 110,000 product images.

 

The effect of exhaust inodes available is that you can not create new directories and files to the disk, try disk full error will be obtained , although in reality it several gigs of space to spare. And of course, in that situation is the web (and our hosting) virtually unusable .

 

Sometimes , the limits imposed by the hosting providers are too low , especially compared with other housing characteristics , such as disk space or transfer , but that does not mean that PrestaShop development team could have devised a method much less "generous" with the consumption of inodes. For example , simply by following the same idea, but using the product ID instead of the image to determine the directory to save the files , and would significantly reduce the consumption. Each product would correspond to a directory, where all files of their images would be put instead of a directory for each image . We would be saving on directories and files ( the index.php not be created ) .

 

Of course, the image files must exist , hence no savings, so if you 're going to open a store with PrestaShop and to sell a large number of products , select either your hosting. Also, this is just one example of the problems of consumption and system performance PrestaShop online store , which , despite all this , it's great and I recommend it, but always knowing all there is to consider.

 

 

[js73]

I have the same problem. But I have inodes limit of 250 000. 

[Dh42]

Clearing your smarty cache should help with the inode issue. I know what you are saying about the images are true, but the smarty cache does the same thing and it can be cleared to reduce inode usage. 

[js73]

Clearing your smarty cache should help with the inode issue. I know what you are saying about the images are true, but the smarty cache does the same thing and it can be cleared to reduce inode usage. 

 

I have no smart cache enable. Am only have "Memcached" enable

 

I have now near 240 000 inodes at my account. And just my PS 1.6 have ~160 000 inodes. 

[Dh42]

That sounds reasonable. 

[js73]

I have about 1-2000 product to import. But I can't do it. Have no inodes left. 

 

Are currently looking for a new web hosting company, that has unlimit inodes and disk space.

[Dh42]

How many products do you currently have in your shop? 

[js73]

I have about 7900 products currently.

[Dh42]

With wanting around 10k products I would suggest for many reasons that you are going to need a large server than a shared server. 

[papaprix]

I have about 1-2000 product to import. But I can't do it. Have no inodes left. 

 

Are currently looking for a new web hosting company, that has unlimit inodes and disk space.

 

Check Arvixe Web hosting, I don't have inode problems anymore. Arvixe has unlimited inodes restrictions.

[Leohvive]

Tengo instalado prestashop 1.6.0.9 con una tienda que tiene 18,000 productos en febrero  tenia alrededor de 400,000 inodos sin embargo algo sucedió que  ahora se están generando carpetas nuevas ( que alojarian en un futuro imágenes de productos) pero ya ha generado la carpeta 1,000,000 esto hace que el consumo de inodos se eleve exponencialmente y sigue generando mas carpetas, ahora mi tienda tiene 1,239,000 inodos y sigue creciendo,  en donde corrijo esto? Podrían ser amables de ayudarme para corregir.