When uploading files via PHP, you might get this PHP error,
Warning: mkdir() [function.mkdir]: Too many links in … online …
Don’t you love PHP error messages? I’ll tell you what this means and a solution to solve it.
On some operating systems, there are limits to how many things you can have in a folder at any given time. For instance, the filesystem ext3 allows only 32,000 files/folders in a single folder. Seems like a lot, doesn’t it? What if you’re allowing users to upload images? If you allow it all in a single folder, you’ll run out of room at some point. The error means you have too many files in a folder. (Or too many folders in a folder.)
Of course, you could keep track in a database and split them up by date. Or make them go into folders based on their names. For example, we’ll use myimage.png as a filename. Your folder structure could be like, m/y/image.png. I bet there are a lot more mycat.png’s than there are yellowbananas.png’s. You get an uneven balance of images located in folders.
My favorite solution is to get the sha1 of the filename or file itself. So again using myimage.png as the filename, let’s get the sha1 hash of that. You get, 6FEA6CFABA9103E54988DF65D93A8E074F8214AB, so your new path could as simple as, 6/f/e/a/myimage.png.
Looking at this from a math perspective, you know there are 16 folders in your parent directory, because there are 16 hexadecimal characters. Then, in each sub-folder, you get another 16 folders. Just two levels deep now, folder wise, and you’re already at 256 folders. Let’s go two more levels deep. You’re now at 65536 folders by just going four levels deep. I doubt you’ll ever fill 65536 folders with 32,000 files!
There is overhead, but then there is overhead in countless emails being sent to your support desk because you have too many files in your upload folder. Sha1 on PHP5 might not be the fastest thing you could use, but it does work. Even md5 works. Only it’s smaller in length and has collisions.
More reading
The Web Guy: mkdir – too many links and Serving Images from a databases – don’t! and handy utility, Hash’em all – Free Text/File hashing
Updates
I was doing some research for the limits of files in a particular volume/folder. So the numbers below are from Wikipedia and the represent the number of files in a volume/folder
- ext2 – 1018
- ext3 – variable
- ZFS (a sun filesystem) – 248
- HSF Plus (Apple file system for Mac/iPod) – Unlimited
- NTFS – 4,294,967,295 (232-1)
- FAT – Apparently, this goes by cluster rather than file
Hello,
are you sure that ext3 allows only 32.000 files per folder? I have a folder with 480.189 files and it seems to work…
@Eugenio, I’m not sure exactly, however most sources believe ext3 had something to do with it.
This post saved my bacon this morning. I think that ext3’s limit is variable, but that Ubuntu specifically is built with the 32,000 limit. That’s what I have deduced from our situation.
Glad I could help save your bacon! I looked on wikipedia again it does say ext3 is variable and allocated at creation time but also goes on to say,
I’m not a linux guru but that seems to be where the 32,000 issue is coming from.
The 32000 limit in ext3 has to do with subdirectories in a directory and not files.
I am reaching this problem soon in my web app. I have until now organized image uploads in directories, sorted by username.
Your solution looks well.
Hi,
This is very helpful. It took me a while to figure out that this was the problem on my project. Thank you for the solution. I surely will adopt it.