Notes from the field on ColdFusion (or related) technical issues.

Wednesday, March 30, 2005

CFDirectory can be Slow with Many Files

CFDirectory can be rather slow when working with directories containing very many files (as in, thousands or tens of thousands of files.)

The underlying method used, java.io.File.listFiles(), does not allow a Java application to pass a filter to the underlying operating system, but rather forces the application to filter the directory listing on a case-by-case basis by creating a FileFilter object.

If the thousands of file names exist across a network share, the problem is compounded by the added latency of the network file system.

If you're only looking for, say, a date stamp for a single file, the entire directory is iterated looking for the file, and only then are the details of the file made available to the ColdFusion application.

If you only want information about a single file, it's fairly easy to get that directly via direct calls to Java from within ColdFusion. Here is a ColdFusion function that will efficiently get the date stamp from a single file:

<cfscript>
function getFileDate(fn) {
    theFile = createObject("java","java.io.File");
    theFile.init(fn);
    if( theFile.exists() ){
        theDate=createObject("java","java.util.Date");
        theDate.init(theFile.lastModified());
        dateString = 1+theDate.getMonth() & "/" & theDate.getDate() & "/" & 1900+theDate.getYear() & " " & theDate.getHours() & ":" & theDate.getMinutes() & ":" & theDate.getSeconds();
    }
    else {
        dateString="";
    }
    return dateString;
}
function getFileSize(fn) {
    theFile = createObject("java","java.io.File");
    theFile.init(fn);
    if( theFile.exists() ){
        return theFile.length();
    else {
        dateString="";
    }
    return dateString;
}
</cfscript>

1 comment:

  1. I hope you don't mind if I use a modified version of your function.

    I changed the return string to:

    dateString = DateFormat(ListGetAt(theDate.toGMTString(),1," ")&"-"&ListGetAt(theDate.toGMTString(),2," ")&"-"&ListGetAt(theDate.toGMTString(),3," "),"ddd")&", "&theDate.toGMTString();

    which is in the date format that HTML uses. I did this because I am using your function to serve files and using ColdFusion to do specific authentication. I then set the HTTP Header variable "Last-modified" to this value and use cfcontent to server the files. I have a few other tricks up my sleeve in this template, as well. It's very cool.

    Thanks for the help.

    ReplyDelete