WinBatch Tech Support Home

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

WIL Extenders
File Searcher
File And Folder Extender

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

FAFFind Returns Duplicate Directories

 Keywords: FAFFind Returns Duplicate Directories Documents and Settings Application Data Reparse Point Directory Junction Tracverse Recurse Recursive Hang Crash Issue GetFileAttributesW FILE_ATTRIBUTE_REPARSE_POINT

UPDATE:

The File and Folder Finder Extender has been updated to support a new flag to IGNORE reparse points. 
File and Folder Finder Extender: Version 44005 Aug. 28, 2013
      
      Added support for a new flag value of 128 (no reparse points)to the fafOpen 
      function's flags parameter.  When set this flag prevents the extender from 
      including file system objects implemented with reparse points in searches 
      performed using the returned seach context. Reparse points are used by the 
      NTFS file system to implement symbolic links, directory junctions and directory 
      mount points among other things. Reparse points, and their associated 
      filters and applications provided by 3rd party vender may also be present on a
      system.

Question:

I have a WinBatch script running as a service, under a Local System Account, that uses FAFOpen() to scan the System returning File information. When run on Windows XP this works fine but on Windows 7 it is returning the same Folders over and over again?

The issue appears to relate to "Application Data" related directories for each User on the Computer. Each user has Application data folders which FAFOpen reports contains the same files yet when you use Windows Explorer and look at the directories it reports them as empty, 'Access Denied' or that the Directories do not exist?

For Example: 

C:\Documents and Settings\FRED\AppData\Local\Application Data\Application Data\Microsoft\Windows\Temporary Internet Files
C:\Documents and Settings\FRED\AppData\Local\Application Data\Application Data\Application Data\Microsoft\Windows\Temporary Internet Files
C:\ProgramData\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Documents\My Music\Sample Music\
These directories get longer and longer with another level of \Application Data\ until the Windows reports that the WinBatch Application has stopped responding and kills the application. 

 ; Initialize flags for readability
fsHidden = 1   ; Include hidden files
fsSystem = 2   ; Include system files
fsRecurse = 16  ; Look in sub directories
File = FileOpen(DatFile, "WRITE")
Drives = DiskScan(2)

For A = 1 to ItemCount(Drives, @Tab)
        DriveID = ItemExtract(A, Drives, @Tab)

        ; Open a search handle
        fsHandle = fafOpen("%DriveID%\", "*.*", fsHidden|fsSystem|fsRecurse)
        LastFilePath = ""

        ; Perform the search

        While @TRUE
           sFound = fafFind(fsHandle)
           If sFound == "" Then Break
                FPath = FilePAth(SFound)

                If LastFilePath <> FPath Then
                        FileWrite(File, "")
                        FileWrite(File, "[%FPath%]")
                        LastFilePath = FPath
                EndIf
                        
                If FileExist(SFound) Then
                        FName = FileRoot(SFound)
                        Extension = StrUpper(FileExtension(SFound))
                        FileName = StrCat(FName, ".", Extension)

                        If Extension == "EXE" || Extension == "COM" || Extension == "DLL" Then
                                FVersion = FileVerInfo(SFound, "", "FileVersion")
                        Else
                                FVersion = ""
                        Endif

                        Details = StrCat(FileName, "=F|", FileSizeEx(SFound, 1), "|", FileYMDHMS(SFound), "|", FVersion)
                EndIf

                FileWrite(File, Details)
        EndWhile
        FafClose(fsHandle)
Next A
FileClose(File)

Answer:

Directories like 'Application Data' are a form of reparse point called a directory junction. Normally, junctions like Application Data under users\whatever have a 'deny' Everyone ACE for file read access. This means that even an Administrator should not be able to access the junctions target using FafFind. The extender has no more privileges than the account that is running it so it shouldn't be able to follow the junction.

The extender has been tested with all the flavors of reparse points and there are no known issues with reparse points. Directory junctions are presented to the extender by the system as normal directory paths.

Since your script is running as a native NT service, as local system, then it is going to have a level of access that is greater than what members of the Administrators group have, and the restrictive deny permissions that are present on some of the junctions under user profiles will not prevent the script from traversing those junctions. With that said, it is almost a certainty that there is, in fact, a recursive junction or directory symbolic link present in the path on the affected computer. If you want to investigate this further, use the "psexec" utility in the SysInternals Suite to run an instance of "cmd.exe" as local system, then use the CD and DIR commands to interrogate the file system contents under the user profile in question.

Note that you would need to use "dir /a" to see hidden directory junctions.

User Reply:

Thanks for the information. The NT Service is running as a "Local System" so I think that explains why it is traversing these folders. Running as Local System does allow me access to Directories that are normally "Access Denied". It looks like Application Data \ Temporary Folders are interlinked? If I look at the file list returned from each of the C:\Users\{User Name}\ directories they contain the same list of files so they must all be pointing to the same location of the HDD. I have confirmed this by deleting some files from one User Area and they disappear from the file list returned from another User's Area? This also explains why I am getting so many files returned. 8 different User Directories each with Multiple \Application Data\ references returning 17,000+ files in each \Application Data\ directory.

Answer:

Reparse points have been around and used for more than 25 years on Windows and the extender has been around for 5 years. No users has ever requested we change how reparse points are handled. This suggests that circular reparse point are rare for extender users and/or extender users are not concerned about the extender following circular references. Projecting ones preferences unto an entire user group is seldom a good software development practice. This is particularly so when the desire is to keep things simple when possible. There is a long standing bullet on the extender's enhancement list to add an ignore flag for reparse points but there has been no interest until now. 'Report but don't traverse' is something that has not been considered until now.

Possible workaround: There may be better ones. You would need to modify it according to your needs, of course. You will need to check for then ignore all reparse points. 

FILE_ATTRIBUTE_REPARSE_POINT  = 1024
hKernel32 = DllLoad("kernel32")
strDir= "C:\Users\username\Application Data"
if DllCall(hKernel32, long:"GetFileAttributesW", lpwstr:strDir) & FILE_ATTRIBUTE_REPARSE_POINT then strText = "is"
else strText = "is not"
DllFree(hKernel32)
Message( strDir, strText:" a reparse point")

Article ID:   W17572

Filename:   FAFFind Returns duplicated Directories.txt
File Created: 2013:10:16:08:34:12
Last Updated: 2013:10:16:08:34:12