Long File Path Support

 Keywords:  fafFind Long File Path Prefix Skip Ignore Directory File Filename Namespace Designation Notation Missing Incomplete Limit Limitation MAX_PATH 260 257  \\?\ CreateDirectoryW DirMake

As of WinBatch 2012C, WinBatch does not support 'Long File Paths'. For what it is worth, we do hope to add full long path support to WinBatch at some future date.

In order for long filename support to work reliably, the following conditions must be met:

The input value is a Unicode string that specifies an arbitrarily long path specification with the appropriate extended syntax notation prefix in use for drive letter paths or UNC paths.
The WIL Interpreter or extender function code must be process the path specification value such that it is always kept in Unicode, is never truncated or otherwise length-limited, and it must be aware of the extended syntax notation and must not remove it while manipulating the path specification value.
Only Unicode-enabled forms of Win32 API functions can be called to perform file system operations using the path specification value.

If any of these conditions are not met, then expect things to not work properly.

Possible Workaround.

By default, WIL strings are ANSI [narrow character] strings using a specific code page. Use the ChrStringToUnicode() function to convert the paths to wide Unicode characters.

Also use the "\\?\" prefix used in front of a drive letter path. This causes underlying Win32 API functions that accept directory & file names that are as long as 32,000 characters, but, the caveat is that only the Unicode versions of the Win32 API functions do this. By doing these 2 things in combination, you are forcing WinBatch to use a Unicode string and to call the appropriate Unicode form of a Win32 API function.

If the WIL Function doesn't support unicode then you will need to use DLLCall to call the Unicode version of the API.

No 'Long File Path' Support:
Any function that is implemented as Unicode should recognize the name space moniker (\\?\). Although many WinBatch file functions are implemented in Unicode, some are not able to parse file paths greater than MAX_PATH. This means that any functions needing to parse a path will not work with long paths. So even though some WinBatch functions do support long paths. It is a side-effect of adding Unicode support and not the result of a specific plan to provide that long path support.

Any function that accepts a file-list parameter (because they must be parsed): i.e. FileCopy, DirItemize
NON Unicode functions: Shell Operation Extender functions (i.e. aFileCopy)

Long File Path Support:

File and Folder Extender functions: fafClose, fafFind, fafGetInfo, fafOpen

Question:

Is there a limitation on path depth for the faffind function?
The reason I ask is that fafFind seems to be skipping some of the directories. I used faffind with both * and *.* to get everyfile in the subdirectories and then copy those files to a alternate share. I made sure to include the flags to find Hidden and System files. I log each file it finds. This script is used for data migrations to new user data spaces.

Once the migration is complete, using Beyond Compare I can see that there are files that were skipped. If I check the log for the faffind activity, it has never even processed the skipped files. They were not 'found'. I find 168 more files than I do with faffind. One thing I noticed is that it skipping entire directories, not individual random files.

Answer:

The FAF extender is specifically written to support long paths. The only limit on files paths in fafFind is the limit imposed by the Windows OS's max Unicode path limit which is 32K Unicode characters. More info can be found here http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

IMPORTANT: If you carefully read the MSDN article about file paths in the Windows OS, you will note that that to access directory paths greater than 260 characters you have to prefix your paths with either "\\?\" or "\\?\UNC\" in the case of UNC paths. You cannot use relative paths with these device name space designations. The "FAF" extender therefore requires these designations in order to access paths greater than 260 characters in length.

Question:

Okay great. I changed the starting path from x: to \\?\x: and fafFind found everything.
One more question. When I use this \\?\ filename designation the fafFind returns all the files names with the same \\?\ prefix. What functions if any in WinBatch can handle this namespace designation?
For example I am using the Shell Operation Extender function aFileCopy. However I have to use shortpaths and/or subst to be able to actually copy the files that fafFind was finding but were too deep for the other functions.

Answer:

Unfortunately I don't have a complete list of I/O related API functions that support this notation, nor do I have any correlation between WIL functions and these API functions to know for certain whether or not it's safe to use this notation from ALL functions within a WinBatch script. I suspect that any functions that have been converted to support Unicode will *probably* support this notation.

Based on some testing it looks like the standard FileCopy operation does support this notation, however the Shell Operation extenders aFileCopy does not. So I would recommend using FileCopy instead of aFileCopy. Note FileCopy has been updated to support Unicode.

In Summary
Not every WinBatch file or directory function supports long path/file names. Any function that parses the path for any reason is still restricted to ~260 character paths. WinBatch functions that do support long paths do so as a side-effect of supporting Unicode.

Although many WinBatch file functions are implemented in Unicode, some are not able to parse file paths greater than MAX_PATH. This means that any functions needing to parse a path will not work with long paths. This also explains in part why the faf extender's ability to handle long paths is not advertised. So even though some WinBatch functions do support long paths. It is a side-effect of adding Unicode support and not the result of a specific plan to provide that long path support. For what it is worth, we do hope to add full long path support to WinBatch at some future date.

More
Recognizing the name space moniker (\\?\) is necessary but not the same thing as supporting long paths. For example, the WinBatch DirMake function can be used to make a long path by performing the task incrementally but it cannot be said to support long paths. Try passing a long (>260) path to the function to see what I mean. Or try creating a new sub folder in a Windows 7 Explorer window displaying a folder with a path of ~255 characters or so. If you must create directories using the long file name designation, then you will need to use this dllCall:

; DirMake for long file paths
strDir = "\\?\C:\long path goes here..."
nResult = DllCall(DirWindows(1):'KERNEL32.DLL',long:'CreateDirectoryW', lpwstr:strdir, long:0)

Workaround UDFs

Article ID:   W17574

Filename:   Long File Path Support.txt

File Created: 2013:04:19:09:17:46

Last Updated: 2013:04:19:09:17:46

Database Search

TechHome