Database Search

TechHome

• Build a Delimited Dir List for FileAppend File List
• Case Conversion Algorithm
• Change Old File and Path to New One
• Check File Date and Delete Older Files in Directory
• Check if File is Open or Exists
• Compare Directories
• Compute File Sizes
• Copy Files and Archive Attribute
• Copying Files from a Given Date
• Copying files XCOPY
• Count Keywords in Several Files
• Delete First Few Lines of a File
• Delete Temp Files Older Than
• Delimit a List of Files
• Display - Write File Contents to Screen
• Display Files in a Dialog with Case Sensitivity
• Fastest File Search Methods
• File Append for ASCII files with Ctrl-Zs in them
• File Counting and Item Counting Loop
• File List Sorted by Size
• File Mask Matching
• File or Item Extraction Loop
• FileCopy Error Handling
• FileOpen Error 1077 - Capture Additional Error Information
• FileVerInfo Constants
• FilVerInfo Returns Incorrect Version
• Get a List of Hidden Files in a Directory
• Get Date when File was Last Accessed Created Modified
• Get File Ownership in NT
• Get File Sizes Recursively
• Get Oldest or Newest File
• Get Users Home Directory
• Isolate Filename in a File Path
• List Files with Alternate Data Streams
• Merge Files with Binary Functions
• Parse Data from ASCII file using FileRead
• Progress Bar during FileCopy
• Read Part of File with BinaryRead
• Removing EOFs
• Renaming Files on an Incremental Basis
• Search for Upper Ascii Chars in File
• Sort a Flat File
• Tricky Bulk FileRename
• Write Various Quote Marks to Files

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

Fastest File Search Method

 Keywords:  searcher files

Question:

hand=SrchInit(share,"*.cls","","",16)
hand=SrchInit(share,"*.txt","","",16)

hand=SrchInit(share,"*.cls|*.txt","","",16)

Answer:

The attached script does some timing tests on various search methodologies. The results of the test are:

Search Extender:     22.39
DOS Extender:         2.59
Recursive DirItemize: 7.77
FIleItemRecursive:    5.29 

What I did was look for *.exe and *.dll files in my c:\Program Files\ directory (which has a total of about 21,000 files comprising about 4GB).

; This tests four different file-search methodologies,
; when looking for two or more extension groups. The
; four routines are each called "max_tests" times, in
; random order (to minimize buffer effects, etc.) in
; groups of four; on their last call, each method writes
; out the files it has found.
;
; FileItemRecursive is from a post from Deana
;
; The recursive_diritemize subroutine is something that I
; wrote years back, with some help from WB forum members
; and staff.
;
; The File Search Extender is the standard extender.
;
; The DOS Capture Extender can be found by searching for
; "doscapture" in the WB Tech Database.  
http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsupt.web+WIL~Extenders/Third~Party~Extenders+Dos~Capture~Extender.txt
Its particulars
; are listed thusly:
;
;   Dos Capture Extender written by Kent Ruddick.
;   Posted to the WebBoard BBS on October 31st, 2004
;
;
; Bill Pollack



GoSub initialize

For test_run=1 To max_tests
   search_extender_done     =@FALSE
   dos_extender_done        =@FALSE
   recursive_diritemize_done=@FALSE
   FileItemRecursive_done   =@FALSE

   While @TRUE
      If search_extender_done&&dos_extender_done&&recursive_diritemize_done&&FileItemRecursive_done Then Break

      Select Int(Random(3))
         Case 0
            If !search_extender_done
               GoSub search_via_search_extender
               search_extender_done=@TRUE
            EndIf

            Break
         Case 1
            If !dos_extender_done
               GoSub search_via_dos_extender
               dos_extender_done=@TRUE
            EndIf

            Break
         Case 2
            If !recursive_diritemize_done
               GoSub search_via_recursive_diritemize
               recursive_diritemize_done=@TRUE
            EndIf

            Break
         Case 3
            If !FileItemRecursive_done
               GoSub search_via_FileItemRecursive
               FileItemRecursive_done=@TRUE
            EndIf

            Break
      EndSelect
   EndWhile
Next

BoxShut()
Message('Timing Results (Seconds)',StrCat('Search Extender: ',search_extender_seconds,@CRLF,'DOS Extender: ',dos_extender_seconds,@CRLF,'Recursive Diritemize: ',recursive_diritemize_seconds,@CRLF,'FileItemRecursive: ',FileItemRecursive_seconds))

Exit

;==========
:initialize
;==========

starting_directory='C:\Program Files\' ; end with a slash
max_tests         =5

search_extender_seconds     =0
dos_extender_seconds        =0
recursive_diritemize_seconds=0
FileItemRecursive_seconds   =0

GoSub define_functions

Decimals(2)
IntControl(5,1,0,0,0)

BoxOpen('Search Test','Starting searches...')

Return

;=======================
:search_via_dos_extender
;=======================

AddExtender('C:\Program Files\WinBatch\DOSCapture\wbDosCapture.dll')
DirChange(starting_directory)

BoxText(StrCat('Searching via DOS Extender (',test_run,')...'))
start_time=GetTickCount()

result=doscapture('cmd.exe /c dir /S/B/A/W *.exe,*.dll')

dos_extender_seconds=(dos_extender_seconds+((GetTickCount()-start_time)/1000.0))/test_run

If test_run==max_tests Then FilePut('c:\test_dos_capture.txt',StrReplace(ItemSort(StrReplace(result,@CRLF,@TAB),@TAB),@TAB,@CRLF))

Drop(result)

Return

;===============================
:search_via_recursive_diritemize
;===============================

start_list_position=1
stop_list_position =1
directory_list     =starting_directory
result             =''

BoxText(StrCat('Searching via Recursive DirItemize (',test_run,')...'))
start_time=GetTickCount()

GoSub build_directory_list
GoSub add_files

recursive_diritemize_seconds=(recursive_diritemize_seconds+((GetTickCount()-start_time)/1000.0))/test_run

If test_run==max_tests Then FilePut('c:\test_recursive_diritemize.txt',StrReplace(ItemSort(result,@TAB),@TAB,@CRLF))

Drop(result)
Drop(directory_list)
Drop(subdirectory_list)

Return

;====================
:build_directory_list
;====================

For directory_number=start_list_position To stop_list_position
   working_directory =ItemExtract(directory_number,directory_list,@TAB)
   subdirectory_list =DirItemize(StrCat(working_directory,'\*.*'))
   subdirectory_count=ItemCount(subdirectory_list,@TAB)

   For subdirectory_number=1 To subdirectory_count
      directory_list=ItemInsert(StrCat(working_directory,ItemExtract(subdirectory_number,subdirectory_list,@TAB),'\'),-1,directory_list,@TAB)
   Next
Next

directory_count=ItemCount(directory_list,@TAB)

If directory_count>stop_list_position
   start_list_position=stop_list_position+1
   stop_list_position =directory_count
   GoSub build_directory_list
EndIf

Return

;=========
:add_files
;=========

For directory_number=1 To directory_count
   working_directory=ItemExtract(directory_number,directory_list,@TAB)

   DirChange(working_directory)
   file_list        =FileItemize('*.dll|*.exe')

; I don't know why the two lines above can't be replaced by the single line...
;
;      file_list=fileitemize(strcat(working_directory,'*.dll|*.exe'))
;
; ...but that doesn't seem to work for me (WB 2005B).

   file_count       =ItemCount(file_list,@TAB)

   For file_number=1 To file_count
      result=ItemInsert(StrCat(working_directory,ItemExtract(file_number,file_list,@TAB)),-1,result,@TAB)
   Next
Next

Return

;==========================
:search_via_search_extender
;==========================

result=''
AddExtender('C:\Program Files\WinBatch\System\wsrch34i.dll')

BoxText(StrCat('Searching via Search Extender (',test_run,')...'))
start_time=GetTickCount()

search_handle=SrchInit(starting_directory,'*.exe','','',8+16)

While @TRUE
   path_and_file_found=SrchNext(search_handle)

   If path_and_file_found=='' Then Break

   If StrSub(path_and_file_found,StrLen(path_and_file_found),1)!='\'
      result=ItemInsert(path_and_file_found,-1,result,@TAB)
   EndIf
EndWhile

SrchFree(search_handle)

search_handle=SrchInit(starting_directory,'*.dll','','',8+16)

While @TRUE
   path_and_file_found=SrchNext(search_handle)

   If path_and_file_found=='' Then Break

   If StrSub(path_and_file_found,StrLen(path_and_file_found),1)!='\'
      result=ItemInsert(path_and_file_found,-1,result,@TAB)
   EndIf
EndWhile

SrchFree(search_handle)

search_extender_seconds=(search_extender_seconds+((GetTickCount()-start_time)/1000.0))/test_run

If test_run==max_tests Then FilePut('c:\test_search_extender.txt',StrReplace(ItemSort(result,@TAB),@TAB,@CRLF))

Drop(result)

Return

;============================
:search_via_FileItemRecursive
;============================

BoxText(StrCat('Searching via FileItemRecursive (',test_run,')...'))
start_time=GetTickCount()

result=FileItemRecursive('',starting_directory,'*.exe||*.dll',0,9999)

FileItemRecursive_seconds=(FileItemRecursive_seconds+((GetTickCount()-start_time)/1000.0))/test_run

If test_run==max_tests Then FilePut('c:\test_FileItemRecursive.txt',StrReplace(ItemSort(result,@TAB),@TAB,@CRLF))

Drop(result)

Return

;================
:define_functions
;================

#DefineFunction FileItemRecursive(tree,dir,mask,currentlevel,maxlevel)

If currentlevel>=maxlevel Then Return(tree)
currentlevel=currentlevel+1

DirChange(dir)
dlist=ItemSort(DirItemize("*.*"),@TAB)
dcount=ItemCount(dlist,@TAB)

For xx=1 To dcount
thisdir=ItemExtract(xx,dlist,@TAB)
tree=FileItemRecursive(tree,thisdir,mask,currentlevel,maxlevel)
Next

If tree=="" Then tt=""
Else tt=@TAB

flist=FileItemPath(mask)
If flist != ""
flist=ItemSort(flist,@TAB)
tree=StrCat(tree,tt,flist)
EndIf

DirChange("..")
Return(tree)

#EndFunction

Return

Article ID:   W16960

File Created: 2014:07:18:09:51:38

Last Updated: 2014:07:18:09:51:38