WinBatch Tech Support Home

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

FileMath UDFs

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

FileMath - Logical File Operations

 Keywords:  Duplicates Dups compare synchronize 

These functions allow logical operations on simple data files. Complete examples in documentation below.

;File Math UDFs
;
;These set of functions allow certain useful operations on files.
;These operations take one or two input files and creates an output file.
;These operations include:
;
;fmSortFile     Sorts one file in acesnding ascii order
;fmRemoveDups   Remove duplicate lines from a file (also sorts it)
;fmGetDups      Get only the duplicate lines from a file.
;fmFileMath     This is where the real power is.  Functions include
;
;               ADD      Makes a combination of both input files.  
;                        Not sorted.  Dups may exist
;               SUBTRACT Subtracts line in second file from the first file..
;                        Sorted.  Dups may exist
;               OR       Returns lines found in both files
;                        Sorted.  No dups
;               XOR      Returns lines not found in both file
;               AND      Returns lines only found in both files

;At the end of the UDFs, will be found a runnable example and
;sample output of each example



;
;
;The whole point of these UDFs is that you can use them without a 100%
;understanding of how they work

;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;fmGetFileObject
;Syntax   object=fmGetFileObject(filename,flag)
;
;         object     a handle to be passed to other FileMath functions when they need a file
;         filename   normal filename of a file ot make a object for
;         flag       0 for new, not necessarily existing output files  0=Output
;                    1 for existing, input type files  1=Input
;
;In order to reduce the processing time required, an array-type
;variable, that we will name a "FileObject" retains information
;aboout the current file. THis information includes
;    0-Filename, 
;    1-FileSize, 
;    2-Record Count, 
;    3-Max record size, 
;    4-id num  1234567
;    5-Sorted Status  0=Not sorted (or unknown)  1=Sorted
;    6-De=Duped Status  0=Not Deduped (or unknown) 1=DeDuped
;
;The FileObjects have been designed as "magic cookies" or "handles"
;that you pass to the other functions.

#DefineFunction fmGetFileObject( FileNameIn ,flag) ;flag0=normal 1=newfile
   ;THis functions returns an array object of 7 elements
   ;0=Filename 1=FileSize 2=Record-count 3=max-record-size 4=FileObjectCode 5=sortedflag 6=dedupedflag
   FileObject=ArrDimension(7)
   FileObject[0]=FileFullName(FileNameIn)
   FileObject[1]=FileSize(FileNameIn)
   if flag==1
       Terminate(!FileExist(FileNameIn),"File not found",FileFullName(FileNameIn))
       handle=FileOpen(FileNameIn,"READ")
       reccount=0
       maxsize=0
       while 1
          line=FileRead(handle)
          if line=="*EOF*" then break
          reccount=reccount+1
          maxsize=max(maxsize,strlen(line))
       endwhile
       FileClose(handle)
       FileObject[2]=reccount
       FileObject[3]=maxsize
   else
       FileObject[2]=0
       FileObject[3]=0
   endif
   FileObject[4]=1234567
   FileObject[5]=0 ; sorted=false
   FileObject[6]=0; 'deduped=false
   return (FileObject)
#EndFunction

;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;
; fmVerifyObject
; syntax    flag=fmVerifyObject(Object)
;   
;           flag      0=Bad object  1=OK object
;           object    Object handle from the fmGetFileObject function
;
;This UDF is used to verify passed parameters to the various Filemath functions
;to make sure the fmGetFileObject was properly called for each file and that
;we are dealing with a good file object.

#DefineFunction fmVerifyFileObject(OBJ)
  ; check  to make sure passed parameter is afile object
  if VarType(OBJ)==256 
     if OBJ[4]==1234567
        return(1)
     endif
  endif
  return(0)
#EndFunction

;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;
; fmSortFile
;
; syntax   fmSortFile( Object-In, Object-Out )
;
;          Object-In   Object for inout file
;          Object-Out  Object for output file

; Sorts a file.  Dictionary String Sort.  Ascending
;
;Note: input and output object may be the same object.  Not recommended.
;
;fmSortFile Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***              ***COMMON DUP***    
;      ***COMMON***                    ***COMMON DUP***    
;      ***File A DUP***                ***COMMON***        
;      ***COMMON DUP***                ***File A 1***      
;      ***File A DUP***                ***File A 2***      
;      ***COMMON DUP***                ***File A 2***      
;      ***File A 1***                  ***File A 3***      
;      ***File X 1***                  ***File A 3***      
;      ***File A 2***                  ***File A 3***      
;      ***File X 2***                  ***File A DUP***    
;      ***File A 2***                  ***File A DUP***    
;      ***File X 2***                  ***File X 1***      
;      ***File A 3***                  ***File X 2***      
;      ***File X 3***                  ***File X 2***      
;      ***File A 3***                  ***File X 3***      
;      ***File X 3***                  ***File X 3***      
;      ***File A 3***                  ***File X 3***      
;      ***File X 3***                  ***FILEA UNIQUE***  


#DefineFunction fmSortFile( FileIn, FileOut)
   Terminate(!fmVerifyFileObject(FileIn),"SortFile Error","Param1 not a FileObject")
   Terminate(!fmVerifyFileObject(FileOut),"SortFile Error","Param2 not a FileObject")
   if FileIn[5]==1   ; is it already sorted?
       FileCopy(FileIn[0],FileOut[0],0)
   else
       reccount=Filein[2]
       maxsize=FileIn[3]
       bb=BinaryAlloc(reccount*maxsize) 
       handle=FileOpen(FileIn[0],"READ")
       ptr=0
       while 1
          line=FileRead(handle)
          if line=="*EOF*" then break
          BinaryPokeStr(bb,ptr,line)
          ptr=ptr+maxsize
       endwhile
       ptr=ptr-maxsize
       FileClose(handle)
       BinaryEODSet(bb,reccount*maxsize)
       BinarySort(bb, maxsize, 0, maxsize, 0 )
       handle=Fileopen(FileOut[0],"WRITE")
       for x=0 to ptr by maxsize
           FileWrite(handle,BinaryPeekStr(bb,x,maxsize))
       next
       FileClose(handle)
   endif

   FileOut[1]=Filesize(FileOut[0])
   FileOut[2]=FileIn[2]
   FileOut[3]=FileIn[3]
   FileOut[5]=1 ; sorted=TRUE
   FileOut[6]=FileIn[6]  ; preserve deduped status
   return
#EndFunction
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
; fmRemoveDups
;
; syntax   fmRemoveDups( Object-In, Object-Out )
;
;          Object-In   Object for inout file
;          Object-Out  Object for output file
;
; Removes duplicate lines from a file, first by sorting the file, then making
; a new file containing at most one copy of each line.
;
;Note: input and output files may NOT be the same file.
;
;fmRemoveDups Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***            ***COMMON DUP***       
;      ***COMMON***                  ***COMMON***           
;      ***File A DUP***              ***File A 1***         
;      ***COMMON DUP***              ***File A 2***         
;      ***File A DUP***              ***File A 3***         
;      ***COMMON DUP***              ***File A DUP***       
;      ***File A 1***                ***File X 1***         
;      ***File X 1***                ***File X 2***         
;      ***File A 2***                ***File X 3***         
;      ***File X 2***                ***FILEA UNIQUE***     
;      ***File A 2***                
;      ***File X 2***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***   

#DefineFunction fmRemoveDups( FileIn, FileOut)
   Terminate(!fmVerifyFileObject(FileIn),"RemoveDups Error","Param1 not a FileObject")
   Terminate(!fmVerifyFileObject(FileOut),"RemoveDups Error","Param2 not a FileObject")
   Terminate(stricmp(FileIn[0],FileOut[0])==0,"RemoveDups Error","Input file may not equal Output file")
   if FileIn[6]==1 ;pre=deduped
       FileCopy(FileIn[0],FileOut[0],0)
       FileOut[1]=Filesize(FileOut[0])
       FileOut[2]=FileIn[2]
       FileOut[3]=FileIn[3]
       FileOut[5]=FileIn[5]
       FileOut[6]=FileIn[6]
   else
       tempout=fmGetFileObject(FileCreateTemp("~fm"),0)
       fmSortFile(FileIn,tempout)
       handle=FileOpen(tempout[0],"READ")
       outhandle=FileOpen(FileOut[0],"WRITE")
       lastrec="dfrt456tgsdr436yter57857456345tefg577878765gdfgsdf345346rthdrg4352efhtyiu67uy4t"
       reccount=0
       maxsize=0
       while 1
          line=Fileread(handle)
          if line=="*EOF*" then break
          if line != lastrec
             lastrec=line
             reccount=reccount+1
             maxsize=max(maxsize,strlen(line))
             FileWrite(outhandle,line)
          endif
       endwhile
       FileClose(handle)
       FileClose(outhandle)
       FileDelete(tempout[0])
       FileOut[1]=Filesize(FileOut[0])
       FileOut[2]=reccount
       FileOut[3]=maxsize
       FileOut[5]=1 ; sorted=true
       FileOut[6]=1 ; 'deduped=true
   endif
   return
#EndFunction
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
; fmGetDups
;
; syntax   fmGetDups( Object-In, Object-Out )
;
;          Object-In   Object for inout file
;          Object-Out  Object for output file
;
; Get the duplicate lines from a file, first by sorting the file, then making
; a new file containing all but the first occurance of each different line.
;
;Note: input and output files may NOT be the same file.
;
;fmGetDups Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***             ***COMMON DUP***    
;      ***COMMON***                   ***File A 2***      
;      ***File A DUP***               ***File A 3***      
;      ***COMMON DUP***               ***File A 3***      
;      ***File A DUP***               ***File A DUP***    
;      ***COMMON DUP***               ***File X 2***      
;      ***File A 1***                 ***File X 3***      
;      ***File X 1***                 ***File X 3***      
;      ***File A 2***               
;      ***File X 2***               
;      ***File A 2***               
;      ***File X 2***               
;      ***File A 3***               
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***   

#DefineFunction fmGetDups( FileIn, FileOut)
   Terminate(!fmVerifyFileObject(FileIn),"fmGetDups Error","Param1 not a FileObject")
   Terminate(!fmVerifyFileObject(FileOut),"fmGetDups Error","Param2 not a FileObject")
   Terminate(stricmp(FileIn[0],FileOut[0])==0,"RemoveDups Error","Input file may not equal Output file")
   tempout=fmGetFileObject(FileCreateTemp("~fm"),0)
   fmSortFile(FileIn,tempout)
   handle=FileOpen(tempout[0],"READ")
   outhandle=FileOpen(FileOut[0],"WRITE")
   lastrec="dfrt456tgsdr436yter57857456345tefg577878765gdfgsdf345346rthdrg4352efhtyiu67uy4t"
   reccount=0
   maxsize=0
   while 1
      line=Fileread(handle)
      if line=="*EOF*" then break
      if line == lastrec
         reccount=reccount+1
         maxsize=max(maxsize,strlen(line))
         FileWrite(outhandle,line)
      else         
         lastrec=line         
      endif
   endwhile
   FileClose(handle)
   FileClose(outhandle)
   FileDelete(tempout[0])
   FileOut[1]=Filesize(FileOut[0])
   FileOut[2]=reccount
   FileOut[3]=maxsize
   FileOut[5]=1   ; tis sorted
   FileOut[6]=0   ; probably not dup free
   return
#EndFunction


;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;
; fmFileMath
;
; syntax   fmFilemath( Object-In-A , Object-In-B, Object-Out, OpType)
;
;          Object-In-A   Object of the "A" input file
;          Object-In-B   Object of the "B" input file
;          Object-Out    Object for output file
;          OpType        Flag indicating desired operation
;                         1 ADD           @fmADD=1
;                         2 SUBTRACT      @fmSubtract=2
;                         3 OR            @fmOR=3
;                         4 XOR           @fmXOR=4
;                         5 AND           @fmAND=5


;Examples
;      ===INPUT FILE A====================INPUT FILE B=====
;      ***FILEA UNIQUE***              ***FILEB UNIQUE***  
;      ***COMMON***                    ***COMMON***        
;      ***File A DUP***                ***File B DUP***    
;      ***COMMON DUP***                ***COMMON DUP***    
;      ***File A DUP***                ***File B DUP***    
;      ***COMMON DUP***                ***COMMON DUP***    
;      ***File A 1***                  ***File B 3***      
;      ***File X 1***                  ***File X 3***      
;      ***File A 2***                  ***File B 2***      
;      ***File X 2***                  ***File X 2***      
;      ***File A 2***                  ***File B 2***      
;      ***File X 2***                  ***File X 2***      
;      ***File A 3***                  ***File B 1***      
;      ***File X 3***                  ***File X 1***      
;      ***File A 3***                  ***File B 1***      
;      ***File X 3***                  ***File X 1***      
;      ***File A 3***                  ***File B 1***      
;      ***File X 3***                  ***File X 1***      
;                                                          
;========ADD==========  =======SUBTRACT====   ======OR=========
;  ***FILEA UNIQUE***     ***File A 1***       ***COMMON DUP***   
;  ***COMMON***           ***File A 2***       ***COMMON***       
;  ***File A DUP***       ***File A 2***       ***File A 1***     
;  ***COMMON DUP***       ***File A 3***       ***File A 2***     
;  ***File A DUP***       ***File A 3***       ***File A 3***     
;  ***COMMON DUP***       ***File A 3***       ***File A DUP***   
;  ***File A 1***         ***File A DUP***     ***File B 1***     
;  ***File X 1***         ***File A DUP***     ***File B 2***     
;  ***File A 2***         ***File X 3***       ***File B 3***     
;  ***File X 2***         ***File X 3***       ***File B DUP***   
;  ***File A 2***         ***FILEA UNIQUE***   ***File X 1***     
;  ***File X 2***                              ***File X 2***     
;  ***File A 3***                              ***File X 3***     
;  ***File X 3***                              ***FILEA UNIQUE*** 
;  ***File A 3***                              ***FILEB UNIQUE*** 
;  ***File X 3***
;  ***File A 3***
;  ***File X 3***
;  ***FILEB UNIQUE***       ======XOR=======    ======AND========  
;  ***COMMON***             ***File A 1***      ***COMMON DUP***  
;  ***File B DUP***         ***File A 2***      ***COMMON***      
;  ***COMMON DUP***         ***File A 3***      ***File X 1***    
;  ***File B DUP***         ***File A DUP***    ***File X 2***    
;  ***COMMON DUP***         ***File B 1***      ***File X 3***    
;  ***File B 3***           ***File B 2***                        
;  ***File X 3***           ***File B 3***                        
;  ***File B 2***           ***File B DUP***                      
;  ***File X 2***           ***FILEA UNIQUE***                    
;  ***File B 2***           ***FILEB UNIQUE***                    
;  ***File X 2***                                                 
;  ***File B 1***                                                 
;  ***File X 1***                                                 
;  ***File B 1***                                                 
;  ***File X 1***                                                 
;  ***File B 1***
;  ***File X 1***
;
;


#DefineFunction fmFileMath( FileA, FileB, FileOut, OpType)
;@fmADD=1
;@fmSubtract=2
;@fmOR=3
;@fmXOR=4
;@fmAND=5

   ;Only write lines in BOTH files
   Terminate(!fmVerifyFileObject(FileA),"fmFileMath Error","Param1 not a FileObject")
   Terminate(!fmVerifyFileObject(FileB),"fmFileMath Error","Param2 not a FileObject")
   Terminate(!fmVerifyFileObject(FileOut),"fmFileMath Error","Param3 not a FileObject")
   Terminate( (OpType<1 || OpType>5), "fmFileMath Error","param4 is not 1 2 3 4 or 5")

   ;Hack to optimize ADD(1)
   if OpType==1
       FileCopy(FileA[0],FileOut[0],0)
       FileAppend(FileB[0], FileOut[0])
       FileOut[1]=FileSize(FileOut[0])
       FileOut[2]=FileA[2]+FileB[2]  ; recordcount
       FileOut[3]=max(FileA[3],FileB[3])  ; maxsize
       FileOut[5]=0 ; sorted=false
       FileOut[6]=0 ; 'deduped=false
       return
   endif
;
   ;Hack to optimize "OR"
   if OpType==3  ; OR
       tempout=fmGetFileObject(FileCreateTemp("~fm"),0)
       fmFileMath(FileA,FileB,tempout,1)
       fmRemoveDups(tempout,FileOut)
       FileDelete(tempout[0])
       ;sorted and deduped status set by fmRemoveDups routine
       return
   endif
;

      switch optype
;       case 1  ; add                 ; ADD optimized above
;            tempA=FileA
;            tempB=FileB
;            FileOut[5]=0  ; not sorted
;            Fileout[6]=0  ; not deduped
;            break
       case 2  ; subtract            
            tempA=fmGetFileObject(FileCreateTemp("~fm"),0)
            tempB=fmGetFileObject(FileCreateTemp("~fm"),0)
            fmSortFile(FileA,tempA)
            fmSortFile(FileB,tempB)
            FileOut[5]=1  ;  sorted
            Fileout[6]=0  ;  deduped
            break                     
;       case 3  ; OR                  ; OR optimized above
       case 4  ; XOR
       case 5  ; AND
            tempA=fmGetFileObject(FileCreateTemp("~fm"),0)
            tempB=fmGetFileObject(FileCreateTemp("~fm"),0)
            fmRemoveDups(FileA,tempA)
            fmRemoveDups(FileB,tempB)
            FileOut[5]=1  ;  sorted
            Fileout[6]=1  ;  deduped
   endswitch


   handleA=FileOpen(tempA[0],"READ")
   handleB=FileOpen(tempB[0],"READ")
   out=FileOpen(FileOut[0],"WRITE")

   ;state machine    0 need both lines
   ;                 1 compare 2 lines
   reccount=0
   maxsize=0
   state=OpType
   while 1
   switch state


;       case 1       ;ADD   ; ADD replaced by optimization above
;                 lineA=strtrim(FileRead(handleA))
;                 lineB=strtrim(FileRead(handleB))
;                 :XADD
;                 if lineA=="*EOF*"
;                    if LineB!="*EOF*"
;                       FileWrite(out,lineB)
;                       reccount=reccount+1
;                       maxsize=max(maxsize,strlen(lineB))
;                       state = 97  ; dump rest of B
;                       continue
;                    endif
;                    state=99
;                    continue
;                 endif
;                 if lineB=="*EOF*"
;                    FileWrite(out,lineA)
;                    reccount=reccount+1
;                    maxsize=max(maxsize,strlen(lineA))
;                    state=98     ; dump rest of A
;                    continue
;                 endif
;
;                 FileWrite(out,lineA)
;                 reccount=reccount+1
;                 maxsize=max(maxsize,strlen(lineA))
;                 FileWrite(out,lineB)
;                 reccount=reccount+1
;                 maxsize=max(maxsize,strlen(lineB))
;                 lineA=strtrim(FileRead(handleA))
;                 lineB=strtrim(FileRead(handleB))
;                 GOTO XADD        ;NB   No structure levels being crossed.  DON'T USE GOTOs!!!!!!
;                 break
;

;
       case 2      ; SUBTRACT
                 lineA=strtrim(FileRead(handleA))
                 lineB=strtrim(FileRead(handleB))

                :XSUBTRACT
                 if lineA=="*EOF*"
                    state = 99  ; all done
                    continue
                 endif
                 if lineB=="*EOF*"
                    FileWrite(out, lineA)
                    reccount=reccount+1
                    maxsize=max(maxsize,strlen(lineA))
                    state=98     ; no more minus lines
                    continue
                 endif
        
                 test=strcmp(lineA,lineB)
                 if test==0    ; lines match.  toast both of them
                    lineA=strtrim(FileRead(handleA))
                    lineB=strtrim(FileRead(handleB))
                 else
                    if test > 0   ; lineA > lineB  Need new lineB
                       lineB=FileRead(handleB)
                    else          ; lineA < lineB
                       FileWrite(out,lineA)
                       reccount=reccount+1
                       maxsize=max(maxsize,strlen(lineA))
                       lineA=FileRead(handleA)
                       state=OpType
                    endif
                 endif
                 GOTO XSUBTRACT     ;NB   No structure levels being crossed.  DON'T USE GOTOs!!!!!!
                 break


;       case 3      ; OR    ; OR replace by optimization aboce
;                 lineA=strtrim(FileRead(handleA))
;                 lineB=strtrim(FileRead(handleB))
;                 :OROR
;                 if lineA=="*EOF*"
;                    state = 97  ; Add rest of B
;                    continue
;                 endif
;                 if lineB=="*EOF*"
;                    state = 98   ; add rest of A
;                    continue
;                 endif
;        
;                 test=strcmp(lineA,lineB)
;                 if test==0    ; lines match.  the only write one of them
;                       FileWrite(out,lineA)
;                       reccount=reccount+1
;                       maxsize=max(maxsize,strlen(lineA))
;                       lineA=strtrim(FileRead(handleA))
;                       lineB=strtrim(FileRead(handleB))
;                 else
;                    if test > 0   ; lineA > lineB  Need new lineB                      
;                       FileWrite(out,lineB)
;                       reccount=reccount+1
;                       maxsize=max(maxsize,strlen(lineB))
;                       lineB=FileRead(handleB)
;                    else          ; lineA < lineB
;                       FileWrite(out,lineA)
;                       reccount=reccount+1
;                       maxsize=max(maxsize,strlen(lineB))
;                       lineA=FileRead(handleA)
;                    endif
;                 endif
;                 GOTO OROR         ;NB   No structure levels being crossed.  DON'T USE GOTOs!!!!!!
;                 break
;;


       case 4      ; XOR
                 lineA=strtrim(FileRead(handleA))
                 lineB=strtrim(FileRead(handleB))
                 :XXOR
                 if lineA=="*EOF*"
                    if LineB!="*EOF*"
                       FileWrite(out,lineB)
                       reccount=reccount+1
                       maxsize=max(maxsize,strlen(lineB))
                       state = 97  ; dump rest of B
                       continue
                    endif
                    state=99
                    continue                 
                 endif
                 if lineB=="*EOF*"
                    FileWrite(out,lineA)
                    reccount=reccount+1
                    maxsize=max(maxsize,strlen(lineA))
                    state=98     ; dump rest of A
                    continue
                 endif
        
                 test=strcmp(lineA,lineB)
                 if test==0    ; lines match.  toast both of them
                    lineA=strtrim(FileRead(handleA))
                    lineB=strtrim(FileRead(handleB))
                 else
                    if test > 0   ; lineA > lineB  Need new lineB
                       FileWrite(out,lineB)
                       reccount=reccount+1
                       maxsize=max(maxsize,strlen(lineB))
                       lineB=FileRead(handleB)
                    else          ; lineA < lineB
                       FileWrite(out,lineA)
                       reccount=reccount+1
                       maxsize=max(maxsize,strlen(lineA))
                       lineA=FileRead(handleA)
                    endif
                 endif
                 goto XXOR          ;NB   No structure levels being crossed.  DON'T USE GOTOs!!!!!!
                 break




       case 5      ; AND
                 lineA=strtrim(FileRead(handleA))
                 lineB=strtrim(FileRead(handleB))
                 :XAND
                 if lineA=="*EOF*"
                    state = 99
                    continue
                 endif
                 if lineB=="*EOF*"
                    state = 99
                    continue
                 endif
        
                 test=strcmp(lineA,lineB)
                 if test==0    ; lines match.  the only case where we write a line
                       FileWrite(out,lineA)
                       reccount=reccount+1
                       maxsize=max(maxsize,strlen(lineA))
                       lineA=strtrim(FileRead(handleA))
                       lineB=strtrim(FileRead(handleB))                   
                 else
                    if test > 0   ; lineA > lineB  Need new lineB
                       lineB=FileRead(handleB)
                    else          ; lineA < lineB
                       lineA=FileRead(handleA)
                    endif
                 endif
                 goto XAND          ;NB   No structure levels being crossed.  DON'T USE GOTOs!!!!!!
                 break




        case 97        ; Dump rest of B to the outfile
                   while 1
                      lineB=FileRead(handleB)
                      if lineB=="*EOF*" then break
                      FileWrite(out,lineB)
                      reccount=reccount+1
                      maxsize=max(maxsize,strlen(lineB))
                   endwhile
                   state=99
                   break


        case 98          ; dump rest of A to the outfile           
                   while 1
                      lineA=FileRead(handleA)
                      if lineA=="*EOF*" then break
                      FileWrite(out,lineA)
                      reccount=reccount+1
                      maxsize=max(maxsize,strlen(lineA))
                   endwhile
                   state=99
                   break



   endswitch
   if state==99 then break
   endwhile

   FileClose(handleA)
   FileCLose(handleB)
   FileCLose(out)
   if OpType!=1  ; not for adds
      FileDelete(tempA[0])
      FileDelete(tempB[0])
   endif
   FileOut[1]=Filesize(FileOut[0])
   FileOut[2]=reccount
   FileOut[3]=maxsize
   return
#EndFunction
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;****************************************************************************************
;Examples begin here
Exclusive(@on)   ; Speed compute intensive processes up

;Using normal File Operations, build the two input test files

;Test file A
aaa=FileOpen("fm-aaa.txt","WRITE")
FileWrite(aaa,"***FILEA UNIQUE***")
FileWrite(aaa,"***COMMON***")
FileWrite(aaa,"***File A DUP***")
FileWrite(aaa,"***COMMON DUP***")
FileWrite(aaa,"***File A DUP***")
FileWrite(aaa,"***COMMON DUP***")
FileWrite(aaa,"***File A 1***")
FileWrite(aaa,"***File X 1***")
FileWrite(aaa,"***File A 2***")
FileWrite(aaa,"***File X 2***")
FileWrite(aaa,"***File A 2***")
FileWrite(aaa,"***File X 2***")
FileWrite(aaa,"***File A 3***")
FileWrite(aaa,"***File X 3***")
FileWrite(aaa,"***File A 3***")
FileWrite(aaa,"***File X 3***")
FileWrite(aaa,"***File A 3***")
FileWrite(aaa,"***File X 3***")
FileClose(aaa)

;Test file B
bbb=FileOpen("fm-bbb.txt","WRITE")
FileWrite(bbb,"***FILEB UNIQUE***")
FileWrite(bbb,"***COMMON***")
FileWrite(bbb,"***File B DUP***")
FileWrite(bbb,"***COMMON DUP***")
FileWrite(bbb,"***File B DUP***")
FileWrite(bbb,"***COMMON DUP***")
FileWrite(bbb,"***File B 3***")
FileWrite(bbb,"***File X 3***")
FileWrite(bbb,"***File B 2***")
FileWrite(bbb,"***File X 2***")
FileWrite(bbb,"***File B 2***")
FileWrite(bbb,"***File X 2***")
FileWrite(bbb,"***File B 1***")
FileWrite(bbb,"***File X 1***")
FileWrite(bbb,"***File B 1***")
FileWrite(bbb,"***File X 1***")
FileWrite(bbb,"***File B 1***")
FileWrite(bbb,"***File X 1***")
FileClose(bbb)


;Define the Filemath constants to made the code more readable
@fmADD=1
@fmSUBTRACT=2
@fmOR=3
@fmXOR=4
@fmAND=5



innya=fmGetFileObject("fm-aaa.txt",1)
innyb=fmGetFileObject("fm-bbb.txt",1)

out1=fmGetFileObject("fm-sort.txt",0)
fmSortFile(innya,out1)
;fmSortFile Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***              ***COMMON DUP***    
;      ***COMMON***                    ***COMMON DUP***    
;      ***File A DUP***                ***COMMON***        
;      ***COMMON DUP***                ***File A 1***      
;      ***File A DUP***                ***File A 2***      
;      ***COMMON DUP***                ***File A 2***      
;      ***File A 1***                  ***File A 3***      
;      ***File X 1***                  ***File A 3***      
;      ***File A 2***                  ***File A 3***      
;      ***File X 2***                  ***File A DUP***    
;      ***File A 2***                  ***File A DUP***    
;      ***File X 2***                  ***File X 1***      
;      ***File A 3***                  ***File X 2***      
;      ***File X 3***                  ***File X 2***      
;      ***File A 3***                  ***File X 3***      
;      ***File X 3***                  ***File X 3***      
;      ***File A 3***                  ***File X 3***      
;      ***File X 3***                  ***FILEA UNIQUE***  

;
out2=fmGetFileObject("fm-nodup.txt",0)
fmRemoveDups(innya,out2)
;fmRemoveDups Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***            ***COMMON DUP***       
;      ***COMMON***                  ***COMMON***           
;      ***File A DUP***              ***File A 1***         
;      ***COMMON DUP***              ***File A 2***         
;      ***File A DUP***              ***File A 3***         
;      ***COMMON DUP***              ***File A DUP***       
;      ***File A 1***                ***File X 1***         
;      ***File X 1***                ***File X 2***         
;      ***File A 2***                ***File X 3***         
;      ***File X 2***                ***FILEA UNIQUE***     
;      ***File A 2***                
;      ***File X 2***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***   

;
out3=fmGetFileObject("fm-dups.txt",0)
fmGetDups(innya,out3)
;fmGetDups Example
;      ===INPUT FILE====================OUTPUT FILE========
;      ***FILEA UNIQUE***             ***COMMON DUP***    
;      ***COMMON***                   ***File A 2***      
;      ***File A DUP***               ***File A 3***      
;      ***COMMON DUP***               ***File A 3***      
;      ***File A DUP***               ***File A DUP***    
;      ***COMMON DUP***               ***File X 2***      
;      ***File A 1***                 ***File X 3***      
;      ***File X 1***                 ***File X 3***      
;      ***File A 2***               
;      ***File X 2***               
;      ***File A 2***               
;      ***File X 2***               
;      ***File A 3***               
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***                
;      ***File A 3***                
;      ***File X 3***  
;
out4=fmGetFileObject("fm-add.txt",0)
fmFileMath(innya,innyb,out4,@fmADD)
;fmFileMath ADD Example
;      ===INPUT FILE A==============INPUT FILE B=============ADD==========  
;      ***FILEA UNIQUE***        ***FILEB UNIQUE***    ***FILEA UNIQUE***   
;      ***COMMON***              ***COMMON***          ***COMMON***         
;      ***File A DUP***          ***File B DUP***      ***File A DUP***     
;      ***COMMON DUP***          ***COMMON DUP***      ***COMMON DUP***     
;      ***File A DUP***          ***File B DUP***      ***File A DUP***     
;      ***COMMON DUP***          ***COMMON DUP***      ***COMMON DUP***     
;      ***File A 1***            ***File B 3***        ***File A 1***       
;      ***File X 1***            ***File X 3***        ***File X 1***       
;      ***File A 2***            ***File B 2***        ***File A 2***       
;      ***File X 2***            ***File X 2***        ***File X 2***       
;      ***File A 2***            ***File B 2***        ***File A 2***       
;      ***File X 2***            ***File X 2***        ***File X 2***       
;      ***File A 3***            ***File B 1***        ***File A 3***       
;      ***File X 3***            ***File X 1***        ***File X 3***       
;      ***File A 3***            ***File B 1***        ***File A 3***       
;      ***File X 3***            ***File X 1***        ***File X 3***       
;      ***File A 3***            ***File B 1***        ***File A 3***       
;      ***File X 3***            ***File X 1***        ***File X 3***       
;                                                      ***FILEB UNIQUE***   
;                                                      ***COMMON***         
;                                                      ***File B DUP***     
;                                                      ***COMMON DUP***     
;                                                      ***File B DUP***     
;                                                      ***COMMON DUP***     
;                                                      ***File B 3***       
;                                                      ***File X 3***       
;                                                      ***File B 2***       
;                                                      ***File X 2***       
;                                                      ***File B 2***       
;                                                      ***File X 2***       
;                                                      ***File B 1***       
;                                                      ***File X 1***       
;                                                      ***File B 1***       
;                                                      ***File X 1***       
;                                                      ***File B 1***       
;                                                      ***File X 1***       


out5=fmGetFileObject("fm-subtract.txt",0)
fmFileMath(innya,innyb,out5,@fmSUBTRACT)
;fmFileMath SUBTRACT Example
;      ===INPUT FILE A==============INPUT FILE B=============SUBTRACT====  
;      ***FILEA UNIQUE***        ***FILEB UNIQUE***       ***File A 1***     
;      ***COMMON***              ***COMMON***             ***File A 2***     
;      ***File A DUP***          ***File B DUP***         ***File A 2***     
;      ***COMMON DUP***          ***COMMON DUP***         ***File A 3***     
;      ***File A DUP***          ***File B DUP***         ***File A 3***     
;      ***COMMON DUP***          ***COMMON DUP***         ***File A 3***     
;      ***File A 1***            ***File B 3***           ***File A DUP***   
;      ***File X 1***            ***File X 3***           ***File A DUP***   
;      ***File A 2***            ***File B 2***           ***File X 3***     
;      ***File X 2***            ***File X 2***           ***File X 3***     
;      ***File A 2***            ***File B 2***           ***FILEA UNIQUE*** 
;      ***File X 2***            ***File X 2***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
                                                  

out6=fmGetFileObject("fm-OR.txt",0)
fmFileMath(innya,innyb,out6,@fmOR)
;fmMath OR Example
;      ===INPUT FILE A==============INPUT FILE B==============OR=========  
;      ***FILEA UNIQUE***        ***FILEB UNIQUE***      ***COMMON DUP***  
;      ***COMMON***              ***COMMON***            ***COMMON***      
;      ***File A DUP***          ***File B DUP***        ***File A 1***    
;      ***COMMON DUP***          ***COMMON DUP***        ***File A 2***    
;      ***File A DUP***          ***File B DUP***        ***File A 3***    
;      ***COMMON DUP***          ***COMMON DUP***        ***File A DUP***  
;      ***File A 1***            ***File B 3***          ***File B 1***    
;      ***File X 1***            ***File X 3***          ***File B 2***    
;      ***File A 2***            ***File B 2***          ***File B 3***    
;      ***File X 2***            ***File X 2***          ***File B DUP***  
;      ***File A 2***            ***File B 2***          ***File X 1***    
;      ***File X 2***            ***File X 2***          ***File X 2***    
;      ***File A 3***            ***File B 1***          ***File X 3***    
;      ***File X 3***            ***File X 1***          ***FILEA UNIQUE***
;      ***File A 3***            ***File B 1***          ***FILEB UNIQUE***
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***  

out7=fmGetFileObject("fm-XOR.txt",0)
fmFileMath(innya,innyb,out7,@fmXOR)
;fmFileMath XOR Example
;      ===INPUT FILE A==============INPUT FILE B==============XOR=======  
;      ***FILEA UNIQUE***        ***FILEB UNIQUE***       ***File A 1***    
;      ***COMMON***              ***COMMON***             ***File A 2***    
;      ***File A DUP***          ***File B DUP***         ***File A 3***    
;      ***COMMON DUP***          ***COMMON DUP***         ***File A DUP***  
;      ***File A DUP***          ***File B DUP***         ***File B 1***    
;      ***COMMON DUP***          ***COMMON DUP***         ***File B 2***    
;      ***File A 1***            ***File B 3***           ***File B 3***    
;      ***File X 1***            ***File X 3***           ***File B DUP***  
;      ***File A 2***            ***File B 2***           ***FILEA UNIQUE***
;      ***File X 2***            ***File X 2***           ***FILEB UNIQUE***
;      ***File A 2***            ***File B 2***         
;      ***File X 2***            ***File X 2***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***   
;
out8=fmGetFileObject("fm-AND.txt",0)
fmFileMath(innya,innyb,out8,@fmAND)
;fmFileMath AND Example
;      ===INPUT FILE A==============INPUT FILE B===============AND======== 
;      ***FILEA UNIQUE***        ***FILEB UNIQUE***       ***COMMON DUP***  
;      ***COMMON***              ***COMMON***             ***COMMON***      
;      ***File A DUP***          ***File B DUP***         ***File X 1***    
;      ***COMMON DUP***          ***COMMON DUP***         ***File X 2***    
;      ***File A DUP***          ***File B DUP***         ***File X 3***    
;      ***COMMON DUP***          ***COMMON DUP***        
;      ***File A 1***            ***File B 3***          
;      ***File X 1***            ***File X 3***          
;      ***File A 2***            ***File B 2***          
;      ***File X 2***            ***File X 2***          
;      ***File A 2***            ***File B 2***         
;      ***File X 2***            ***File X 2***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***         
;      ***File A 3***            ***File B 1***         
;      ***File X 3***            ***File X 1***  
;

Message("FileMath","Example run complete.")






Article ID:   W14749
Filename:   FileMath - Logical File Operations.txt
File Created: 2002:08:12:12:29:34
Last Updated: 2002:08:12:12:29:34