Syncronize FTP

Database Search

If you can't find the information using the categories below, post a question over in our WinBatch Tech Support Forum.

TechHome

WIL Extenders

WinInet

Samples from Users

Can't find the information you are looking for here? Then leave a message over on our WinBatch Tech Support Forum.

Syncronize FTP

Question:

I have a directory on my desktop pc I'd like to synchronize with a specific directory on my website via FTP. In other words, anytime I add or delete files from the directory on my pc, I'd like a winbatch script to reflect those changes on the specified directory on my website. Is that possible?

Answer:

It's sort of a work-in-progress, but this should get you pretty far, with little effort. You'll probably want to strip out a lot of code, as I developed it to meet very specific requirements. This app is made to run via the task scheduler. Also, the script FTP's from one remote server, to a local computer, then to a remote web server. Confusing as it is, it works really well for me.

FTP File Sync by CowboyDan

;***added function to skip the multimedia directory at wwwroot
;***added log file management that works (email at the begining of each day)
;***changed For loops with ItemCount() to be 2 steps now. 11/13/2006
;***Added file age checking to limit the number of times a file will transfer to one day. (288 times) 11/24/2006
;***file xfer tracking using date check(so that the files only transfer once) 11/28/06
;***---This was done by comparing the date time stamp for the file on the public ftp server vs
;***---the date time stamp for the file on the cms / web2 server
;***ToDo:
;****Clearer Logging.
;****"skip" variable made into a subroutine (like test_file_extension). This would allow for multiple folders to be skiped
;****wishlist: add user specific emailing (maybe keyed of from directory list)
AddExtender("WWINT44I.DLL")
IntControl(39, 2, 0, 0, 0)
IntControl(73, 2, 0, 0, 0)

;variable init section
log = ""
fix=""
listput=""
ftpgeterror=0
xx=0
remote_srcfilename=""
handle=0
testwrite=""
tophandle=0
conhandle1=0
mode="APPEND"

;ini read section
filename=StrCat(DirScript(), "FileFTP_Staging.ini")
If IniReadPvt("Local", "Run_state", "run", filename) == 0 Then Exit
wwwroot=IniReadPvt("Local", "wwwroot_path", "d:\inetpub\wwwroot", filename)
xferLogPath=IniReadPvt("Local", "transfer_log_path", "d:\inetpub\", filename)
xferLogName=IniReadPvt("Local", "transfer_log_Name", "Staging_filesync.log", filename)
errorLogName=IniReadPvt("Local", "Error_log_name", "staging_errorfile.log", filename)
errorlog=StrCat(xferlogpath, errorLogName)
xferlog=StrCat(xferlogpath, xferlogname)

fetch_server=IniReadPvt ("Remote_fetch", "server", "Public FTP server", filename) ; server address
fetch_uname=IniReadPvt ("Remote_fetch", "username", "anonymous", filename)
fetch_pwd=IniReadPvt ("Remote_fetch", "userpasswd", "password", filename)
fetch_start_folder=IniReadPvt ("Remote_fetch", "Home_Folder", "/WebAssetsToStaging/", filename)
skip=IniReadPvt ("Remote_fetch", "skip", "multimedia", filename)
filex=IniReadPvt ("Remote_fetch", "file_extension", "jpg,gif,pdf,xls,doc,ppt,xml,wpd,rtf,swf", filename)

put_server=IniReadPvt ("Remote_put", "server", "1.2.3.4", filename) ;Web server address
put_server_name=IniReadPvt ("Remote_put", "server_name", "webserver1", filename)
put_uname=IniReadPvt ("Remote_put", "username", "anonymous", filename)
put_pwd=IniReadPvt ("Remote_put", "userpasswd", "password", filename)

;Sub Routine to filter file extensions to transfer
#DefineSubRoutine test_file_extension()
filex=IniReadPvt ("Remote_fetch", "file_extention", "jpg,gif,pdf,xls,doc,ppt,xml,wpd,rtf,swf", filename)  ; list of extension from the ini file
compare=FileExtension(ItemExtract(1, remote_srcfilename, @TAB)) ; grabs the file extension from remote_srcfilename
   x=ItemCount(filex, ",")
   For fx = 1 To x ; count the number of extensions in the list
      file_extension_ini=ItemExtract(fx, filex, ",") ;extract a file extension
      If compare == file_extension_ini ;test if it matches
         Return 1 ; if it matches, return 1 (one) and exit
         Exit
      EndIf
   Next fx  ;loop through the next extension
   Return 0  ; if no matches, return 0 (zero)
#EndSubRoutine

;function to email errors and log files
#DefineFunction email_ftp_error (error, attach)
   filename=StrCat(DirScript(), "FileFTP_Staging.ini")
   host=IniReadPvt ("Local", "email_server", "My Mail Server", filename)   ; SMTP mail server address
   fromaddr=IniReadPvt ("Local", "email_from", "admin@mydomain.com", filename) ;from address
   tolist=IniReadPvt ("Local", "email_to", "myemail@mydomain.com", filename)     ; to address
   AddExtender("WWPST44I.DLL") ;extender to make it work
   name=ComputerNameGet(0)
   userid=""
   password=""
   port=""
   cclist=""
   bcclist=""
   If error=="Yesterday's Staging transfer log" Then subject=error
   Else subject="FTP ERROR on %name%"
   msg=StrCat("On ", TimeDate(),": ", error)
   attachments=attach
   flags=""
   kInit(host,fromaddr,userid,password,port)
   kDest(tolist,cclist,bcclist)
   test=kSendText(subject,msg,attachments,flags)
   If test == 1 Then Return 1
   Else Return 0
#EndFunction

;create log file if none is available
If FileExist(xferlog) == @FALSE
   handle=FileOpen(xferlog, "WRITE")
   FileWrite(handle, "File Transfer log for staging")
   FileClose(handle)
EndIf

;email log file then start new one at the begining of the day
If ItemExtract(3, FileYmdHms(xferlog), ":") !=  ItemExtract(3, TimeYmdHms(), ":")
   mode="WRITE"
   email_ftp_error("Yesterday's Staging transfer log", xferlog)
   FileTimeTouch (xferlog)
EndIf

;connects to ftp server to GET files
handle=FileOpen(xferlog, mode)
Header=StrCat(@CRLF, TimeDate())
testwrite=FileWrite(handle, Header)
tophandle=iBegin(0,"","")
conhandle1=iHostConnect(tophandle, fetch_server, @FTP, fetch_uname, fetch_pwd)
If conhandle1==0
 FileWrite(handle, "ERROR: Unable to connect to %fetch_server%")
 FileClose(handle)
 fix=email_ftp_error("iHostConnect: Unable to connect to %fetch_server%", "")
 Exit
EndIf

;initialize recursive 'find'
init=iFtpFindInit(conhandle1, fetch_start_folder, "*.*", @TRUE, @FALSE)
 If init==0
   FileWrite(handle, "iFtpFindInit Failed: could not find %fetch_start_folder% - or something else is wrong")
   FileClose(handle)
   fix=email_ftp_error("iFtpFindInit Failed: could not find %fetch_start_folder% - or something else is wrong", "")
   Exit
 EndIf

;recursively searches all the sub-directories. xx var is used for debugging and error reporting only
While 1
   xx=xx+1
   remote_srcfilename=iFtpFindNext(@TRUE);include attributes
   If remote_srcfilename==0 || remote_srcfilename=="" Then Break
   If test_file_extension() == 0
      email_ftp_error("%remote_srcfilename% does not fit included extensions", "")
      remote_srcfilename=""
      Continue
   EndIf
   bill=ItemExtract(6, remote_srcfilename, @TAB)
   bill_plus5=TimeAdd(bill, "00:00:00:05:00:00")
   tempfile=StrReplace(ItemExtract(1,remote_srcfilename,@TAB), fetch_start_folder, "")
   destfile=StrReplace(tempfile, "/", "\")
   destfile=StrCat("\", destfile)
   If ItemExtract (1, destfile, "\") == skip Then Continue ; skips over the multimedia directory at wwwroot
   destfile=StrCat(wwwroot, destfile)  ;local file
   If FileExist(destfile) Then ted=FileYmdHms(destfile)
   Else ted = 0
   If Ted > bill_plus5 Then Continue
   getfile=iFtpGet(conhandle1,ItemExtract(1,remote_srcfilename,@TAB),destfile,0,@BINARY, @TRUE)
   If getfile==0
      FileWrite(handle, "#1->iFtpGet Failed on File: %remote_srcfilename% - Check receiving file path")
      ftpgeterror=remote_srcfilename
   Else
   ;      log=StrCat(remote_srcfilename, " ==> ", destfile, @CRLF)
      log=StrCat(ItemExtract(1,remote_srcfilename,@TAB)," >> ", bill_plus5, " ==> ", destfile, @CRLF)
      FileWrite(handle, log)
      listput=StrCat(listput, destfile, @TAB)
   EndIf
EndWhile
iClose(init)
iClose(conhandle1)
iClose(tophandle)

If ftpgeterror != 0 Then email_ftp_error("iFtpGet Failed on File: %ftpgeterror% - Check Transfer log", "")
If listput == ""
   FileWrite(handle, "All Done ==> No files to tansfer")
   FileClose(handle)  ;takes care of the 2 "filewrite" commands above
   Exit                ;exit here if there is nothing to transfer
EndIf

; put files on one remote server
tophandle2=iBegin(0,"","")
conhandle2=iHostConnect(tophandle2, put_server, @FTP, put_uname, put_pwd)
If conhandle2==0
 FileWrite(handle, "ERROR: Unable to connect to %put_server_name%")
 FileClose(handle)
 fix=email_ftp_error("iHostConnect: Unable to connect to %put_server_name% (%put_server%)", "")
 Exit
EndIf
x=ItemCount(listput, @TAB)
For y=1 To x
locfilename=ItemExtract(y,listput,@TAB)
   If locfilename=="" Then Break
intfilename=StrReplace(locfilename, wwwroot, "")
targfilename=StrReplace(intfilename, "\", "/")
ret=iFtpPut(conhandle2,locfilename,targfilename,0,@BINARY, @TRUE)
   If ret==0
     fix=email_ftp_error("iftpPut Failed to %put_server_name%: %targfilename%", "")
     listput=ItemRemove(y, listput, @TAB)
   EndIf
   FileWrite(handle, StrCat("iFtpPut: %put_server_name% ==>", targfilename))
Next y
iClose(conhandle2)
iClose(tophandle2)
Exit



:WBERRORHANDLER
errhand=FileOpen(errorlog, "WRITE")
Error=LastError()
FileWrite(errhand, StrCat("LastError Value is ",Error))
FileWrite(errhand, StrCat("wberrorhandlerline ",wberrorhandlerline))
FileWrite(errhand, StrCat("wberrorhandleroffset ",wberrorhandleroffset))
FileWrite(errhand, StrCat("wberrorhandlerassignment ",wberrorhandlerassignment))
FileWrite(errhand, StrCat("wberrortextstring ", wberrortextstring))
FileWrite(errhand, StrCat("wberroradditionalinfo ", wberroradditionalinfo))
FileWrite(errhand, StrCat("Number of times the While Loop ran ", xx ))
FileWrite(errhand, StrCat("iFtpFindNext output: ", remote_srcfilename))
FileWrite(errhand, StrCat("Log File Check (File Handle Number): ", handle))
FileWrite(errhand, StrCat("Log File Header successful?: ", testwrite))
FileWrite(errhand, StrCat("connection strings, group 1 (should be 2 numbers):", tophandle, ", ", conhandle1))
FileClose(errhand)
If Error != 321 Then fix=email_ftp_error("Error Handler: PANIC!", errorlog)
Return
</PRE>

<HR>
FTP File Sync.ini
<PRE>
; This is the configuration file for the FileFTP_Staging.exe program.
; This file must stay in the same directory as the FileFTP_Staging program to be used.
; Stuff in < > is for you to fill in
;

[Local]
; local path to the local wwwroot directory on the local computer.
; This is used as a temp dumping place for all the files that are being transfered.
; this needs to be local to the program, because I don't know how to do
; remote <==> remote transfers (like any decent FTP client can)
wwwroot_path=d:\inetpub\wwwroot

; "run_state" will not allow the program to run if 0  (zero) is used. The number 1 (one) will allow execution.
Run_state=1

; email server must be an SMTP. ESMTP with username and passwd is not currently supported
email_server=<my email SERVER>

; address to be displayed in the "From" field of the email
email_from=<admin@mydomain.com>

; address to send the email to. Specify Multiple addresses by "|" (pipe) character between them
email_to=<me@mydomain.com>

; Path and file name for the activity log. This log is emailed once a day.
transfer_log_path=d:\inetpub\
transfer_log_name=Staging_filesync.log

; Error log file name to use.
; This log file is overwritten on every error and is emailed on a program error.
; It shouldn't be used much. If it is, it's because there's a bug in the code,
; or something is not configured correctly in this ini file.
Error_log_name=staging_errorfile.log


[Remote_fetch]
; The public ftp server address (or FQDN)
SERVER=<Public FTP SERVER address>

; User name of the account to use
username=<FTP SERVER ID>
userpasswd=<FTP SERVER password>

; Starting folder from which the directory structure will be searched.
; Must include leading and trailing slashes
Home_Folder=/WebAssetsToStaging/

; If "file_rename" is set to "1" (the number one)
; files will have a date tag appended to the extention so that they will only transfer once
;File_rename=0

; Skip directory listing - Used to skip directories that will be handled in a special manner.
; List directory names without slashes,
; more than one directory can be specified by using "," (comma) between the directory names
skip="Multimedia"

[Remote_put]
; Server address or (FQDN) name. This is where we are dumping the files to.
SERVER=<Web srever address>

; "server_name" is the common name for the server. It's used only in the emails and log files.
server_name=<webserver name>

; User credentials for logging into the server
Username=<Web SERVER user account>
userpasswd=<web SERVER user password>

Article ID:   W16894

File Created: 2007:07:03:14:26:44

Last Updated: 2007:07:03:14:26:44