Read a Large File in Multiple Parts

 Keywords: Binary BinaryReadEx huge_Multiply huge_Add huge_Subtract udfHuge_Compare


; Detlev's Example how to read big file in multiple parts.


#DefineFunction udfHuge_Compare (hugeNum1, hugeNum2)

;This UDF assumes that the HugeMath extender has been previously loaded.
hugeNum = huge_Subtract (hugeNum1, hugeNum2)
If StrSub (hugeNum, 1, 1) == "-" Then Return -1

If hugeNum == "0" Then Return 0

Return 1
#EndFunction


AddExtender ("WWHUG34i.DLL")

strMsgTitle = "Demo: Read big file in multiple parts."
BoxOpen (strMsgTitle, "")

; Create testfile, please wait ......
strFilename = "K:\TEST.BIN"
hugeFileSize = huge_Multiply (1 << 30, 40) ; "42949672960" ; 40,0 GB (42.949.672.960 Bytes).

BoxText ("Writing test file: " : strFilename : @LF : "Size: " : hugeFileSize)

BinaryWriteEx (0, 0, strFilename, hugeFileSize, -1)


; Define chunk size.
hugeBBSize = huge_Multiply (1 << 20, 100) ; 100 MB chunk size.

hdlBB = BinaryAlloc (hugeBBSize) ; Physical memory is the limit.


; Loop through the parts.
intCount = 0

hugeOffset = "0"
While @TRUE
   intCount = intCount + 1

   hugeBytesRead = "" : BinaryReadEx (hdlBB, 0, strFilename, hugeOffset, hugeBBSize)

   BoxText ("Count: " : intCount : @LF : "Offset: " : hugeOffset : @LF : "Bytes read: " : hugeBytesRead)

   ; Do something with the data in the binary buffer.

   ; Calculate start address for the next part.
   hugeOffset = huge_Add (hugeOffset, hugeBBSize)

   If udfHuge_Compare (hugeOffset, hugeFileSize) == 1 Then Break

EndWhile

hdlBB = BinaryFree (hdlBB)

Message (strMsgTitle, "Ready.")

BoxShut ()

Exit

Article ID:   W17579

Filename:   Read a Large File in Multiple Parts.txt

File Created: 2009:08:31:13:23:32

Last Updated: 2009:08:31:13:23:32

Database Search

TechHome

Read a Large File in Multiple Parts