If you have several similar large files to upload over a slow link you can use ‘rdiff’ to optimise the transfer. ‘rdiff’ compares files block by block and produces a delta file that contains only the blocks that differ.
\nIt is therefore of most use when the two files are mostly similar but some parts different such as powerpoint product presentations\u00c2\u00a0 targeted for two different customers.<\/p>\n
In this example we have two similar but different documents:<\/p>\n
$ dir *.doc<\/p>\n
643,584 document-edition-two.doc
\n634,880 document-original.doc<\/p>\n
Generate an MD5 hash to be used later to verify the file integrity
\n$ md5 document-edition-two.doc
\n8280AEAFABC0833D5FEC64CE5FEF6237\u00c2\u00a0 document-edition-two.doc<\/p>\n
Prepare a “signature” file which contains hash codes of each block in the base file.<\/p>\n
$ rdiff signature document-original.doc document.sig<\/p>\n
Next I use that signature file to see which blocks are different in the second file and extract them to a delta file.<\/p>\n
$ rdiff delta document.sig document-edition-two.doc document.delta<\/p>\n
$ dir document.delta<\/p>\n
78,504 document.delta<\/p>\n
Note that the “delta” file is only 12% of the size of “document-edition-two.doc”, the relative file size depends on how similar the two documents are.<\/p>\n
Now I upload the files “document-original.doc” and “document.delta”<\/p>\n
On the server I (or the recipients of the files) run ‘rdiff’ to generate the second document from the first and the delta.
\n$ rdiff patch document-original.doc document.delta document-edition-two-reconstructed.doc<\/p>\n
Check the MD5 hash to confirm that the second document has been faithfully reproduced.<\/p>\n
$
\n$ md5 document-edition-two-reconstructed.doc
\n8280AEAFABC0833D5FEC64CE5FEF6237\u00c2\u00a0 document-edition-two-reconstructed.doc<\/p>\n
Download rdiff<\/a> for Windows, compiled with Cygwin\u00c2\u00a0\u00c2\u00a0<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" If you have several similar large files to upload over a slow link you can use ‘rdiff’ to optimise the transfer. ‘rdiff’ compares files block by block and produces a delta file that contains only the blocks that differ. It is therefore of most use when the two files are mostly similar but some parts […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-54","post","type-post","status-publish","format-standard","hentry","category-tools"],"_links":{"self":[{"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=\/wp\/v2\/posts\/54","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=54"}],"version-history":[{"count":0,"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=\/wp\/v2\/posts\/54\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=54"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=54"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.eriugena.org\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=54"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}