Russell Gadd
2015-04-10 14:05:10 UTC
I'm having difficulty understanding this:
continue-put
"Continue uploading partially uploaded files or
multipart upload parts. Restarts/parts files that
don't have matching size and md5. Skips files/parts
that do. Note: md5sum checks are not always
sufficient to check (part) file equality. Enable this
at your own risk."
It reads ok up to the note but then I'm stumped as to why "md5sum checks
are not always sufficient to check (part) file equality". I understood md5
collisions are virtually impossible in practice. Furthermore I experimented
with the feature within sync which copies files across the bucket in s3
when it detects that a file already is in s3, rather than uploading it,
even if the filename and timestamp are different. So I assume you can only
detect identical files using the MD5 which implies your program relies on
them.
Can you explain why md5sum checks aren't necessarily sufficient and what
the risk is? How would I avoid the risk?
Regards
Russell
continue-put
"Continue uploading partially uploaded files or
multipart upload parts. Restarts/parts files that
don't have matching size and md5. Skips files/parts
that do. Note: md5sum checks are not always
sufficient to check (part) file equality. Enable this
at your own risk."
It reads ok up to the note but then I'm stumped as to why "md5sum checks
are not always sufficient to check (part) file equality". I understood md5
collisions are virtually impossible in practice. Furthermore I experimented
with the feature within sync which copies files across the bucket in s3
when it detects that a file already is in s3, rather than uploading it,
even if the filename and timestamp are different. So I assume you can only
detect identical files using the MD5 which implies your program relies on
them.
Can you explain why md5sum checks aren't necessarily sufficient and what
the risk is? How would I avoid the risk?
Regards
Russell