[S3tools-general] Calculation of local elements is broken when local target is not terminated with /

[Tilman Baumann]

Hi list,

I just found out something quite weird and petty unexpected. I thought I
let you know in case I'm the first to discover this.

There is a subtle difference between those two commands:
s3cmd sync s3://BUCKET/PREFIX target
s3cmd sync s3://BUCKET/PREFIX target/

Sync will work just fine without the trailing slash.
However, once you try to sync a second time, you notice that it will
re-download everything again!
With --verbose you see why. It fails generating the local list "INFO:
Compiling list of local files...". Therefore everything seen remote is new.

I can see why this happens. But it's obviously highly confusing because
it kind of half works. I think sync should always assume (or enforce)
that the target is a directory and treat it as such.


Cheers
Tilman Baumann

PS: I just cerry-picked Matt Domschs commit
2720431cf385c2af8f73a50fd2c1d517bdc4fffd which fixes the md5 checksum
check. It seems to work for the verification of downloaded files.
("WARNING: MD5 signatures do not match:" message)
Sync however still can't md5 check if files need to be synced if they
are multi-part. Line 508 in S3/FileLists.py
Looks ugly to fix. But would be quite cool to have :)