[S3tools-general] problems uploading .zip files (but not other kinds of files)

Jeremy Wadsack

2016-10-14 18:55:18 UTC

Now that you mention this I saw this as well for a time. I just tested a
small file again and still get the same errors.

upload: 'test.zip' -> 's3://test.bucket/test.zip' [1 of 1]
2375680 of 9642480 24% in 4s 507.10 kB/s failed
WARNING: Upload failed: /test.zip ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
upload: 'test.zip' -> 's3://test.bucket/test.zip' [1 of 1]
2383872 of 9642480 24% in 3s 593.05 kB/s failed
WARNING: Upload failed: /test.zip ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.01)
WARNING: Waiting 6 sec...
upload: 'test.zip' -> 's3://test.bucket/test.zip' [1 of 1]
1904640 of 9642480 19% in 6s 300.84 kB/s failed
WARNING: Upload failed: /test.zip ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.05)
WARNING: Waiting 9 sec...
upload: 'test.zip' -> 's3://test.bucket/test.zip' [1 of 1]
1605632 of 9642480 16% in 21s 73.33 kB/s failed
WARNING: Upload failed: /test.zip ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.25)
WARNING: Waiting 12 sec...

I chocked it up to S3 or network issues, not s3cmd. In fact, I'm pretty
sure that I resolve this last time by moving to another network and doing
the transfer there. Or maybe it was a time of day issue.

This SO post
<http://stackoverflow.com/questions/5774808/s3cmd-failed-too-many-times>
implies lots of different causes for this message including files too
large, time of day being incorrect, bucket region. It seems the common
solution was to switch to aws cli. :(

Jeremy Wadsack

Post by Allen Hancock
I've been using s3cmd happily for about 2 years, from my Mac OS computer
in an automated script I run.
Lately, I've been having issues with .zip uploads timing out, but other
files (pkg and png) don't have this issue.
/usr/local/bin/s3cmd put --acl-public --guess-mime-type
"/path/to/thearchive.zip"
It uploads some, then fails, the tries again 5 times
2646016 of 3569354 74% in 2s 1099.92 kB/s failed
2539520 of 3569354 71% in 2s 1118.48 kB/s failed
1847296 of 3569354 51% in 6s 278.84 kB/s failed
1691648 of 3569354 47% in 22s 74.51 kB/s failed
1499136 of 3569354 42% in 92s 15.76 kB/s failed
1486848 of 3569354 41% in 453s 3.20 kB/s failed
the pkg files I send are of similar size and upload fine.
Has anyone else see this issue?
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
S3tools-general mailing list
https://lists.sourceforge.net/lists/listinfo/s3tools-general