[S3tools-general] A few questions

[Dylan]

Hello,

Tonight I started playing around with amazon s3 as a backup solution for
my company. I quickly stumbled upon your website for s3cmd and its
providing exactly what I need. After playing around with it for a while
I have a few questions. I'd like to say I'm using this as a replacement
for rsync'ing files between a backup server and client servers.


1.) When using the sync command with the -r option it doesn't seem to
sync sub directories of the directory I'm syncing? I'm sure I'm just
missing something here.

2.) If I create 4 files all owned by root:root then sync with s3. Then
change one to dylan:dylan when I run the sync command again it doesn't
sync this file as it thinks nothing has changed? Any way to fix this?
I'm running the sync command with the -p option.

3.) When I sync using the -p option I can verify through the s3 console
that the ownership is storing correctly on amazons end through the
metadata. But when I perform a get to retrieve the file it is owned by
the user that retrieved the file. This would be a major pain when having
to restore backups as I will have files owned by 100's if not 1000's of
users.


Overall this utility is awesome and has made s3 much easier to tackle
than I originally thought.

Also as a bit of a side discussion does anyone use the reduced
redundancy storage? Does that mean amazon doesn't take backups of your
info on s3? How much money does it save you?

Thank you in advance,

Dylan K

[Jeremy Wadsack]

S3 doesn't have the same ownership/permissions that your Unix system does
so backing up to S3 won't retain the permissions and restoring is done with
the permissions of the user running the command.

There are probably third-party tools (that you could pay for) that would
handle backups to S3 that preserve permissions.

If you're making backups do you really want reduced redundancy? It will
save you money as you can see in the pricing
<http://aws.amazon.com/s3/pricing/>.

Amazon doesn't make backups unless you request it. S3 is hosted on
redundant media already and provides the most durable storage in the world
(empirically). You can backup your S3 by having Amazon archive it to Glacier
<http://aws.amazon.com/glacier/> which is much cheaper at the cost of
slower access times (in hours). You can configure a bucket policy to move
data to Glacier after a period of time, which is a command archival
pattern.

You might also consider sending data directly to Glacier (and skipping S3)
if you don't need immediate access to backups. s3cmd will restore from
Glacier but not write directly (that I know of). But you could backup to an
S3 bucket and then bounce that to Glacier in a day or in hours if you want
to reduce S3 costs. Here's a post
<http://gregsramblings.com/2013/06/07/simple-server-backups-with-amazon-s3-lifecycle-rules-and-glacier/>
about doing this.

Jeremy Wadsack

Post by Dylan
Hello,
Tonight I started playing around with amazon s3 as a backup solution for
my company. I quickly stumbled upon your website for s3cmd and its
providing exactly what I need. After playing around with it for a while I
have a few questions. I'd like to say I'm using this as a replacement for
rsync'ing files between a backup server and client servers.
1.) When using the sync command with the -r option it doesn't seem to sync
sub directories of the directory I'm syncing? I'm sure I'm just missing
something here.
2.) If I create 4 files all owned by root:root then sync with s3. Then
change one to dylan:dylan when I run the sync command again it doesn't sync
this file as it thinks nothing has changed? Any way to fix this? I'm
running the sync command with the -p option.
3.) When I sync using the -p option I can verify through the s3 console
that the ownership is storing correctly on amazons end through the
metadata. But when I perform a get to retrieve the file it is owned by the
user that retrieved the file. This would be a major pain when having to
restore backups as I will have files owned by 100's if not 1000's of users.
Overall this utility is awesome and has made s3 much easier to tackle than
I originally thought.
Also as a bit of a side discussion does anyone use the reduced redundancy
storage? Does that mean amazon doesn't take backups of your info on s3? How
much money does it save you?
Thank you in advance,
Dylan K
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website,
sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for
all
things parallel software development, from weekly thought leadership blogs
to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
S3tools-general mailing list
https://lists.sourceforge.net/lists/listinfo/s3tools-general

[Will McCown]

Post by Jeremy Wadsack
S3 doesn't have the same ownership/permissions that your Unix system
does so backing up to S3 won't retain the permissions and restoring is
done with the permissions of the user running the command.
There are probably third-party tools (that you could pay for) that would
handle backups to S3 that preserve permissions.

I use duplicity (http://duplicity.nongnu.org/) to backup up my linux
boxes to S3. It supports a variety of back ends, uses Gnupg encryption,
etc. It's only downside is that if you are ever in a situation of
needing to recover files without duplicity, it is possible but not
easy.