[S3tools-general] Server backup to s3 using s3cmd sync

Discussion:

sumeet dembra

2015-08-18 10:44:52 UTC

Hi,

We have to take daily server backup of around 70-80 GB to s3.
Currently we are using duply for this.
I tried to restore this. But it seems that duply restore is not working for
too large backups.

So we were thinking of using s3cmd sync for backing local files to s3.
Is this the right approach?

Please let me know.

Thanks in advance.

- Regards,
Sumeet

Marc Teichtahl

2015-08-18 10:54:25 UTC

Permalink

It is certainly one approach.

Is the data ephemeral ? Does it change a lot ? If not you may want to consider an incremental approach which would keep transfer size lower. Also, if your holding the data for long periods and restore duration is not an issue for you then you may want to consider glacier to keep your costs down.

Sent from my iPhone

Post by sumeet dembra
Hi,
We have to take daily server backup of around 70-80 GB to s3.
Currently we are using duply for this.
I tried to restore this. But it seems that duply restore is not working for
too large backups.
So we were thinking of using s3cmd sync for backing local files to s3.
Is this the right approach?
Please let me know.
Thanks in advance.
- Regards,
Sumeet
------------------------------------------------------------------------------
_______________________________________________
S3tools-general mailing list
https://lists.sourceforge.net/lists/listinfo/s3tools-general

------------------------------------------------------------------------------

sumeet dembra

2015-08-18 13:55:37 UTC

Permalink

Yes. We want an incremental approach.
I just wanted to confirm whether s3cmd sync
is a good approach for this?

Post by Marc Teichtahl
It is certainly one approach.
Is the data ephemeral ? Does it change a lot ? If not you may want to
consider an incremental approach which would keep transfer size lower.
Also, if your holding the data for long periods and restore duration is not
an issue for you then you may want to consider glacier to keep your costs
down.
Sent from my iPhone

for

Post by sumeet dembra
too large backups.
So we were thinking of using s3cmd sync for backing local files to s3.
Is this the right approach?
Please let me know.
Thanks in advance.
- Regards,
Sumeet

------------------------------------------------------------------------------

Post by sumeet dembra
_______________________________________________
S3tools-general mailing list
https://lists.sourceforge.net/lists/listinfo/s3tools-general

------------------------------------------------------------------------------
_______________________________________________
S3tools-general mailing list
https://lists.sourceforge.net/lists/listinfo/s3tools-general

Marc Teichtahl

2015-08-18 14:01:23 UTC

Permalink

Should be fine

Experiment with a subset of your data or smaller dataset

Sent from my iPhone

Post by sumeet dembra
Yes. We want an incremental approach.
I just wanted to confirm whether s3cmd sync
is a good approach for this?

Post by Marc Teichtahl
It is certainly one approach.
Is the data ephemeral ? Does it change a lot ? If not you may want to consider an incremental approach which would keep transfer size lower. Also, if your holding the data for long periods and restore duration is not an issue for you then you may want to consider glacier to keep your costs down.
Sent from my iPhone