Find solution
How to sequence events while uploading large files to Amazon S3?

Is uploading through a browser a viable option for files in excess of 15GB?

  • I recently installed an s3 direct upload in a rails app, but just found out the files the client needs to put on amazon range from 5GB to 20GB. In addition to being posted they also need to be retrieved. At what point is an FTP solution a necessity. What are the inherent speed differences like?

  • Answer:

    No, the likelihood that this is going to work is very, very low.  Here are some considerations: You (and the client connecting to you) need to keep the socket open for the full duration of the 15GB upload.  For users with 1Mbps DSL, this will be about 36 hours.   The likelihood of a single socket staying open this long is very low. RAM usage on the Rails side.  You absolutely need to make sure that the POST data comes in from your client and goes directly to DISK.  Do not pass go, do not collect 200GB RAM.

Steve Lacy at Quora Visit the source

Was this solution helpful to you?

Other answers

It's viable, except that: 1. you need to configure the server to accept large POST request (the default limit is usually ~ 2GB).  2. you need to somehow make sure that the clients won't close their browsers during the upload. Since uploading a 15GB file will take a long time, it's hard to make sure the client won't accidentally close his browser during the upload, and that's why it's better to have a native software that handles the upload through either HTTP or FTP. FTP is usually recommended for large file uploads, but if it's not available, you can look into HTTP file system API as well: http://www.html5rocks.com/en/tutorials/file/filesystem/

Derek Chiang

You won't get a big speed difference bit given the size, a browser upload it, at less, risky. FTP is more reliable and with this kind of upload, you definitely need a "resume" feature, wich is built in FTP.

Alain Mevellec

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.