best connection suggestions or best practices for multi site data pull, expert advice needed

i will be having a NAS box that will be receiving data from 20 sites in 20 cities. the computers will be sending about 500MB a night from each site. What are the requirements for equipment hardware that is needed to achieve this?

thanks gd

From serverfault dasko

• A DSL connection should be enough; if we assume that a "night" is 8 hours, the average bitrate required to transfer 10GB in 8 hours is about 3.5Mbps, which any decent DSL connection should be able to sustain in the downstream direction (which is what you're going with). You might need a special data plan from your DSL provider, since 300GB of traffic is a fair bit (at least in my part of the world) and you really don't want to get shaped.

  From womble

• Some allowances for failures both on the sending and receiving side should be considered and the assumption should be that the transfer will be running 24x7. One missed "night" and the data has doubled for the next night, a missed weekend and there is 30GB + 10GB for Monday night. It's easy to get behind and have a lot of trouble catching up. This may mean you'll more or dedicated bandwidth at the receive site.

  On the client side you'll need to calculate upload times, if using asymmetric DSL then consider what happens if a client skips a few days and has to catch up. The upload will probably need to be throttled or some type of QOS implemented to give your "normal" traffic priority during working hours.

  The hardware really depends on how long you need to keep the data stored at each location, which will allow you to calculate storage capacities needed. Most current firewalls will allow for QOS, most consumer grade routers/modems will not.

  If the data will be all new each day then you'll need to copy the entire set each night. However if the data will be a change from the previous day then consider software that will let you copy just the changes/delta. The delta can be either block or file level depnding on the systems involved. In Linux RSync will do block level diffs. In Windows Server 2008 there is Remote Differential Compression.

  From Ed Fries