Friday, January 21, 2011

Squid utility command to force a cache load from a local hard drive?

Is there a Squid utility command to force a cache load from a local hard drive?

              xww.example.com
                /w /x /y /z
                  Apache
         ------------------------
         Slow Internet Connection
         ------------------------
          lan1  |  lan2  |  lan3
                |        |
          cache |  cache |  cache
          /x /y |  /x /z |  /y /z
          Squid |  Squid |  Squid
         ------ | ------ | ------
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser
         Browser| Browser|Browser

For example, plug the USB hard drive into the machine on lan1 that is running Squid. Then invoke a command like:

$ suqid-util\
 --force-cache-load\
 --url-root http://www.example.com/x\
 --local-root /mnt/usb1\
 --recursive

This would be useful when a 2TB hard drive can be hand carried to a location with slow Internet connection.

OR

A better solution using something other than Squid? I am still hunting.

  • I have no experience with this but it seems others have used wget to spider the content via the proxy, thus populating the cache.

    I can't think of any reason why you wouldn't be able to copy the cache_dir to your hard drive.

    Out of curiosity, what exactly are you trying to do?

    Using wget to populate SQUID cache

    C.W.Holeman II : Mirror subsets of an increasing collection of static audio-video content to remote locations with slow Internet connections where the opportunistic delivery of microSD cards on the backs of pigeons can used to improve through-put. Well, actually people with hard drives may substitute for the pigeons under management control.
    Warner : It seems like there should be a better solution than SQUID for something like this. I hear there's heavy latency with pigeons but with modern storage densities the throughput isn't too bad.
    C.W.Holeman II : wget link should be placed in the answers to: http://serverfault.com/questions/118518/configure-squid-to-permanently-cache-specific-directories
    Warner : I appreciate you accepting. Did this work for you?
    From Warner

0 comments:

Post a Comment