Incremental backup of site

I want to make periodical archive of my site. I have lftp script to download site content via ftp to today (date +%Y%m%d) directory. What is best way to make an incremental compressed backup without many duplicates?

From serverfault Vlad

• did you try rsync?

  Vlad : i can not use rsync - access only via ftp

  Vlad : and i want make compressed incremental backup
  From quamis

• Duplicity may fit your needs.

  It is incremental: After a full backup is performed, all future backups are simply difference files. It's important to note that it is the opposite of other backup solutions which store a mirror of the latest state, and difference files to recreate the previous backup points.

  It is compressed: Duplicity is an encrypted backup (perhaps good for you, since you're stuck with FTP?) - and the encrypted file is compressed (as I understand it). You can also bypass the encryption, and simply get a gzipped backup. (--no-encryption)

  It works over FTP: Duplicity can use many remote protocols (including FTP), the problem in you case is that duplicity would need to be run from your server. I do not believe you can use duplicity to backup a remote source to a local destination (just local source to remote destination).

  In your case, if you're not looking for the compression benefit in transferring the data, only storing the data, then you could keep your FTP script, and after the current 'image' is transfered have duplicity backup that temporary image to your existing backup, the delete the image. This way you would have a series of backup files that could be used to restore you site at any backup point, and those files would be gziped archives of only the changes from the last backup point.

  Just a note, every so often it would be wise to do a 'full' backup, since duplicity relies on each incremental backup going forward from a full backup.

  Another solution (assuming again that temporarily storing a FTP'd copy locally is acceptable), would be to simply use rdiff-backup. This would give you a mirror of you site (as of the last backup), and past backups would be stored as the differences going backwards. I'm not sure if those are compressed, but even if they aren't, you would be only storing the changes to files for each backup point.

  Vlad : so i can not use it from local linux?

  Tim Lytle : Not sure what you mean there. I don't believe you can run duplicity locally and backup a remote path (it would make the encryption somewhat meaningless). But you can run duplicity locally and backup to a local path (essentially making gzipped archives of the changes since the last full backup).
  From Tim Lytle

• backup2l is a very simple tool that builds an incremental zip file, which you can then download via FTP.

  From Randolph Potter

MySQL databases and Windows Server Backup (Windows 2008)

Hello,

I have strange problem. I've turned on incremental backup in Windows Server Backup so if I'm not wrong every backup allows to retrieve files from certain point in time. Does anyone can tell me why when I'm recovering MySQL database data folder I'm getting latest backup instead of backup from point of time which I selected to recover?

Thanks

From serverfault GrZeCh

• I would schedule the MySQL backups separately to the Windows Server Backup, for a number of reasons, not least of all the consistency checks and avoiding corruption if there are I/O errors and the data files are corrupt.

  That way, your Windows incremental backups can take the backup files (as opposed to the data files) and you will be covered.

  From Randolph Potter

bond0 and xen = crash

Bonding with xen 1 - Stop all guests. Reboot dom0 after running "chkconfig xend off" and "chkconfig xendomains off". 2 - Configure bond0 by enslaving eth0 and eth1 to it. I added the below two entries to /etc/modprobe.conf.

alias bond0 bonding options bond0 mode=6,miimon=100

Content of /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none

Content of /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none

Content of /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0 IPADDR= NETMASK= ONBOOT=yes BOOTPROTO=static USERCTL=no

Did "modprobe bond0" and "service network restart" after that.

3 - Edit /etc/xen/xend-config.sxp

Change

(network-script network-bridge)

To

(network-script 'network-bridge netdev=bond0')

4 - Start xend. "service xend start".

5 - chkconfig xend on.

6 - modprode bond0

7 - more /proc/net/bonding/bond0

8 - Create guest images as usual and bridge it to xenbr0.

about config i did for my xen kernel rhel 5.3 after i reboot the host server i get in place bond0 get pbond0 and its get disconnect from network only i ping to my vm's on the host server any one have any idea why xen bond0 is acting like that or what is solutions to come out of pbond0 to bond0.

From serverfault Rajat

• I think I had the same problem but it is hard to know without seeing the stack trace from the kernel oops. I think mine was related to a driver issue. Bonding with xen crashed the machine. I found a bug report from RedHat on this. Upgrading to kernel-2.6.18-160.el5.x86_64.rpm or newer fixed it. You can get -160 from:

  http://people.redhat.com/dzickus/el5/160.el5/x86%5F64/

  Try this out and see if it fixes your problem.

  Rajat : thanks i m using kernel-2.6.18-164.el5.x86_64.rpm i got fix that.
  From Tracy Reed

Connecting multiple switches to a router

Hi there,

We have an office network that consists of the following:

1x Vigor 2950 5-port (WAN Load Balancer) 2x Netgear 24-port Managed Switch FSM726 1x PowerConnect 2724 24-port

• The Vigor has our two ADSL lines hanging off it.
• Our patch panel connects into the 2 Netgear (for all desktops, laptops, etc...)
• All servers plug in to the PowerConnect

Currently the configuration is:

• Netgear 1 connected to Netgear 2 using GB ports
• PowerConnect connected to Netgear 2 using GB ports
• Netgear 2 connected to Vigor 2950 using GB ports

Basically, the question I have is this the correct way we should be doing it? We had an instance last week when a user was copying 10GBs of large files from a server on the PowerConnect to his machine on Netgear 1 and it basically killed the network for everyone else except him.

Should I infact be connecting each Netgear and the PowerConnect into the Vigor instead?

I'm not sure what the rules are for connecting multiple switches together and I don't seem to be able to find anything good on Google.

Thanks.

Niklas

From serverfault $pageUsers[$entry.posts[0].author].name

• If you've got more than one connection between the same two switches, either:

  a) Make sure that you're using spanning tree, or
  b) Don't do it.

  Remember, the LAN ports on the Vigor box count as a switch as well. I'm not sure what sort of bandwidth management options the NetGear switches give you, but you may also want to look into that. As far as the actual setup goes, it looks pretty solid.

  sybreon : +1 for STP. Recently had an organisation wide black-out when someone plugged a cable into the wrong port.

  RainyRat : I've done something similar myself. Once the lack of STP bites you, you tend to stay bit...

  kmarsh : STP doesn't work that great on my PowerConnects, or between different brands. See my answer about daisy chaining.
  From RainyRat

• The proper way of doing it is:

  1. Don't use Netgear and Dell PowerConnect switch for mission critical network operations (Been there, done that, moved on.) Their advanced features just don't work that well, especially when using more then one feature at once.

  2. Don't use a conglomeration of cheap switches for your network backbone. Invest in at least one large managed Layer 3 switch, with real phone tech support and 4 hour replacement. They exist and cost more for a very good reason.

  3. Don't use cheap switches for port aggregation to combine 10/100 and Gigabit Ethernet clients. They will drag down the performance of all Gigabit connections the moment the first 10 or 100Mbit is connected.

  4. Now that you have real equipment that doesn't choke, use EtherChannel (siamesed ports) or Stacking to connect backbone switches together. This will allow more than one user full Gigabit throughput internally.

  5. As RainyRat said, implement Spanning tree on EVERYTHING, even if it slows down recognition of new devices (30 sec instead of 3 sec).

  As you have already discovered, cheap SOHO switches simply don't have the internal backbone to handle serious network traffic. Daisy-chaining them multiplies their limitations.

  EDIT: If you can't afford that, you can try: EtherChannel the two NetGear switches together with a 2xGigabit link, and turn STP on. You can use 10/100 ports to limit your power user's throughput.

  The PowerConnect is a managed switch, but I have found difficulties in utilizing more than one managed feature at a time. You can try to STP on the PowerConnect and EtherChannel to the NetGears, but I'm not optimistic about throughput. When I tried fixed port speed plus VLANs on my PowerConnects, they bricked and had to be hard reset.

  kmarsh : Upper management couldn't understand or believe the price of good managed switches + tech support. Explaining 4 hour replacement helped, but they still didn't get it. Explaining "I can call tech support, put in a research request how to connect X # of switches with Y # of 4-port EtherChannels, STP+ loop-back protection and lock out physical cross-connects of VLANs, and get the correct answer back in a couple of days", that got through.

  Neil Middleton : I don't think this answer is actually helpful - whilst we would all like to be able to replace kit, most people can't to solve a simple problem

  kmarsh : I never said "top of the range kit". HP ProCurve costs 1/10 of Cisco equipment and gives you what you need. See edit above for other ideas.

  kmarsh : Yes, assuming the Vigor+ADSL can handle this. Or, if you wish to bandwidth-limit the PowerConnect users, you can put the PowerConnect downstream of a NetGear. This forces the switches to limit throughput to 10/100 before it puts pressure on the Vigor. In other words, if you have limited throughput, giving power users the best and fastest pathways can be counterproductive. By forcing their traffic down to 10/100 speeds you can keep them from dominating your limited resources.
  From kmarsh

crontab: login name too long

When I try to edit do crontab for a user with a long username on solaris 10 I get this error:

crontab: login name too long

Is this a known problem and is there a solution for it (without changing the username)?

The username is 27 characters long.

From serverfault $pageUsers[$entry.posts[0].author].name

• Hi. I never saw that before.

  But, while we don't discover how to fix this issue, try to edit the crontab file by vi command.

  vi /var/spool/cron/crontabs/"username"
  

  It will help you meanwhile.

  From Pinho

• After some quick googling it appears that many of Sun's unix tools follow the Unix convention of 8 or less character usernames. It seems that Solaris will allow you to use longer usernames, but it is not a certified or supported configuration.

  innaM : True enough. But not really helpful, is it?
  From Josh Budde

How to report less memory to 32 bit program so it will work on 64 bit Vista?

I have a legacy setup program that will not install on a 64 bit version of Vista with 4GB of RAM. The setup program performs a check at the beginning of the installation to see if there is enough memory. It determines there is "less than 256K of RAM." I assume this is because of a signed 32bit number being used in there math.

I imagine I could take some memory out of the computer and try it. I will as a last resort. But, I was hoping there may be some setting or command line option to get Vista to report less than 4GB to the setup.exe process.

Does anyone know of a way to do this?

From serverfault Matt Spradley

• Use BCDEdit to set truncatememory option. That will limit your memory.

  To use it first check what BCD entries you have with

  BCDEDIT /v
  

  Remember id of wanted entry and then use

  BCDEDIT /set "{id}" truncatememory 1073741824
  

  This will limit it to 1 GB.

  TomTom : Dont forget to undo that after install. This msut be the most ridiculous incompatibility eve.... I hope the software runs at all.
  From Josip Medved

• Another alternative is to run Windows in a virtual machine like Virtualbox. Then you can sandbox the application and run it with as much or as little memory as you'd like, as well as run with an older version of Windows if you have licensing available to do so (if it's a compatibility issue).

  From Bart Silverstrim

• One of the available compatibility shims in Windows is "GlobalMemoryStatus2GB". This might be enough. Look in the Application Compatibility Toolkit.

  From Roger Lipscombe

How to pipe stderr without piping stdout

How do I pipe the standard error stream without piping the standard out stream?

I know this command works, but it also writes the standard out.

Command 2>&1 | tee -a $LOG

How do I get just the standard error?

Note: What I want out of this is to just write the stderr stream to a log and write both stderr and stdout to the console.

From serverfault C. Ross

• To do that, use one extra file descriptor to switch stderr and stdout:

  find /var/log 3>&1 1>&2 2>&3 | tee foo.file
  

  Basically, it works, or at least I think it works, as follows:
  The re-directions are evaluated left-to-right.

  3>&1 Makes a new file descriptor, 3 a duplicate (copy) of fd 1 (stdout).

  1>&2 Make stdout (1) a duplicate of fd 2 (stderr)

  2>&3 Make fd 2, a duplicate (copy) of 3, which was previously made a copy of stdout.

  So now stderr and stdout are switched.

  | tee foo.file tee duplicates file descriptor 1 which was made into stderr.

  Kyle Brandt : Oh, not tested with ksh, works with bash though ...

  C. Ross : Thanks, works in ksh too. I think most of the pipe and stream things are posix standard.
  From Kyle Brandt

• according to the man page for ksh (pdksh), you can just do:

  Command 2>&1 >/dev/null | cat -n

  i.e. dup stderr to stdout, redirect stdout to /dev/null, then pipe into 'cat -n'

  works on pdksh on my system:

  $ errorecho(){ echo "$@" >&2;}
  
  $ errorecho foo
  foo
  
  $ errorecho foo >/dev/null   # should still display even with stdout redirected
  foo
  
  $ errorecho foo 2>&1 >/dev/null | cat -n
       1  foo
  $   
  

  From Craig Sanders