alexalmazan.com

Rsync can be used effectively to transition services amongst server host(s) who share common distribution roots in regards to Linux distrubution in use. ie. ‘Fedora FCx Vs. Fedora Latest’

Please note that if certain exclusions are not granted, you can easily destroy libraries and or other precious data serving your customer base. Please ensure you are full aware of the command syntax, especially when running rsync as root!

If you are after a single directory, on the server you are migrating from, you can transport it with the following syntax

/usr/bin/rsync -ave ssh user@ip.address:/home/user /home/user

If the servers you are moving data amongst are similar or greater in disk geometry/capacity and identical operating system, you can clone or replicate the server you are migrating from to your new platform with the following:

/usr/bin/rsync -az -e ssh / root@ip.address:/

The method just described has it s consequences. First and foremost, all IP address information and or user information configured has now been modified to be that of the server you are migrating from. If this occurs within a datacenter environment, both servers involved will stay online and active, however manual console intervention may be required to address the IP address issue upon the next reboot of the server receiving the rsync feed.

It also requires that SSH be permitted to the server you are migrating to. SSH root access is only recommended if you are in full control of the IP sources available to interact with your server via SSH. IF you are unable to filter or have no filters in place .ie firewall then SSH access for root should not be permitted for extended periods if at all. Root access is required to preserve data permissions when migrating data in such scenarios.

Here is another examples which shows how it can be used to mirror MySQL database information amongst similar server hosts.(mysql server version and OS should be identical )

# rsync -avru -e ssh root@ip.address:/var/lib/mysql/DB1 /var/lib/mysql 
 

The above example demonostrates the movement of one database, here is a method for the entire datadir. (examples recommended with both database servers off)

# rsync -avru -e ssh root@ip.address:/var/lib/mysql/ /var/lib/mysql

Placement of ’/’ makes a huge difference with the outcome when using rsync. WIth it, you are copying the contents of a filesystem or directory, without it, you are copying entire file structures or directories.

To overcome limitations such as overwriting libraries and or devices files, or everything in ’/etc’ when using rysnc, I recommend the following syntax when migrating from similar server OS platforms.

# rsync --rsh="ssh " -avz --exclude="/etc" --exclude="/proc" --exclude="/dev" --exclude="/lost+found" --exclude="/var/log" / root@ip.address:/
 

Usage of ‘—exclude’ will vary from migration to migration.

Migrating data can be a quick and easy process. The following example will provide a method for archiving and transferring data directories simultaneously. The biggest difference is the compression used in these examples.

tar zcvf - /home | ssh user@ip.address "cat > homearchive.tgz" 

Here is another:

tar cjvf - /home | ssh user@ip.address "cat > homearchive.tar.bz2" 

These examples demonstrate that you can begin a tar archive and pipe it into an SSH session for transport. It works great if the users in question for which the data you are moving exist. If this is not the case, then file permission preservation is probably going to be an issue with the above examples.

Rsync is certainly better suited for this quick hack.

A quick guide to transferring Urchin data from one host to the next. First, bundle the urchin directory on the server you are going to transfer/migrate from.

 tar -cvjf urchin.tar.bz2 /usr/local/urchin

From the new server, you should then connect to your migration platform and conduct the following

sftp <old server>
cd /usr/local
get urchin.tar.bz2
cd /etc/init.d
get urchin
exit

Now, on the server in which you are getting this migrated to, perform the following actions as root

mv urchin /etc/init.d
tar -xvjf urchin.tar.bz2
mv urchin /usr/local
chkconfig --list urchin
/etc/init.d/urchin start
http://<ip>:10000

For the former installation, you will have to determine the serial key so that it can be reset.

• Copy serial key

• access secure.urchin.com/var/login (login with credentials from internal Rack sources)

• Click on “serial codes” link
• Put serial key from Urchin and paste it and search for it
• Click on “reset”
• Go back to Urchin web interface & click re-activate license

When you see the profiles list, you are almost there . Might want to make sure the whole /var/www/vhosts thing vs /home/httpd/vhosts won’t fsck things up.

Greylisting is an alternative to the psa-spamassassin method for dealing with spam/UCE. These instructions work with Plesk 7.54 and 8.x, however there is no guarantee that this setup will keep active post updates to Plesk.

Obtain the sources

wget http://carbonblock.net/files/qmail-...greylist.tar.gz

The source contains patches by SW Soft, please do not distribute.

tar xvfz qmail-1.03-psa-greylist.tar.gz
cd qmail-1.03

Edit local_scan.c and change values as appropriate.

#define MYSQLHOST "localhost" 
#define MYSQLUSER "greylist" 
#define MYSQLPASS "rdePmee7" 
#define MYSQLDB "qmail" 
#define BLOCK_EXPIRE 2 /* minutes until email is accepted */
#define RECORD_EXPIRE 1480 /* minutes until record expires */
#define RECORD_EXPIRE_GOOD 36 /* days until record expires after accepting email */

Setup MySQL

mysql -u admin -p`cat /etc/psa/.psa.shadow`

mysql>CREATE DATABASE qmail;
mysql>GRANT ALL ON qmail.* TO 'greylist'@'localhost' IDENTIFIED BY 'rdePmee7';
mysql>FLUSH PRIVILEGES;

Table structure for table `relaytofrom`

CREATE TABLE relaytofrom (
id bigint(20) NOT NULL auto_increment,
relay_ip varchar(16) default NULL,
mail_from varchar(255) default NULL,
rcpt_to varchar(255) default NULL,
block_expires datetime NOT NULL default '0000-00-00 00:00:00',
record_expires datetime NOT NULL default '0000-00-00 00:00:00',
blocked_count bigint(20) NOT NULL default '0',
passed_count bigint(20) NOT NULL default '0',
aborted_count bigint(20) NOT NULL default '0',
origin_type enum('MANUAL','AUTO') NOT NULL default 'MANUAL',
create_time datetime NOT NULL default '0000-00-00 00:00:00',
last_update timestamp(14) NOT NULL,
PRIMARY KEY (id),
KEY relay_ip (relay_ip),
KEY mail_from (mail_from(20)),
KEY rcpt_to (rcpt_to(20))
) TYPE=MyISAM;

Edit conf-ld and change it to:

cc -s -lssl

Next

make
/etc/init.d/qmail stop
/etc/init.d/xinetd stop
mkdir /home/user/ticket-#/BACKUP
cp /var/qmail/bin/qmail-smtpd /home/user/ticket-#/BACKUP

(this backups the base qmail binary, only do this on first install)

cp qmail-envelope-scanner /var/qmail/bin/.
cp -f qmail-smtpd /var/qmail/bin/.
chown root.qmail /var/qmail/bin/qmail-envelope-scanner
chown root.qmail /var/qmail/bin/qmail-smtpd

Next

/etc/init.d/qmail start
/etc/init.d/xinetd start

Create a quick perl script to clean up the database and place in /etc/cron.daily named greylist.clean:

#!/usr/bin/perl
use strict;
use warnings;

use constant DBD => 'DBI:mysql:qmail:localhost:3306';
use constant DBUSER => 'greylist';
use constant DBPASS => 'rdePmee7';

use DBI;

system ("cat /dev/null > /tmp/greylist_dbg.txt");

my $dbh = DBI->connect(DBD,DBUSER,DBPASS) or die "can't connect to db ", $DBI::errstr, ":$!";

$dbh->do("DELETE FROM relaytofrom WHERE record_expires < NOW() - INTERVAL 1 HOUR AND origin_type = 'AUTO'");
$dbh->do("OPTIMIZE TABLE relaytofrom");

$dbh->disconnect;

exit;

Here is a way for determining domains which recieve the most SMTP traffic:

# cat /usr/local/psa/var/log/maillog | grep -o 'to=.*@.*' | sed 's/to=.*@//' | sort | uniq -c |sort -rn | head