Archive for category Unix

Simple script to add a new Virtual Host on Mac OSX

Posted by Andrea De Pirro in bash, Unix on June 23, 2011

Ideally when you develop a lot of websites on a local machine on OSX you have to create a virtual host for each website.
Below we will describe a really simple script to do this boring operation automatically.

Firstly a simple control to remember you have to choose a host name:

#!/bin/sh
if [ $# -lt 1 ]
then
echo
echo "You have to choose a host name!"
exit 0
fi

Then append a new host to the hosts file and verify it’s all right with a cat:

echo "\n" >> /etc/hosts
echo "127.0.0.1 $1" >> /etc/hosts
cat /etc/hosts

Now you can append the new Virtual Host to the apache configuration file httpd-vhosts.conf:

echo "\n" >> /etc/apache2/extra/httpd-vhosts.conf
echo "<VirtualHost *:80>" >> /etc/apache2/extra/httpd-vhosts.conf
echo " ServerAdmin yourmail@email.com" >> /etc/apache2/extra/httpd-vhosts.conf
echo " DocumentRoot /Library/WebServer/Documents/$1/" >> /etc/apache2/extra/httpd-vhosts.conf
echo " ServerName $1" >> /etc/apache2/extra/httpd-vhosts.conf
echo " ErrorLog \"/private/var/log/apache2/$1-error_log\"" >> /etc/apache2/extra/httpd-vhosts.conf
echo " CustomLog \"/private/var/log/apache2/$1-access_log\" common" >> /etc/apache2/extra/httpd-vhosts.conf
echo " <Directory \"/Library/WebServer/Documents/$1/\">" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Options All" >> /etc/apache2/extra/httpd-vhosts.conf
echo " AllowOverride All" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Order allow,deny" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Allow from all" >> /etc/apache2/extra/httpd-vhosts.conf
echo " </Directory>" >> /etc/apache2/extra/httpd-vhosts.conf
echo "</VirtualHost>" >> /etc/apache2/extra/httpd-vhosts.conf
cat /etc/apache2/extra/httpd-vhosts.conf

Finally restart apache:

apachectl restart

This is the final script:

#!/bin/sh
if [ $# -lt 1 ]
then
echo
echo "You have to choose a host name!"
exit 0
fi
echo "\n" >> /etc/hosts
echo "127.0.0.1 " >> /etc/hosts
cat /etc/hosts
echo "\n" >> /etc/apache2/extra/httpd-vhosts.conf
echo "<VirtualHost *:80>" >> /etc/apache2/extra/httpd-vhosts.conf
echo " ServerAdmin yourmail@email.com" >> /etc/apache2/extra/httpd-vhosts.conf
echo " DocumentRoot /Library/WebServer/Documents//" >> /etc/apache2/extra/httpd-vhosts.conf
echo " ServerName " >> /etc/apache2/extra/httpd-vhosts.conf
echo " ErrorLog \"/private/var/log/apache2/-error_log\"" >> /etc/apache2/extra/httpd-vhosts.conf
echo " CustomLog \"/private/var/log/apache2/-access_log\" common" >> /etc/apache2/extra/httpd-vhosts.conf
echo " <Directory \"/Library/WebServer/Documents//\">" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Options All" >> /etc/apache2/extra/httpd-vhosts.conf
echo " AllowOverride All" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Order allow,deny" >> /etc/apache2/extra/httpd-vhosts.conf
echo " Allow from all" >> /etc/apache2/extra/httpd-vhosts.conf
echo " </Directory>" >> /etc/apache2/extra/httpd-vhosts.conf
echo "</VirtualHost>" >> /etc/apache2/extra/httpd-vhosts.conf
cat /etc/apache2/extra/httpd-vhosts.conf
apachectl restart

download it here.
Once unzipped remember to edit the email on the script, and to make it executable with:

chmod +x newHost.sh

To execute the command you have to sudo (apache and the 2 files are root property):

sudo ./newHost chosenHostName

, , , , ,

No Comments

How to execute remote ssh commands without interactive password

Posted by Andrea De Pirro in bash, ssh, Unix on February 4, 2011

When you manage a lot of remote machines sometimes you need to write scripts to automate commands to be executed or open multiple shells to look at logs.
For security reasons SSH doesn’t give you a -p option to set a password to launch a session, but this can be circumvented with a command line tool called sshpass (you can find it in several repositories, Google is your friend). Mind you! SSH doesn’t give you this ability because is really difficult to store password in a secure way! So don’t blame me if your system will be compromised :D
Basically sshpass is a wrapper that add the -p option, so if you want to launch an ssh session just type this:
sshpass -p password ssh user@host

Sometimes you want to execute commands on the remote machine (like open a given folder) and keep the connection open. To do this just add the -t option and the command followed by the command bash separated by a semi-colon:
sshpass -p password ssh user@host -t "cd /some/folder/you/want/to/open;bash"

Another cool trick if you are under gnome is to open a separated terminal window:
gnome-terminal -e "sshpass -p password ssh user@host -t \"cd /some/folder/you/want/to/open;bash\""

You can even open multiple tabs in the same window with a title for each one:
gnome-terminal --tab -e "sshpass -p password ssh user@host1 -t \"cd /some/folder/you/want/to/open;bash\"" --title "Server1" --tab -e "sshpass -p password ssh user@host2 -t \"cd /some/folder/you/want/to/open;bash\"" --title "Server2"

For convenience you can add the –maximize option at the end to open the terminal full screen.
Just keep in mind that this method is quite unsecure, so use it at your own risk.

, , , ,

No Comments

Awstats installation on Red Hat 5.3

Posted by Enrico Aillaud in Unix on September 7, 2009

System requirements:

  • A Web Server which must have access to log file had to analyse
  • PERL

Steps for installing awstats

  • Go in the /home/awstat folder and uncompress the tar.gz file with the following command:
    #tar xvf awstats-6.9.tar.gz
  • After the decompression move the folder awstats-version in /var/www/html folder with the command:
    #mv awstats-version /var/www/html/awstats
  • go in the var/www/html/awstats/tools folder and type the following command:
    #perl awstats_configure.pl
  • Setup steps (the installation will ask for the following directives):
    • Do you want to continue setup from this standard directory [yN]?
      Type y and press enter to install in the /var/www/html folder
    • Enter full config file path of your web server.
      Type none and press enter
    • Do you want me to build a new AWStats config/profile file [y/N]
      Type y and press enter
    • What is the name of your web site or profile analysis?
      Type the name of the website you want to analyze (ServerName) and press enter
    • In which directory do you plan to store your config file(s)?
      Press enter to use the default one (etc/awstats)
    • Press enter until you return to shell prompt
  • Create a folder in which store the log file you want to analyze:
    #mkdir /etc/awstats/log_servername
  • Copy the log file from the server in which are stored the apache access_log:
    #scp root@ip_remote_server:/folderInWhichIsStoredAccessLog /etc/awstats/log
    (i.e. #scp root@ip-remote-server:/var/log/apache2/access_log /etc/awstats/log)
  • The system will ask for a password; insert the root password of the remote host.
  • Edit the config file in /etc/awstats (in this case we create the mysite.conf file) with vi:
    vi mysite.conf
  • Go in the LogFile option and edit it to setup the right logfile that awstats has to analyze:
    LogFile = “/etc/awstats/access_log”
  • Create the folder /var/lib/awstats
    #mkdir /var/lib/awstats
  • Set the permissions in the folders created for apache:
    #chown -R apache.apache /var/lib/awstats
    #chown -R apache.apache /etc/awstats
    #chown -R apache.apache /var/www/html/awstats
    (If SELinux activated execute the following 2 commands)
    #chcon -R -h -t httpd_sys_content_t /var/www/html/awstats/
    #chcon -R -h -t httpd_sys_content_t /var/lib/awstats/
  • go in /var/www/html/awstats/wwwroot/cgi-bin/ and type
    #perl awstats.pl -config=mysite-update
  • restart apache:
    #service httpd restart
  • Access from the browser the awstats statistics typing in the address bar:
    localhost/awstats/awstats.pl?config=mysite

Notes for old log file:

A fresh awstats installation will not provide full statistics history but there is a way thank to which users could set up awstats to allow a full history statistics (previous log files must be available)

How load previous log file

(Usually previous log files are available in a compress format; in this case for the remote-host we suppose to have a bz2 file extension for old files)

  • Copy the access_log.bz2 from the remote host to the local host running the command:
    #scp root@remote-server:/var/log/apache2/access_log-*.bz2 /etc/awstats/log
  • Go in the following folder:
    #cd /var/www/html/awstats/tools
  • Type:
    #perl logresolvemerge.pl /folder/access_log-*.* > /folder/access_log
  • Run the command to update the log file awstats use to make the statistics:
    cd /var/www/html/awstats/wwwroot/cgi-bin/
    #perl awstats.pl -config=mysite -update

Note for access_log automatic update

In order to have an automatic update of the log file which resides in the remote host we set up a cron job which will update our local log file which resides in the /etc/awstats/log folder.

To accomplish this work we must avoid the password request in ssh setting up the private-public keys security.

Setup SSH without password

  • In the localhost run ssh-keygen
  • Write the folder in which save the key: /root/.ssh/id_rsa
  • Rename id_rsa.pub:
    #mv /root/.ssh/id_rsa.pub /root/.ssh/redhat.pub
  • Run:
    scp /root/.ssh/id_rsa root@remote-server:/root/.ssh/
  • Access remote server in ssh
  • Go in the .ssh folder
    #cd /root/.ssh
  • Type
    #cat redhat.pub >> authorized_keys
  • Run the command:
    #service sshd restart

After this setup the connection with the host will be without password

Updating access_log with  crontab

#crontab -e

10 9 * * * scp root@remote-host:/var/log/apache2/access_log /etc/awstats/log

11 9 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -config=mysite –update

, , ,

No Comments

How to convert Squid log file to Apache format in order to use Awstats

Posted by Enrico Aillaud in Unix on September 3, 2009

The following guide is the result of a problem I met while trying to analyze old Squid files log with Awstats.

The problem was that the Squid option to write the log format in HTTPD common log file format was disabled and so it was impossible analyzed all previous Squid log files.

So I post this guide to explain how allow Awstats to analyze Squid log format formatted in native mode.

First of all there is a useful free software available at: http://pwebstats.gleeson.net/

The author is Martin Gleeson and he wrote the perl script we need to convert the squid log file.

Save the zipped application file, uncompress it and save the squid2common.pl file in /unix-folder/.

To convert the file simply run the perl script:

perl /unix-folder/squid2common.pl squid-access-log-file

The script will generate two different log files (cache.convert and proxy.convert), the proxy.convert one contains the information Awstats needs to make the reports.

Personally I run the scrip for all old Squid log file and I create a unique access_log file (set up in the awstats.site.com)  needed by Awstats to make the statistics with the command:

less cache.convert >> access_log

Finally we must set up the correct LogFormat option on awstats.site.conf in order to allow Awstats to read the squid log file converted:

LogFormat=4

In this way, running the Awstats perl command to update the statistics, we will be able to have in our reports the old squid log files not directly processable by Awstats.

Enrico Aillaud

, ,

1 Comment