intro

home - top of the page -

A shell is a command interpreter. You type commands on the prompt. Shell interprets them and passes further for the execution. Over the years unix developers created many shells. Here are manpages for 5 most important ones:
• www.engin.umich.edu/htbin/mangate?manpage=sh - sh (bourne shell) - first, simpliest.
• www.engin.umich.edu/htbin/mangate?manpage=csh - csh - from Berkley. Use tcsh instead of it.
• www.engin.umich.edu/htbin/mangate?manpage=ksh - ksh (korn shell) - common, many programming features
• www.engin.umich.edu/htbin/mangate?manpage=tcsh - tcsh - very good convenient shell (enchanced csh)
• www.engin.umich.edu/htbin/mangate?manpage=bash - bash (born again shell) - main shell in the Linux world

• www.whereisit.com/shellscript/index2.html - shellscript.com - some good links

If you have any experience at all with Unix - you worked with one of those. The first one (Bourne shell) is very common - you can use it to write scripts. The C-shell (csh) is common, but should be avoided. You better use tcsh for command line editing, and bourne shell for installation scripts (read Tom Christiansen's article why csh (and tcsh) should not be used for writing shell scripts).

The last 3 scripts (ksh,tcsh, bash) are much more convenient to work with and you can use one of them as your command-line shell.

To find which shell you are running login to the prompt and use ps command with no options.
Often $SHELL variable will show the current shell too.
Also in the /etc/passwd file you can see what is your default shell.
Here is what you can see on the prompt (your login name is "mylogin"):

> ps   PID  TT  STAT      TIME COMMAND  6111  p0  Ss     0:00.06 -bash (bash)  6124  p0  R+     0:00.01 ps > echo $SHELL /bin/bash > which bash /usr/local/bin/bash > which tcsh /bin/tcsh > which ksh /bin/ksh > which sh /bin/sh > which csh /bin/csh > which zsh /bin/zsh > whoami mylogin > cat /etc/passwd | grep mylogin mylogin:*:1487:102:,,,,:/home/mylogin:/bin/bash >

• www.opennc.org/onlinepubs/7908799/xcu/sh.html - using sh (includes vi-mode command line editing).
• http://codedata.box.sk/ghostrider/bash.tutor - bash tutorial
• http://uracil.cmc.uab.edu/MolModLab/tutor/irix/basecmd.shtml - intro into unix, includes shell scripting

2 ways to run shel scripts:
  1.start a new shell (from command line or from a "shabang" line (first line of your script)).
  2.source the file (using source or '.' (dot command)) - this way you can change the enviromental variables in the currently running shell.

To force shell to echo commands:
  use -v option (works with ksh, csh, tcsh & bash), for example:
    tcsh -v myscript.sh

Using shell scripts:
I mostly use Perl instead of shell scripting. But you still need to know at least a little bit of shell scripting for:
  - writing very short scripts or installations scripts (see examples below).
  - setting environment (in .profile or .cshrc file)
  - writing thin wrappers (see examples below)

Again, let me repeat main recommendations:
1. use tcsh, ksh or bash for your command-line editing.
2. use perl for scripting
3. If you need to write a shell script (for installation, for example) - use bourne shell (sh) and NOT csh.

 

setting environment

home - top of the page -

ksh:   .profile

.profile

# ------------------------------------------ #  .profile for ksh on Solaris # ------------------------------------------ if [ -f /somedir/setEnv.sh ]; then     . /somedir/setEnv.sh fi set -o vi EDITOR=vi; export EDITOR VISUAL=vi; export VISUAL ulimit -c 0     # LIMIT CORE DUMP SIZE rev=`uname -r` case $rev in 5.*)  echo Solaris Profile  PATH=$PATH:/usr/bin:/usr/sbin:/opt/bin:/opt/systools:/opt/sybase/bin:/usr/ucb:/usr/ccs/bin:/usr/java/bin:.  export PATH  LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib:/usr/java/lib:.:/opt/sybase/openclient/11.11/lib:/opt/sybase/lib  export LD_LIBRARY_PATH  CLASSPATH=$CLASSPATH:/opt/jconnect-5.2/classes/jconn2.jar:/usr/java/lib:.:/opt/devshare/htdocs/classes:/somedir/selector.jar  export CLASSPATH  ;; 4.*) echo SunOS Profile  PATH=$PATH:/bin:/usr/bin:/opt/bin:/etc:/usr/ucb:/usr/bin/X11:/usr/lib:/usr/etc:/usr/man:$HOME:/opt/systools:/usr/openwin/bin  export PATH  ;; esac # ------------------------------------------ #         set up ksh # ------------------------------------------ TERM=xterm;  export TERM set -o emacs umask 002 PS1=`/usr/ucb/whoami`:'$PWD:'" " PS2=">" HISTSIZE=1024 HISTFILE=$HOME/.sh_history stty erase ^H # ------------------------------------------ #         set up MAN path # ------------------------------------------ MANPATH=$MANPATH:/usr/local/man:/usr/dt/man:/usr/openwin/man MANPATH=/usr/share/man:$MANPATH:/opt/perl/perl5.003/man export MANPATH ##### Check printer ### if [ -f $HOME/.printer ] ; then  PRINTER=`cat $HOME/.printer` ; export PRINTER fi # ------------------------------------------ if [ -f $HOME/.profile.local ]; then  . $HOME/.profile.local fi # ------------------------------------------ LD_LIBRARY_PATH=/opt/sybase/lib:/usr/lib:/usr/local/lib:/usr/lib/X11:/usr/local/openwin/lib:/wrk/staging/FrameWork/rogue/lib export LD_LIBRARY_PATH export SYBASE=/opt/sybase PATH=$PATH:$HOME/bin:$HOME/util export PATH CVSEDITOR=vi; export CVSEDITOR CVSROOT=/home/proj/cvs_repository; export CVSROOT alias l='ls -alF' alias ii='isql -S PRODSERV -U mylogname -P mypasswd' alias ss='sqsh -S PRODSERV -U mylogname -P mypasswd'  alias cgi="cd ~/proj/www/cgi-bin" alias myupdate="cvs update -d -P"

Settings for your xterm:
/usr/local/bin/dtterm -d 10.3.3.101:0.0 -fg black -bg white &
xterm -d daddypc:0.0 -fg green -bg black &
xemacs -d daddypc:0.0 &
alias term='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-120-75-75-*-*-ISO8859-1 -bg DarkCyan -fg White -geom 90x30 -cr red &'
alias term0='xterm -d daddypc:0.0 -fg black -bg white &'
alias term1='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-120-75-75-*-*-ISO8859-1 -bg black -fg LemonChiffon -geom 90x30 -cr red'
alias term2='xterm -d daddypc:0.0 -fn -*-Fixed-Medium-R-*-*-*-140-75-75-*-*-ISO8859-1 -bg black -fg LemonChiffon -geom 100x40 -cr red'
# export DISPLAY=daddypc:0.0
/usr/openwin/bin/xfontsel -d daddypc:0.0 &
/usr/openwin/bin/xlsfonts
CTRL-(right-mouse-button)

tcsh:   .cshrc  .login   .aliases

.cshrc

# ------------------------------------------ # .cshrc file for tcsh # ------------------------------------------ if ( $?prompt ) then     echo "Running cshrc..." endif if ($?DEBUGCSHRC) set echo umask 2 # figure out what os we're in set os=`/usr/bin/uname -r -s | /usr/bin/sed -e 's/ \([0-9]\).*$/\1/'` switch ($os)  case "SunOS4":   setenv VERSION sunos   breaksw  case "SunOS5":   setenv VERSION sparc   breaksw  default: endsw if ( $?prompt ) then     test -f ~hallro/.aliases && source ~/.aliases     set notify     set filec     set nobeep     set history=100 endif if ($?HAVEPATH == 0) then    if ($?OPENWINHOME == 0) setenv OPENWINHOME /usr/local/openwin    set path=(/home/hallro/usr/bin \               /home/oedev/usr/bin \               /home/slsdev/bin \               /opt/perl-5.004/bin \               /opt/workshop/SUNWspro/bin \               $OPENWINHOME/bin \               /bin \               /usr/bin \               /usr/ucb \               /usr/sbin \               /opt/bin \               /opt/systools \               /opt/sybase/bin \               /usr/ucb \               /usr/ccs/bin \               /usr/java/bin \              /opt/java/jdk1.2.2/bin \               . \    )    if (-d /home/nysid10/sqllib/) then         source /home/nysid10/sqllib/db2cshrc    endif   setenv HOSTNAME `/bin/hostname`   setenv HAVEPATH endif if ($?SYBASE == 0) then     if (-d /opt/sybase) then        if ($?LD_LIBRARY_PATH == 0) then            setenv LD_LIBRARY_PATH ""        endif        setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\/opt/sybase/lib:        setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH\/opt/sybase/openclient/11.11/lib        setenv SYBASE /opt/sybase        setenv PATH $PATH\/opt/systools:/opt/sybase/bin:     else if (-d /usr/local/sybase) then        setenv SYBASE /usr/local/sybase     endif endif if ($?OPENWINHOME) then     if (-d /usr/local/openwin) then         setenv OPENWINHOME /usr/local/openwin         setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH$OPENWINHOME/lib         setenv PATH $PATH\/usr/openwin/bin:     endif endif

.login

# ------------------------------------------ # .login file for tcsh # ------------------------------------------ if ( $?prompt ) then     echo "Running login..." endif if ($?DISPLAY == 0) then     if ($?REMOTEHOST != 0) then         setenv DISPLAY $REMOTEHOST":0.0"     endif endif if ($?MANPATH == 0) then     setenv MANPATH "" endif setenv MANPATH $MANPATH\/usr/local/man:/usr/openwin/man:/usr/man:/opt/man: setenv MANPATH $MANPATH\/home/hallro/usr/man: if ($?PRINTER) then     setenv PRINTER pusg25a endif setenv EDITOR vi setenv CVSROOT /home/mydev/cvs_repository setenv CVSEDITOR vi

.aliases

# ------------------------------------------ # .aliases file for tcsh # ------------------------------------------ if ( $?prompt ) then   echo "Running aliases..."   alias ls      "\ls -F"   alias l       "\ls -alF"   alias cls     "\clear"   alias cl      "\clear; ls"   alias h       history   alias DISPLAY "setenv DISPLAY fcfrnb419.jany.gs.com:0.0"   alias hw      "\hostname; \whoami"   # ---- Find Magic :-)   alias fpcgi 'find . -name "*.cgi" -print -exec grep "\!^" {} \;'   alias fcgi  'find . -name "*.cgi" -exec grep "\!^" {} \;'   alias fppl  'find . -name "*.pl" -print -exec grep "\!^" {} \;'   alias fpl   'find . -name "*.pl" -exec grep "\!^" {} \;'   alias fppm  'find . -name "*.pm" -print -exec grep "\!^" {} \;'   alias fpm   'find . -name "*.pm" -exec grep "\!^" {} \;'   # ---- Web development...   alias GET    "setenv REQUEST_METHOD GET"   alias POST   "setenv REQUEST_METHOD POST"   # ---- Project Specific   alias pf "cd ~someuser/somedir; pwd"   alias DEVSERV "echo 'DEVSERV as mylogname'; \                    isql -S DEVSERV -U mylogname"   alias PRODSERV "echo 'PRODSERV as mylogname'; \                     isql -S PRODSERV -U mylogname"   alias sybdev "echo 'DEVSERV as mylogname'; \                 isql -S DEVSERV -U mylogname"   alias sybprd "echo 'PRODSERV as mylogname'; \                 isql -S PRODSERV -U mylogname"   alias iifiam="isql -U mylogname -P mylogname -S SERV2"   alias cdcom "cd ~/dir/common; pwd"   alias cdpr "cd ~/dir/project/www/cgi-bin; pwd"   alias london "rlogin -l mylogin  serv.domain.com"   # ---- Host specific   if (`hostname` == "host1.dom1.com") then        alias  wl  "tail  -f  somepath/logs/errors"   endif   alias   psgrep  'ps -auxgww | grep $1'   alias   psapi   'ps -auxgww | grep api'   alias   ldir    'ls -al | grep "^d"'   alias   tm     '(set d = `date` ; echo $d[4])'   alias   since   'set t = `who | grep $USER` ; echo $t[5]; unset t'   alias   fixt    'stty erase'    alias   sett    'setenv TERM vt100'   alias   l       ls -alF   alias   em  'emacs -fg white -bg black -fn 6x13' endif # setting nice prompt (see www.engin.umich.edu/caen/faqs/setprompt.html )    if( ${?prompt} ) then       alias setprompt 'set prompt = `whoami`:"%/-> "' #      alias setprompt 'set prompt = `whoami`:"%~-> "'       alias cwdcmd 'setprompt'       setprompt    endif # -------------------- # and some more # -------------------- set addsuffix set ampm set autolist=ambiguous set autoexpand #set autologout=600 unset autologout set cdpath=(~/user ~/user/procmon ~/user/src ~) set complete = enhance set symlinks=chase                        # $cwd has real path (follows links) #set correct=all set correct=cmd set echo_style=both                       # an hpux thing to make -n opt work set fignore = (\- .o CVS)                 # suffixes of files for tab to ignore set filec set history = 500  set hislit                                #tcsh-show history as typed set host = "`hostname`"; set host=$host:r; set host=$host:r #set hostfull = (`nslookup $host | sed -n -e "/Name:/s/\(Name: \)\(.*\)/\2/p" `) set listjobs = long                       #tcsh-list all jobs when suspending #set mail=(10 /usr/mail/lane) set nobeep set nonomatch set nostat = (~/xstuff)                   #tcsh-dirs not to stat for completion set notify set printexitvalue                        #tcsh-exit val for interactive prog set prompt2 set prompt3="CORRECT> %R (y|n|e)?" set rmstar set savehist = 500

bash    .bashrc     .bash_profile

.bashrc

# -------------------- # .bashrc # -------------------- alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # ---- User specific aliases and functions # ---- Source global definitions if [ -f /etc/bashrc ]; then         . /etc/bashrc fi

.bash_profile

# -------------------- # .bash_profile # -------------------- # ---- Get the aliases and functions if [ -f ~/.bashrc ]; then         . ~/.bashrc fi # ---- User specific environment and startup programs umask 022 ORACLE_BASE=/usr/local/oracle/8i/u01/app/oracle ORACLE_HOME=$ORACLE_BASE/product/8.1.6 ORACLE_SID=starter BASH_ENV=$HOME/.bashrc USERNAME="" JAVA_HOME=/usr/local/java JSDK_HOME=/usr/local/jsdk CLASSPATH=.:$JAVA_HOME/lib/dt.jar CLASSPATH=$CLASSPATH:$JSDK_HOME/servlet-2.0.jar CLASSPATH=$CLASSPATH:/usr/local/java/lib/xalan.jar CLASSPATH=$CLASSPATH:/usr/local/java/lib/xerces.jar CLASSPATH=$CLASSPATH:/usr/local/oracle/8i/u01/app/oracle/product/8.1.6/jdbc/lib/classes111.zip CLASSPATH=$CLASSPATH:/usr/local/oracle/8i/u01/app/oracle/product/8.1.6/jdbc/lib/nls_charset11.zip PATH=.:$HOME/util:$PATH:$HOME/bin:$ORACLE_HOME/bin:$JAVA_HOME/bin export USERNAME BASH_ENV PATH JAVA_HOME JSDK_HOME CLASSPATH ORACLE_BASE ORACLE_HOME ORACLE_SID

 

writing thin wrappers

home - top of the page -

Example: a thin wrapper running a perl script (note how syntax is different in 2 shells):

C-shell

Bourne shell

#! /bin/csh set mailTo='me@somedomain.com' set bindir='/home/some_directory' set log1file='/tmp/myfile.log' rm -f $log1file touch $log1file echo "Start time:" >> $log1file date >> $log1file $bindir/some_script.pl >>& $log1file if ( "$status" != "0" ) then   echo "Error at:" >> $log1file   date >> $log1file   /opt/fax/bin/sendmime.pl --to=$mailTo  \   --subject='myjob failed' --msg_file=$log1file   exit(1) endif echo "Successful end time:" >> $log1file date >> $log1file /opt/fax/bin/sendmime.pl --to=$mailTo \ --subject='myjob Successful' --msg_file=$log1file exit(0)

#!/bin/sh mailTo='Lev.Selector@gmail.com' bindir='/home/selecl/tests' log1file="/tmp/myfile.log" rm -f $log1file touch $log1file echo "Start time:" >> $log1file date >> $log1file $bindir/some_script.pl  1>>$log1file 2>>$log1file if [ $? -ne 0 ]; then    echo "Error at:" >> $log1file    date >> $log1file    /opt/fax/bin/sendmime.pl --to=$mailTo \    --subject='myjob failed' --msg_file=$log1file    exit 1 fi echo "Successful end time:" >> $log1file date >> $log1file /opt/fax/bin/sendmime.pl --to=$mailTo \ --subject='myjob Successful' --msg_file=$log1file exit 0

Example: a simple shell script I use to install tables and stored procedures into Sybase database.

#! /bin/csh set  dbServer="-S MY_DB_SERVER -U someuser -P somepasswd" isql $dbServer -ifile1.sp isql $dbServer -ifile2.sp isql $dbServer -ifile3.sp isql $dbServer -ifile4.sp

Two small examples of including text into the script as "here doc" - or reading from a file  (csh syntax):

$SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w300 <<EOF > $resultFile  use $DATABASE  go  exec some_stored_procedure go  EOF    $SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w300 <<!  :r /home/somefile.sql go  !

More sophisticated version - requires one parameter - a name of a text file with SQL to run

#!/usr/bin/sh SERVER="MY_DB_SERVER" DATABASE="mydatabase" DB_USER=someuser DB_PASSWD=somepasswd if [ $# != 1 ] then      echo "Usage: $0 <file>"     exit 1 fi file=$1 if [ ! -f $file ] then     echo "File \"$file\" Does Not Exist"     exit 1 fi PATH="/bin:/usr/bin:/usr/local/bin:/usr/local/sybase/bin:someotherdirectories" SYBASE="/usr/local/sybase" export PATH export SYBASE $SYBASE/bin/isql -U$DB_USER -P$DB_PASSWD -S$SERVER -w250 <<EOF print "Installing $file in Database: $DATABASE Server: $SERVER" go use $DATABASE go `cat $file` EOF exit 0

# db.sh - wrapper to prevent password from showing in ps:

#!/bin/sh MYSQL="sql.txt" MYPASS="somepaswd" isql -S SOMESERVER -U someuser <<EOF $MYPASS `cat $MYSQL` EOF exit(0)

Here "sql.txt" is a file with SQL commands, for example:

use somedatabase go select * from sometable go

 

writing thin wrappers

home - top of the page -

Automating FTP:

#!/bin/sh ftp -n <<EOF open www.LevSelector.com user myname mypass get sleep.pl quit EOF

Note: "-n" option prevents ftp from asking the login and passwd on open command.
Thus you can supply them both using the user command.
Here is another way of doing the same plus some minimal error checking

#!/bin/sh  echo "open webprod       user login passwd        verbose       cd dir1/dir2       bin       get file1.cgi       close       quit" | ftp -n > myftplog  # Check FTP log to see if the file was retrieved okay  if grep "^226" myftplog  then    echo "Successfully got the file from remote server" >> myftplog else    echo "Failed to get the file" >> myftplog  fi

By the way, you can always use Perl:
Example using  Net::FTP

use Net::FTP; $ftp = Net::FTP->new("some.host.name"); $ftp->login("anonymous","me@here.there"); $ftp->cwd("/pub"); $ftp->binary; $ftp->get("that.file"); $ftp->quit;

Example using LWP::Simple

use LWP::Simple; my $status = getstore('ftp://user:password@host/foo/file', 'localfile');

 

Automating telnet

home - top of the page -

Sending telnet from a script - I don't know how to do this from a shell script, because it doesn't have options for that. I think.
Usually people use something like remote shell (rsh)
    http://www.engin.umich.edu/htbin/mangate?manpage=rsh
or kermit
    http://www.columbia.edu/kermit/ckermit.html
    http://www.columbia.edu/kermit/telnetd.html
    http://www.columbia.edu/kermit/ck80.html

I would prefer to use a Perl script using NET::Telnet module from CPAN:
    http://www.perldoc.com/cpan/Net/Telnet.html

Here are examples right from documentation of NET::Telnet  :
 

use Net::Telnet (); $t = new Net::Telnet (Timeout => 10,  Prompt => '/bash\$ $/'); $t->open("sparky"); $t->login($username, $passwd); @lines = $t->cmd("/usr/bin/who");   # give command print @lines;      # collect response    # etc.

 

my ($forecast, $t); use Net::Telnet (); $t = new Net::Telnet; $t->open("rainmaker.wunderground.com"); ## Wait for first prompt and "hit return". $t->waitfor('/continue:.*$/'); $t->print(""); ## Wait for second prompt and respond with city code. $t->waitfor('/city code.*$/'); $t->print("BRD"); ## Read and print the first page of forecast. ($forecast) = $t->waitfor('/[ \t]+press return to continue/i'); print $forecast; exit;

 

# ----------------------------------------------------------- # example checking mail on a pop server # ----------------------------------------------------------- my ($hostname, $line, $passwd, $pop, $username); $hostname = "your_destination_host_here"; $username = "your_username_here"; $passwd = "your_password_here"; use Net::Telnet (); $pop = new Net::Telnet (Telnetmode => 0); $pop->open(Host => $hostname, Port => 110); ## Read connection message. $line = $pop->getline; die $line unless $line =~ /^\+OK/; ## Send user name. $pop->print("user $username"); $line = $pop->getline; die $line unless $line =~ /^\+OK/; ## Send password. $pop->print("pass $passwd"); $line = $pop->getline; die $line unless $line =~ /^\+OK/; ## Request status of messages. $pop->print("list"); $line = $pop->getline; print $line; exit;

- www.gnu.org/software/bash/manual/bashref.html - Bash Reference

An event designator is a reference to a command line entry in the history list.

!	Start a history substitution, except when followed by a space, tab, the end of the line, `=' or `('
!n	Refer to command line n
!-n	Refer to the command n lines back
!!	Refer to the previous command. This is a synonym for `!-1'
!string	Refer to the most recent command starting with string
!?string[?]	Refer to the most recent command containing string. The trailing `?' may be omitted if the string is followed immediately by a newline
^string1^string2^	Quick Substitution. Repeat the last command, replacing string1 with string2. Equivalent to !!:s/string1/string2/
!#	The entire command line typed so far