Red Hat EX200 Objectives and solutions

I. Understand and Use Essential Tools

• Access a shell prompt and issue commands with correct syntax

         Applications --> System Tools ---> Terminal

         or

         press function keys to access pseudo consoles to type commands

         shell prompt (command line) refers to the Linux command prompt whre we enter commands for execution.

          $ cal 2011

• Use input-output redirection (>, >>, |, 2>, etc.)

       The BASH shell allows you to redirect input, output and error messages to allow programs and commands to read input from something other than the keyboard and send output and errors to something other than the terminal window.

File Descriptor	Symbol	Associated Digit	Descriptor
stdin	<	0	Standard input
stdout	>	1	Standard output
stderr	>	2	Standard error

         Table: I/O/E Redirection Symbols

   

      Redirecting Standard Input

    

      $ mailx user2 < file1    ( Get input to mailx command from file1 file)

      Redirecting Standard Output

      $ sort file1 > sort.out   ( Redirect the output to sort.out file)     

      $ sort file1 >> sort.out  ( appent to output to sort.out file) 

     Redirecting Standard Error

     Error redirection sends any error messages generated to an alternate destination such as a file, instead of sending them to the terminal window. 

     $ find / -name core -print 2> /dev/null 

     Redirecting both Standard Output and Error

      $ ls /etc/ /cdr 1> testfile1 2>&1

      $ ls /etc/ /cdr &> testfile1

• Use grep and regular expressions to analyze text

     grep (global regular expression print) searches contents of one or more specified files for a regular expression. If found, it prints every line containing the expression on the screen without changing the original file contents.

   

      for example, to search for the pattern "user1" in the /etc/passwd file:

       $ grep user1 /etc/passwd
        user1:x:501:501::/home/user1:/bin/bash

    

     to search for all the users beginning with user:

       $ grep "user*" /etc/passwd
       usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
       saslauth:x:497:495:"Saslauthd user":/var/empty/saslauth:/sbin/nologin
       user1:x:501:501::/home/user1:/bin/bash
       user2:x:502:502::/home/user2:/bin/bash

     To search for all occurrences of the pattern "user1" in both the /etc/passwd and /etc/group files:

     $ grep user1 /etc/passwd /etc/group
    /etc/passwd:user1:x:501:501::/home/user1:/bin/bash
    /etc/group:user1:x:501:

    To display only the names of those files that contain the pattern "user1" from the specified file list use :

    $ grep -l  

   

• Access remote systems using ssh and VNC

 check whether the required packages are installed. then we can use ssh and vnc as follows:

$ ssh user@remotesystem

we can either use vncviewer  in commandline and X windows session to connect to the vnc server running on remote systems.

• Log in and switch users in multi-user runlevels

log in using the username and passwords in runlevel 3 (in text mode) and graphically in run level 5

• Archive, compress, unpack and uncompress files using tar, star, gzip, and bzip2

star -xattr -H=exustar -c -f all_web.star public_html/ web_files/

tar xvzf file.tar.gz

tar cvzf file.tar.gz files

gunzip patch-2.5.28.gz

• Create and edit text files

Use vi, nano, emacs, gedit etc  text editors

• Create, delete, copy and move files and directories

touch, mv, rm , mv , cp

for directories use -r for recursive option

• Create hard and soft links

ln hardlink1 file1

ln -s softlink1 file1

• List, set and change standard ugo/rwx permissions

ls -l or ll

chmod

numerics -- symbolic

1 -- execute-- x

2 -- write-- w

4 -- read-- r

• Locate, read and use system documentation including man, info, and files in /usr/share/doc .
  [Note: Red Hat may use applications during the exam that are not included in Red Hat Enterprise Linux for the purpose of evaluating candidate's abilities to meet this objective.]

read the documentation accordingly using:

man command

info commnad

and files in /usr/share/doc

II. Operate Running Systems

• Boot, reboot, and shut down a system normally

power on the machine to boot it

reboot command can be used to reboot the machine

shutdown command with -h option to halt or -r option to reboot with time mostly 'now'

• Boot systems into different runlevels manually

init runlevel

• Use single-user mode to gain access to a system

init 1 or telinit 1

at the boot time edit the grub menu and at the end of the kernel entry append 1 or S to boot into single user mode

• Identify CPU/memory intensive processes, adjust process priority with renice, and kill processes

ps aux

ps -ef

top

kill 

nice

renice 

• Locate and interpret system log files

/var/log/messages

/var/log/*

• Access a virtual machine's console

xm console

virsh console

virt-viewer

• Start and stop virtual machines

xm create < vm >

xm destroy

xm shutdown

xm restart

virsh option

• Start, stop and check the status of network services

/sbin/service status/start/stop/restart

/etc/init.d/service status/start/stop/restart

III. Configure Local Storage

• List, create, delete and set partition type for primary, extended, and logical partitions 

Use fdisk diskname and type m for help on the commands and use the neccesssary command

for listing the partitions in fdisk: type p

for creating a new partition: type n , select whether it should be primary/extended and  specify the required amount of space

for creating the system id of a partition  in fdisk:  type t, and for help type L then select the required id such as 83 for linux  and 8e for Linux LVM etc. Then type w to save the changes to the partition table.

Further we can create the required filesystem using 'mkfs.ext(x) partition' or use LVM tools for creating logical volumes accordingly then format with neccessary filesystem.

• Create and remove physical volumes, assign physical volumes to volume groups, create and delete logical volumes

to create a physical volume create a partition on the disk using fdisk utility as mentioned above and select the system id for the partition as 8e which is Linux LVM

then use the following commands to create the logical volume groups:

# pvcreate -v /dev/sda1 /dev/sda2
# vgcreate -v -s 32 vg-testvm /dev/sda1 /dev/sda2
# lvcreate -v -L 4g -n lv-home vg-testvm
# lvcreate -v -L 2g -n lv-var vg-testvm

to remove 

# lvremove /dev/vg-testvm/lv-home

# vgremove /dev/vg-testvm

# pvremove /dev/sda1

Use vgdisplay -v and lvdisplay -v to see your new creations and complete details. My own naming convention is to use "vg" to indicate a volume group, and "lv" for a logical volume. So you see the structure here: the volume group is your total LVM storage space, which is comprised of several physical disk partitions, and then you have to divide your volume group into logical groups, or even just one logical group.

The -v switch turns on verbosity so you know what it's doing, and -s 32 creates physical extents that are 32 megabytes in size. Extents are often shrouded in mystery because no one bothers to explain them, but actually they're not mysterious at all. Physical extents are LVM's individual storage blocks, so the smallest possible size for a logical volume is a single extent. There is a maximum of 65,536 extents available per Linux kernel. The default size is 4 MB, which limits the maximum size of your volume group to about 256 GB. You can calculate a reasonable extent size by dividing the desired size of your volume by 65K. Extent sizes must be a power of 2, so round up to the next one and leave room for growth. Extent size doesn't affect performance, just your storage allocations. Extents are fixed when you create your volume group, so you can't change them later.

You have to increase or decrease the size of your volumes according to your extents, so here we're limited to 32 MB increments. The maximum possible size of a logical volume for 2.6 kernels is 16 terabytes on 32-bit systems, and 8 exabytes on 64-bit systems.

Now it's time to put filesystems and mountpoints on your logical volumes. Logical volumes are akin to physical disk partitions, so "lv-home" is going to be /home, and "lv-var" is /var:

# mkfs.xfs /dev/vg-testvm/lv-home
# mkfs.ext3 /dev/vg-testvm/lv-var

You may use any filesystem you want. Now create your mountpoints, adjust permissions and ownership, and then create your /etc/fstab entries. You can use either the /dev names or UUIDs:

/dev/vg-testvm/lv-home /home xfs defaults 0 2
/dev/vg-testvm/lv-var /var ext3 defaults 0 2

UUID=8d566d0e /dev/vg-testvm/lv-home /home xfs defaults 0 2
UUID=681919d5 /dev/vg-testvm/lv-var /var ext3 defaults 0 2

The UUIDs are truncated to conserve pixels. vgdisplay -v shows your UUIDs. Now you can reboot or manually mount your new logical volumes, and you're ready to start using them just like physical disk partitions.

Increasing the Size of a Logical Volume

Follow these steps to add a physical disk partition to an existing logical volume:

# pvcreate -v /dev/sdb1
# vgextend vg-testvm /dev/sdb1
# lvextend -L+10G

Then you must resize your filesystem using the resizing command specific to your filesystem. ReiserFS can be safely resized while mounted, and XFS must be mounted. Ext2/3 should be unmounted first:

# umount /var
# resize2fs -p /dev/vg-testvm/lv-var
# mount /var

The others look like this:

# resize_reiserfs /dev/volumegroup/logical-volume
# xfs_growfs /home

ReiserFS uses the /dev name, and XFS uses the name of the mountpoint. JFS is rather complicated

• Create and configure LUKS-encrypted partitions and logical volumes to prompt for password and mount a decrypted file system at boot

The following procedure will reconfigure and format your /home. The procedure is for single-user computers or computers that are shared between trusted users.

The following procedure will wipe all your existing data, so be sure to have a tested backup before you start. This also requires you to have a separate partition for /home (in my case that is /dev/VG00/LV_home). All the following must be done as root. Any of these steps failing means you must not continue until the step succeeded.

Step-by-Step Instructions

1. enter runlevel 1: telinit 1
2. unmount your existing /home: umount /home
3. if it fails use fuser to find and kill processes hogging /home: fuser -mvk /home
4. verify /home is not mounted any longer: cat /proc/mounts | grep home
5. fill your partition with random data: dd if=/dev/urandom of=/dev/VG00/LV_home

You're looking at a process that takes many hours, but it is imperative to do this in order to have good protection against break-in attempts. Just let it run overnight.

1. initialize your partition: cryptsetup --verbose --verify-passphrase luksFormat /dev/VG00/LV_home
2. open the newly encrypted device: cryptsetup luksOpen /dev/VG00/LV_home home
3. check it's there: ls -l /dev/mapper | grep home
4. create a filesystem: mkfs.ext3 /dev/mapper/home
5. mount it: mount /dev/mapper/home /home
6. check it's visible: df -h | grep home
7. add the following to /etc/crypttab: home /dev/VG00/LV_home none
8. edit your /etc/fstab, removing the old entry for /home and adding /dev/mapper/home /home ext3 defaults 1 2
9. verify your fstab entry: mount /home
10. restore default SELinux security contexts: /sbin/restorecon -v -R /home
11. reboot: shutdown -r now
12. The entry into /etc/crypttab makes your computer ask your luks passphrase on boot.
13. Log in as root and restore your backup.

we can create encrypted partitions at the time of installation by selecting the option available and providing the passphrase

• Configure systems to mount file systems at boot by Universally Unique ID (UUID) or label

to check UUID use blkid command on the device or partition

then edit the /etc/fstab file accordingly as follows:

UUID=d39b074c-9d46-4eab-a116-38d1a751c6fb /                       ext4    defaults        1 1

UUID=ded0bcce-2769-411d-8eaa-453233a5b9e4 /boot                   ext4    defaults        1 2

UUID=31379ba4-dce7-43f2-8679-5488aff9e82b /home                   ext4    defaults        1 2

UUID=1e4a6b5d-5d10-44d6-a5ff-334cdc63e4a4 /putty                  ext4    defaults        1 2

UUID=951da975-f039-42d4-86d5-42875a3d8b70 swap                    swap    defaults        0 0

• Add new partitions, logical volumes and swap to a system non-destructively

create the partiontions, logical volumes and swap using the fdisk utitiliy and add them while unmounted  

IV. Create and Configure File Systems

• Create, mount, unmount and use ext2, ext3 and ext4 file systems
• Mount, unmount and use LUKS-encrypted file systems
• Mount and unmount CIFS and NFS network file systems

• Configure systems to mount ext4, LUKS-encrypted and network file systems automatically

add entries in /etc/fstab and /etc/crypttab files using the UUID's and dev mapper paths respectively

• Extend existing unencrypted ext4-formatted logical volumes

lvextend

• Create and configure set-GID directories for collaboration

• Create and manage Access Control Lists (ACLs)

mount -t ext3 -o acl  

For example:

mount -t ext3 -o acl /dev/VolGroup00/LogVol02 /work

Alternatively, if the partition is listed in the /etc/fstab file, the entry for the partition can include the acloption:

LABEL=/work      /work       ext3    acl        1 2

If an ext3 file system is accessed via Samba and ACLs have

setfacl -m u:andrius:rw /project/somefile

To remove all the permissions for a user, group, or others, use the -x option and do not specify any permissions:

setfacl -x  

For example, to remove all permissions from the user with UID 500:

setfacl -x u:500 /project/somefile

To set a default ACL, add d: before the rule and specify a directory instead of a file name.
For example, to set the default ACL for the /share/ directory to read and execute for users not in the user group (an access ACL for an individual file can override it):

To determine the existing ACLs for a file or directory, use the getfacl command. In the example below, thegetfacl is used to determine the existing ACLs for a file.

getfacl home/john/picture.png

The above command returns the following output:

# file: home/john/picture.png 
# owner: john 
# group: john 
user::rw- 
group::r-- 
other::r--

If a directory with a default ACL is specified, the default ACL is also displayed as illustrated below.

[john@main /]$ getfacl home/sales/
# file: home/sales/ 
# owner: john 
# group: john 
user::rw- 
user:barryg:r-- 

setfacl -m d:o:rx /share

• Diagnose and correct file permission problems

V. Deploy, Configure and Maintain Systems

• Configure networking and hostname resolution statically or dynamically

edit the files in 

/etc/sysconfig/network-scripts/ifcfg-eth* 

with neccessary options such as

IPADDR, BOOTPROTO, NETMASK, NETWORK, BROADCAST, GATEWAY, ONBOOT, HWADDR etc

Edit /etc/sysconfig/network  file for specifying the hostname

hostname can be assigned using the hostname command temporarily

Edit /etc/resolv.conf file and specify 

search parameters and nameserver parameters accordingly

• Schedule tasks using cron

crontab -e for editing specific user crontab

# Each task to run has to be defined through a single line

# indicating with different fields when the task will be run

# and what command to run for the task

#

# To define the time you can provide concrete values for

# minute (m), hour (h), day of month (dom), month (mon),

# and day of week (dow) or use '*' in these fields (for 'any').#

# Notice that tasks will be started based on the cron's system

# daemon's notion of time and timezones.

#

# Output of the crontab jobs (including errors) is sent through

# email to the user the crontab file belongs to (unless redirected).

#

# For example, you can run a backup of all your user accounts

# at 5 a.m every week with:

# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/

#

# For more information see the manual pages of crontab(5) and cron(8)

# minutes  hour  dayofmonthy month dayofweek   command

• Configure systems to boot into a specific runlevel automatically

edit /etc/inittab file and specify the runlevel at the bottom entry which looks like:

id:3:initdefault:

• Install Red Hat Enterprise Linux automatically using Kickstart

PXE boot  (Install tftp which is a xinetd managed service  then configure tftpboot with pxelinux configuration and pxelinux images)

share the kickstart file using either nfs/http 

provide the following information in kickstart file

• Configure a physical machine to host virtual guests
• Install Red Hat Enterprise Linux systems as virtual guests
• Configure systems to launch virtual machines at boot

• Configure network services to start automatically at boot

chkconfig service on

• Configure a system to run a default configuration HTTP server
• Configure a system to run a default configuration FTP server

• Install and update software packages from Red Hat Network, a remote repository, or from the local filesystem

rpm -ivh 

rpm -e

yum install 

yum groupinstall

yum remove

/etc/yum/repos.d for configuring repositories

• Update the kernel package appropriately to ensure a bootable system

#yum -y update kernel

# rpm -Uvh /var/yum/repos.d/kernel/kernel-2xxxx

• Modify the system bootloader

Edit  /bootgrub/grub.conf file to change the bootloader and its options

VI. Manage Users and Groups

• Create, delete, and modify local user accounts

useradd

userdel

usermod

id

• Change passwords and adjust password aging for local user accounts

passwd

chage 'user'

• Create, delete and modify local groups and group memberships

usermod -a -G unixadm balaji

groupadd

groupmod

gpasswd

• Configure a system to use an existing LDAP directory service for user and group information

Install the Necessary LDAP Packages.

First, make sure that the appropriate packages are installed on both the LDAP server and the LDAP client machines. The LDAP server needs the openldap-servers package.

The openldap, openldap-clients, and nss_ldap packages need to be installed on all LDAP client machines.

Edit the Configuration Files.

• On the server, edit the /etc/openldap/slapd.conf file on the LDAP server to make sure it matches the specifics of the organization. 
• On the client machines, both /etc/ldap.conf and /etc/openldap/ldap.conf need to contain the proper server and search base information for the organization.
  To do this, run the graphical Authentication Configuration Tool (system-config-authentication) and select Enable LDAP Support under the User Information tab.
  It is also possible to edit these files by hand.
• On the client machines, the /etc/nsswitch.conf must be edited to use LDAP.
  To do this, run the Authentication Configuration Tool (system-config-authentication) and selectEnable LDAP Support under the User Information tab.
  If editing /etc/nsswitch.conf by hand, add ldap to the appropriate lines.
  For example:
  

  
  passwd: files ldap 
  shadow: files ldap 
  group: files ldap

VII. Manage Security

• Configure firewall settings using system-config-firewall or iptables

run system-config-firewall as root and select the required configuration if X windows is installed

/etc/sysconfig/iptables file can be edited according to the rules. 

• Set enforcing and permissive modes for SELinux

 we can either edit /etc/sysconfig/selinux or run system-config-selinux or

use

/usr/sbin/setenforce — Modifies in real-time the mode in which SELinux runs.

For example:

setenforce 1 — SELinux runs in enforcing mode.

setenforce 0 — SELinux runs in permissive mode.

To actually disable SELinux, you need to either specify the appropriate setenforce parameter in/etc/sysconfig/selinux or pass the parameter selinux=0 to the kernel, either in /etc/grub.conf or at boot time.

• List and identify SELinux file and process context

ll -Z 

ps -ef -Z

cp /mv -Z

id -Z

This section covers the specific roles enabled for the targeted policy. The unconfined_t type exists in every role, which significantly reduces the usefulness of roles in the targeted policy. More extensive use of roles requires a change to the strict policy paradigm, where every process runs in an individually considered domain.

Effectively, there are only two roles in the targeted policy: system_r and object_r. The initial role is system_r, and everything else inherits that role. The remaining roles are defined for compatibility purposes between the targeted policy and the strict policy.^[21]

Three of the four roles are defined by the policy. The fourth role, object_r, is an implied role and is not found in policy source. Because roles are created and populated by types using one or more declarations in the policy, there is no single file that declares all roles. (Remember that the policy itself is generated from a number of separate files.)

system_r

This role is for all system processes except user processes:

system_r (28 types)
    dhcpd_t
    httpd_helper_t
    httpd_php_t
    httpd_suexec_t
    httpd_sys_script_t
    httpd_t
    httpd_unconfined_script_t
    initrc_t
    ldconfig_t
    mailman_cgi_t
    mailman_mail_t
    mailman_queue_t
    mysqld_t
    named_t
    ndc_t
    nscd_t
    ntpd_t
    pegasus_t
    portmap_t
    postgresql_t
    snmpd_t
    squid_t
    syslogd_t
    system_mail_t
    unconfined_t
    winbind_helper_t
    winbind_t
    ypbind_t

user_r

This is the default user role for regular Linux users. In a strict policy, individual users might be used, allowing for the users to have special roles to perform privileged operations. In the targeted policy, all users run in the unconfined_t domain.

object_r

In SELinux, roles are not utilized for objects when RBAC is being used. Roles are strictly for subjects. This is because roles are task-oriented and they group together entities which perform actions (for example, processes). All such entities are collectively referred to as subjects. For this reason, all objects have the roleobject_r, and the role is only used as a placeholder in the label.

sysadm_r

This is the system administrator role in a strict policy. If you log in directly as the root user, the default role may actually be staff_r. If this is true, use the newrole -r sysadm_r command to change to the SELinux system administrator role to perform system administration tasks. In the targeted policy, the following retain sysadm_r for compatibility:

sysadm_r (6 types)
    httpd_helper_t
    httpd_sys_script_t
    initrc_t
    ldconfig_t
    ndc_t
    unconfined_t

There is effectively only one user identity in the targeted policy. The user_u identity was chosen becauselibselinux falls back to user_u as the default SELinux user identity. This occurs when there is no matching SELinux user for the Linux user who is logging in. Using user_u as the single user in the targeted policy makes it easier to change to the strict policy. The remaining users exist for compatibility with the strict policy.^[22]

The one exception is the SELinux user root. You may notice root as the user identity in a process's context. This occurs when the SELinux user root starts daemons from the command line, or restarts a daemon originally started by init.

• Restore default file contexts

chcon

ex:  #chcon -R -t httpd_user_content_t public_html/

# ls -Z

restorecon

Use the restorecon command to restore files to the default values according to the policy. There are two other methods for performing this operation that work on the entire file system: fixfiles or a policy relabeling operation

ex: #/sbin/restorecon -R archives

• Use boolean settings to modify system SELinux settings

sestatus

getsebool -a

Use the getsebool command to get the current status of the boolean:

[root@host2a ~]# getsebool named_disable_trans
named_disable_trans --> off

Use the following command to disable enforcing mode for this daemon:

[root@host2a ~]# setsebool named_disable_trans 1

[root@host2a ~]# getsebool named_disable_trans
named_disable_trans --> on

Use the following command to find which of these booleans are set:

getsebool -a | grep disable.*on

httpd_disable_trans=1
mysqld_disable_trans=1
ntpd_disable_trans=1

You can set any number of boolean values using the setsebool command:

setsebool -P httpd_disable_trans=1 mysqld_disable_trans=1 ntpd_disable_trans=1

You can also use togglesebool  to change the value of a specific boolean:

[root@host2a ~]# getsebool httpd_disable_trans
httpd_disable_trans --> off

[root@host2a ~]# togglesebool httpd_disable_trans
httpd_disable_trans: active

You can configure all of these settings using system-config-selinux. The same configuration files are used, so changes appear bidirectionally.

• Diagnose and address routine SELinux policy violations

his section describes some common tasks that a security analyst might need to perform on an SELinux system.

44.3.1. Enabling Kernel Auditing

As part of an SELinux analysis or troubleshooting exercise, you might choose to enable complete kernel-level auditing. This can be quite verbose, because it generates one or more additional audit messages for each AVC audit message. To enable this level of auditing, append the audit=1 parameter to your kernel boot line, either in the /etc/grub.conf file or on the GRUB menu at boot time.
This is an example of a full audit log entry when httpd is denied access to ~/public_html because the directory is not labeled as Web content. Notice that the time and serial number stamps in the audit(...) field are identical in each case. This makes it easier to track a specific event in the audit logs:

Jan 15 08:03:56 hostname kernel: audit(1105805036.075:2392892): \
 avc:  denied  { getattr } for  pid=2239 exe=/usr/sbin/httpd \
 path=/home/auser/public_html dev=hdb2 ino=921135 \
 scontext=user_u:system_r:httpd_t \
 tcontext=system_u:object_r:user_home_t tclass=dir

The following audit message tells more about the source, including the kind of system call involved, showing that httpd tried to stat the directory:

Jan 15 08:03:56 hostname kernel: audit(1105805036.075:2392892): \
 syscall=195 exit=4294967283 a0=9ef88e0 a1=bfecc0d4 a2=a97ff4 \
 a3=bfecc0d4 items=1 pid=2239 loginuid=-1 uid=48 gid=48 euid=48 \
 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48

The following message provides more information about the target:

Jan 15 08:03:56 hostname kernel: audit(1105805036.075:2392892): \
 item=0 name=/home/auser/public_html inode=921135 dev=00:00

The serial number stamp is always identical for a particular audited event. The time stamp may or may not be identical.

Note

If you are using an audit daemon for troubleshooting, the daemon may capture audit messages into a location other than /var/log/messages, such as /var/log/audit/audit.log. Red Hat Enterprise Linux 5 does not currently ship with an audit daemon.

44.3.2. Dumping and Viewing Logs

The Red Hat Enterprise Linux 5 implementation of SELinux routes AVC audit messages to /var/log/messages. You can use any of the standard search utilities (for example, grep), to search for lines containing avc or audit

Red Hat Linux Boot Process

Boot Process, Init, and Shutdown

An important and powerful aspect of Red Hat Enterprise Linux is the open, user-configurable method it uses for starting the operating system. Users are free to configure many aspects of the boot process, including specifying the programs launched at boot-time. Similarly, system shutdown gracefully terminates processes in an organized and configurable way, although customization of this process is rarely required.
Understanding how the boot and shutdown processes work not only allows customization, but also makes it easier to troubleshoot problems related to starting or shutting down the system.


The Boot Process

Below are the basic stages of the boot process for an x86 system:

1.The system BIOS checks the system and launches the first stage boot loader on the MBR of the primary hard disk.
2.The first stage boot loader loads itself into memory and launches the second stage boot loader from the /boot/ partition.
3. The second stage boot loader loads the kernel into memory, which in turn loads any necessary modules and mounts the root partition read-only.
4. The kernel transfers control of the boot process to the /sbin/init program.
5. The /sbin/init program loads all services and user-space tools, and mounts all partitions listed in /etc/fstab.
6. The user is presented with a login screen for the freshly booted Linux system.


The BIOS

When an x86 computer is booted, the processor looks at the end of system memory for the Basic Input/Output System or BIOS program and runs it. The BIOS controls not only the first step of the boot process, but also provides the lowest level interface to peripheral devices. For this reason it is written into read-only, permanent memory and is always available for use.

Other platforms use different programs to perform low-level tasks roughly equivalent to those of the BIOS on an x86 system.

Once loaded, the BIOS tests the system, looks for and checks peripherals, and then locates a valid device with which to boot the system. Usually, it checks any diskette drives and optical drives present for bootable media, then, failing that, looks to the system's hard drives. In most cases, the order of the drives searched while booting is controlled with a setting in the BIOS, and it looks on the master IDE device on the primary IDE bus. The BIOS then loads into memory whatever program is residing in the first sector of this device, called the Master Boot Record or MBR. The MBR is only 512 bytes in size and contains machine code instructions for booting the machine, called a boot loader, along with the partition table. Once the BIOS finds and loads the boot loader program into memory, it yields control of the boot process to it.

The first thing to occur in the boot process is the Power-On Self Test (POST) that is performed by the compter's Basic Input/Output System (BIOS), when it is powered on. This self test is just that -- an internal check of the system's internal components. Other architectures may use different ways to accomplish this, but the purpose is identical. This checking normally involves the system RAM, CPUs, video card, hard drives, and other motherboard components. Many other types of internal cards such as RAID cards and various other types of controller cards perform their own self-testing. If errors occur, some form of alert will be generated or sent to a panel or console; the POST uses a series of beeps to indicate the specific error encountered. If all goes well, the BIOS reads the master boot record on the hard disk and loads the program that is found there into memory.

Master Boot Record

The Master Boot Record (MBR) is the first 512 bytes of the boot drive that is read into memory by the BIOS. (This is assuming we are using an x86 architecture.) The first 446 bytes of that 512 will normally contain a low-level boot code that points to a boot loader somewhere else on the disk - it can even point to another hard disk. The next 64 bytes contain the partition table for the disk (four 16-byte entries known as the IBM Partition Table Scheme). Finally, the remaining 2 bytes are the "Magic Number" (used for error detection).

The Boot Loader


A boot loader for the x86 platform is broken into at least two stages. The first stage is a small machine code binary on the MBR. Its sole job is to locate the second stage boot loader and load the first part of it into memory.
GRUB has the advantage of being able to read ext2, ext3, and ext4 partitions and load its configuration file — /boot/grub/grub.conf — at boot time.

he purpose of a bootloader is to load the operating system. Many boot loaders are available; however, LILO and GRUB are the most common for Linux. Windows has its own, called the New Technology Loader (NTLDR). You may have at one time or another seen the message "NTLDR is missing"; this is a Windows bootloader error.
Regardless of the bootloader being used, it is important to keep in mind that bootloaders are very complex and easily rendered inoperative by inexperienced users with root access. In my opinion, the best practice is to explore their workings only on machines you are using as sandboxes or in virtual machines to sharpen your skills. One slight typographical error while working with, e.g., grub.conf, and you may be spending more time than you anticipated trying to fix your error; however, this too, can be a great learning experience, as long as it is on a sandbox and you can afford the time.

GRUB

The most common bootloader program in use today for modern Linux systems is the GRand Unified Bootloader (GRUB). This is the bootloader we will be talking about, here. The GRUB bootloader is a program written to the master boot record and /boot partition of a hard drive that loads the operating system. The bootloader code used to fit in the first 446 bytes of the MBR, but, owing to progressively increasing complexity of operating systems and the need to boot almost any operating system, it has grown in size. Currently, a subset of the bootloader code is written to the MBRi, and the remainder is written to the /boot partition. In addition, the GRUB bootloader is modular in design and works in stages which I will only cover briefly.
The stages of GRUB
GRUB works in stages called Stage1, Stage1_5, and Stage 2. I will give a brief overview of what each stage does.
Stage1
The stage1 code of GRUB is written within the 512 bytes of the master boot record. Because of the limited size of the master boot record area, GRUB stage1 will usually point to the next stage of GRUB, stage1_5 or stage2. GRUB may or may not need to load stage1_5 depending on the types of filesystems present.

[root@localhost grub]# file stage1
stage1: x86 boot sector, code offset 0x48

Stage1_5
Stage1_5 is the intermediate stage between stage1 and stage2. If you look at your /boot/grub area, you should see various stage1_5 files with names of various types of filesystems. Stage1_5 deals with specific types of filesystems, and is named after them. This code will allow the filesystem to be interpreted appropriately.
These are the stage1_5 files located in my /boot/grub directory:

e2fs_stage1_5
fat_stage1_5
ffs_stage1_5
iso9660_stage1_5
jfs_stage1_5
minix_stage1_5
reiserfs_stage1_5
ufs2_stage1_5
vstafs_stage1_5
xfs_stage1_5

Stage2
This is the main GRUB image, and will usually reside on the filesystem in the /boot partition at /boot/grub/stage2. It reads the /boot/grub/grub.conf file for configuration information that details how it will load the kernel. It also has an interactive interface that will allow you to troubleshoot, re-install, or modify how GRUB works. Stage2 will present the user with a graphical boot menu entry. If the appropriate key is not pressed in the time allotted to enter the hidden menu, or nothing is selected in the time frame allowed, GRUB will boot the default entry.
GRUB has a really helpful tab-completion feature that you can use to assist you in getting your system up and running if, for example, you have a non-bootable system due to a improperly specified line in your grub.conf file. I have used it in this way many times to get unbootable systems working again.
/boot/grub/grub.conf
This file specifies the kernel to load and the initrd image file with all of the modules to load for your system.
Here is a typical grub.conf file for Red Hat Linux:

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/hdb9
#          initrd /initrd-version.img
#boot=/dev/hdb
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server   (2.6.18-53.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-53.el5 ro root=LABEL=/ rhgb quiet
        initrd /initrd-2.6.18-53.el5.img

This file is fairly easy to edit; because of this, it is a common location for errors.

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

 [Minimal BASH-like line editing is supported.  For the first word, TAB
   lists possible command completions.  Anywhere else, TAB lists the possible
   completions of a device/filename.]

grub> 

Nothing but a GRUB prompt - now what do we do? If you're making any changes to this file, remember to print it out first, and make a backup copy; it may save you if you make a typo. You can use the hardcopy to recall the options necessary to boot your machine and fix your mistake. There are various errors, and it pays to become familiar with some of the common ones such as an incorrectly specified kernel name, or a root partition that is improperly designated in this file. If you do see this GRUB prompt, ask yourself this question, "What do I do now to recover my system?" If you have no idea at this point, and you are managing Linux systems, it would probably be a good idea to get on a practice box and create some errors within the grub.conf, and get to know your bootloader. Here's what could be done in the above situation - notice where I hit the key to view my options.
Booting manually with GRUB tab completion

grub> root (hd0,0)                               
 Filesystem type is ext2fs, partition type 0x83

grub> kernel /               
 Possible files are: lost+found grub initrd-2.6.18-8.el5.img System.map-2.6.18-
8.el5 config-2.6.18-8.el5 symvers-2.6.18-8.el5.gz vmlinuz-2.6.18-8.el5

grub> kernel /vmlinuz-2.6.18-8.el5  ro root=LABEL=/                                                                                                                    
     [Linux-bzImage, setup=0x1e00, size=0x1ad054]

grub> initrd /                  
 Possible files are: lost+found grub initrd-2.6.18-8.el5.img System.map-2.6.18-
8.el5 config-2.6.18-8.el5 symvers-2.6.18-8.el5.gz vmlinuz-2.6.18-8.el5

grub> initrd /initrd-2.6.18-8.el5.img
   [Linux-initrd @ 0xfe8a000, 0x15547c bytes]

grub> boot               

The "root (hd0,0)" specifies the first drive, and the first partition on that drive. On the kernel line, you can type '/', hit , and view your options; after selecting one, type one or more letters (whatever's necessary for a unique name) and let GRUB complete it for you. The additional option of "ro root=LABEL=/" came from my experience of working with Red Hat systems; yours may be different - so you need to examine the options in your grub.conf file. The same thing happens on the initrd line: after the /, hit and let GRUB do the work by listing your possible choices. You may also have to further specify modules and additional options.

Kernel

When GRUB transfers control to the kernel and the kernel is booting, you may see a lot of text output. On Red Hat-based systems, the line in grub.conf specifying the kernel may have an "rhgb quiet" appended to it that prevents this. It stands for "Red Hat graphical boot quiet" and will suppress kernel boot messages. When I need to see boot messages, I interrupt the GRUB cycle by hitting the escape key, hitting 'e' for edit (the procedure to modify the grub kernel arguments is at the bottom of the GRUB screen), and editing the kernel line. Removing the "rhgb quiet" allows me to see the kernel messages so I can determine if any of them are relevant to, e.g., a kernel panic or a similar problem.
What is actually going on at this time is that the kernel is probing your hardware and configuring itself for your hardware. The kernel is also loading modules in the initrd image that it needs to operate your hardware. Note that the information will scroll very quickly up your screen - so, if you believe your problem exists at the kernel level, be sure to watch closely, as this is sure to give you a clue about where your problem lies. Once the kernel is done with its initialization, it starts the system's first process, which is /sbin/init.

[ Much, but not all, of the boot information is available in the /var/log/dmesg file, once you've booted. -- Ben ]

INIT

Init is the first process running on your system. It reads the /etc/inittab file, executes /etc/rc.d/rc.sysinit, then boots into the runlevel as defined in /etc/inittab.
Init starts out with a Process ID (PID) of 1. In the image above, there's a line saying "INIT: version 2.86 booting"; this is /sbin/init taking over at this point in the boot process. On the line right after that one, you see the messages being displayed by the /etc/rc.d/rc.sysinit shell script; as a matter of fact, the entire screen contains messages from that script, so you can get an idea of some of the functions it performs. Init will also normally start several instances of /sbin/getty or /sbin/mingetty, which are your virtual terminals. This is why you can hit + + < F1> through and come up with a virtual terminal.
Next, we will look at the /etc/inittab file, specifically, at the configuration file for Init:

#
# inittab       This file describes how the INIT process should set up
#               the system in a certain run-level.
#
# Author:       Miquel van Smoorenburg, 
#               Modified for RHS Linux by Marc Ewing and Donnie Barnes
#

# Default runlevel. The runlevels used by RHS are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
#
id:5:initdefault:

# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

# Trap CTRL-alt-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now

# When our UPS tells us power has failed, assume we have a few minutes
# of power left.  Schedule a shutdown for 2 minutes from now.
# This does, of course, assume you have powerd installed and your
# UPS connected and working correctly.
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"

# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Cancelled"


# Run gettys in standard runlevels
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6

# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon

Depending on what Init is doing, you may see typical Init script verification messages getting printed to the screen, i.e., [ OK ] or [Failed] to aid in troubleshooting. You may see a message like "Press 'I' to enter interactive startup" (on Red Hat-based systems); this is an indication of rc.sysinit executing, and allows the operator a certain level of control over the still-booting system. rc.sysinit ends with your default runlevel (as defined in /etc/inittab) being started. This is another common place for errors, because servers will usually have the "id:5:initdefault:" line set to 3, so the machine boots to runlevel 3 instead of runlevel 5. Another common place for errors is the line pertaining to "ca::ctrlaltdel:/sbin/shutdown -t3 -r now" commented out to prevent the 3-finger salute ( + + ) from restarting the server. People are human and make mistakes; I have seen typographical errors in both places that can cause problems. Unlike the "mount -a" command, which will alert you if errors exist in your mount points in the /etc/fstab file, executing the command "init q" will reread your /etc/inittab, but will not check for errors in the runlevels themselves; the best way to know if errors exist is to learn this file and to be very, very careful if you decide to modify any of the /etc/rc*/* files.

Runlevels

When you enter your runlevel, you will see further Init messages being printed to the screen (depending if your machine is configured to do so), again ending with a [ OK ] or [Failed] depending on whether it started successfully or not. These are your startup services within your runlevels. When you look at your /etc/inittab file, you will see a line like "id:5:initdefault:"; this is your default runlevel. The default runlevel on most servers will be set to 3; on desktops, of course, it's set to 5, so we can have an X Window System session start as soon as the system boots up.
To get an overview of what processes get started or stopped for any particular runlevel, we should look within the /etc/rcX.d (where X is your runlevel) directory. Inside these directories, you will see symbolic links to the files in your /etc/init.d/ directory. The file names will be prefixed with either a 'K' or an 'S' (signifying kill or start) for the given daemon at that runlevel. The number immediately after the letter positions the script in the start order, because the processes are started alphabetically. With Red Hat-based systems, the "chkconfig" command will alter the symbolic links to start or stop the daemon in a desired runlevel; the 'S' or 'K' will change appropriately and the number will most likely change as well.

Login

That sums up the outline of the boot process for a typical Linux machine. At this point, you should have a better understanding of what goes on up until the login prompt or dialog on your screen. Last of all, always pay close attention to the details: the machine will most likely tell you what is wrong if you're experiencing problems.


Resources


1) http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/s2-boot-init-shutdown-loader.html

2)http://linuxgazette.net/156/prestia.html