Setup and manage RAID devices

The use of RAID (Redundant Array of Independent Disks) spreads I/O over multiple physical disks. It’s purpose is to enhance data integrity and recover-ability in case of failure. Three essential features of RAID:
a) mirroring: writing the same data to more than one disks
b) stripping: splitting of data to more than one disks
c) parity: extra data is stored to allow problem detection and repair

There are number of RAID specifications of increasing complexity and use:
RAID 0: uses only stripping. Data is spread across multiple disks. However, there’s no redundancy and there’s no stability or recovery capabilities.
RAID 1: uses only mirroring, each disk has a duplicate. At least two disk are required.
RAID 5: uses a rotating parity stripe, a single drive failure cause no data loss. At least 3 disks are required.
RAID 6: has stripped disks with dual parity. It can handle loss of two disks. It requires at least four disks.
RAID 10: is a mirrored and stripped data set. It needs at least four drives.

1. First create two partitions using fdisk utility:
a) fdisk /dev/sdb
b) partprobe -s /dev/sdb; parted /dev/sdb print

2. Create RAID device:
a) mdadm –create /dev/md0 –level=1 –raid-disks=2 /dev/sdb5   /dev/sdb6

3. Format the RAID device:
a) mkfs.ext4   /dev/md0

4. Mount the RAID device:
a) mkdir /mnt/raid
b) echo “/dev/md0  /mnt/raid  ext4  defaults  0  0″  >>  /etc/fstab
c) mount -a; df -hT

5. Capture the RAID device details to ensure persistence:
mdadm –detail –scan >> /etc/mdadm.conf

6. Verify the RAID device status:
a) mdadm –detail /dev/md0
b) cat /proc/mdstat

Stop RAID device:
mdadm -S /dev/md0

Advertisements

Setup and Manage LVM in Linux

LVM (Logical Volume Management) permits having one logical filesystem span multiple physical volumes and partitions while appearing as a simple partition for normal usage. Disk partitions are converted into physical volumes and multiple physical volumes are grouped into a volume group. Then the volume group is subdivided into logical volumes.

1. Create two logical partitions inside an extended partition and set their type to 8e using fdisk utility:
a) fdisk /dev/sdb
b) Inside fdisk utility, type ‘t‘ to set partition type to 8e (Linux LVM)
c) partprobe -s /dev/sdb; parted /dev/sdb print 

2. Create two Physical Volumes from the partitions:
a) pvcreate    /dev/sdb5    /dev/sdb6
b) pvdisplay; pvs

3. Create a Volume Group:
a) vgcreate vg1   /dev/sdb5   /dev/sdb6
b) vgdisplay; vgs

4. Allocate a Logical Volume from the volume group:
a) lvcreate -n   lv1   -L   300M    vg1
b) lvdisplay; lvs

5. Format the Logical Volume:
a) mkfs.ext4   /dev/vg1/lv1

6. Mount the Logical Volume:
a) mkdir /mnt/lv1
b) echo “/dev/vg1/lv1  /mnt/lv1  ext4   defaults  0  0″ >> /etc/fstab
c) mount -a; df -hT

Extend or increase a logical volume relative size by 100MB:

a) lvextend -L +100M /dev/vg1/lv1
b) resize2fs /dev/vg1/lv1

P.S. The above two commands can be combined in:
lvextend -r   -L   +100M   /dev/vg1/lv1

Reduce or shrink the logical volume size to absolute 200MB:

a) umount   /mnt/lv1
b) fsck -f   /dev/vg1/lv1
c) resize2fs   /dev/vg1/lv1   200M
d) lvreduce   -L   200M   /dev/vg/lv1
e) mount   /dev/vg1/lv1

P.S. The above commands can be combined in just one command as:
lvredue   -r   -L   200M   /dev/vg/lv1

Remove Physical Volumes, Volume Group and Logical Volum:

a) umount   /dev/vg1/lv1
b) lvremove   /dev/vg1/lv1
c) vgremove   /dev/vg1
d) pvremove   /dev/sdb5   /dev/sdb6

Create LVM snapshot:

a) lvcreate  -L 100M  -s -n lv1snap  /dev/vg1/lv1

 

Setup a encrypted swap partition in Linux

1. Create a partition using the following utilities:
a) fdisk /dev/sdb or parted /dev/sdb
b) partprobe -s /dev/sdb

2. Initializes a LUKS partition and sets the initial passphrase:
a) cryptsetup luksFormat /dev/sdb1 –force-password –verbose
b) Type  YES and enter password as pass1234

3. Opens the LUKS device and sets up a mapping:
a) cryptsetup luksOpen /dev/sdb1 secretsdb1 –verbose
b) ls -l /dev/mapper/

4. Setup a linux swap area on the device:
    mkswap /dev/mapper/secretsdb1

5. Enable the device swapping:
    swapon /dev/mapper/secret-sdb1

6. Enter the following line in /etc/fstab:
echo “/dev/mapper/secretsdb1 none swap defaults 0 0″ >> /etc/fstab

7. Enter the following line in /etc/crypttab:
echo “secretsdb1  /dev/sdb1   /dev/urandom   swap” >> /etc/crypttab

8. Reboot the system and verify the swap space using the following utilities:
    cat /proc/swaps; free -mh

P.S. If you want to use a loopback file, then create it as:
dd if=/dev/urandom of=/root/swapfile bs=1M count=1024
And replace /dev/sdb1 with /root/swapfile in the above steps.  

Encrypt a partition in Linux

1. Create a partition using the following utilities:
a) fdisk /dev/sdb or parted /dev/sdb
b) partprobe -s /dev/sdb

2. Initializes a LUKS partition and sets the initial passphrase:
a) cryptsetup luksFormat /dev/sdb1 –force-password –verbose
b) Type  YES and enter password as pass1234

3. Opens the LUKS device /dev/sdb1 and sets up a mapping:
a) cryptsetup luksOpen /dev/sdb1 secretsdb1 –verbose
b) ls -l /dev/mapper/

4. Format the encrypted partition using the following:
mkfs.ext4 /dev/mapper/secret-sdb1

5. Enter the following line in /etc/crypttab:
echo “secretsdb1  /dev/sdb1″ >> /etc/crypttab

6. Create a mount directory and mount it in /etc/fstab:
a) mkdir /secret; chmod a+rw /secret
b) echo “/dev/mapper/secretsdb1  /secret  ext4  defaults  1  2″ >> /etc/fstab
c) mount -a; df -hT

7. (Optional) Setup a keyfile for an encrypted partition:
a) dd if=/dev/urandom of=/root/secretkey bs=512 count=4
b) cryptsetup luksAddKey /dev/sdb1 /root/secretkey
c) (Replace Step-5 with) echo “secretsdb1  /dev/sdb1  /root/secretkey  luks” >> /etc/crypttab
P.S: Above steps will remove password at the boot for an encrypted partition.

Encrypt a loopback device:

Create a 1GB looback file and setup it with the loopback device using the following commands, and then replace /dev/sdb1 in above commands with the loopback device. e.g. /dev/sdb1 -> /dev/loop0
dd if=/dev/zero of=cryptfile bs=1M count=1024
losetup -f cryptfile

Remove the encrypted partition:

a) umount /secret
b) Remove entry from /etc/fstab and /etc/crypttab
c) cryptsetup luksClose /dev/mapper/secret-disk
d) Delete the partition /dev/sdb1 using fdisk or parted utilities.
If using the loopback device then detached it using losetup -d /dev/loop0 and remove the file /root/imagefile.

Configure user disk quotas in Linux

1. Create a 1 GB partition or a image file:
   dd if=/dev/zero of=imgfile bs=1M count=1000
   parted -s /dev/sdb unit MB mkpart primary ext4 0 1024
2. mkfs.ext4 imgfile  OR mkfs.ext4 /dev/sdb1
3. Enter one of the following line in /etc/fstab
   /imgfile       /mnt    ext4    loop,usrquota  1  2
   /dev/sdb1   /mnt     ext4    usrquota          1  2
4. mount -o remount /mnt
5. quotacheck -vu /mnt
6. quotaon -vu /mnt
7. edquota -u user
8. quota -u user
9. quotaoff -v /mnt   (Optional)
10. Remove the line in step-3 /etc/fstab  (Optional)
11. umount /mnt; mount -a         (Optional)

Monitoring utilities / commands in Linux

Process and Load Monitoring Utilities:

• top (procps): Process activity, dynamically updated
• uptime (procps): How long the system is running and average load
• ps (procps): Detailed information about process
• pstree (psmisc or pstree): A tree of process and their connections
• mpstat (sysstat): Multiple process usage
• iostat (sysstat): CPU utilization and IO states
• sar (sysstat): Display and collect information about system activity
• numastat (numactl): Information about NUMA (Non-Uniform Memory Architecture)
• strace (strace): Information about all system calls a process makes

P.S. In parenthesis it shows the package name, in which the utility is located.

Memory Monitoring Utilities:

• free (procps): Brief summary of memory usage
• vmstat (procps): Detailed virtual memory statistics and block IO, dynamically updated
• pmap (procps): Process memory map

I/O Monitoring Utilities:

• iostat (sysstat): CPU utilization and I/O statistics
• vmstat (procps): Detailed virtual memory statistics and block I/O, dynamically updated
• sar (sysstat): Display and collect information about system activity

Networking Monitoring Utilities:

• netstat (netstat): Detailed network statistics
• iptraf (iptraf): Gather information on network interfaces
• tcpdump (tcpdump): Detailed analysis of network packets and traffic
• wireshark (wireshark): Detailed network traffic analysis

 

Linux networking basic

ip:   ( ip [options] Object-> addr|link|route {command | help} )

• ip addr show [eth0]
• ip addr add|del 192.168.1.3  dev eth0
• ip link show [eth0]
• ip link set eth0 down|up    (Stop & Start network device)
• ip route show; route -n; netstat -r
• ip route add|del 10.10.10.0/24  via  192.168.1.1  [dev eth1]

P.S. 10.10.10.0/24 should be external network address, wheres 192.168.1.1 need to be the  address that must be accessible by the host.

ifconfig:

• ifconfig            (Show all the interfaces)
• ifconfig eth0
• ifconfig eth0 down|up  (Stop & Start network device)
• ifconfig eth0 192.168.1.3   (Set ip address)
• ifconfig eth0 netmask 255.255.255.0
• ifconfig eth0 mtu 1480

Miscellaneous:

• hostname | hostnamectl status
• hostnamectl set-hostname newhost1.mydomain (–transient|–static|–pretty)
• systemctl enable|disable NetworkManager
• systemctl status|start|stop NetworkManager
• service network status|stop|start
• ping google.com
• netstat -tulpan
• route
  • route -n; netstat -r
  • route add|del default gw 192.168.1.1
  • route add|del -net 10.10.10.0 netmask 255.255.255.0 gw 192.168.1.1
• traceroute google.com
• [ dig | host | nslookup ] google.com
• mtr google.com

Network Manager:

• systemctl status|restart NetworkManager
• systemctl enable|disable NetworkManager
• systemctl status|restart network
• nmtui                (Graphical tool)
• nmcli;  nmcli connection
• nmcli con show|up|down|reload eth0
• nmcli con add|del eth0
• nmcli con modify eth0 +|-ipv4.address 192.168.1.3/24
• nmcli con modify eth0 +|-ipv4.routes “10.10.10.0/24   192.168.1.1“

Static host resolution -> /etc/hosts 

• 192.168.1.3  host   host.domain

Static DNS resolution -> /etc/resolv.conf

• search localdomain
• nameserver 192.168.1.1
• nameserver 8.8.8.8
• systemctl restart network

Assign static IP
(1) Add the following lines into /etc/sysconfig/network-scripts/ifcfg-eth0 

• NAME=eth0                          (Optional)
• DEVICE=eth0
• BOOTPROTO=static
• ONBOOT=yes
• IPADDR=192.168.1.3
• NETMASK=255.255.255.0
• GATEWAY=192.168.1.1

(2) Add the following line into /etc/resolv.conf:

• nameserver 192.168.1.1

( 3) Restart the network interface:

• systemctl restart network

P.S. If you are using Network Manager then you may run the following commands to add static IP:
(1)  nmcli con add con-name eth0 ifname eth0 type ethernet ip4 192.168.1.3/24 gw4 192.168.1.1
(2) nmcli con show; cat /etc/sysconfig/network-scripts/ifcfg-eth0

 Assign static routes -> /etc/sysconfig/network-scripts/route-eth0

• 10.10.10.0/24 via 192.168.1.1  [dev eth0]
• default via 192.168.1.1  dev  eth0
• systemctl restart network

P.S. If you are using Network Manager then you may run the following commands to add static IPs and route:
-> nmcli con modify  eth0  ipv4.routes  “10.10.10.0/24  192.168.1.1”
-> nmcli con up eth0

Firewalld:

• systemctl status|start|stop|enable|disable firewalld
• firewall-cmd –state
• firewall-cmd –reload
• firewall-cmd –list-all|–list-services |–list-ports
• firewall-cmd  –get-zones|–get-services|–get-default-zone|–get-active-zone
• firewall-cmd   [–permanent] –set-default-zone=trusted
• firewall-cmd [–permanent] –add-service|–remove-service=pop3  [–zone=home]
• firewall-cmd [–permanent] –add-port|–remove-port=21/tcp
• Config-> /etc/firewalld/firewalld.conf
• Enable following if using multiple network interfaces:
  • echo “net.ipv4.ip_forward=1” >> /etc/sysctl.conf
  • sysctl -p