Skip to content

Redirection

  • STDIN < or 0<
  • STDOUT > or 1>. >> or 1>> to append.

  • STDERR 2>. 2>> to append.

  • Redirect STDERR to SDTOUT 2>&1

  • Redirect STDERR and STDOUT to different files
command 2> stderr > stdout
  • /dev/null

Find your way

  • which shows the full path of (shell) commands.
  • type bash builtin. Alternative of which.
  • realpath print the resolved path of a file.
  • pwd print name of current/working directory.
  • apropos search the manual page names and descriptions. Equivalent to man -k.
  • man an interface to the system reference manuals.
  • alias bash builtin. ll="ls -lAh".
  • history bash builtin. Manipulate the history list.
  • env run a program in a modified environment and show the current environment variables.
  • set read and write variables.

  • unset delete variables.

  • ls list directory contents.

  • cd change working directory.
  • mv move (rename) files.
  • cp
  • ln
  • mkdir
  • rmdir
  • rm

  • find

  • cat concatenate files and print on the standard output.

  • head output the first part of files.
  • file determine file type.
  • tail output the last part of files.
  • more file perusal filter for crt viewing (pager).
  • less opposite of more (pager).
  • du
  • touch
  • cut
  • sort
  • wc

  • grep

  • df

  • mount
  • lsblk
  • findmnt
  • fdisk
  • gdisk
  • blkid
  • parted
  • partprobe
  • mkfs
  • tune2fs
  • e2label
  • xfs_admin
  • mkswap
  • swapon and swapoff
  • findfs
  • fsck
  • showmount

  • smbclient

  • w show who is logged on and what they are doing.

  • who show who is logged on.
  • last and lastb show a listing of last logged in users.
  • su run a command with substitute user and group ID.
  • sudo & sudoedit execute a command as another user
  • whoami
  • who show who is logged on
  • getent get entries from administrative database.
  • users print the user names of users currently logged in to the current host
  • id
  • passwd
  • chage change user password expiry information
  • vipw & vigr edit the password or group file
  • useradd create a new user or update default new user information
  • usermod modify a user account
  • userdel delete a user account and related files
  • userdbctl inspect users, groups and group memberships (good visibility)
  • groupadd create a new group
  • groupdel delete a group
  • groupmems administer members of a user's primary group
  • groupmod modify a group definition on the system

  • groups display current group names

  • chown

  • chmod
  • chgrp
  • newgrp
  • getfacl
  • setfacl
  • umask
  • chattr - change file attributes on a Linux file system

  • lsattr

  • ip

  • ss
  • netstat
  • ping
  • ifconfig
  • nmcli & nmtui tools for controlling NetworkManager

  • hostnamectl

  • reboot

  • halt
  • poweroff

  • uptime

  • lscpu

  • lspci

  • lsusb

  • uptime

  • free

  • timedatectl

  • date
  • hwclock

  • tzselect

  • compgen bash builtin. Display possible completions depending on the options.

    • compgen -c list binaries.
    • compgen -a list aliases.
    • compgen -b list builtins.
    • compgen -k list keywords.
    • compgen -A function list functions.
    • compgen -A function -abck list all the above.
  • ls -AH ${PATH//:/ } list all files in $PATH directories. Or find ${PATH//:/ } -maxdepth 1 -executable. Or IFS=: ; ls -AH $PATH
  • whence -pm '*' zsh builtin.
  • print -rC1 -- $commands

Process management

  • shell job commands started from the command line. interactive processes
  • daemon processes in the background that provide services.
  • kernel thread part of the kernel.
  • When a process is started from a shell, it becomes the child of that shell. When the shell stops all children are terminated.
  • PID Process ID. The name is represented between brakets [].

  • Priority is between 0 priority (-20 niceness) (maximum priority), 20 (0 niceness) (default) and 39 (19 niceness) (minimum). using -20 priority may have bad consequences.

  • & append to a command to run in the background.

  • jobs show running jobs and ID.
  • fg bring last job to foreground.
  • bg run last job in the background.
  • Ctrl+z stop currently running job.
  • Ctrl+d sends EOF to the current job indicating it should stop waiting for input.
  • Ctrl+c cancel current interactive job.
  • kill
  • killall
  • pkill
  • nohup
  • disown
  • ps snapshot of the current processes. ps -ef --forest or ps aux
  • pgrep
  • nice & renice
  • top
  • trap

Example to use jobs:

$ jobs
[1]- Stopped sleep 20
[2]+ Stopped vi

$ fg %v
vi

$ bg %?eep
sleep

$ fg %2
vi

Process signals

Some process signals; man 7 signal for more information: - SIGTERM (15) ask a process to stop - SIGKILL (9) force stop a process - SIGHUP (1) hung up a process. Some processes reload their configuration this way. - SIGUSR1 (10) custom user signal. Some processes reserve this signal for custom actions.

Process states

  • Running R - Active and using CPU or in the queue.
  • Sleeping S - Waiting for an event to complete.
  • Uninterruptible sleep D - Sleep state that can not be stopped.
  • Stopped T - Process stopped.
  • Zombie Z - Process stopped but could not be removed from memory by it's parent.

More info - kill may be a shell builtin in some cases. You may have it installed at /usr/bin/kill.

Files

  • /etc/profile, ~/.profile and ~/.bash_profile processed upon login.
  • /etc/bashrc and ~/.bashrc processed when subshells start.
  • /etc/motd is displayed after login.
  • /etc/issue is displayed before login.
  • /etc/passwd user information. Fields:
    • Username
    • Password (empty as /etc/shadow is used)
    • UID
    • GID
    • Comment
    • Home directory
    • Shell
  • /etc/shadow hashed user passwords. Fields:
    • Login name
    • Encrypted password
    • Days since epoch from last password change
    • Days before password may be changed
    • Days after which password must be changed
    • Days before password is to expire that user is warned
    • Days after password expires that account is disabled'
    • Days since epoch that account is disabled
    • Reserved field for future use
  • /etc/group groups. Fields:
    • Group name
    • Group password
    • GID
    • Members (user1,user2,user3)
  • /etc/gshadow group passwords
  • /etc/skel skeleton directory for new users
  • /etc/default

File system tree.

  • / root.
  • /boot files needed to boot the kernel.
  • /dev device files. Used to access physical devices. Created by the kernel.
  • /etc configuration files.
  • /home user(s) files.
  • /media and /mnt used to mount devices.
  • /opt optional packages installed.
  • /proc information about processes. Created by the kernel.
  • /root user root home.
  • /run information about user and processes from last boot. Created by the kernel.
  • /srv to serve files. Used by NFS, FTP and HTTP.
  • /sys files to interface with hardware devices. Created by the kernel.
  • /tmp temporal files. Deleted at boot.
  • /usr user usable programs and data. User System Resources.

  • /var files that variate in size. Logs, mail and spools.

  • /usr/share/doc documentation for larger software.

  • /proc/sysrq-trigger triger magic-sysrq commands.

  • /boot, /home, /usr and /var can be in separate file systems.

File system

An inode stores the data block where the file contents are stored; the creation, access and modification time; permissions and owners.

A hard link must be in the same file system; you can not create a hard link of a directory and when the last hard link to an inode is removed, the data is also removed.

A symbolic link (soft link or symlink (symbolic links)) link to a file name and no the inode. When the original file is removed the symlink becomes invalid.

symlinks of directories are files. If you remove link/ instead of link you risk deleting all the contents of the linked directory.

Consider the following situation
mkdir original ; touch original/file
ln -s original link
ls link/file
Remove the link. This will not delete the files inside `original` 
rm link
Remove the *symlink* as a folder instead of a file. 
u@h ~> rm link/
rm: cannot remove 'link/': Is a directory
u@h ~> rm -r link/
rm: cannot remove 'link/': Is a directory
u@h ~> rm -rf link/
u@h ~> ls -lA *
lrwxrwxrwx 1 yu yu  8 Jul  3 18:14 link -> original

original:
total 0
drwxr-xr-x 2 yu yu 40 Jul  3 18:15 .
drwx------ 3 yu yu 80 Jul  3 18:14 ..

Permissions

  • SUID - u+s - only on files - user executes file with permissions of file owner.
  • SGID - g+s - on files executes file with permissions of owner group; on directories files created in directory get the same group owner.
  • Sticky bit - +t - only folders - prevents users from deleting files from other users.

ACLs

Default ACL: permissions for all new items.

Mask

mask of 022 gives 644 for files and 755 for folders. Defaults to 666 for files and 777 for folders.

value files directories
0 6 RW 7 Everything
1 6 RW 6 RW
2 4 RO 5 RX
3 4 RO 4 RO
4 2 W 3 WX
5 2 W 2 W
6 0 - 1 X
7 0 - 0 -

Extended attributes

man chattr

  • A when a file is accessed, its atime record is not modified.
  • a the file can only be opened in append mode for writing.
  • c automatically compressed on the disk by the kernel.
  • i sets the immutable bit (cannot be modified).
  • s when a file is deleted, its blocks are zeroed and written back to the disk.
  • u when a file is deleted, its contents are saved.

Interfaces

  • console as in the old CRTs connected to the mainframe or your computer. Hardware devices that allows basic IO.
  • terminal or TTY is a device that does more than basic IO (read and write data). Now we use Pseudo Terminals like a terminal emulator; xterm for example.
  • shell or prompt is a program that processes commands. bash for example.
  • TTY means TeleTYpewriter
  • chvt change foreground virtual terminal

Tasks

Cron

  • crond is the service. There are multiple packages that provide crond.
  • crontab is used to show (-l) and edit (-e) the crontab for the current user or another user (-u username).
  • /var/spool/cron the location of all user crontabs. Always edit with crontab.
  • /etc/crontab system-wide crontab.
  • /etc/cron.hourly directory that contains executable files (not crontab files), like shell scripts, that get executed every hour. Also exist versions that run daily, weekly and monthly. They do not run at a fixed time.
  • For /etc/crontab you have to specify the user name.
  • /etc/cron.d location of custom crontab files. Drop your here instead of editing /etc/crontab.
  • /etc/cron.allow file that contains user names (one by line). Only listed users can use cron.
  • /etc/cron.deny file that contains user names (one by line). All users of the system can use cron except those listed in the file.
  • If neither cron.allow nor cron.deny exist only root can use cron.

Example of /etc/crontab

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name command to be executed
* */4 10-20 1,2,3,4 sat,sun user-name command to be executed
  • * every single one
  • */n once every n being n a number
  • n-m from n to m being n and m numbers
  • n,m,j,k,l at n, m, j, k and l

anacron

If the computer is off when a cron task should have been execute it does not get executed. With anacron it makes sure all tasks are executed when the computer turns on again. The main and only file is /etc/anacrontab.

man 8 anacrontab and man 5 anacrontab

Example /etc/anacrontab. Pretty self explanatory.

#period in days   delay in minutes   job-identifier   command
1       5       cron.daily              nice run-parts /etc/cron.daily
7       25      cron.weekly             nice run-parts /etc/cron.weekly
@monthly 45     cron.monthly            nice run-parts /etc/cron.monthly

at

  • atd is the daemon that runs the jobs.
  • at is the command line utility to schedule a job.
  • atrm is the cli tool used to remove a job.
  • atq query currently queued jobs. Same as at -l.
  • batch executes commands when system load levels permit; when the load average drops below 0.8, or the value specified in the invocation of atd.

man 1 batch, man 1 at and man 8 atd

Some examples

at now +5 minutes -f /path/to/script
echo "curl http://127.0.0.1/api/test" | at 14:22
echo "reboot" | at teatime
echo "logger hello world" | batch
atq
atrm 2

Logging

Location of logs

  • /var/log/messages common log file.
  • /var/log/dmesg kernel log messages.
  • /var/log/secure authentication-related messages.
  • /var/log/boot.log related to system startup.
  • /var/log/audit/audit.log audit messages like SELinux.
  • /var/log/maillog mail-related messages.
  • /var/log/journal/$(cat /etc/machine-id) journald messages (not stored in plain text) when local storage (Storage=auto for example) is enabled for journald. More info in man 5 journald.conf

Software

  • journalctl Query the systemd journal. journalctl -f to follow.
  • logger enter messages into the system log
  • dmesg print or control the kernel ring buffer. dmesg -w to follow.
  • logrotate rotates, compresses, and mails system logs

Explaining a log line

There are two RFCs for logging that container in-depth information on formatting: RFC5424 "The Syslog Protocol" and RFC3164 "The BSD syslog Protocol" 

Jul 31 20:55:44 computer sudo[46711]: pam_unix(sudo:session): session closed for user root
  • Jul 31 20:55:44 date and time
  • computer hostname
  • sudo[] process name
  • 46711 process id
  • pam_unix(sudo:session): session closed for user root message content

journalctl commands

journalctl -xe
journalctl -f
journalctl --no-pager
journalctl _UID=1000
journalctl -n 20
journalctl -p err.
journalctl --since yesterday
journalctl -o verbose
journalctl --dmesg
journalctl _SYSTEMD_UNIT=sshd.service
journalctl -p 3 -xb

Partitioning schemes

Device types

  • /dev/hda IDE.
  • /dev/sda SCSI and SATA. Also used for virtio (qemu/kvm) devices.
  • /dev/nvmeXnY NVMe where X is the device and Y is the partition. Y is normally 1.
  • /dev/xvda Xen virtual disk.

Partition types

  • 82 Linux Swap
  • 83 Linux
  • 8e Linux LVM

File systems

  • XFS
  • ext4
  • ext3
  • ext2
  • BtrFS
  • NTFS
  • VFAT

How to know the file system used in a partition?

  • df -T (only for mounted file systems)
  • sudo file -sL /dev/sda1 (too much information)
  • sudo lsblk -f
  • sudo blkid
  • sudo parted /dev/sda p

mount

man mount

mount /dev/sda /mnt
mount -t vfat /dev/sda /mnt -o uid=1000,gid=1000
mount UUID="..." /mnt -o ro,nosuid
mount LABEL="My Drive" /mnt/drive

fstab

  • Dump: set to 1 to enable the user of the dump utility. Used for some backup tools.
  • Check: 0 to disable (do not enable for network file systems); 1 to enable check on boot for the root file system and 2 to enable it for other file systems.

Some examples:

Device              Mount point     FS type     Mount options   Dump    Check
------------------------------------------------------------------------------
/dev/mapper/root    /               ext4        rw,noatime      0       1
UUID=...            /               ext4        defaults        0       2
tmpfs               /tmp            tmpfs       size=2G         0       0
/path/to/swap       swap            swap        defaults        0       0
192.168.0.1:/nfs    /mnt            nfs         noauto,user     0       0

LVM

  • LVM means Logical Volume Manager
  • Dynamic partitions:
  • Snapshots:
  • physical extent size:
  • building blocks:
  • device mapper:

Consists on:

  • Physical volumes (PV): a disk like /dev/sda. Mark the disk to be used in LVM
  • Volume group (VG): abstraction of all available storage. A bunch of disks grouped together.
  • Logical volume (LV):

Use LVM:

  • Create a partition.
  • Set the partition type to 8e00. [is this redundant?]
  • pvcreate to makes a disk usable in LVM. [is this redundant?]
  • pvs or pvdisplay to display current physical volumes.
  • vgcreate vgname /dev/sda1 to add a physical volume (/dev/sda1) to a volume group (vgname).
  • vgs or vgdisplay to display current volume groups.
  • lvcreate -n lvname -L 5G vgname to create a logical volume (lvcreate) named lvname inside the volume group vgname with a size of 5G.
  • lvs or lvdisplay to display current logical volumes.
  • Create a filesystem in the drive mkfs.ext4 /dev/mapper/lvname

  • Mount and use it.

  • You can add all partitions in a disk to a volume group with vgcreate vgname /dev/sda.

  • You can remove any of them with pvremove, vgremove and lvremove.
  • The actual logical volumes devices are /dev/dm-X. Other representations are just symlinks.
  • You can find logical volumes in /dev/mapper/ named like vgname-lvname.
  • You can also find logival volumes in /dev/vgname/lvname.
root@host:~# ls -l /dev/dm-0 
brw-rw---- 1 root disk 253, 0 Aug  8 20:35 /dev/dm-0
root@host:~# ls -lA /dev/mapper/
lrwxrwxrwx 1 root root 7 Aug  8 20:35 vgname-lvname -> ../dm-0
root@host:~# ls -l /dev/vgname/lvname 
lrwxrwxrwx 1 root root 7 Aug  8 20:35 /dev/vgname/lvname -> ../dm-0
  • You can extend a volume group with vgextend and reduce it with vgreduce.
  • For logical volumes you have lvresize, lvextend and lvreduce. You also have to resize the filesystem.

Kernel management

  • tainted kernel: kernel that contains closed source drivers.

  • modules are drivers. modules can be loaded and removed while running.

  • uname

  • procinfo
  • modprobe
  • journalctl -k and journalctl --dmesg shows clock time.
  • systemd-udev loads the appropriate drivers and udev rules at boot.
  • udevadm

Kernel modules

  • /etc/modules-load.d and /usr/lib/modules-load.d
  • lsmod
  • modinfo
  • modprobe
  • insmod and rmmod they do not load dependencies.
  • lspci -k show drivers loaded for each device.
  • /etc/modprobe.d

Boot

Secuence:

  • POST
  • UEFI/BIOS selects the bootable device
  • Bootloader
  • Kernel
  • Init -> initrd.target
  • Root File System
  • Default target

GRUB2

  • vmlinuz:
  • initramfs: contains drivers needed to boot.
  • dracut: creates a initramfs image.
  • entry:
  • kernel boot parameters: man 7 bootparam

Files and folders:

  • /etc/default/grub GRUB configuration file
  • /etc/grub.d/ scripts to configure entries.
  • /usr/lib/dracut/dracut.conf.d system default configuration.
  • /etc/dracut.conf.d custom configuration files.
  • /etc/dracut.conf main dracut configuration file.

Commands:

  • Reconfigure GRUB: grub-mkconfig -o /boot/grub/grub.cfg
  • Install GRUB: grub-install /dev/sda

GRUB prompt

  • Enter pressing e in the boot menu.
  • rd.break stops the boot procedure.
  • init=/bin/sh changes init for a shell.
  • systemd.unit=emergency.target changes the target to load.

Recover root password

  • Pres e on GRUB
  • Ennter rd.break and press Ctrl + x to boot
  • mount -o remount,rw /sysroot
  • chroot /sysroot
  • passwd
  • If you have SELinux run chcon -t shadow_t /etc/shadow
  • Reboot

Time and date

  • Hardware clock refferes to the same as Real-time clock.
  • Usually the time in the HW clock is in UTC.
  • Epoch time is the number of seconds since January 1, 1970 in UTC.
  • Hardware clock is the hardware clock that resides on the main board of a computer
  • System time time maintained by the OS. Read from HW on boot.
  • Software clock is similar to system time
  • UTC is Coordinated Universal Time
  • DST is Daylight saving time
  • Local time time of current time zone
  • Stratum is the reliability of an NTP time source, the lower the better. 1 or 2 is common.

Commands

  • Enable NTP: timedatectl set-ntp 1

  • Seconds from Epoch to human readable date: date --date '@1420987251'

  • Manage local time: date

  • Current system time: date
  • Current system day of month, month, and year: date +%d/%m/%y

  • Set the current time: date -s 12:45

  • Manage hardware time: hwclock

  • Synchronize current system time to the hardware clock: hwclock --systohc

  • Synchronize current hardware time to the system clock: hwclock --hctosys

  • Manage time: timedatectl

  • Shows current time settings: timedatectl status
  • Sets current time: timedatectl set-time TIME
  • Sets current time zone: timedatectl set-timezone ZONE
  • Shows a list of all time zones: timedatectl list-timezone
  • Change the RTC from/to local time from/to UTC: timedatectl set-local-rtc [0|1]
  • Disable or enable NTP: timedatectl set-ntp [0|1]

  • Set the local time: ln -sf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

  • Change the time zone: tzselect

Locales

Files are parsed one after another in the order of "C" locale.

Locales also specify the order or files.

firewalld

  • firewalld service that can configure firewall rules by using different interfaces.
  • Uses nftables or iptables
  • Applications can request ports to be opened using DBus A zone is a collection of rules that are applied to incoming packets matching a specific source address or network interface. Firewalld applies to incoming packets only by default, and no filtering happens on outgoing packets Firewalld service specifies what exactly should be accepted as incoming and outgoing traffic in the firewall. It typically includes ports to be opened, as well as supporting kernel modules that should be loaded. Behind each service is a configuration file that explains which UDP or TCP ports are involved and, if so required, which kernel modules must be loaded. To get a list of all services available on your computer, you can use the command firewall-cmd --get-services Default (RPM installed) XML files are stored in /usr/lib/firewalld/services. Custom XML files can be added to the /etc/firewalld/services directory

Tools

  • CLI: firewall-cmd
  • GUI: firewall-config

Show commands

  • Show available zones: firewall-cmd --get-zones
  • Show current default zone: firewall-cmd --get-default-zone
  • Set default zone: firewall-cmd --set-default-zone=<zone>
  • show available services (all services): firewall-cmd --get-services
  • Show services available in the current zone (in use): firewall-cmd --list-services
  • Show a lot more information: firewall-cmd --list-all
  • Show a lot more information about a specific zone: firewall-cmd --list-all --zone=public

Add commands

  • Temporally add the service "vnc-server" to the default zone: firewall-cmd --add-service=vnc-server
  • Temporally add the service "vnc-server" to the specified zone: firewall-cmd --add-service=vnc-server --zone=<zone>
  • Permanently add a service to the default zone: firewall-cmd --add-service vnc-server --permanent
  • Permanently add a service to the specified zone: firewall-cmd --add-service vnc-server --zone=<zone> --permanent

Remove commands

  • Remove the service from the default zone: firewall-cmd --remove-service vnc-server
  • Remove the service from the default zone: firewall-cmd --remove-service vnc-server [--zone=<zone>] [--permanent]
  • This does not apply it to the currently running firewall; you have to reload the configuration: firewall-cmd --add-port=2022/tcp --permanent
  • This does not apply it to the currently running firewall; you have to reload the configuration: firewall-cmd --remove-port=2022/tcp --permanent
  • Add an interface to the defatul zone: firewall-cmd --add-interface=<inferface>
  • Remove an interface from the defatul zone: firewall-cmd --remove-interface=<inferface>
  • Add a specific IP adress: firewall-cmd --add-source=<ipaddress/netmask>
  • Removes a specific IP adress: firewall-cmd --remove-source=<ipaddress/netmask>

Reload commands

  • Write current configuration to disk: firewall-cmd --permanent
  • Reload the configuration: firewall-cmd --reload
  • Reload the configuration: systemctl restart firewalld