Posts
Hardware:
Detectable Hardware:
HAL(Hardware Abstraction Layer)
/sys/block -> hard drive block devices
/sys/class -> device classes
/sys/devices -> hierarchy of detected devices
/sys/firmware -> drivers
/sys/modules -> loaded kernel modules
Dynamic Settings in /sys/proc. Its a virtual file system as it is created and loaded when linux boots
cat /proc/cpuinfo -> CPU Information
Network forwarding enabled? /proc/sys/net/ipv4/ip_forward
-> Boolean: 1=enabled, 0=disabled
cat /etc/sysctl.conf -> Configuration file for setting system variables
cat /proc/modules -> loaded kernel modules
cat /proc/mounts -> mounted directories and their filesystems
cat /proc/meminfo -> information on memory allocation
ls /proc/ -> contains numbered directories for each process id, inside those directories are details about the process.
Hardware Info:
lsusb -> Connected USB devices
lspci -> Connected internal hardware
lsmod -> List installed and loaded drivers
Mass Storage Device Files:
/dev/sda1 : attached to primary SATA/SCSI cable as master drive on 1st partition
/dev/sdb1 : attached to primary SATA/SCSI cable as slave drive on 1st partition
/dev/sdc1 : attached to secondary SATA/SCSI cable as master drive on 1st partition
/dev/sdd1 : attached to secondary SATA/SCSI cable as slave drive on 1st partition
Logs:
cat /var/log -> contains all system logs
cat /var/log/apache2/access.log
cat /var/log/apache2/error.log
cd /var/log/mysql
cat /var/log/mysql.err etc
cat /var/log/messages -> boot event logs
cat /var/log/debug -> system debug logs
Process Management:
Linux is filled with services which are managed through scripts in etc/init.d directory.
These scripts may be started, stopped, reloaded, etc depending on current runlevel.
The root user can run scripts in /etc/init.d directory.
Most scripts start with "K" (Kill) and "S" (Start). The parameter "stop"/"start" is passed to the scripts invoked by K or S named scripts.
Runlevels:
Standard runlevels - 0,1,2,3,4,5,6,S ...etc
shell > init <run_level_number
shell > telinit <run_level_number>
Runlevel 0 -> halt/shutdown the system
run scripts in /etc/rc0.d (most scripts start with name "K" --> Kill, "S" -> Start)
ls /etc/rc0.d -l
Runlevel 1 -> single user mode (root user without password -> very dangerous)
run scripts in /etc/rc1.d
ls /etc/rc1.d -l
Runlevel S -> single user mode
does NOT run scripts in /etc/rc1.d
Runlevel 6 -> reboot
run scripts in /etc/rc6.d
ls /etc/rc6.d -l
Runlevels 2-5 are multi-user.
aliases: init 6 => shutdown -r now
init 0 => shutdown -h now
FileSystem:
Filesystem, Volumes and Directories:
shell > df
- Local Filesystem -> Formatted partition, volumn, RAID Array
- A file system is mounted on a directory
- A file system is always mounted on /
Directory Mount Points:
* - can be kept/advisable to keep on dedicated partitions
/bin - basic executable files
/lib - program libraries (/usr/lib too)
/boot* - linux kernel, GRUB
- Common practice to put it on its own dedicated partition of 100-200 MB.
This isolation helps protect the contents of the boot directory including the linux kernel and initial RAM disk from the regular file system.
- Do not mount on a logical volume (coz if logical volume is corrupted, one would not be able to boot the system)
/home* - user home directories
- Facilitates backups. Should be mounted on a separate filesystem(any of partition/volume/RAID) in order to be able
to upgrade to any linux distributions witho loosing personal data.
/etc - most system-wide configuration files accessed during boot process and more
/sbin - system binaries
/dev - hardware and software devices
/media - standard mount point for removable media, its a successor to the /mnt directory which is sometimes still used (automounter uses it)
/opt* - common dir for executables and 3rd party apps. Can be kept on dedicated partitions.
/proc - is a virtual file system which includes kernel parameters only when the system is running
/root - home directory of root user
/sbin - administrative commands
#User sub-directories with commands accessible to all
/srv* - for servers, e.g) Apache, FTP, etc
/tmp* - for temporary, user based GUI config data
/usr - commands, libraries
/var* - log files, FTP services (uploaded files), print spool files, server files (of apache, ftp services, etc..)
- Mounting it on a separate file system protects the system from being overloaded with large log, FTPed files.
These directories are "Mount" points for a given volume/partition/RAID Array
- Linux uses dedicated Swap Space for partition or logical volumes.
- RAID Arrays do not require swap space as redundancy is not an issue.
Shared Libraries:
- are programs, functions, routines
- usually in /lib and /usr/lib directories
ldd - identify library files associated with a program
shell > ldd /bin/ls
ldconfig - reads and caches currently installed libraries from /lib /usr/lib and directories associated with LD_LIBRARY_PATH env variable
- also reads and caches directories configured in /etc/ld.so.conf (it includes conf files from /etc/ld.so.conf.d/ directory)
ldconfig -p => display ALL libraries
Debian Packages:
- The packages have .deb extensions
To see details about passwd package
shell > dpkg -l passwd
To see all Debian based packages installed
shell > dpkg -l
List of files from passwd package
shell > dpkg -L passwd
To identify a package that owns a file
shell > dpkg -S /usr/bin/passwd
dpkg -i
dpkg -r
dpkg -P
dpkg may not work if there are dependencies. Dependencies are managed by apt-* commands.
apt-get -> acquire, download, install (also automatically include dependencies)
apt-cache -> search through repositories
aptitude -> user interface to apt-* commands
shell > apt-get install
shell > apt-get remove
shell > apt-get purge
shell > apt-get update => Installs available updates for all packages
The repositories used by apt and aptitude commands are configured in /etc/apt/sources.list
RPM - Red Hat Package Manager.
The files have .rpm extensions
Linux Commands:
Profile and Environment Variables:
- In bash shell, Shell Variables = Environment Variables
shell > env => To view all environment variables
For each user, the shell that will be used is specified in /etc/passwd file
shell > cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
Systemwide profile variables are configured in /etc/profile configuration file.
The file may refer to other files in /etc/profile.d/ depending on the linux distro.
User specific variables are specified in .bashrc, .bashrc_profile, .profile hidden files in users home directory.
Current Variables:
PS1: default command prompt
value of PS1: echo $PS1 (PS1 is a variable associated with the prompt)
PAGER: default text reader
setting a variable in bash shell: export PAGER = less
setting a variable in all other shells: set PAGER = less
unsetting a variable: unset PAGER
PATH: variable is defined in profile file and can be modified in run time. It may have diff values for a root user and a normal user.
HISTSIZE: size of history
History and Command Completion:
When a user logs out, the latest history is transferred to the ~/.bash_history hidden file
~ denotes the user's HOME folder
When user types first few letters of a command and hits "tab" key, the Shell checks directories in the PATH (echo $PATH) and if a matching
command is found, it is completed.
Basic Commands:
exec : this command overrides a shell process
eg) find . -name "pra.*" -exec rm {} \;
The above command first finds all the files that start with pic and for all these files, the exec command runs an 'rm' command.
Globbing:
Say if you have 100 files with names pra00.txt to pra99.txt then:
* wildcard is used for globbing all characters
? wildcard is used for globbing 1 character
[] is used for limiting conditions
! is used for negation
- List all files starting with name pra ==> ls pra*
- List all txt files starting with pra1 ==> ls pra1?.txt
- List all txt files starting with pra ==> ls pra??.txt
- List pra01.txt through pra39.txt ==> ls pra[1-3]?.txt
- List pra01.txt through pra39.txt ==> ls pra[1-3][1-9].txt
- List all files that dont include pra10 through pra39 ==> ls pra[!1-3]?.txt
File:
- List all files last changed ==> ls -t
- List all files with symbols ==> ls -F
symbols - * (executable), / (directories), @ (symbolic links), = (sockets), | (named pipes)
- List all file/directories with content type ==> file *
Archives and Compression:
tar creates/extracts archives
tar -czf home_bak.tar.gz ~
c - create
z - compress with gzip algorithm
f - use the given file name
tar cf home_bak.tar ~ --> creates an uncompressed archive file
gzip home_bak.tar --> compresses the file and adds a .gz extension, so it becomes home_baj.tar.gz
gunzip home_bak.tar.gz --> uncompress the file
cpio --> This command copies input and outputs an archive
eg) Find all doc files from current directory and archive and compress them
shell > find / -name *.doc | cpio -o > docfiles
extract them:
shell > cpio -i < docfiles
Basic Data Redirection:
3 streams of data
- Standard Input (stdin) > or 1>
- Standard Output (stdout) <
- Standard Error (stderr) 2>
Use >, 1>, 3>, <, etc for redirection
eg) ls > filelist.txt (output of ls command to a file)
ls >> filelist.txt (output of ls command appended to a file)
cd nonexistingdir 2> error.log (redirect errors to error.log file)
cd nonexistingdir 2>> error.log (redirect and append errors to error.log file)
xargs --> This command can help commands that cannot accept standard inputs from other commands
eg) delete all txt files
ls *.txt | xargs rm --> (Takes the list of txt files and applies rm command to them)
Text Filter Commands:
- cut, join (columns)
- split (divide by number of rows)
- sort, uniq (manipulates file contents)
Text Search:
shell > ls My Documents --> May not work so use the following to escape the space
shell > ls My\ Documents --> or
shell > ls 'My Documents'
'\' character can also be used for searching for special characters like *, ?, etc..
Back quotes('') can enclose commands.
Commands: grep, egrep, fgrep
egrep = grep -E (supports multiple search terms)
fgrep = grep -F (supports search of multiple files)
shell > grep bash /etc/passwd ==> Search for lines with the string 'bash' in /etc/passwd
shell > grep -v bash /etc/passwd ==> Search for the lines not containing the string 'bash' in /etc/passwd
shell > ls -la | grep pratik ==> Search for files having string 'pratik'
shell > grep -l pratik /etc/* ==> Search for files containing string 'pratik' in /etc directory
shell > grep -L pratik /etc/* ==> Search for files not containing string 'pratik' in /etc directory
shell > grep -i Bash /etc/passwd ==> Search for lines with the non-case-sensitive string 'bash' in /etc/passwd
shell > grep -l root /etc/*
/etc/aliases
/etc/aliases.db
/etc/bash.bashrc
/etc/ca-certificates.conf
/etc/complete.tcsh
/etc/crontab
/etc/ftpusers
/etc/group
/etc/group-
/etc/gshadow
/etc/gshadow-
/etc/inetd.conf
/etc/inetd.conf~
/etc/inetd.conz~
/etc/logrotate.conf
/etc/mailcap.order
/etc/passwd
/etc/passwd-
/etc/quotagrpadmins
/etc/securetty
/etc/services
/etc/shadow
/etc/shadow-
/etc/sudoers
/etc/warnquota.conf
Text Search with Special characters:
shell > grep "My Documents" < ls -l ==> Search for files containing the search term "My Documents" in the list of files
shell > grep bash < ls /etc/*
/etc/adduser.conf:DSHELL=/bin/bash
/etc/bash.bashrc:# System-wide .bashrc file for interactive bash(1) shells.
/etc/bash.bashrc:# enable bash completion in interactive shells
/etc/bash.bashrc:#if [ -f /etc/bash_completion ]; then
shell > grep \: /etc/passwd ==> Searches for ':' in /etc/passwd
shell > grep * ==> It takes list of all the files in current dir and uses the first file name as a search term in the text of all other
files that follow
shell > grep -e -pratik /etc/passwd ==> If searching for a search term starting with a dash ('-')
shell > egrep /bin/*sh /etc/passwd ==> Search for users with all kinds of shells
shell > fgrep pratik /etc/* ==> Search for 'pratik' in all files under /etc
Stream Editor:
sed - an editor for streams of text data
syntax: sed -switch 'action/term/sub/flag' filename
(An action where a term 'term' in a file, is substituted with the term 'sub' based on some flag and output to a file called filename')
shell > sed '20d' test1.txt ==> Deletes lines containing 20 from text1.txt
shell > sed '20,25d' test1.txt ==> Deletes lines containing 20 through 25 from text1.txt
shell > sed 'y/abcde/uvwxy' test1.txt ==> 'y' for yank, the letters a,b,c,d,e are replaced with u,v,w,x,y in the file
shell > sed 's/abcde/uvwxy' test1.txt ==> 's' for substitute, the letters 'abcde' are replaced with 'uvwxy' in the file
shell > sed 's/us.archive.ubuntu.com/mirrors.kernel.org' /etc/apt/sources.list > tempfile
Vi Editor:
Monitor Processes:
ps, top, uptime, free
top ==> lists processes, order by CPU and RAM use, includes swap space information
uptime ==> includes current runtime, number of users, load average
free ==> shows memory capacity, usage w.r.t RAM, Swap space
ps => lists processes in current shell
ps a => lists all running shells
ps -u pratik => lists that user's processes
ps aux => lists all running processes
ps -C command => lists processes by command/daemon
shell > ps -C apache2
pstree => lists processes in a hierarchy
shell > pstree
init-+-apache2-+-ApplicationPool-+-ruby
| | `-2*[{ApplicationPool}]
| `-3*[apache2]
|-cron
|-dovecot-+-dovecot-auth
| |-3*[imap-login]
| `-3*[pop3-login]
|-freshclam
|-master-+-pickup
| |-qmgr
| `-tlsmgr
|-2*[miniserv.pl]
|-mysqld_safe-+-logger
| `-mysqld---2*[{mysqld}]
|-nmbd
|-proftpd
|-saslauthd---4*[saslauthd]
|-smbd---smbd
|-sshd---sshd---bash---pstree
|-syslogd
`-xinetd
Kill:
Command signals for kill
Termination: signal 15 (TERM)
Restart: signal 1 (HUP)
Unclean stop: signal 9
Killall
shell > ps aux | grep smb ==> Searches for Samba process
shell > sudo kill -1 5394 ==> Restart samba process (PID = 5394)
shell > ps aux | grep apache2 ==> Searches for all apache process
shell > sudo killall apache2 ==> Kills all apache processes
Foreground and Background processes:
Create a background job by adding an '&'
eg) sleep 10000 & ==> The sleep process is created in background
shell > sleep 10000 &
[1] 31781
List background jobs with jobs command
shell > jobs
[1]+ Running sleep 10000 &
Get PID of background Job
shell > jobs -p
31781
Bring the background job to foreground
shell > fg
sleep 10000
Bring a foreground job back to background
shell > bg [jobnumber]
Keep a job running even after logout:
nohup /path/to/somescript
More on FileSystems:
Partitions:
1. Primary
2. Extended (conversion from a primary partition)
3. Logical (logical partitions have to be fully contained within that extended partition)
shell > fdisk -l ==> Review partition types, device files, boot flag (asterisk *), cylinders (start, end), ID (of partition)
shell > fdisk /dev/sdc
press 'm' for available commands
press 'p' to print configured partitions
press 't' / 'l' to list / change partition types
press 'n' to create a new partition
-> select primary/extended/logical partition
-> select partition number
-> select partition type, cylinder, specify size ( eg +200M )
-> accept default start cylinder
-> specify end cylinder or desied size e.g) 200M
-> 'p' to print configured partitions again
-> 't' to change the partition type if desired
-> 'w' to write changes to disk
Once saved, the new partition is ready for formatting, or converting to a logical volume or RAID array.
Filesystem Format:
Local filesystem is either -> formatted partition, logical volume, RAID Array
Standard Formats ->
-> ext2, ext3: 2nd, 3rd extended filesystems
-> ext3 = ext2 + journaling (minimises corruption in the event of a filesystem crash)
-> ext4 now available
Filesystem format commands /sbin/mk* + 2 times tab key
Filesystem Integrity Commands:
df -> to monitor capacity, free space on mounted file systems
It gives, fs_name, size, used, available, used %, mount point
du -> for space taken by directories and individual files that maybe overloading the system
fsck -> to check actual filesystem integrity
shell > du => lists all files usage
shell > du /isofiles/ => lists all files usage under a folder
shell > du 'find / -name "*.iso"' => first runs find command and finds all files under root with extension .iso and then feeds to du
fsck command:
- apply only to un-mounted filesystems else risk of filesystem damage
- automatically runs during boot process typically once per 30 mounts
- can minimise corruption issues
- fcsk -N for demo on filesystems in /etc/fstab
shell > dumpe2fs /dev/sda1 | less => to get details
Mounting:
#Mounting a newly created partition/volume/RAID on test directory
shell > mount /dev/sdb1 /test/
#downloaded cd/dvd .iso file
shell > mount -o loop ubuntu-9.4.iso /test => mount, loopback devices work with iso files
shell > umount /test => un-mount
shell > mount -o remount -o ro /dev/sda6 /etc/backups => remounting the filesystem with read only privileges
shell > mount -o remount -o rw /dev/sda6 /etc/backups => remounting the filesystem with read write privileges
shell > mount => listing mounts
Mount shared network directories:
Network File System:
shell > mount -t nfs ubuntuserver:/share /test
shell > umount /test
Samba:
shell > mount -t cifs //ubuntuserver/share /test => (cifs - common interface file system - standard for microsoft directory sharing)
shell > umount /test
Quotas:
File Permissions and Ownership:
chown:
shell > chown pratik test1 # change owner of test1 to pratik
shell > chown devgroup test1 # change group owner of test1 to devgroup
shell > chown pratik.devgroup test1 # change user and group simultaneously
shell > chown pratik:devgroup test1 # change user and group simultaneously - alternate syntax
shell > chown -R pratik:devgroup test1 # recursively change ownership of all files and subdirectories also along with changing ownership of this directory
shell > groups <user_name> #To check how many groups a user belongs
umask: sets default permissions when you create a new file or directory
shell > umask
0022
- First number is not used
- Last three numbers are substracted from 666 to get the default file permissions, in this case 644 (-rw-r--r--)
- No execute permissions on newly created files
Special File Permissions:
-Super User ID (SUID) - generally applied on root commands executable by regular users, this 's' bit is in place of executable bit of user
-Super Group ID (SGID) - supports file sharing with group ownership, this 's' bit is in place of executable bit of group
-Sticky Bit - supports file sharing with user ownership, this 't' bit is in place of executable bit of other
shell > ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 29K Dec 8 2008 /usr/bin/passwd*
shell > ls -l /home/shared
drwsr-sr-x 1 root root 29K Dec 8 2008 /usr/bin/passwd*
shell > ls -l /temp1
drwsrwxrwt
Implementation:
SUID - chmod u+s /path/to/script
SGID - chmod g+s /path/to/dir
Sticky Bit - chmod o+t /path/to/dir
Reverse process: u-s, g-s, o-t
Octal Format: SUID=4, SGID=2, Sticky=1
#First number in the command: shell> chmod 4755 test1.txt
Links:
Hard Link: Same inode, CANNOT span partitions, volumes and RAID Arrays
Soft Link: CAN span partitions, volumes and RAID Arrays
shell> ln -s
Identify System File Locations:
1. Find
shell > find /path -user
Also, -uid, -gid, -perm
2. Locate Command
The locate database is updated by script in /etc/cron.daily as per /etc/updatedb.conf
shell > locate searchterm
#Commands which focus more directly on system files
3. type - alias
shell > type ls
ls is aliased to `ls -hF --color'
4. which - alias + to find full path to command
shell > which ls
ls is aliased to `ls -hF --color'
ls is /bin/ls
5. whereis - full path + full path to associated man pages and other files
shell > whereis ls
ls: /bin/ls /usr/share/man/man1/ls.1.gz
Shells/Scripting & Data Management:
Environment Variables:
shell > cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
shell > cat /etc/profile
# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
if [ "$PS1" ]; then
if [ "$BASH" ]; then
PS1='\u@\h:\w\$ '
if [ -f /etc/bash.bashrc ]; then
. /etc/bash.bashrc
fi
else
if [ "`id -u`" -eq 0 ]; then
PS1='# '
else
PS1='$ '
fi
fi
fi
umask 022
User Specific Configuration Files:
Tre located in /etc/skel/. These files are copied to user's home dir when a new user is created.
shell > la /etc/skel
-rw-r--r-- 1 root root 220 May 12 2008 .bash_logout
-rw-r--r-- 1 root root 2.9K May 12 2008 .bashrc
-rw-r--r-- 1 root root 586 May 12 2008 .profile
Aliases:
Functions:
- Similar to aliases
- eg) pst () { ps; top; } #pst function runs, ps command and then top command
Basics of Shell Scripts:
eg)
shell > cat /etc/crontab
shell > cat /etc/profile
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
Operator Switches:
For if, for, test, and more script commands
-d looks for a directory
-e checks for existence
-f looks for a regular file
-r checks for read permission
-w looks for write permission
-x inspects for execute permission
-eq checks for equality
-ge greater than or equal to
|| is the previous expression false?
&& is the previous expression true?
i as a variable in a for list
=> for n in 1 2 (just 1 and 2)
=> for n in 'seq 10' (from 1 to 10)
#! => 'shebang' character
#! /bin/sh => specifies the shell to be used
shell > cat /etc/cron.daily/logrotate
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0 => (if the executable file /usr/sbin/logrotate is not found then exit)
/usr/sbin/logrotate /etc/logrotate.conf => (run /usr/sbin/logrotate based on config in /etc/logrotate.conf)
Administrative Tasks:
Local Authentication Files:
/etc/passwd, /etc/group #Users and Groups
/etc/shadow, /etc/gshadow #Passwords
shell > cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
username:password('x' refers to /etc/shadow):UserID:GroupID:UserInfo:HomeDir:LoginShell
shell > cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
groupname:password('x' refers to /etc/gshadow):GroupID:UserMembers
shell > cat /etc/shadow
root:hkNDZjs.RQlk2:14749:0:99999:7:::
daemon:*:14518:0:99999:7:::
bin:*:14518:0:99999:7:::
username:password(encrypted):days since last pwd change:minimum pwd lifetime:max pwd lifetime:warn period:account exp period:disable date in days
shell > cat /etc/gshadow
root:*::
daemon:*::
bin:*::
groupname:group password(not required):group admin users:group members(users)
Group Management:
commands: groupadd, groupdel, groupmod
shell > groupadd newgroupname #Creates a new group in /etc/group
shell > groupadd -g 1000000 newgroupname #Creates a new group in /etc/group with specified group id
shell > groupmod -g 2000000 groupname #Modifies groupid of group
shell > groupmod -n newgroupname groupname #Modifies group_name of group
shell > groupdel groupname #Deletes group
User Management:
commands: useradd, userdel, usermod, chage
Create New User:
- Home directory default: /home/newuser
- Copy files from /etc/skel
shell > useradd newusername
Options for useradd and usermod:
-c : comment in /etc/passwd
-d : non-standard home directory
-e : account expiration date
-g : different standard group id (or group name)
-G : additional groups, by GID
-s : different login shell
-u : different user id number
Lock, Unlock user account:
shell > usermod -L username
shell > usermod -U username
shell > cat /etc/passwd |grep "/bin/bash" |grep "[5-9][0-9][0-9]" |cut -d: -f1 #Lists physical users in system
shell > cat /etc/passwd | cut -d: -f1 #Lists users in system
shell > awk -F":" '{ print "username: " $1 " uid:" $3 " guid: " $4 }' /etc/passwd #Lists all users in system with userid
shell > w #Lists all logged in users
shell > who #Lists all logged in users
shell > users #Lists all logged in users
shell > passwd
options for chage:
-d : sets when password was last changed
-I : sets inactive date
-E : sets expiration date (-1 disables expiration)
-M : sets max days between password changes
-W : sets warn days before expiration
shell > userdel
shell > userdel -r username
Cron Jobs:
1. Administrative cron jobs
2. User defined cron jobs
Administrative crons are defined in /etc/crontab.
The columns are: Minute, Hour, Day of Month, Month, Day of Week, User, Command
eg)
#Run cron.hourly 17 minutes after every hour
17 * * * * root cd / && run-parts --report /etc/cron.hourly
#Run cron.daily everyday at 6:25
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
#Run cron.weekly every Sunday (0 or 7 - Sunday, 1 - Mon, 2 - Tue, etc)
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
#Run cron.monthly on 3rd of every month at 6:52 AM
52 6 3 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#Run tempdelete.pl everyday at 17:56
56 17 * * * /etc/webmin/cron/tempdelete.pl
#Run spamconfig.pl every 21 minutes
21 * * * * /etc/webmin/virtual-server/spamconfig.pl
User Defined Cron:
shell > crontab -e #Opens a textfile in /var/spool/cron/ or /var/spool/cron/crontab/
shell > crontab -e
Usage of cron can be crontolled by user accounts
- If /etc/cron.allow exists, only users listed in file allowed to use crontab and /etc/cron.deny is ignored.
- If only /etc/cron.deny exists, users in this file not allowed to use crontab
Timezone:
Defaut time zone is configured in /etc/timezone
shell > tzselect #To configure timezone
add the following line to /etc/profile
TZ='America/Chicago'; export TZ
System Services:
System Logs:
Logfiles are driven by syslogd and klogd commands. Configuration files /etc/syslog.conf
Most logs collected in /var/log. Some services configured logs in /var/log/apache2, /var/log/samba
7 Message Security Levels
- debug (least severe), info, notice
- warning (warn), err (error)
- crit (critical), emerg (panic - most severe)
Basic Log Configuration:
auth,authpriv.*, security /var/log/auth.log
*.* (standard);auth,authpriv.none /var/log/syslog
cron.* /var/log/cron.log
daemon.* /var/log/daemon.log
kern.* /var/log/kern.log
mail.* /var/log/mail.log (Mail Server Messages)
user.* /var/log/user.log (Hardware Detection)
mail.info /var/log/mail.info
mail.warn /var/log/mail.warn
mail.err /var/log/mail.err
# Some `catch-all' logfiles.
*.=debug;\
auth,authpriv.none;\
news.none;mail.none -/var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none -/var/log/messages
# Emergencies are sent to everybody logged in.
*.emerg
Mail Transfer Agent:
qmail - simple replacement for sendmail (not open source)
Postfix: default MTA for Ubuntu, TLS(Transport Layer Security) successor of SSL(Secure Sockets Layer)
- options for: Mailbox, Virtual domain
- config file: /etc/postfix
sendmail - open source variant of Sendmail
- Huge config files in /etc/mail, uses macro files like sendmail.mc and submit.mc
Networking Fundamentals:
Common TCP/IP Ports and Protocols:
shell > cat /etc/services
port 53 - DNS Services
port 139 - NetBIOS SSN (essential for communication with microsoft systems)
port 143 - port for IMAP (Novell Evolution, Ms Outlook) (Internet Message Protocal, alternative to POP3)
port 161 - SNMP (Simple Network Management Protocol)
port 443 - HTTPS
port 993 - Secure IMAP
port 995 - Secure POP3
ftp-data 20/tcp #Enables file transfers to and from ftp clients
ftp 21/tcp
ssh 22/tcp
telnet 23/tcp
smtp 25/tcp
nameserver 42/tcp
www 80/tcp
www 80/udp
pop3 110/tcp
pop3 110/udp
auth 113/tcp
mailq 174/tcp # Mailer transport queue for Zmailer
mailq 174/udp # Mailer transport queue for Zmailer
log-server 1958/tcp # remstats log server
mysql 3306/tcp
mysql 3306/udp
svn 3690/tcp subversion # Subversion protocol
svn 3690/udp subversion
Firewall may be configured to allow access.
Basic Network Commands:
host, ping, dig, traceroute, tracepath
The ping message is associated with the ICMP protocol, some servers block ping messages.
shell > dig www.google.com #For more DNS info
Network Configuration Files:
shell > cat /etc/hostname #Hostname of local system
abc.railsbook.net
shell > hostname
abc.railsbook.net
shell > cat /etc/hosts
127.0.0.1 localhost.localdomain localhost
# Auto-generated hostname. Please do not remove this comment.
142.55.35.54 abc.railsbook.net imageupdate abc
shell > cat /etc/nsswitch.conf #Beyond networking
#password authentication database
passwd: compat
group: compat
shadow: compat
#Look for hostnames first in /etc/hosts then DNS
hosts: files dns
#Look for local files for network configuration
networks: files
shell > cat /etc/resolv.conf
nameserver 42.64.98.221
nameserver 289.67.20.21
#Static route, ip-addresses or network mass is configured in /etc/network/interfaces
Network Status Commands:
ifconfig, route, ifup, ifdown,
iwconfig, iwlist (for wireless)
route command lists the current routing tables.
shell > route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
24.0.24.2 * 255.255.255.255 UH 0 0 0 venet0
default 56.0.3.1 0.0.0.0 UG 0 0 0 venet0
Network Troubleshooting:
Systematically check network, step by step:
1. ifconfig -a # Check network card detection, lists all cards currently on system. If active they are listed with the word "UP"
2. ping 127.0.0.1 # Check loopback adapter, network software status
3. ping local_ipaddress (ip address of network card) # Make sure address is bound to network card. The ip of network card is found by 'ifconfig -a'
4. ping one remote_ipaddress (from routing table, say IP of gateway) # Check communication on local network,can find remote ip_address with 'route -n' or 'netstat -nr'
5. ping one external_ip_address # Check for connection with DNS server
-eg) ping DNS server mentioned in /etc/resolv.conf
6. traceroute google.com # Check for connection to external network
ping google.com
Troubleshooting Network and Client DNS:
Open ports are shown with netstat
- specific ports: netstat -atun
DNS Client Management:
Client DNS Commands-
host google.com # Lists IP Addresses, email servers
host -v google.com # Gives more information
host -v google.com 195.5.5.4 # Takes information from a root DNS server (cross checking with this DNS server)
dig @195.5.5.4 google.com
General Security:
Sudo:
- Give users sudo privileges, never login with root user
change /etc/sudoers file
eg)
%admin ALL=(ALL) ALL #Members of the admin group may get root privileges
#Power users
%users ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom #Allows the members of 'users' group to mount/umount the cdrom as root
%users localhost=/sbin/shutdown -h now #Allows members of the users group to shutdown this system
sample sudoers file: http://www.sudo.ws/sudo/sample.sudoers, http://www.webune.com/forums/20100328cbtz.html
Port Security:
netstat, nmap, lsof
shell > netstat -atun #gives list of open local ports
nmap : unauthorised use of nmap on someone else's system, even on your ISPs system can get you sued. Limit nmap use to your own system.
shell > nmap localhost # Shows open ports with associated services
shell > lsof -i # Lists open network files/commands. Pipe to grep to identify specific service. lsof -i | grep apache
Resource Limits:
are associated with Pluggable Authentication Modules (PAM) implemented via pam_limits.so
configuration file: /etc/pam.d/* files. They use /etc/security/limits.conf file (can also limit user logins with soft/hard limit)
shell > ulimit -a # Lists default SOFT resource limits
shell > ulimit -aH # Lists default HARD resource limits
Different defaults can be specified in /etc/profile
File Audits:
Check for files/scripts with elevated permissions like ones having SUID or SGID permissions
shell > find / -perm /u+s # Find files with SUID privileges
shell > find / -perm 4755 # Find files with SUID privileges (with permissions 755=rwx-xr-x
A rouge script with these permissions can be a trouble.
Unused Services:
- Stop, uninstall unused services
- Stop service with /etc/init.d/ script
- Then make sure that service does not start on next boot
eg)
shell > /etc/init.d/portmap stop #Stopping portmap will prevent attacks from some NFS and NIS ports.
- The services that start when you boot a system is based on the default runlevel
- The default runlevel is specified in /etc/inittab
shell > update-rc.d -f
shell > ls /etc/rc2.d/
TCP Wrappers:
- Provides another layer of security for those services which communicate with TCP packets
- Limited to TCP (not UDP (video, audio), or ICMP (ping))
- To identify TCP Wrapper Services run
shell > lsof /lib/libwrap.so.0 (or /usr/lib/libwrap.so.0)
Configuration file: /etc/hosts.allow and /etc/hosts.deny
AppArmor, iptables can add more security
TCP Wrappers file format:
Daemon : Clients : Command
eg) in hosts.allow
1. ALL:ALL
2. in.telnetd:.example.org # For telnet server service, this limits access for *.example.org domain, comp1.example.org can connect to telnet service)
3. ssh:192.176.3. # ssh is allowed for 192.176.3.* (the wild card is implied)
4. ssh:.example.org EXCEPT crack.example.org # Exceptions can be made
5. in.telnetd: 192.176.3. EXCEPT 192.176.3.7
# Commands are often used with ruleset. ex: if you have a rule associated with access attempts to an ftp server, you could add the spawn command, which allows
# the use of regular shell commands
6. vsftpd : ALL : spawn(some_command) &
# Similarly the twist command can send a message to the user who attempts to access the FTP server
7. vsftpd : ALL : twist /bin/echo "access denied"
Super Server:
-This special service can work with other servers, and can take 2 forms Regular(inetd.conf) and new(xinetd.conf)
-Services can be configured within
-Regular Super Server
-configured in /etc/inetd.conf
-TCP packets
-ports based on /etc/services
-Extended internet Super Server
-configured in /etc/xinetd.conf
-includes logging information
-control with /etc/init.d/xinetd script
-uses configuration files in /etc/xinetd.d
Secure Shell Server:
Configuration:
-/etc/ssh/sshd_config
-Avoid insecure protocol 1
-Port 22 (or something else)
-PermitRootLogin no
-X11 Forwarding yes (allows access to remote GUI tools)
-PubKeyAuthentication yes
Update changes with: /etc/init.d/ssh reload (if server is already running)
or using /etc/init.d/sshd reload
Passphrase and Authentication Agents:
DSA - Digital Secure Algorithm
RSA - Rivest, Shamir and Adleman (lastnames of developers who developed it)
(DSA keys = 1024 bits, RSA Keys range 768 <-> 2048 bits)
1. Create a private and public key
shell > ssh-keygen -t dsa
or
shell > ssh-keygen -t rsa -b 2048
#The default locations for the keys are in ~/.ssh directory
#Private Key: id_dsa or id_rsa
#Private Key: id_dsa.pub or id_rsa.pub
2. Safely Copy (SFTP/SSH Copy) public key to remote system
- Remote home directory with SSH server
- ssh-copy-id -i .ssh/id_rsa.pub remoteserver
- SSH Copy transmits and automatically appends public key to:
- .ssh/authorized_keys on the remote system
3. Assign appropriate permissions (if you give too many permissions, SSH dosent work)
chmod 700 ~; chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
4. Use SSH Client:
ssh pratiK@remotesystem
-password not required
-may be prompted for paraphrase
No comments:
Post a Comment