List of LINUX troubleshooting commands/tools part # 1

Generic troubleshooting tips

1) Use tail –f to watch log file in real time, advantage is simple you can spot error or warning message in real time.
tail –f /path/to/log/file
Example(s):
# tail –f /var/log/maillog

2) Use telnet command to see if you get response or not. Sometime you will also see some informative message:
telnet ip port

Example(s):
# telnet localhost 53
# telnet localhost 25

3) Make sure you can see PID of your service.
pidof service-name
cat /var/run/service.pid
Example(s):
# pidof sshd
# cat /var/run/sshd.pid

4) You need to make sure that your DNS server or third party DNS server (ISP) is accessible. This is an important step, as many network services depend upon DNS; especially sendmail/postfix or Squid etc for example. Run dig or nslookup. No timeout should occurred.

# dig your-domain.com
# nslookup gw.isp.com
# more /etc/resolv.conf

5) For networking troubleshooting, make sure your ip address configuration is right, gateway, routine, hostname etc all configured. Here is list of tools on RedHat Linux to verify or modify information:

Hostname verification or setup tools

• hostname : To get hostname of server.
• hostname –s : To get FQDN hostname of server
• more /etc/sysconfig/network : To setup hostname and networking can enabled or disabled.
• dnsdomainname : List or setup domainname.
• more /etc/hosts :Make sure at least localhost entry do exist.

Ethernet configuration tools

• ifconfig : To see running network card information.
• ifconfig eth0 up|down : To enable|disable network interface
• service network reload|restart|stop|start : To reload (after changed made in ip config file)|restart|stop|start network interface with all properties.
• route|netstat –rn : To print routing table
• ping ip-address : To see if host is alive or dead
• more /etc/modules.conf : To see your network card configuration alias for eth0 exists or not.
• lsmod : To list loaded modules (read as drivers), here you need to see that eth0 module is loaded or not, if not loaded then use insmod to insert (load) driver.
• dhclient : Dynamic Host Configuration Protocol Client, run this if your Ethernet card is not getting ip from DHCP box on startup; this command does by default shows useful information.

To see if service blocked because of access control

• iptables –n –L : To list all iptable rules; useful to see if firewall blocks service or not.
• service iptables stop|start : To start|stop iptables
• more /etc/xinetd.conf

OR

• more /etc/xinetd.conf/SERVICENAME = To list configuration of xinetd server. Again useful to see if firewall xinetd based security blocks service or not (xinetd includes host-based and time-based access control)
• more /etc/hosts.allow : To see list of hosts allowed to access service.
• more /etc/hosts.deny : To see list of hosts NOT allowed to access service. NOTE first TCP wrappers (hosts.allow|hosts.deny) checked and then xinetd-based access control checked.
• more /etc/path/to/application.conf : See your application configuration file for access control. For example smb.conf and many other applications/services got own access control list in application. You need to check that as well.

Read man pages of all above command for more details on syntax and usage.