Introduction
Network troubleshooting is fastest when performed layer by layer. Changing DNS while the interface is down adds confusion; prove each prerequisite before moving upward.
What you should be able to do after this lesson:
- Follow a repeatable network diagnostic sequence.
- Identify hardware, link, address, route, DNS, and service failures.
- Inspect modern and legacy persistent configuration.
- Use logs and packet paths to locate a fault.
- Recognize interference from NetworkManager or another manager.
Big Idea: Find the Smallest Failing Boundary
Do not ask only whether “the network” works. Test a sequence of increasingly broad boundaries:
device -> link -> local address -> local neighbor -> gateway -> remote IP
-> name resolution -> remote port -> application protocol
The first failed boundary determines the next command. This prevents unrelated changes and makes the result reproducible for another administrator.
1. Hardware and Link
lspci -nnk
ip link show
ip -s link show dev eth0
ethtool eth0
dmesg | grep -i -E 'eth|link|firmware'
Check that the expected driver is bound, the interface is up, carrier exists, and errors or drops are not increasing.
2. Addresses and Neighbors
ip addr show dev eth0
ip neigh show dev eth0
Confirm the address and prefix match the network. Duplicate addresses, wrong masks, and failed neighbor entries can all prevent local communication.
3. Routing
ip route
ip -6 route
ip route get 203.0.113.10
traceroute 203.0.113.10
mtr 203.0.113.10
Test the local address, gateway, and remote address in order. traceroute and mtr show where responses stop, but firewalls may hide hops without blocking final traffic.
4. Name Resolution
cat /etc/nsswitch.conf
cat /etc/resolv.conf
getent hosts example.com
dig example.com
dig @192.0.2.53 example.com
getent follows the system's Name Service Switch policy; dig queries DNS directly. A mismatch between them can point to /etc/hosts, NSS, caching, or resolver-manager behavior.
5. Transport and Service
ss -tulpn
nc -vz server.example 443
curl -v https://server.example/
Check whether the process listens on the expected address and port, then test from both local and remote hosts. Review host and network firewalls.
Persistent Configuration
Configuration paths differ:
- Debian-style files below
/etc/network/ - historical Red Hat-style files below
/etc/sysconfig/network-scripts/ - NetworkManager connection profiles
- systemd-networkd
.networkfiles
Determine which manager owns the interface before editing files:
nmcli device status
networkctl status
systemctl is-active NetworkManager systemd-networkd
Two managers attempting to configure the same interface can create intermittent behavior.
Logs
journalctl -b -u NetworkManager
journalctl -b -u systemd-networkd
journalctl -k -b
Match timestamps across DHCP, authentication, kernel, and service logs. Preserve the original error before restarting networking.
Legacy Access Files
/etc/hosts.allow and /etc/hosts.deny belong to TCP Wrappers. They affect only programs compiled with that support and are not a general firewall.
Guided Practice: Create a Failure Worksheet
Choose a reachable test service and fill in evidence rather than writing only “works” or “fails”:
| Boundary | Command | Expected evidence |
|---|---|---|
| Driver | lspci -nnk or ethtool -i <iface> |
expected driver bound |
| Link | ip -s link |
interface up, carrier, stable counters |
| Address | ip addr |
correct address and prefix |
| Neighbor | ip neigh |
reachable gateway MAC or IPv6 neighbor |
| Route | ip route get <ip> |
expected source, gateway, and interface |
| DNS | getent hosts <name> |
expected address through NSS |
| Port | nc -vz <host> <port> |
transport connection succeeds |
| Protocol | protocol client | valid application response |
Now introduce one safe lab failure, such as an incorrect resolver address in an isolated namespace or a stopped test listener. Record which boundaries remain healthy and which first fails. Restore the original state and repeat the same checks.
Troubleshooting Scenario
dig @192.0.2.53 example.test succeeds, but getent hosts example.test fails. Link, route, and UDP/TCP 53 are healthy.
Because direct DNS works, inspect /etc/nsswitch.conf, the effective /etc/resolv.conf, local resolver services, search domains, and caches. The failure lies in the system resolver path, not the authoritative DNS data or interface driver.
Exam Focus
- Know the location and purpose of
/etc/network/,/etc/sysconfig/network-scripts/,/etc/hosts,/etc/resolv.conf, and hostname files. - Recognize NetworkManager and systemd-networkd ownership before editing persistent files.
- Use
ping6,traceroute6,mtr, logs, anddmesgas well as IPv4 tools. - TCP Wrappers access files apply only to supporting applications.
Recap
- Diagnose from hardware upward: link, address, route, DNS, transport, application.
getentanddiganswer different resolver questions.- Know which service owns persistent network configuration.
- Logs plus route and packet evidence are stronger than repeated restarts.
Test Your Knowledge
Complete the quiz to assess your understanding of this course's concepts.
