Recognizing the Signs
Malicious traffic rarely announces itself. It blends with legitimate traffic, exploits normal protocols, and operates at hours when security teams are least attentive. However, it always leaves traces. The key is knowing what patterns to look for and having the tools to surface them from the noise of normal operations.
Before diving into specific tools and commands, here are the universal indicators that something is wrong on your network:
- Unusual port activity — Connections on ports you don't expect. Outbound connections on port 4444 (Metasploit default), 6667 (IRC, used by botnets), or high-numbered ports from servers that should only be serving HTTP/HTTPS. Inbound probes on ports 22 (SSH), 3389 (RDP), 445 (SMB), and 23 (Telnet) from IPs you don't recognize.
- Geographic anomalies — Traffic from countries where you have no customers or business relationships. If your web application serves Norwegian users and you see sustained connections from IP ranges in countries with high cybercrime activity, that warrants investigation. IP geolocation data from tools like WAYSCloud can automate this analysis.
- Rate spikes — Sudden increases in failed login attempts, DNS queries, or HTTP requests that don't correlate with business activity. A normal day might show 50 failed SSH logins; 5,000 in an hour indicates a brute force campaign.
- Known-bad user agents — Web scanners often use distinctive user-agent strings:
Nikto,sqlmap,masscan,ZmEu, or the genericpython-requestshitting endpoints that real users never access. - Data exfiltration patterns — Large outbound data transfers at unusual hours, especially over DNS (DNS tunneling), HTTPS to recently registered domains, or ICMP (ping tunneling). These can indicate an active breach where the attacker is extracting data.
Log Analysis: Your First Line of Detection
Server logs contain a goldmine of security information. The challenge is extracting signals from noise. Here are the essential log files and what to look for in each.
SSH authentication log (auth.log)
# Top failed login IPs (brute force detection)
grep "Failed password" /var/log/auth.log \
| awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -20
# Invalid users (dictionary attack detection)
grep "Invalid user" /var/log/auth.log \
| awk '{print $8}' | sort | uniq -c | sort -rn | head -20
# Successful logins from unexpected IPs
grep "Accepted" /var/log/auth.log \
| awk '{print $9, $11}' | sort | uniq -c | sort -rn
# Check for successful root logins (should be zero!)
grep "Accepted.*root" /var/log/auth.log
# On RHEL/CentOS, use /var/log/secure instead
Web server access log (access.log)
# Top IPs by request volume (detect crawlers/DDoS)
awk '{print $1}' /var/log/nginx/access.log \
| sort | uniq -c | sort -rn | head -20
# Requests to common attack targets
grep -E "wp-login|xmlrpc|phpmyadmin|\.env|wp-config" \
/var/log/nginx/access.log | awk '{print $1}' \
| sort | uniq -c | sort -rn | head -10
# High rate of 4xx errors from single IP (scanning behavior)
awk '$9 ~ /^4/ {print $1}' /var/log/nginx/access.log \
| sort | uniq -c | sort -rn | head -10
# Suspicious user agents
grep -iE "nikto|sqlmap|masscan|zmeu|nmap|dirbuster" \
/var/log/nginx/access.log | head -20
# POST flood detection (brute force against login forms)
grep "POST" /var/log/nginx/access.log \
| awk '{print $1}' | sort | uniq -c | sort -rn | head -10
System log (syslog / journald)
# Firewall blocks (iptables/nftables logged to syslog) grep -i "blocked\|drop\|reject" /var/log/syslog | tail -20 # Kernel-level network anomalies dmesg | grep -i "possible syn\|martian\|drop" # systemd journal for service failures journalctl -u sshd --since "1 hour ago" --no-pager | grep -i "fail\|error"
Network Monitoring: Real-Time Visibility
When log analysis shows suspicious activity, network monitoring tools give you real-time visibility into active connections and traffic flows.
Connection inspection with ss and netstat
# Show all active connections with process info
ss -tunapo
# Count connections per remote IP (detect DDoS/scanning)
ss -tna | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head -20
# Find established connections to unusual ports
ss -tna state established | awk '$4 !~ /:80$|:443$|:22$/ {print}'
# Show which processes have network connections
ss -tulnp
# Find SYN_RECV connections (possible SYN flood)
ss -tna state syn-recv | wc -l
Packet capture with tcpdump
# Capture traffic from a specific suspicious IP sudo tcpdump -i eth0 host 185.220.101.1 -n -c 100 # Capture DNS queries (detect DNS tunneling) sudo tcpdump -i eth0 port 53 -n -l | grep -v "your-dns-server" # Capture SYN packets only (detect port scanning) sudo tcpdump -i eth0 'tcp[tcpflags] & (tcp-syn) != 0 and tcp[tcpflags] & (tcp-ack) == 0' -n -c 50 # Capture and save for later analysis in Wireshark sudo tcpdump -i eth0 -w /tmp/capture.pcap -c 10000
Content inspection with ngrep
# Search for cleartext passwords in network traffic sudo ngrep -d eth0 -i "password\|passwd\|login" port 80 # Detect HTTP requests to suspicious paths sudo ngrep -d eth0 "wp-login\|phpmyadmin\|\.env" port 80 # Watch for IRC traffic (botnet C2 indicator) sudo ngrep -d eth0 -q "" port 6667
Automated Detection Systems
Manual log analysis and packet capture don't scale. For continuous monitoring, deploy automated intrusion detection and prevention systems.
fail2ban (host-based intrusion prevention)
fail2ban is the simplest automated defense: it watches log files for patterns (failed logins, error pages, scanning signatures) and bans offending IPs via the firewall. It works out of the box for SSH and can be extended to any service that logs authentication failures. See our brute force protection guide for detailed configuration.
OSSEC / Wazuh (host-based intrusion detection)
Wazuh (the actively maintained fork of OSSEC) provides file integrity monitoring, log analysis, rootkit detection, and vulnerability scanning. It correlates events across multiple log sources and generates alerts based on customizable rules. Unlike fail2ban which only blocks based on patterns, Wazuh understands context — it can detect privilege escalation chains, lateral movement, and subtle indicators of compromise that simple pattern matching misses.
# Wazuh agent installation (connects to Wazuh manager) curl -s https://packages.wazuh.com/4.x/apt/doc/install-agent.sh | sudo bash # Key capabilities: # - File integrity monitoring (detect unauthorized changes) # - Log correlation across auth.log, syslog, web logs # - Rootkit detection # - Vulnerability scanning # - CIS benchmark compliance checking
Suricata (network-based intrusion detection)
Suricata inspects network traffic in real time against a ruleset of known attack signatures and protocol anomalies. It operates at the network level, catching threats that host-based tools miss — like exploit attempts, malware communications, and lateral movement between machines. It uses the same rule format as Snort, giving access to thousands of community and commercial detection rules.
# Install Suricata sudo apt install suricata # Update rules (Emerging Threats community ruleset - free) sudo suricata-update # Run in IDS mode (detection only, no blocking) sudo suricata -c /etc/suricata/suricata.yaml -i eth0 # Check alerts tail -f /var/log/suricata/fast.log
Using Threat Intelligence for Detection
The techniques above detect malicious behavior patterns. Threat intelligence takes a different approach: it identifies known malicious infrastructure by reputation, allowing you to flag or block connections before malicious behavior even occurs.
Check IPs from your logs against WAYSCloud
#!/bin/bash
# Extract unique IPs from auth.log and check against threat intelligence
grep "Failed password" /var/log/auth.log \
| awk '{print $(NF-3)}' | sort -u | while read ip; do
result=$(curl -s "https://ip.wayscloud.services/api/v1/ip/$ip")
score=$(echo "$result" | jq -r '.intelligence_data.threat_score // 0')
risk=$(echo "$result" | jq -r '.intelligence_data.risk_level // "unknown"')
if [ "$score" != "0" ] && [ "$score" != "null" ]; then
echo "THREAT: $ip (score: $score, risk: $risk)"
fi
sleep 0.5 # Respect rate limits
done
SIEM integration
For organizations running a SIEM (Splunk, Elastic SIEM, QRadar, or similar), WAYSCloud threat intelligence can be integrated as an enrichment source. When security events are ingested, the SIEM looks up the source IP against the threat intelligence feed. This turns a generic "failed login from 185.220.101.1" into "failed login from a CRITICAL-risk IP with 47 reports for SSH brute force across 4 independent sources" — dramatically accelerating analyst triage.
ASN-Level Analysis
Individual IP analysis is important, but patterns often emerge at the network level. Certain hosting providers and autonomous systems have disproportionately high rates of malicious traffic. Understanding these patterns helps you make informed decisions about network-level blocking or enhanced monitoring.
WAYSCloud tracks threat activity by ASN (Autonomous System Number), allowing you to identify hosting providers with high abuse rates. If 20% of your failed login attempts come from a single cloud hosting provider known for tolerating abuse, you might choose to rate-limit or challenge all traffic from that ASN rather than blocking individual IPs one at a time.
The ASN threat ranking page shows the autonomous systems with the most malicious IPs in the WAYSCloud database. Use the ASN intelligence page to drill into specific networks and see their threat profile over time.
Country-Level Patterns
Geographic analysis of your traffic reveals patterns that may warrant network-level controls. If your service operates exclusively in the Nordic region but 40% of your failed authentication attempts come from a few specific countries, geo-blocking or geo-rate-limiting those sources can reduce your attack surface significantly.
Important caveats: VPNs and cloud providers mean that geographic origin is not always the true source. Blocking entire countries affects legitimate travelers and expats. Use country-level analysis as one input to your security posture, not the sole determinant. Rate limiting by country (allowing some traffic but slowing down bulk requests) is often a better approach than hard blocking.
Explore country-level threat data on the country risk trends page, which tracks how threat activity from each country changes over time.
Building a Detection Pipeline
Effective threat detection is not about any single tool — it's about building a pipeline that combines multiple detection methods. A practical pipeline for a small-to-medium server looks like this:
- Layer 1: Proactive blocking — Pre-block known malicious IPs using threat intelligence feeds (WAYSCloud API, run every 15 minutes via cron)
- Layer 2: Automated response — fail2ban for immediate banning of brute force attempts detected in real-time logs
- Layer 3: Pattern detection — Wazuh or OSSEC for host-level intrusion detection and correlation
- Layer 4: Network monitoring — Suricata for signature-based detection of known exploits and malware communications
- Layer 5: Manual investigation — Daily review of aggregated alerts, enriched with threat intelligence for analyst triage
Each layer catches threats that others miss. Proactive blocking prevents known attackers from reaching your services. Automated response handles new attackers within minutes. Pattern detection finds slow, distributed attacks. Network monitoring catches exploits and malware. Manual investigation provides the context that automated systems can't.