6.3 Network Detection and Resilience: IDS, Traffic Analysis, and DDoS
Network control defines what should happen; detection systems observe what actually happens. With increasingly prevalent encrypted traffic, detection can no longer rely solely on deep payload inspection. Connection metadata, DNS queries, identity information, endpoint behavior, and application telemetry must be correlated to enable effective detection.
IDS and IPS Have Different Failure Modes
IDS operates in bypass mode, observing and alerting, misfires primarily consume analysis capacity. IPS is placed directly in the traffic path and can actively block; misfires can directly cause availability incidents. Only rules with high confidence and fast rollback capabilities should be used for automatic blocking; all others should first be observed, correlated, and manually verified.
Signature detection excels at identifying known patterns, while anomaly detection looks for behavior that deviates from a baseline. An anomaly does not necessarily mean malicious activity: scheduled releases, batch jobs, and holiday traffic can all shift the baseline. Every high-priority detection must include asset scope, attack hypothesis, expected false positives, and defined response actions.
Metadata Can Still Answer Key Questions
Even when TLS payloads are invisible, flow and endpoint data can still provide answers to critical questions such as:
Which host first connected to a new destination?
Which machine suddenly scanned multiple ports?
Which workload consistently uploads to non-standard regions?
Are DNS queries showing high entropy or newly registered domains?
Is the same identity simultaneously accessing from impossible devices or locations?TLS fingerprints, SNI, certificate details, and traffic volume can be helpful, but they may change with ECH, proxies, or normal client updates. These should be treated as contextual signals, not standalone justifications for arbitrary blocking.
Recording controlled application metadata at TLS termination points is generally more reliable than broadly decrypting traffic. Full-scale TLS interception introduces new root CA trust chains, private keys, privacy risks, compatibility issues, and single points of failure, risks that must be strictly scoped and limited to specific use cases.
Detection-as-Code Requires Test Data
Rules must be versioned and include:
- Data sources and required fields;
- ATT&CK technique or internal attack assumption;
- Thresholds and aggregation windows;
- Allowed business exceptions;
- Owner, severity, and runbook;
- Positive, negative, and historical replay test samples.
The absence of data from a source should not be interpreted as "zero events, therefore secure." Monitoring must include sensor coverage, log latency, field parsing failures, and rule execution status.
DDoS is a problem of capacity and state exhaustion
An attack can exhaust bandwidth, connection tables, TLS handshake CPU, threads, queues, database connections, or expensive business operations. Layered defense:
Upstream/CDN/Anycast cleans large-scale traffic
→ L4 SYN/connection protection
→ L7 rate limiting and identity quotas
→ Application-level concurrency, queues, and timeouts
→ Dependency isolation and fallbacksLimiting by source IP alone harms users behind NAT and can be circumvented by distributed sources. Instead, combine account, API key, device, tenant, operational cost, and global capacity, and apply different budgets for anonymous versus authenticated traffic.
Rate limiters themselves must make trade-offs around distributed consistency: overly strong consistency can become a bottleneck, while fully local approaches can amplify attack traffic by the number of instances. The key is to limit the most expensive resources after minimal, low-cost filtering, such as screening traffic before expensive operations like password hashing, complex queries, or file transcoding.
Response Must Preserve Evidence and Control Blast Radius
After a high-confidence event is triggered, actions such as revoking short-lived credentials, isolating workloads, tightening egress rules, switching to read-only mode, blocking malicious destinations, or enhancing authentication assurances should be executed. Each action must have a clearly defined scope, expiration time, approval workflow, and automated rollback conditions.
Do not restart or clear all state during an investigation. Instead, preserve evidence across key flows, DNS records, identities, processes, image digests, and memory/disk artifacts, ensuring time synchronization is maintained. Network teams see IPs, identity teams see subjects, and application teams see traces. Only through shared correlation and a unified asset inventory can the full attack chain be reconstructed.
Exercise List
- Simulate an application host transmitting to a new external domain, how long until detection triggers?
- Simulate internal port scanning: does segmentation block the traffic, and does the IDS correctly identify the initiating entity?
- Disable one sensor, does the platform generate an alert for missing data?
- Replay a normal peak traffic event, does any rule incorrectly block traffic?
- Perform load testing on an expensive anonymous interface, on which layer does saturation first occur?
- After isolation, is the recovery and cleanup of temporary policies fully auditable?
Network security is not a single boundary wall, but a combination of provable allowed paths, explicit identity decisions, continuous monitoring, and an exerciseable, recoverable posture.