Security Chaos Engineering VS. traditional security approaches 3/4

After introducing Security Chaos Engineering (SCE) as a potential approach to cloud security, we look at important differences between SCE and traditional security approaches. We discuss vulnerability assessment, penetration testing, and red teaming briefly and end with an overview of the properties and differences.
Vulnerability assessment is one of the first steps in threat management.

It aims to identify potential vulnerabilities in a system (i.e., misconfigured network devices or the execution of arbitrary code) to provide an overview. Vulnerability assessments focuses on detecting vulnerabilities in the system, generally assessing consequences and considering remediation efforts. External experts perform the vulnerability assessment. The first step for the company is to define the target. Subsequently, automated tools scan the system. Once the tester identifies vulnerabilities, he analyzes the results, prioritizes them based on the severity of their impact, and generates a report documenting the operations performed and the results obtained. Usually a vulnerability assessments take less than a week and occur at the end of the development process. Regular reperformance is necessary.

Penetration testing is about identifying, and testing as many known vulnerabilities of a system as possible in more detail within in a certain amount time.

The tester mimics an attacker by using their tools and trying to adopt the attacker’s perspective. Penetration testing focuses on the exploitability and assessing the business impact of vulnerabilities. Most often, external companies with specific expertise perform penetration testing. However, the target company’s employees are informed about the penetration test. It usually starts with a vulnerability assessment and continues with the development of a test plan that includes parameters such as timeframe, deliverables, or tools . During the detection and penetration phase, the expert attempts to compromise the target system by exploiting logical and physical vulnerabilities. Afterwards, the tester prepares a report for the client that includes the vulnerabilities found, the exploitation procedure, and remediation instructions. Usually a penetration testing occurs at the end of the development phase if necessary and lasts 1-3 weeks.

Red Teaming takes the level of detail of vulnerability management one step further.

The goal is to test an organization’s attack detection and response capabilities. A large team of security experts simulates a malicious attack, using the same knowledge, assets, tools, and resources attackers would use. Focus is detecting vulnerabilities which would otherwise go undetected. In most cases, external experts perform red teaming. First, testers perform an Attack Assessment, in which attack target and strategy is narrowly defined. Hence, testers examine significantly fewer vulnerabilities than in penetration testing, but with more depth. Afterwards, they evaluate the system in detail and launch their attack. The attack can take a long time, as the testers adapt their strategy to defenses, they find. At the end, the red team reports to the company about the exploited vulnerabilities and the attack process. Red Teaming takes at least 3-6 weeks, depending on scope significantly longer. Due to immense resources, it is the costliest approach.

Security Chaos Engineering is a complement to existing security approaches.

The goal of SCE is to proactively identify unknown vulnerabilities that would otherwise go undetected. SCE occues either in-house or by external IT security experts. We discussed the process of how to conduct security chaos experiments in the previous post. Generally, the process does not take much time in the long run because once the developers have programmed the experiments, they can run automatically without manual effort. Compared to Read Teaming cost is limited.


Comparison of Security Tests
The above table summarizes the most crucial properties of each approach.
Traditional approaches have problems identifying relevant vulnerabilities in large and complex cloud systems.

The vulnerabilities under investigation are already known, or the testers try to identify unknown vulnerabilities manually. However, due to the complexity and size, it is almost impossible to consider all relevant components and dependencies. Therefore, SCE as an automated approach offers the potential to discover additional vulnerabilities. In the following article, we will discuss tools for implementing SCE in enterprises as the endpoint of our SCE series.

If you are interested in more details on how A&B security experts can help establish a Security Chaos Engineering culture in your company havc a look at our SCE Program or contact us at Alice&Bob.Company!
Resources used and interesting content on this topic:
  1. Rinehart, Aaron, and Nwatu, Charles – Security Chaos Engineering: A new paradigm for cybersecurity (2018) ( last accessed 13.06.2022)
  2. Rinehart, Aaron, and Shortridge, Kelly – Security Chaos Engineering (2020)
  3. Combs, Veronica (2021): Security chaos engineering helps you find weak links in your cyber defenses before attackers do (, last accessed 14.06.2022)
  4. Arkin, Brad, Scott Stender, and Gary McGraw. “Software penetration testing.” IEEE Security & Privacy 3.1 (2005): 84-87
  5. Bishop, Matt. “About penetration testing.” IEEE Security & Privacy 5.6 (2007): 84-87.
  6. Thompson, Herbert H. “Application penetration testing.” IEEE Security & Privacy 3.1 (2005): 66-69.
  7. Whitaker, Andrew, and Daniel P. Newman. Penetration Testing and Network Defense: Penetration Testing. Cisco Press, 2005.
  8. Al Shebli, Hessa Mohammed Zaher, and Babak D. Beheshti. “A study on penetration testing process and tools.” 2018 IEEE Long Island Systems, Applications and Technology Conference (LISAT). IEEE, 2018.
  9. Denis, Matthew, Carlos Zena, and Thaier Hayajneh. “Penetration testing: Concepts, attack methods, and defense strategies.” 2016 IEEE Long Island Systems, Applications and Technology Conference (LISAT). IEEE, 2016.
  10. Yeo, John. “Using penetration testing to enhance your company’s security.” Computer Fraud & Security 2013.4 (2013): 17-20.
  11. Rajendran, Jeyavijayan, Vinayaka Jyothi, and Ramesh Karri. “Blue team red team approach to hardware trust assessment.” 2011 IEEE 29th international conference on computer design (ICCD). IEEE, 2011.
  12. Abbass, Hussein, et al. “Computational red teaming: Past, present and future.” IEEE Computational Intelligence Magazine 6.1 (2011): 30-42.
  13. Ray, Helayne T., Raghunath Vemuri, and Hariprasad R. Kantubhukta. “Toward an automated attack model for red teams.” IEEE Security & Privacy 3.4 (2005): 18-25.
  14. Kahmen, Jan (2020) ( last accessed 14.06.2022)
  15. Packetlabs (2019) ( last accessed 14.06.2022)
  16. Mitnick Security (2021) ( last accessed 14.06.2022)
  17. Rana, Pramod (2021) ( last accessed 14.06.2022)
  18. Guedez, Alexander (2021) ( last accessed 14.06.2022)