So, let’s assume we want as "realistic" pen test as our budget allows. What are some of the things to look for from a pen test service company during the selection and agreement phases? Here are some suggestions gained from experience:
Hire the right talent. Ultimately, you are hiring a team of people with experience, skills, and tools to do the job right. Pen testing is an inherently high-risk endeavor. Things break, stuff goes wrong, alarms go off (hopefully) and that is the whole point. Make sure the team you are hiring is experienced and ask them detailed questions about how they come up with a test plan, rules of engagement, and the final reporting content. If an un-experienced penetration tester is hired, you’ll have just as many alarms go off, but you may not have any positive test results to show for their efforts. The last thing you want after a penetration test is no actionable results that come out of it. This is not the time to feel good about your security after a pen test doesn’t uncover weaknesses on your network!
Pay attention to scope. This is one of the trickiest parts of any penetration test, and the right team will be the one that helps you both determine what should be scoped into the target environment and what should be scoped out. Before the test begins, there should be clearly defined IP address ranges, external URLs and IP addresses, and applications both internal and external that are defined. Other scope considerations include the degree to which social engineering is acceptable and if there are any off-limits people that should not be targeted. Similarly, physical access to everything from buildings to dumpsters should be defined at the outset. By limiting scope, you effectively focus more effort on those areas of you organization you want to be tested. And you also prevent unacceptable actions from being taken against resources that are deemed off-limits. Often over-looked, scope should also be prioritized as much as possible so that the test team spends focused time on high value assets, etc. You want to strike a balance between too broad and too narrow a scope, based in part on your budget. If it is defined too broadly, efforts will not be focused properly in the allotted time. If it is too narrow, however, the testers may not be given enough lateral flexibility to explore alternate paths towards real-world exploitation.
Blackbox vs. Whitebox. There are advantages and disadvantages to both. A Whitebox test (in which the attacker is provided with information or network access going into the engagement that would be difficult to obtain on their own) has two advantages: 1) less time and money is spent on the discovery, reconnaissance and enumeration portions of the test, leaving more time and money to be spent on breaking applications, network devices, people, etc. 2) The threat posed by insiders is often underestimated by organizations that entrust them to physical and logical access to IT resources. By its very nature, whitebox testing allows the attacker to be one step closer to the internal environment and may help uncover vulnerabilities in internal applications that a blackbox test might not. The advantages of a blackbox test (in which only a small amount of an organization’s information is provided, or only that which is readably uncovered via Internet searches and making phone calls into the organization) include: 1) it provides the best ‘real-world’ perspective of the organization from an external attacker’s perspective 2) it naturally forces the attacker to spend time uncovering information on the organization that is public or able to be social engineered out of employees or partners. By analyzing the results of this process, an organization will learn a tremendous amount about how an attacker can gain a foothold in the organization starting from scratch, and then be able to take steps to mitigate or remediate those vulnerabilities.
Goals and Objectives. By establishing what the overall goals of the test are going in, you will allow the test team to produce a report that caters to those goals and addresses them. If there is a particular hot button you want to make sure is addressed, be sure to include it outright in the goals. Understand that not all of the goals may be met during the test, and in some cases this may be a good thing! (e.g. test the ability to access the development environment from the production network and attempt to access source code or other intellectual property)
Recommendations. Before choosing a test team, be sure to discuss whether or not, and to what extent, recommendations will be made in the report. Don’t assume that a pen test report will include detailed recommendations about how to mitigate or remediate every finding. Ask for a sanitized example of a report and review the recommendations. Are they written in a way that is actionable by your staff after the engagement? Avoid recommendation examples that read like this: “We recommend that your firewall is configured using industry best practices using the concept of least privilege”. That’s simply too high-level to be of value and won’t help your firewall admin know what needs to be changed on the firewall from how it is already configured.
Schedule the events properly. Work with the test team to determine when certain systems should be tested. You might not want your online payment system to be tested during peak purchase hours, for example. Conversely, you definitely DO want the test team to run a sniffer on the network during normal business hours. The test team should be able to guide the conversation to account for any scheduling considerations before the test begins. If this doesn’t happen, or if the question never even gets asked, it’s a sign you may be headed for a painful experience.