The purpose of this post is to provide a comprehensive guide specific to Application Penetration Testing (AST) and to provide answers to some commonly asked questions that surround AST and its value.
This guide includes:
In the modern digital age, it is increasingly important for organizations to develop reliable security programs designed to achieve defensive security goals and mitigate risk. Whether an organization develops their own internal business applications, outsources the development of web or mobile applications for use by their customers, or relies on open-source applications or libraries, AST is required to gain a high degree of cybersecurity assurance. Even full-fledged software development firms with strong source code review processes may want to have their products tested to reduce risk.
A bug or malicious code within an application could allow an attacker to gain initial access, achieve high-level privileges, disable security or critical business services, steal important company secrets, encrypt critical data and demand ransom for its return, or outright destroy a company's data. There are several different types of AST, and this guide will help stakeholders understand the difference between them and identify the right testing procedures for each one of their assets.
Application Penetration Testing (also known as "Application Pentesting", "Application Security Testing", or "AST") evaluates web, mobile, and native desktop applications and packages to identify exploitable vulnerabilities and protect against cyber-attacks. In a "black-box test", the penetration testers start with no information about the target application and attempt to exploit it the same way a real-world attacker would. By simulating a real-world cyberattack, the security of an application and its environment's security controls (such as web-server configuration) can be examined. In a "white-box" test, testers are provided information about an application's internal functionality, which may include providing the full source code for manual review. "grey-box" testing is somewhere between black-box and white-box testing where some limited information about the target application is provided to help the penetration tester verify specific security goals, including vulnerability to insider attacks.
The process of developing software is known as the Software Development Life-Cycle (SDLC) and the environment that software is developed in is known as Development Operations (DevOps). The stages of the SDLC are: Design, Build, Document, Test, and Deploy. Ideally, IT and application security should be considered during all stages of the SDLC and DevOps (DevSecOps), however, this is not always the feasible reality. The complexity of software development, combined with business interests such as developing software cost effectively means that bugs often find their way into the source code. It's also a real possibility that an insider may attempt to plant malicious code into an application. Application Pentesting is purposed to remedy both inadvertent bugs or planted malicious code.
App Pentesting may also be warranted for software released years ago if the application has not been tested before, or has had changes. In more extreme cases where risk mitigation requirements are very high, Application Pentesting can be a continuous process that stops at nothing to circumvent security, achieve exploitation, and improve the security of a software product.
There are many potential ways that a software application may fail or be exploited and various impacts that a failed software application could have, so App Pentesting needs to approach the problem from many different angles in order to hunt and identify all of the potential points of failure.
Application Penetration Testing should attempt to identify, but is not limited to:
Companies incurred an estimated total of six trillion dollars in damages due to cybercrime in 2021. In response to the fear of incurring staggering damages, companies are increasing their cybersecurity budgets and taking a more proactive approach to reduce their cyber-risk exposure. The average cost of a data breach was $6.75 M CAD per incident in 2021 according to the IBM report, up from roughly $4M CAD in 2018. The consequences of cyberattacks include operational downtime, loss of brand reputation, loss of business relationships, and large fines and class action lawsuits.
The average cost of a data breach was $6.75 M CAD per incident in 2021.
All organizations depend on software to some degree, and for the modern enterprise, operational resiliency hinges on the security of software applications. Considering how damaging a cyber-breach can be, companies need to take cybersecurity challenges head on with a proactive approach, by testing and verifying application security.
Companies that rely on third party applications put a significant amount of trust into a commercial vendor or open-sourced development process that they do not directly oversee. Considering that vendors face budget and resource restraints in order to remain profitable and competitive, it becomes obvious how security can take a backseat to other priorities. AST provides stronger assurances regarding the reliability of a particular software application.
Finally, when publishing or selling software as a service (SaaS), such as a website, mobile, or desktop application, it's important to consider the producer's brand reputation. Users and customers place their trust in a producer's software, and failure to deliver on security could erode brand confidence, profits, and cause users to flee to other alternatives.
Software applications can embody a vast spectrum of different technologies, fill many different purposes, and contain a wide array of complexity. Therefore, the amount of time and resources required to test an application also varies significantly. Applications with many features and functions generally require more time to test than applications with simple functionality. Also, certain software technologies such as the language an app is written in and which software constructs are used, determine which experience, tools, skills, and amount of information gathering is required to complete an effective evaluation.
The duration of an Application Pentest is also impacted by the depth of testing and assurances that an organization requires to satisfy its risk requirements. Black-box testing is closest to simulating a real-world cyberattack environment, but requires time for information to be gathered manually by the penetration testing organization. In white-box testing, full information including source-code is disclosed to the penetration testing entity, allowing source code to be manually reviewed for potential exploitability, however this manual review of code is also time intensive.
A grey-box pentest achieves a good balance by increasing the efficiency of a black-box test by providing some information beforehand, thus allowing an engagement to approach the depth of a white-box approach. In most cases, grey and white-box testing may include "credentialed" testing, in which account credentials are provided to penetration testers to simulate an insider attack. Credentialed testing may increase the testing process's efficiency by honing focus on critical aspects of an application.
Separately conducting first black-box and then white-box testing as two separate phases provides the highest degree of security assurance, but requires the most time and resources.
Web-application penetration testing is the process of conducting penetration tests on a website and hosting infrastructure. The tests can be conducted as both black-box tests which test the application's resilience against a simulated real-world cyber-attack, white-box tests that can expedite the testing process or allow deeper testing into key areas, or grey-box tests in which limited information is provided about the target application to pentesters before the testing process begins to expedite the process and allows testers to focus on specific goals.
The general process of a web-application penetration test is:
The assessment of a web-application should include all items in the OWASP Top Ten, ensuring that IT security best-practices are implemented, testing API endpoints that the application relies on, and testing infrastructure configuration and services. This should include the server application (Apache, Nginx, Microsoft IIS), and any exposed services on the infrastructure such as remote access services (SSH, SFTP, or SQL).
In the case that multiple web-applications are hosted on the same server, all applications should be tested, since a breach of one of the applications could result in any and all of the hosted web-applications and their data being compromised.
Many companies have launched their own mobile apps for internal use and for their customers. Mobile app testing may involve testing the mobile version of a web-application, and native mobile applications that are installed directly to iOS or Android. Mobile app testing includes many of the same approaches as web-application testing, such as testing for OWASP Mobile Top Ten vulnerabilities, verifying best-practices, and testing API endpoints and infrastructure.
However, native mobile apps face some unique security challenges. For example, "rooting" a mobile device provides the owner with admin privileges and ability to inspect the file and memory contents of any installed applications. Therefore, mobile app security testing should include testing how the target app reacts when a device is operating under these conditions, since it is a security vulnerability that could expose credentials or package source code.
Testing native Windows, macOS or Linux applications ensures they have been designed securely and reduces the risk of an application being used to gain initial access, escalate system privileges, or read or modify unauthorized data. In general, native desktop applications should be tested for proper input sanitation, safe use of any functions that can execute system commands, map memory, and deserialize objects, and the application logic should be tested to verify it can properly implement type-checking and type-assignment of variables, and that there are no bugs in the authentication process. Also, if the application is to be run as administrator or root account, the risks of a vulnerability are much higher and the application should be tested more rigorously.
Each operating system has its own set of potential vulnerabilities and therefore the testing process for native desktop applications depends on which OS it is compiled for. For native Windows applications, service paths used to load built-in functions and DLLs should be verified to ensure that these critical files cannot be replaced with a malicious version.
Many applications are developed using open source software (OSS) libraries because using OSS can expedite the development process, saving time and money. Although by definition OSS's source code is publicly disclosed, there are no guarantees that it has been reviewed by security minded software developers. In fact, many open source software packages have been found to contain malicious code or bugs. Companies may also rely on fully-developed OSS applications for their business operations.
If your applications rely on open source packages, only pentesting the package (which should include a source code assessment) can provide the highest degree of security assurances. This perspective is gaining traction in the software development industry and increasingly more developers, security analysts, and penetration testers are joining together to share OSS related threat intelligence.
Dynamic Application Security Testing (DAST) involves testing an application while in use and can be conducted as white-box, grey-box, or black-box testing. Dynamic analysis evaluates that access control is managed securely, sensitive data is not exposed, the application handles errors properly, and provides resiliency against an attack. Fuzzing is an advanced form of DAST, which tests an application by submitting invalid, unexpected, or random data.
Static Application Security Testing (SAST) is the process of auditing a software application by inspecting its source code and is a type of white-box testing. Automated source code analysis tools can identify functions or packages that present potential security risks, however, the scan should be manually reviewed to verify its results. Source code analysis tools are available for all popular software programming languages and frameworks including iOS and Android mobile applications.
Static Application Security Testing (SAST) is the process of manually inspecting the source code of an application, can identify all forms of vulnerabilities, and is a form of white-box testing because the application source code is provided to testers for evaluation. SAST testing does not execute the code during the testing process. SAST is incorporated into the Software Development LifeCycle (SDLC) to evaluate the security of software structures (functions, classes, APIs).
Dynamic Application Security Testing (DAST) is the process of testing an application as it executes on a production server or in a testing environment and is a form of black or grey-box testing because pentesters are not provided information about an application's internals. DAST can only be done later in the SDLC after a working copy of the application is available.
Together, static and dynamic code analysis allow software engineers and 3rd party vendors to verify an application with a high degree of assurance.
The most important factors that determine the scope of a testing engagement is the type of application being tested, the degree of risk assurance required for the target application, and the unique risks posed by an application. The type of application being tested (web, mobile, desktop, open-source, or proprietary) determines which testing methodologies are relevant. All types of applications should be tested for IT security best practices such as authentication, exposure of sensitive information and files, secure error handling, secure input handling, known package vulnerabilities, and exposed services and API endpoints.
The degree of risk assurance required will determine the depth of testing and which advanced testing procedures should be considered. More focus should be placed on testing systems, features, and functions that present a higher degree of risk. Some examples are payment processing features, and critical operational and business logic. A risk assessment of an application's unique purposes, business operations, design, and components will also determine how pentesters should allocate their resources.
Application Pentesting is not the only process for increasing the security and resilience of applications. DevSecOps and Threat Modelling are two other cybersecurity processes that are worthwhile understanding and comparing to Application Pentesting.
Development Operations (DevOps) is a set of operational practices related to the SDLC, aimed at increasing the productivity, quality, and efficiency of software application development. DevSecOps refers to securing the DevOps process and increasing the security of the final application. In other words, DevSecOps may include AST, but AST may also be used outside of the scope of DevSecOps. For example, AST may be part of a corporate risk management or vulnerability management program if an organization outsources application development and needs to verify the security of the application.
Threat Modelling is a process that seeks to identify contextual risk within an organization's operational IT by modelling how a cyber-attack is likely to take place and thereby enable a "Secure by Design" approach. Threat Modelling identifies the most critical security threats, attack vectors, and vulnerabilities, allowing remediation to be prioritized, and defensive resources to be allocated efficiently. If a Threat Model identifies a particular application as high risk to business operations or sensitive data exposure, then mitigation may include AST to ensure that those critical applications are properly secured.
The OWASP Top Ten is a ranked list of the most critical web-application security vulnerabilities and is ordered according to the current web-application threat environment. It serves as both a fundamental checklist of security concerns for security teams during the design and development phases of an application and for penetration testers to hunt when testing an application. The list also provides a common terminology for security analysts to communicate. The list is updated every three to four years to account for changes to the web-application threat environment.
The current OWASP Top Ten Web Application Security Risks list is shown below for reference:
Scheduling AST regularly is important to effectively manage an enterprise security program. Exposure time refers to the period of time in between tests, when changes to an application may have introduced new vulnerabilities. Large enterprises may have AST programs that are continuous, but for organization's that do not have a continuous penetration testing program, testing should be done prior to an application's release and before major or minor updates, or after other significant changes.
Organizations may be required to conduct AST by law, or to comply with industry standards to improve overall security posture. For financial applications handling payment card data, PCI-DSS requires that companies conduct penetration tests every 3 months or after significant changes are made. Similarly, SOC-2 type 2 is a continuous attestation of IT security compliance by an organization and requires a penetration testing program that is in-line with a businesses unique operational and risk objectives.
Requesting access to AST information such as frequency, reports, and remediation activity is also advisable when a merger or acquisition (M&A) is being considered. This due diligence can provide valuable insight into the risk management practices and security posture of potential partners.
The cost of a pentest can vary greatly depending on the scope and complexity of the engagement, but the typical range of a quality professional Application Penetration Test is between $10K - $60K.
The most significant factors that impact the cost of a Software Security test include the complexity of the target application, whether the target is a web-application, mobile app, or desktop app, the type of testing conducted (SAST / DAST), the amount of manual testing performed, and the duration of the engagement. All these factors are part of the formal discussion between the target organization and penetration testing entity before testing begins.
By limiting focus to a small group of assets, or providing detailed information beforehand, the cost of a test can be reduced. Organizations, who are debating the value of penetration testing could initially contract a narrowly scoped test to assess the return value that penetration testing provides.
The Return on Security Investment (ROSI) metric is the appropriate method of calculating the ROI of penetration testing. ROSI is an alternative ROI equation, designed to accommodate the uniqueness of security-related investments. It compares the total avoided costs of potential security breaches to the cost incurred by penetration testing.
A generalized version of the ROSI equation is:
ROSI = (Security expense avoided – prevention cost) / prevention cost
For example, if your company can expect to avoid even a minor security breach that would cost $100,000 over the next year, and the price of a penetration testing engagement were estimated to be $10,000, then the ROSI calculation would be 9 times the cost:
ROSI = ($100,000 - $10,000) / $10,000 = 9
Packetlabs creates a professional, customized report for each client that includes complete details of the application assessment. Each report contains an executive summary that clearly outlines the goals and scope of the assessment, highlights all the discovered security flaws, and describes the app's overall level of security.
The body of the report includes the methodologies used to conduct each test, technical findings with steps to reproduce, collected evidence, and information about remediation of the exploited vulnerability. Finally, the report is concluded with a list of strategic and tactical security recommendations related to the specific application tested, and informational appendices when necessary.
The pentester role (also known as ethical hacker) is a distinct IT security role that requires specialized training and certification. Ethical hackers may be categorized as generalists who are broadly trained in penetration testing tactics, or specialists with deeper skills in some particular aspect of the pentesting process. Specialists may also be distinguished by the specific exploitation frameworks, protocols, operating systems, or exploitation types they are experts in.
The OSCP is a globally recognized and industry leading ethical hacking certification offered by Offensive Security. Offensive Security offers several certifications but the OSCP is the most broad and well-known. Packetlabs is a passionate team of highly trained ethical hackers with the industry’s most advanced certifications. All PacketLabs pentesters are required to have a minimum of OSCP.
While OSCP is the PacketLabs minimum requirement, many team members go above and beyond to gain additional certified expertise including:
This allows our team of OSCP penetration testing professionals to demonstrate industry leading comprehensive hands-on mastery of penetration testing.
There are many reasons to outsource pentesting, but the primary reason is that putting fresh eyes on a target environment is likely to shed light on potential security weaknesses that an internal security team may overlook. This is because internal teams can develop assumptions and blind spots that the fresh eyes of a motivated and specialized pentesting team will not have.
Many internal enterprise security programs use automated scanning tools that can identify many known vulnerabilities and misconfigurations. However, these automated testing tools are not capable of identifying all vulnerabilities and over-reliance on them may provide a false sense of security. Therefore it is important to evaluate a cybersecurity firm based on their ability to conduct advanced manual testing techniques.
Ransomware has greatly increased the potential value of a cybersecurity breach to cyber-criminals. This results in highly-skilled and pervasive threat actors dedicating their efforts towards developing custom exploits and learning every trick in the cyber-attack playbook. The specialized knowledge, skills, and tools possessed by a professional penetration testing team allow them to simulate a wide range of realistic attacker TTP and provide stronger security assurances.
When selecting a penetration testing consultant many things should be considered such as reputation, trust, size of the entity, their degree of experience and professionalism (including certification requirements and statuses), and specialized skills that apply specifically to the target organization's environment.
Best value and most cost-effective
Although our engagement prices are inline with our competitor's we provide an extensive portfolio of manual testing that our competitors do not provide. Automated testing is only the first step to a pentesting engagement, and manual testing is critical when it comes to uncovering all the security gaps and providing the highest degree of risk assurances.
At PacketLabs we have a unique approach to sourcing talent. Our requirements demand that candidates possess a minimum OSCP certification and pass an intensive 72-hour hacker challenge. Once aboard, our pentesters complete continuous skills training, and practical evaluations.
Packetlab’s ethical hacker team surpasses the skills of our competitors. Firstly, most of our competitors do not make the OSCP a prerequisite. Secondly, our team of highly skilled professional ethical hackers learn an industry-leading methodology to identify hard-to-find vulnerabilities and weaknesses often missed by conventional testing.
At PacketLabs we take the time to understand every in-scope component and its role in the overall system topography. We tailor our approach to each environment we assess and this customized approach goes beyond the industry standard of simply running an automated software scan.
Our reporting methodology follows a thoughtful narrative approach. Instead of just giving a technical description of the problem, we demonstrate the contextual business impact of our findings. Narrative explanations clarify key takeaways and contribute more insight to your organization's cybersecurity posture.
Automated tools such as software scanners do not sufficiently probe an application and should be augmented with extensive manual analysis and testing. To be completely thorough, the pentesting process needs to seek to identify logical vulnerabilities and attempt to exploit all security scenarios. Automated testing accounts for only 5% of what we do. The other 95% consists of manually simulated real-life attacks.
At Packetlabs, we have developed a comprehensive foundational methodology and continuously refine our cyber-threat intelligence (CTI) and skills. Our foundational methodology not only assesses the target environment but also tests exploitable attack vectors that industry standards are not designed to address, without impacting pricing.
