Url Redirector Modified

Performance Testing FAQs

What are the pre-requisites for Performance Testing?

  • Defined and documented performance testing goals and objectives
  • Testable and measurable Non Functional Requirements (NFR) in terms of critical transactions - user intensive, business critical and performance intensive, non critical transaction which can have performance impact in specific instances and measurable SLAs in terms of processing time, throughput and concurrent users
  • Workload which is user behavior in performing critical transactions and distribution of critical transactions among total expected transactions
  • Data in terms of usage patterns and availability of data(unique user name, password etc..) to mimic real user behavior
  • Test Environment - functionally stable software, hardware and software configuration similar to Production and database size similar to Production

Are functional testing requirements good enough to do Performance Testing?

Functional testing requirements can be taken as starting point for Performance Testing. But it is not enough. Functional requirements will contain almost all the functionalities of the applications, but for performance testing most frequently used functionalities need to be identified. Transactions volume, Number of concurrent users also needs to be determined. In summary, non-functional requirements need to be determined and finalized.


How can we collect Performance Testing requirements if it is not available?

Ideally, during the system design phase, business analysts and system architects would have written down the non functional requirements which include performance validation requirements. But in most of the cases, it would not have taken in to consideration in the early stages like application design. If it is not readily available, there are multiple ways to collect the performance requirements. One of the approaches is to have a discussion with Business point of contact and Architect and ask about what needs to be validated by doing performance testing. This will also help to bridge the gap between business expectations and what technology offers. Another approach to collect the requirement if the system is already in production is to get the baseline from the production. There will be application logs available for most of the system which will be helpful in getting the production baseline in terms of response time, number of users, resource utilization like CPU, Memory etc..

How to get performance baseline from the existing system?

If the system is  a web application, it will have access logs in web server which will give the details like usage patterns like number of page views per hour, day or month, response time, number of users etc... This can be analyzed to determine the baseline performance and arrive it system usage patterns.

Can functional tester do Performance Testing?

It is a difficult to answer. Performance Testing is little bit different from functional testing. Functional testing mostly deals with functionality verification at the same time Performance tester verifies the overall system performance with one user and realistic user load. It requires the tool skills. It also requires basic knowledge of the technology in which application is developed.


What is end user experience?

In simple terms, it is the time taken for the end user to view the application. It is the combination of server response time, network time, client rendering time which includes the time to execute the client side Java/VB scripts, Applet, ActiveX etc... Most of the load testing tools do not run the client side script. So the load test tool response time is not really the end user experience, but it is the request response time from the server including the network time but excluding the rendering time.


Load testing environment is not similar to production environment. Will this give correct result?

Answer is No. Ideally, the load testing environment should have same hardware and software configuration to get the correct results.


Is there a way around if the testing environment is not similar to production?

Yes. In most of the companies, testing environment may not be similar to production. For example Production has 4 web servers each with 2 CPU 1.5 GHz speed. But the performance environment has 2 web servers each with 2 CPU 1.5 GHz speed. The best approach to simulate this is to scale down the load. In this case, scale down the load to half of the expected load. For example, the expected load is 100 users creating 20000 transactions, performance test should target for 50 users creating 10000 transactions. But in reality, hardware configuration of performance environment will be different in terms of number of servers, number of CPUs, CPU speed and memory. It will make the scenario more complex.


What is performance bottleneck? 

Performance bottleneck can be anything which is affecting the performance of the system. Some of the common bottlenecks are poorly written code consumes lot of CPU or Memory, less number of application server threads even with less CPU utilization, lack of Physical memory and finally it can be lack of CPU power. Bottlenecks can be in network too.


Can functional testing tools be used for Load Testing?

Functional Testing tools are designed to verify the application functionality including the client side validations. Also, these tools require lot of hardware resources to run. Compared to load testing tools, functional tools will require huge amount of hardware resources. So it is not a good option for Load Testing. But still it can be considered if the numbers of users are very minimal.


Response time from the Load Testing script is different from manual testing. Is it correct?

Before answering this, you need to make sure that test is conducted for same navigation, number of users, same input values. Test for one user both manually and through the script. If the response times are different, it means tool is not simulating what application does. Tools will have its own restrictions. Refer the section for some of the details on tools. Always refer the tools documentation for tool specific details.

Can we do load testing for an application where client-server traffic can not be captured?

Yes. Recording is a feature in load testing tool to help the test engineer to easily capture the client-server traffic. If the tool can not interpret the client-server traffic or the client-server traffic is in a format, say binary, which can’t be readable, we should look for option to manually code the script to simulate the traffic. In order to do that, the performance test engineer should know the APIs using which client talks to the server. The API should be supported in the load testing tool as well. The white paper on SQL Anywhere performance testing in LoadRunner gives an innovative approach for application which can’t be recorded in a traditional approach. 


Performance Testing Fun