top of page
  • Writer's pictureNick

Data Center Performance Testing: A Closer Look at AssertThat BDD & Cucumber Plugin

Updated: Nov 10, 2023

As a vendor providing the Assertthat BDD, Cucumber & Test Management plugin for Jira Data Center, it is crucial for us to ensure that our application meets the highest performance standards. To achieve this, we undergo an annual/periodic performance testing process that complies with Atlassian's Marketplace approval requirements. In this blog post, we'll look into the specifics of this performance testing process and the necessary steps we take to ensure optimal functionality in Data Center environments.


The Data Center Performance Testing Requirement

Atlassian Marketplace's approval process necessitates rigorous performance testing, ensuring that our plugin functions seamlessly in environments equivalent to our Data Center customers. The testing environment, akin to a 4 node Jira instance type, plays a significant role in accurately gauging performance impact. Key requirements for our performance testing include:

  1. Test Environment Setup:

    1. A clustered test host with a 4 node configuration (32 CPU cores minimum).

  2. Preloaded Large Dataset:

    1. Using a base dataset that mirrors the one included in the Data Center App Performance Toolkit to emulate real-world usage.

  3. Test Scenarios:

    1. Performance testing on a 4 node Data Center instance, both with and without our app installed.

    2. Testing various actions on the base Jira product, including login, issue creation, viewing and editing issues, dashboard and board views, commenting, JQL search, and logout.

  4. Concurrency Details:

    1. Simulating a substantial load with 200 concurrent users performing 54,500 actions per hour, reflecting real-world usage patterns.

  5. Test Duration:

    1. Each performance run lasts for 45 minutes, capturing a substantial amount of data to analyze performance effectively.


Conducting Performance Testing

To meet the performance testing requirements, we follow a structured process:

  1. Environment Setup: Configure a test host that mirrors the Data Center environment, meeting the outlined specifications.

  2. Dataset Preparation: Load a preconfigured large dataset similar to the Data Center App Performance Toolkit to simulate realistic usage scenarios.

  3. Test Execution: Conduct performance tests on the Data Center instance, both with and without our app installed, using the specified test scenarios and concurrency levels.

  4. Data Collection and Analysis:

    1. Capture performance metrics for each transaction, listing the response times with and without our app installed.

    2. Analyze the data to identify any performance regressions and areas for optimization.


Performance Regression Report

Following each performance test, we generate a comprehensive performance regression report. This report includes a detailed analysis of the tested transactions, highlighting response times with and without our app.


Scalability Testing

Scalability testing is vital for assessing how well our plugin performs when operating across multiple nodes in a Jira Data Center (DC) deployment. The main goal is to understand the impact as the environment is scaled to different node configurations.


Tests are repeated on Jira DC environments with the following configurations:

  • 1 node Jira DC environment

  • 2 node Jira DC environment

  • 4 node Jira DC environment

By conducting these tests on different node setups, we gain valuable insights into how the app scales. This ensures that our plugin can effectively meet the demands of our users as their requirements grow and evolve.

Comparison of performance transactions for 1,2 and 4 node Data Center instance
Comparison of performance transactions for 1,2 and 4 node Data Center instance

Conclusion

At AssertThat, ensuring top-notch performance of our BDD, Cucumber & Test Management plugin for Jira Data Center is fundamental. Our commitment to compliance with Atlassian's performance testing requirements guarantees a reliable and high-performing experience for our users. If you have any questions or need further information about our performance testing processes, feel free to reach out. Your satisfaction with our high-performance plugin is our priority.

40 views0 comments

Recent Posts

See All

Comments


bottom of page