Skip to content

Add performance test results for API-M 4.7.0#11160

Open
SavinduDimal wants to merge 6 commits intowso2:masterfrom
SavinduDimal:master
Open

Add performance test results for API-M 4.7.0#11160
SavinduDimal wants to merge 6 commits intowso2:masterfrom
SavinduDimal:master

Conversation

@SavinduDimal
Copy link
Copy Markdown
Contributor

@SavinduDimal SavinduDimal commented Apr 13, 2026

This pull request updates the API Manager performance test documentation to reflect newer test environments, revised throughput and latency results, and improved visualizations for performance metrics. The most significant changes include updating the Java version used in tests, revising throughput and percentile data, refreshing result links, and adding new CPU/IOPS graphs under different load conditions.

Test Environment Updates:

  • Updated the Java version used in performance tests from Temurin JDK 17 to Temurin JDK 21 in both performance-test-results-token-persistence.md and performance-tests-results.md. [1] [2]

Performance Results and Data Updates:

  • Revised throughput values for different grant types in the results table to reflect new measurements.
  • Updated response time percentile values (99th percentile for Echo API at 100 users from 133ms to 124ms) and corrected the image path for the corresponding graph.
  • Updated the link to the complete performance test results to point to the latest benchmark summary.

Visualization and Documentation Improvements:

  • Removed API Manager CPU usage graphs for different grant types and added new sections with side-by-side graphs for APIM CPU, DB CPU, and DB IOPS under a fixed throughput scenario (100 TPS), providing clearer insights into system behavior under constrained load. [1] [2]

Summary by CodeRabbit

  • Documentation
    • Updated performance docs to reflect Temurin JDK 21 in the testing environment.
    • Revised throughput table headers and updated throughput numbers for Client Credentials, Password, and Refresh grants.
    • Clarified wording that refresh tokens are generated as JWTs by default without persistence for the Password grant.
    • Replaced some CPU-vs-time charts with new APIM/DB CPU and DB IOPS visualizations under a fixed 100 TPS scenario.
    • Updated linked test results reference and corrected a response-time percentile and image path.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 13, 2026

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)
  • en/docs/assets/img/setup-and-install/performance-test-results/response_time_summary_1kib_0ms.png is excluded by !**/*.png

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 82439a6e-792b-4676-bf1f-8fd74c6a9975

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

Two documentation files updated with new test data: Java runtime bumped to Temurin JDK 21, throughput and percentile values revised, a results repo link updated, some APIM CPU graphs removed, and new CPU/DB IOPS sections added for fixed 100 TPS.

Changes

Cohort / File(s) Summary
Token persistence results
en/docs/reference/performance-test-results-token-persistence.md
Updated Java runtime to Temurin JDK 21; renamed "Without Persistence" header to note "Default since API-M 4.7.0"; revised throughput values for Client Credentials / Password / Refresh; clarified Password grant wording re: JWT refresh token generation without persistence; removed per-grant APIM CPU time-series graphs from the first note; added new APIM CPU, DB CPU, and DB total IOPS charts for Client Credentials, Password, Refresh under a fixed 100 TPS section; ensured trailing newline.
General performance results
en/docs/reference/performance-tests-results.md
Updated Java runtime to Temurin JDK 21; updated linked complete-results path to a new test run ID; adjusted Echo API 99th percentile at 100 concurrent users from 133ms to 124ms; corrected image filename from response_time_summary_1KiB_0ms.png to response_time_summary_1kib_0ms.png.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 I hopped from seventeen up to twenty-one,
Docs refreshed, new graphs beneath the sun,
Throughput tuned and percentiles sleeker,
CPU and IOPS now speak a bit clearer,
A tiny rabbit cheers — the updates are done!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description provides a clear overview of changes (Java version updates, performance data revisions, visualization improvements) and references specific files modified. However, it does not follow the required repository template structure with sections like Purpose, Goals, Approach, User Stories, Release Notes, etc. Reformat the description to match the repository template structure, including sections for Purpose, Goals, Approach, Release Notes, and Test Environment to ensure consistency with project standards.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Add performance test results for API-M 4.7.0' directly and specifically summarizes the main change—updating performance test documentation with new results for the API Manager 4.7.0 version.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@en/docs/reference/performance-tests-results.md`:
- Line 77: Fix the typo in the vendor name on the line that currently reads
"Java version is Termurin JDK 21" by changing "Termurin" to the correct
"Temurin" so the line becomes "Java version is Temurin JDK 21"; locate and
update that exact string in the document to correct the JDK distribution name.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 00dec1dc-cf02-4384-85ad-11046820947d

📥 Commits

Reviewing files that changed from the base of the PR and between 43f6083 and 2bee45b.

⛔ Files ignored due to path filters (23)
  • en/docs/assets/img/setup-and-install/performance-test-results/gc_0ms.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lineplot-average-response-time-2g-1kib-0ms.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lineplot-throughput-2g-1kib-0ms.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lmplot-average-response-time-concurrent-users-2g.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lmplot-average-response-time-message-size-2g.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lmplot-throughput-concurrent-users-2g.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/lmplot-throughput-message-size-2g.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/response_time_summary_1KiB_0ms.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/cc_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/cc_iops.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/cc_fixed_apim_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/cc_fixed_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/cc_fixed_db_iops.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/pwd_fixed_apim_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/pwd_fixed_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/pwd_fixed_db_iops.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/refresh_fixed_apim_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/refresh_fixed_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/fixed_tps_metrics/refresh_fixed_db_iops.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/pwd_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/pwd_iops.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/refresh_db_cpu.png is excluded by !**/*.png
  • en/docs/assets/img/setup-and-install/performance-test-results/token/refresh_iops.png is excluded by !**/*.png
📒 Files selected for processing (2)
  • en/docs/reference/performance-test-results-token-persistence.md
  • en/docs/reference/performance-tests-results.md

Comment thread en/docs/reference/performance-tests-results.md Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
en/docs/reference/performance-tests-results.md (1)

89-89: Consider more descriptive link text.

The link text "here" is non-descriptive. Readers scanning the document or using assistive technologies would benefit from more explicit text.

♻️ Proposed improvement
-The complete results can be found [here](https://github.com/wso2/performance-apim/blob/performance-test-644-2026-04-07_07-39-24/performance/benchmarks/summary.md).
+The [complete benchmark results](https://github.com/wso2/performance-apim/blob/performance-test-644-2026-04-07_07-39-24/performance/benchmarks/summary.md) are available in the performance repository.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@en/docs/reference/performance-tests-results.md` at line 89, Replace the
non-descriptive link text in the sentence "the complete results can be found
[here](https://github.com/wso2/performance-apim/blob/performance-test-644-2026-04-07_07-39-24/performance/benchmarks/summary.md)"
with a clear, accessible label (for example "complete performance test results
on GitHub" or "performance-apim benchmark summary") so the link text describes
the destination and purpose; update only the link text while keeping the same
URL.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@en/docs/reference/performance-tests-results.md`:
- Line 118: Update the image filename in the markdown so its case matches the
actual asset: replace any occurrences of "response_time_summary_1kib_0ms.png"
with "response_time_summary_1KiB_0ms.png" in the document (the image reference
line and any linked/img paths), e.g., update the tag referencing
response_time_summary_1kib_0ms.png to use response_time_summary_1KiB_0ms.png to
avoid broken links.

---

Nitpick comments:
In `@en/docs/reference/performance-tests-results.md`:
- Line 89: Replace the non-descriptive link text in the sentence "the complete
results can be found
[here](https://github.com/wso2/performance-apim/blob/performance-test-644-2026-04-07_07-39-24/performance/benchmarks/summary.md)"
with a clear, accessible label (for example "complete performance test results
on GitHub" or "performance-apim benchmark summary") so the link text describes
the destination and purpose; update only the link text while keeping the same
URL.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b4d45047-4f88-4c24-a948-e85e092b81e5

📥 Commits

Reviewing files that changed from the base of the PR and between 2bee45b and 5bd3228.

📒 Files selected for processing (2)
  • en/docs/reference/performance-test-results-token-persistence.md
  • en/docs/reference/performance-tests-results.md

Comment thread en/docs/reference/performance-tests-results.md
Add temp image name to bypass macOS file name case insensitive issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant