chore(benchmark): align JCasbin benchmarks with go-casbin standard cases #477
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Aligns JCasbin's benchmark suite with go-casbin's 11 standard scenarios to enable direct performance comparisons. Previously, benchmarks had inconsistent data scales, undocumented test parameters, and manual iteration loops that skewed JMH measurements.
Changes
Standardized all 11 benchmark scenarios to match go-casbin:
Fixed JMH benchmark methodology:
Unified JMH parameters across all benchmarks: 2 forks, 3 warmup iterations, 5 measurement iterations, 1 thread
Added comprehensive Javadoc documenting data scale, policy structure, test cases, and recommended parameters for each scenario
Created benchmark suite README with scenario comparison table, execution instructions, and deterministic generation patterns
Technical Details
Data generation uses integer division for predictable distribution:
All benchmarks now produce bit-identical results across runs, enabling reliable performance tracking and cross-implementation comparison.
Original prompt
This section details on the original issue you should resolve
<issue_title>chore(benchmark): align jcasbin benchmarks with go-casbin standard cases</issue_title>
<issue_description>## Role Definition
You will act as an experienced Java/JMH performance engineer, familiar with Casbin access-control models and the differences between Golang and Java.
Background
The official Casbin performance monitoring page lists multiple benchmark scenarios, including ACL, RBAC (large / medium / small), RBAC extended scenarios (resource roles, domains/tenants), ABAC, RESTful, Deny-override, and Priority.
The table describes in detail the number of rules and objects for each scenario. For example:
The current jcasbin repository contains only part of the JMH benchmark tests.
The number of cases and their generation logic are not consistent with go-casbin, which makes fair horizontal comparison impossible.
We need to complete the missing benchmark scenarios in jcasbin and implement deterministic generation logic, so that anyone running the benchmarks with the same command can obtain identical results.
Task Requirements
1. Analyze go-casbin benchmark cases
Identify and implement all of the following standard benchmark scenarios (this is the full required scenario set). For each scenario, treat the rule size and object counts below as canonical and ensure the Java-side benchmark matches these scales:
Make sure the implementation work that follows uses this list as the authoritative checklist of what must exist in
jcasbin(either already present or newly added), and that each scenario’s benchmark logic and data scale match the counts above exactly.2. Compare existing jcasbin benchmark cases
Search the jcasbin repository for existing JMH benchmark classes and identify:
For missing scenarios, new classes must be added.
For inconsistently named scenarios, class names must be adjusted to align with go-casbin.
3. Generate Java benchmark logic
org.openjdk.jmh) as the benchmarking framework.@State(Scope.Benchmark)to manage the Enforcer instance and data initialization.For each scenario, deterministically generate policies and role/user relationships according to go-casbin logic:
ACL
addPolicyandaddGroupingPolicyRBAC
RBAC small
RBAC medium / large
RBAC with resource roles
RBAC with domains/tenants
Enforcer.addGroupingPolicydomain::userformatABAC
enforceinside the@BenchmarkmethodRESTful
Deny-override and Priority
Each class must provide one
@Benchmarkmethod that callsenforcewith the same parameters used in Go(e.g. RBAC small must always use
user501,data9).No randomness is allowed.
Loop variables must generate strings using integer division, ensuring the generated data is bit-for-bit identical.
4. Execution parameters and reproducibility guarantees
This means:
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.