diff --git a/README.md b/README.md index f214b5ff..a2e17308 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,10 @@ [![Maven Central](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-benchmark-runtime.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:%22org.jetbrains.kotlinx%22%20AND%20a:%22kotlinx-benchmark-runtime%22) [![Gradle Plugin Portal](https://img.shields.io/maven-metadata/v?label=Gradle%20Plugin&metadataUrl=https://plugins.gradle.org/m2/org/jetbrains/kotlinx/kotlinx-benchmark-plugin/maven-metadata.xml)](https://plugins.gradle.org/plugin/org.jetbrains.kotlinx.benchmark) -kotlinx-benchmark is a toolkit for running benchmarks for multiplatform code written in Kotlin. +`kotlinx-benchmark` is a toolkit for running benchmarks for multiplatform code written in Kotlin. +It is designed to work with Kotlin/JVM, Kotlin/JS, Kotlin/Native, and Kotlin/Wasm (experimental) targets. + +To get started, ensure you're using Kotlin 1.9.20 or newer and Gradle 7.4 or newer. ## Features @@ -19,8 +22,7 @@ kotlinx-benchmark is a toolkit for running benchmarks for multiplatform code wri -- [Using in Your Projects](#using-in-your-projects) - - [Project Setup](#project-setup) +- [Setting Up a Kotlin Multiplatform Project for Benchmarking](#setting-up-a-kotlin-multiplatform-project-for-benchmarking) - [Target-specific configurations](#target-specific-configurations) - [Kotlin/JVM](#kotlinjvm) - [Kotlin/JS](#kotlinjs) @@ -42,17 +44,13 @@ kotlinx-benchmark is a toolkit for running benchmarks for multiplatform code wri - [Setting Up a Separate Source Set for Benchmarks](docs/separate-benchmark-source-set.md) - [Overview of Tasks Provided by kotlinx-benchmark Gradle Plugin](docs/tasks-overview.md) -## Using in Your Projects - -The `kotlinx-benchmark` library is designed to work with Kotlin/JVM, Kotlin/JS, Kotlin/Native, and Kotlin/Wasm (experimental) targets. -To get started, ensure you're using Kotlin 1.9.20 or newer and Gradle 7.4 or newer. +## Setting Up a Kotlin Multiplatform Project for Benchmarking -### Project Setup +To configure a Kotlin Multiplatform project for benchmarking, follow the steps below. +If you want to benchmark only Kotlin/JVM and Java code, you may refer to our [comprehensive guide](docs/kotlin-jvm-project-setup.md) +dedicated to setting up benchmarking in those specific project types. -Follow the steps below to set up a Kotlin Multiplatform project for benchmarking. - -
-Kotlin DSL +
Kotlin DSL 1. **Applying Benchmark Plugin**: Apply the benchmark plugin. @@ -98,10 +96,9 @@ Follow the steps below to set up a Kotlin Multiplatform project for benchmarking } ``` -
+
-
-Groovy DSL +
Groovy DSL 1. **Applying Benchmark Plugin**: Apply the benchmark plugin. @@ -147,7 +144,7 @@ Follow the steps below to set up a Kotlin Multiplatform project for benchmarking } ``` -
+
### Target-specific configurations @@ -191,8 +188,7 @@ To run benchmarks in Kotlin/JVM: } ``` -
- Explanation +
Explanation Assume that you've annotated each of your benchmark classes with `@State(Scope.Benchmark)`: @@ -208,8 +204,8 @@ To run benchmarks in Kotlin/JVM: } ``` - In Kotlin, classes are `final` by default, which means they can't be overridden. - This is incompatible with the operation of the Java Microbenchmark Harness (JMH), which kotlinx-benchmark uses under the hood for running benchmarks on JVM. + In Kotlin, classes are `final` by default, which means they can't be overridden. + This conflicts with the Java Microbenchmark Harness (JMH) operation, which `kotlinx-benchmark` uses under the hood for running benchmarks on JVM. JMH requires benchmark classes and methods to be `open` to be able to generate subclasses and conduct the benchmark. This is where the `allopen` plugin comes into play. With the plugin applied, any class annotated with `@State` is treated as `open`, which allows JMH to work as intended: @@ -307,7 +303,8 @@ To run benchmarks in Kotlin/Wasm: } ``` -Note: Kotlin/Wasm is an experimental compilation target for Kotlin. It may be dropped or changed at any time. +Note: Kotlin/Wasm is an experimental compilation target for Kotlin. It may be dropped or changed at any time. Refer to +[Kotlin/Wasm documentation](https://kotlinlang.org/docs/wasm-overview.html) for up-to-date information about the target stability. ### Writing Benchmarks @@ -393,11 +390,11 @@ See [writing benchmarks](docs/writing-benchmarks.md) for a complete guide for wr To run your benchmarks in all registered platforms, run `benchmark` Gradle task in your project. To run only on a specific platform, run `Benchmark`, e.g., `jvmBenchmark`. -For more details about the tasks created by the kotlinx-benchmark plugin, refer to [this guide](docs/tasks-overview.md). +For more details about the tasks created by the `kotlinx-benchmark` plugin, refer to [this guide](docs/tasks-overview.md). ### Benchmark Configuration Profiles -The kotlinx-benchmark library provides the ability to create multiple configuration profiles. The `main` configuration is already created by the Toolkit. +The `kotlinx-benchmark` library provides the ability to create multiple configuration profiles. The `main` configuration is already created by the toolkit. Additional profiles can be created as needed in the `configurations` section of the `benchmark` block: ```kotlin @@ -430,9 +427,9 @@ Refer to our [detailed documentation](docs/separate-benchmark-source-set.md) on ## Examples -To help you better understand how to use the kotlinx-benchmark library, we've provided an [examples](examples) subproject. +To help you better understand how to use the `kotlinx-benchmark` library, we've provided an [examples](examples) subproject. These examples showcase various use cases and offer practical insights into the library's functionality. ## Contributing -We welcome contributions to kotlinx-benchmark! If you want to contribute, please refer to our [Contribution Guidelines](CONTRIBUTING.md). \ No newline at end of file +We welcome contributions to `kotlinx-benchmark`! If you want to contribute, please refer to our [Contribution Guidelines](CONTRIBUTING.md). \ No newline at end of file diff --git a/docs/kotlin-jvm-project-setup.md b/docs/kotlin-jvm-project-setup.md new file mode 100644 index 00000000..f34f62e9 --- /dev/null +++ b/docs/kotlin-jvm-project-setup.md @@ -0,0 +1,293 @@ +# Setting Up Kotlin/JVM and Java Projects for Benchmarking + +In this guide, we'll take you through the steps to set up a Kotlin/JVM or Java project +for benchmarking using `kotlinx-benchmark`. + +## Step-by-step Setup Guide + +To configure Kotlin/JVM and Java projects for benchmarking, follow these steps: + +1. Apply the benchmark plugin: + +
Kotlin DSL + + ```kotlin + // build.gradle.kts + plugins { + id("org.jetbrains.kotlinx.benchmark") version "0.4.10" + } + ``` + +
+ +
Groovy DSL + + ```groovy + // build.gradle + plugins { + id 'org.jetbrains.kotlinx.benchmark' version '0.4.10' + } + ``` + +
+ +2. Make sure to include the Gradle Plugin Portal for plugin lookup in the list of repositories: + +
Kotlin DSL + + ```kotlin + // settings.gradle.kts + pluginManagement { + repositories { + gradlePluginPortal() + } + } + ``` + +
+ +
Groovy DSL + + ```groovy + // settings.gradle + pluginManagement { + repositories { + gradlePluginPortal() + } + } + ``` + +
+ +3. Next, add the `kotlinx-benchmark-runtime` dependency to the project: + +
Kotlin DSL + + ```kotlin + // build.gradle.kts + dependencies { + implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.10") + } + ``` + +
+ +
Groovy DSL + + ```groovy + // build.gradle + dependencies { + implementation 'org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.10' + } + ``` + +
+ +4. Include Maven Central in the list of repositories for dependency lookup: + +
Kotlin DSL + + ```kotlin + // build.gradle.kts + repositories { + mavenCentral() + } + ``` + +
+ +
Groovy DSL + + ```groovy + // build.gradle + repositories { + mavenCentral() + } + ``` + +
+ +5. Apply [allopen plugin](https://kotlinlang.org/docs/all-open-plugin.html) if you have benchmark classes in Kotlin: + +
Kotlin DSL + + ```kotlin + // build.gradle.kts + plugins { + kotlin("plugin.allopen") version "1.9.20" + } + + allOpen { + annotation("org.openjdk.jmh.annotations.State") + } + ``` + +
+ +
Groovy DSL + + ```groovy + // build.gradle + plugins { + id 'org.jetbrains.kotlin.plugin.allopen' version "1.9.20" + } + + allOpen { + annotation("org.openjdk.jmh.annotations.State") + } + ``` + +
+ +
Explanation + + Assume that you've annotated each of your benchmark classes with `@State(Scope.Benchmark)`: + + ```kotlin + // MyBenchmark.kt + @State(Scope.Benchmark) + class MyBenchmark { + // Benchmarking-related methods and variables + @Benchmark + fun benchmarkMethod() { + // benchmarking logic + } + } + ``` + + In Kotlin, classes are `final` by default, which means they can't be overridden. + This conflicts with the Java Microbenchmark Harness (JMH) operation, which `kotlinx-benchmark` uses under the hood for running benchmarks on JVM. + JMH requires benchmark classes and methods to be `open` to be able to generate subclasses and conduct the benchmark. + + This is where the `allopen` plugin comes into play. With the plugin applied, any class annotated with `@State` is treated as `open`, which allows JMH to work as intended: + + ```kotlin + // build.gradle.kts + plugins { + kotlin("plugin.allopen") version "1.9.20" + } + + allOpen { + annotation("org.openjdk.jmh.annotations.State") + } + ``` + + This configuration ensures that your `MyBenchmark` class and its `benchmarkMethod` function are treated as `open`. + +
+ + You can alternatively mark your benchmark classes and methods `open` manually, but using the `allopen` plugin enhances code maintainability. + +6. Designate the `main` source set as a benchmark target: + +
Kotlin DSL + + ```kotlin + // build.gradle.kts + benchmark { + targets { + register("main") + } + } + ``` + +
+ +
Groovy DSL + + ```kotlin + // build.gradle + benchmark { + targets { + register("main") + } + } + ``` + +
+ + This informs the `kotlinx-benchmark` tool that benchmarks reside within `main` source set. + +## Writing Benchmarks + +After completing the setup of your project, you're ready to dive into writing benchmarks. +`kotlinx-benchmark` leverages the Java Microbenchmark Harness (JMH) toolkit to execute benchmarks on the JVM. +As a result, it automatically includes the necessary dependency on JMH, allowing you to harness its API for crafting benchmarks: + +```kotlin +import org.openjdk.jmh.annotations.* +import java.util.concurrent.* +import kotlin.math.cos +import kotlin.math.sqrt +import kotlin.random.Random + +@State(Scope.Benchmark) +@Fork(1) +@Warmup(iterations = 10) +@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS) +class MyBenchmark { + private var data = 0.0 + + @Setup + fun setUp() { + data = Random.nextDouble() + } + + @Benchmark + fun sqrtBenchmark(): Double { + return sqrt(data) + } + + @Benchmark + fun cosBenchmark(): Double { + return cos(data) + } +} +``` + +See [writing benchmarks](writing-benchmarks.md) for a complete guide for writing benchmarks. + +### Running Benchmarks + +To run your benchmarks, run `benchmark` Gradle task in the project. +In the terminal, navigate to the project's root directory and run `./gradlew benchmark`. + +For more details about the tasks created by the `kotlinx-benchmark` plugin, refer to [this guide](tasks-overview.md). + +### Benchmark Configuration Profiles + +The `kotlinx-benchmark` library provides the ability to create multiple configuration profiles. The `main` configuration is already created by the toolkit. +Additional profiles can be created as needed in the `configurations` section of the `benchmark` block: + +```kotlin +// build.gradle.kts +benchmark { + configurations { + named("main") { + warmups = 20 + iterations = 10 + iterationTime = 3 + iterationTimeUnit = "s" + } + register("smoke") { + include("") + warmups = 5 + iterations = 3 + iterationTime = 500 + iterationTimeUnit = "ms" + } + } +} +``` + +Refer to our [comprehensive guide](configuration-options.md) to learn about configuration options and how they affect benchmark execution. + +### Separate source set for benchmarks + +Often you want to have benchmarks in the same project, but separated from main code, much like tests. +Refer to our [detailed documentation](separate-benchmark-source-set.md) on configuring your project to set up a separate source set for benchmarks. + +## Examples + +Explore sample [Kotlin/JVM](/examples/kotlin) and [Java](/examples/java) benchmarking projects that use `kotlinx-benchmark`. +These examples showcase how to structure benchmarking projects using `kotlinx-benchmark`.