diff --git a/README.md b/README.md index 63937bf0f9..95490162f9 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ Explore [**documentation**](https://kotlin.github.io/dataframe) for details. You could find the following articles there: * [Guides and Examples](https://kotlin.github.io/dataframe/guides-and-examples.html) -* [Get started with Kotlin DataFrame](https://kotlin.github.io/dataframe/gettingstarted.html) +* [Get started with Kotlin DataFrame](https://kotlin.github.io/dataframe/setup.html) * [Working with Data Schemas](https://kotlin.github.io/dataframe/schemas.html) * [Setup compiler plugin in Gradle project](https://kotlin.github.io/dataframe/compiler-plugin.html) * [Full list of all supported operations](https://kotlin.github.io/dataframe/operations.html) @@ -61,7 +61,7 @@ Check out this [notebook with new features](examples/notebooks/feature_overviews ## Setup > For more detailed instructions on how to get started with Kotlin DataFrame, refer to the -> [Getting Started](https://kotlin.github.io/dataframe/gettingstarted.html). +> [Getting Started](https://kotlin.github.io/dataframe/setup.html). ### Kotlin Notebook diff --git a/docs/StardustDocs/d.tree b/docs/StardustDocs/d.tree index 95f4c84255..8724fc4475 100644 --- a/docs/StardustDocs/d.tree +++ b/docs/StardustDocs/d.tree @@ -14,12 +14,12 @@ - - - - - - + + + + + + @@ -209,4 +209,5 @@ + diff --git a/docs/StardustDocs/images/datalore-1.png b/docs/StardustDocs/images/datalore-1.png deleted file mode 100644 index 9206d144d2..0000000000 Binary files a/docs/StardustDocs/images/datalore-1.png and /dev/null differ diff --git a/docs/StardustDocs/images/datalore-1_dark.png b/docs/StardustDocs/images/datalore-1_dark.png deleted file mode 100644 index 9206d144d2..0000000000 Binary files a/docs/StardustDocs/images/datalore-1_dark.png and /dev/null differ diff --git a/docs/StardustDocs/images/datalore-2.png b/docs/StardustDocs/images/datalore-2.png deleted file mode 100644 index 090727fe8d..0000000000 Binary files a/docs/StardustDocs/images/datalore-2.png and /dev/null differ diff --git a/docs/StardustDocs/images/datalore-2_dark.png b/docs/StardustDocs/images/datalore-2_dark.png deleted file mode 100644 index 090727fe8d..0000000000 Binary files a/docs/StardustDocs/images/datalore-2_dark.png and /dev/null differ diff --git a/docs/StardustDocs/images/dataloreCreateNotebook.png b/docs/StardustDocs/images/dataloreCreateNotebook.png new file mode 100644 index 0000000000..703dd8899e Binary files /dev/null and b/docs/StardustDocs/images/dataloreCreateNotebook.png differ diff --git a/docs/StardustDocs/images/dataloreSetupNotebook.png b/docs/StardustDocs/images/dataloreSetupNotebook.png new file mode 100644 index 0000000000..141a3f807e Binary files /dev/null and b/docs/StardustDocs/images/dataloreSetupNotebook.png differ diff --git a/docs/StardustDocs/topics/FAQ.md b/docs/StardustDocs/topics/FAQ.md index 99ed9ec049..61a80bc0a6 100644 --- a/docs/StardustDocs/topics/FAQ.md +++ b/docs/StardustDocs/topics/FAQ.md @@ -27,7 +27,7 @@ To stay updated on progress, subscribe to the Yes — Kotlin DataFrame can be used in Android projects. There is no dedicated Android artifact yet, but you can include the standard **JVM artifact** -by setting up a [custom Gradle configuration](gettingStartedGradleAdvanced.md). +by setting up a [custom Gradle configuration](SetupAndroid.md). ## How to start with Kotlin DataFrame ? @@ -41,7 +41,7 @@ to explore various use cases and deepen your understanding of Kotlin DataFrame. For the best experience, Kotlin DataFrame is most effective in an interactive environment. -- **[Kotlin Notebook](gettingStartedKotlinNotebook.md)** is ideal for exploring Kotlin DataFrame. +- **[Kotlin Notebook](SetupKotlinNotebook.md)** is ideal for exploring Kotlin DataFrame. Everything works out of the box — interactivity, rich rendering of DataFrames and plots. You can instantly see the results of each operation, view the contents of your DataFrames after every transformation, inspect individual rows and columns, and explore data step-by-step in a live and interactive way. @@ -109,7 +109,7 @@ in a **type-safe** and **name-safe** way. They are generated automatically when working with Kotlin DataFrame in: -- [Kotlin Notebook](gettingStartedKotlinNotebook.md), where extension properties are generated +- [Kotlin Notebook](SetupKotlinNotebook.md), where extension properties are generated after each cell execution. - A Kotlin project in [IntelliJ IDEA](https://www.jetbrains.com/idea/) with the [](Compiler-Plugin.md) enabled, where the properties are generated at compile time. diff --git a/docs/StardustDocs/topics/Home.topic b/docs/StardustDocs/topics/Home.topic index aad8c8f117..70c6bfb631 100644 --- a/docs/StardustDocs/topics/Home.topic +++ b/docs/StardustDocs/topics/Home.topic @@ -19,7 +19,7 @@ First steps - + Reading from files: CSV, JSON, ApacheArrow diff --git a/docs/StardustDocs/topics/collectionsInterop.md b/docs/StardustDocs/topics/collectionsInterop.md index 8684e436b4..169f0f4f9a 100644 --- a/docs/StardustDocs/topics/collectionsInterop.md +++ b/docs/StardustDocs/topics/collectionsInterop.md @@ -78,7 +78,7 @@ val df2 = df.add("c") { a + b } To enable extension properties generation, you should use the [DataFrame plugin](schemasGradle.md) -for Gradle or the [Kotlin Jupyter kernel](gettingStartedJupyterNotebook.md) +for Gradle or the [Kotlin Jupyter kernel](SetupJupyter.md) diff --git a/docs/StardustDocs/topics/dataSources/ApacheArrow.md b/docs/StardustDocs/topics/dataSources/ApacheArrow.md index cf9deb045e..8aaca8bb2e 100644 --- a/docs/StardustDocs/topics/dataSources/ApacheArrow.md +++ b/docs/StardustDocs/topics/dataSources/ApacheArrow.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from and writing to Apache Arrow files. Requires the [`dataframe-arrow` module](Modules.md#dataframe-arrow), which is included by default in the general [`dataframe`](Modules.md#dataframe-general) artifact -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. > Make sure to follow the > [Apache Arrow Java compatibility guide](https://arrow.apache.org/docs/java/install.html#java-compatibility) diff --git a/docs/StardustDocs/topics/dataSources/CSV-TSV.md b/docs/StardustDocs/topics/dataSources/CSV-TSV.md index 1061620d18..0a65c91852 100644 --- a/docs/StardustDocs/topics/dataSources/CSV-TSV.md +++ b/docs/StardustDocs/topics/dataSources/CSV-TSV.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from and writing to CSV and TSV files. Requires the [`dataframe-csv` module](Modules.md#dataframe-csv), which is included by default in the general [`dataframe`](Modules.md#dataframe-general) -artifact and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +artifact and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. ## Read diff --git a/docs/StardustDocs/topics/dataSources/Excel.md b/docs/StardustDocs/topics/dataSources/Excel.md index 349cc81eee..44fc2a7461 100644 --- a/docs/StardustDocs/topics/dataSources/Excel.md +++ b/docs/StardustDocs/topics/dataSources/Excel.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from and writing to Excel files in both `.xls` Requires the [`dataframe-excel` module](Modules.md#dataframe-excel), which is included by default in the general [`dataframe`](Modules.md#dataframe-general) -artifact and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +artifact and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. ## Read diff --git a/docs/StardustDocs/topics/dataSources/JSON.md b/docs/StardustDocs/topics/dataSources/JSON.md index 96a926c1a7..3323d0a4d5 100644 --- a/docs/StardustDocs/topics/dataSources/JSON.md +++ b/docs/StardustDocs/topics/dataSources/JSON.md @@ -16,7 +16,7 @@ Kotlin DataFrame supports reading from and writing to JSON files. Requires the [`dataframe-json` module](Modules.md#dataframe-json), which is included by default in the general [`dataframe`](Modules.md#dataframe-general) -artifact and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) +artifact and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. > Kotlin DataFrame is suitable only for working with table-like structured JSON — diff --git a/docs/StardustDocs/topics/dataSources/sql/H2.md b/docs/StardustDocs/topics/dataSources/sql/H2.md index fc266dce36..a288628fd7 100644 --- a/docs/StardustDocs/topics/dataSources/sql/H2.md +++ b/docs/StardustDocs/topics/dataSources/sql/H2.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from an [H2](https://www.h2database.com/html/m Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [the official H2 JDBC driver](https://www.h2database.com/html/main.html): diff --git a/docs/StardustDocs/topics/dataSources/sql/MariaDB.md b/docs/StardustDocs/topics/dataSources/sql/MariaDB.md index 41d09f58fb..3c23c56c13 100644 --- a/docs/StardustDocs/topics/dataSources/sql/MariaDB.md +++ b/docs/StardustDocs/topics/dataSources/sql/MariaDB.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from [MariaDB](https://mariadb.org) database u Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [the official MariaDB JDBC driver](https://mariadb.com/docs/connectors/mariadb-connector-j): diff --git a/docs/StardustDocs/topics/dataSources/sql/Microsoft-SQL-Server.md b/docs/StardustDocs/topics/dataSources/sql/Microsoft-SQL-Server.md index e42b3bd857..37ecbba24f 100644 --- a/docs/StardustDocs/topics/dataSources/sql/Microsoft-SQL-Server.md +++ b/docs/StardustDocs/topics/dataSources/sql/Microsoft-SQL-Server.md @@ -18,7 +18,7 @@ database using JDBC. Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [the official MS SQL JDBC driver](https://learn.microsoft.com/en-us/sql/connect/jdbc/download-microsoft-jdbc-driver-for-sql-server?view=sql-server-ver17): diff --git a/docs/StardustDocs/topics/dataSources/sql/MySQL.md b/docs/StardustDocs/topics/dataSources/sql/MySQL.md index 2ed3dbc5d6..c845abed81 100644 --- a/docs/StardustDocs/topics/dataSources/sql/MySQL.md +++ b/docs/StardustDocs/topics/dataSources/sql/MySQL.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from [MySQL](https://www.mysql.com) database u Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [the official MySQL JDBC driver](https://dev.mysql.com/downloads/connector/j/): diff --git a/docs/StardustDocs/topics/dataSources/sql/PostgreSQL.md b/docs/StardustDocs/topics/dataSources/sql/PostgreSQL.md index c62cd71fca..a2f80991e1 100644 --- a/docs/StardustDocs/topics/dataSources/sql/PostgreSQL.md +++ b/docs/StardustDocs/topics/dataSources/sql/PostgreSQL.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from [PostgreSQL](https://www.postgresql.org) Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [the official PostgreSQL JDBC driver](https://jdbc.postgresql.org): diff --git a/docs/StardustDocs/topics/dataSources/sql/SQL.md b/docs/StardustDocs/topics/dataSources/sql/SQL.md index 5337903285..c612e16283 100644 --- a/docs/StardustDocs/topics/dataSources/sql/SQL.md +++ b/docs/StardustDocs/topics/dataSources/sql/SQL.md @@ -16,7 +16,7 @@ Kotlin DataFrame supports reading from SQL databases using JDBC. Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need a JDBC driver for the specific database. ## Supported databases diff --git a/docs/StardustDocs/topics/dataSources/sql/SQLite.md b/docs/StardustDocs/topics/dataSources/sql/SQLite.md index 0d4956dd46..0740f87582 100644 --- a/docs/StardustDocs/topics/dataSources/sql/SQLite.md +++ b/docs/StardustDocs/topics/dataSources/sql/SQLite.md @@ -17,7 +17,7 @@ Kotlin DataFrame supports reading from [SQLite](https://www.sqlite.org) database Requires the [`dataframe-jdbc` module](Modules.md#dataframe-jdbc), which is included by default in the general [`dataframe` artifact](Modules.md#dataframe-general) -and in [`%use dataframe`](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. +and in [`%use dataframe`](SetupKotlinNotebook.md#integrate-kotlin-dataframe) for Kotlin Notebook. You’ll also need [SQLite JDBC driver](https://github.com/xerial/sqlite-jdbc): diff --git a/docs/StardustDocs/topics/extensionPropertiesApi.md b/docs/StardustDocs/topics/extensionPropertiesApi.md index 41c79591eb..90a9e86fcc 100644 --- a/docs/StardustDocs/topics/extensionPropertiesApi.md +++ b/docs/StardustDocs/topics/extensionPropertiesApi.md @@ -8,7 +8,7 @@ with the name and type of properties inferred from the name and type of the corr It also works for all types of hierarchical dataframes. > The behavior of data schema generation differs between the -> [Compiler Plugin](Compiler-Plugin.md) and [Kotlin Notebook](gettingStartedKotlinNotebook.md). +> [Compiler Plugin](Compiler-Plugin.md) and [Kotlin Notebook](SetupKotlinNotebook.md). > > * In **Kotlin Notebook**, a schema is generated **only after cell execution** for > `DataFrame` variables defined within that cell. @@ -29,7 +29,7 @@ which is a [**column group**](DataColumn.md#columngroup) containing two nested [value columns](DataColumn.md#valuecolumn) — `age` of type `Int`, and `height` of type `Double`. - +
diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStarted.md b/docs/StardustDocs/topics/gettingStarted/gettingStarted.md deleted file mode 100644 index c1f000ee46..0000000000 --- a/docs/StardustDocs/topics/gettingStarted/gettingStarted.md +++ /dev/null @@ -1,87 +0,0 @@ -[//]: # (title: Get started with Kotlin DataFrame) - -The Kotlin DataFrame library gives you the power to manipulate data in your Kotlin projects. - -To use the Kotlin DataFrame library with a [custom Gradle configuration](gettingStartedGradle.md), -[Jupyter Notebooks](gettingStartedJupyterNotebook.md), or [Datalore](gettingStartedJupyterNotebook.md), -follow the instructions on the appropriate pages. - -## Install Kotlin - -Kotlin is included in each IntelliJ IDEA release. -Download and install [IntelliJ IDEA](https://www.jetbrains.com/idea/download/) to start using Kotlin. - - -## Create your powerful application with the Kotlin DataFrame library - - -Here is how you can take the first steps in developing data-intensive Kotlin applications with Gradle in IntelliJ IDEA. - -1. **Create your first application for data exploration:** - * To start from scratch, - create a [basic JVM application with the IntelliJ IDEA project wizard](gettingStartedGradle.md). - * To set up the custom Gradle configuration or Linter configuration, follow the instruction on this [page](gettingStartedGradleAdvanced.md). - * If you prefer more robust examples, try to use Kotlin DataFrame together with KotlinDL, - like in the [Titanic example](https://github.com/Kotlin/dataframe/tree/master/examples/idea-examples/titanic). -2. **Use Kotlin DataFrame and third-party Kotlin Data Science libraries in your application:** - * You can find a curated list of recommended data science libraries on the JVM [here](https://kotlinlang.org/docs/data-science-overview.html#java-libraries). -3. **Learn more about Kotlin DataFrame usage:** - * [Kotlin DataFrame Preview blogpost](https://blog.jetbrains.com/kotlin/2022/06/kotlin-dataframe-library-preview/) - * [Kotlin DataFrame Overview (video)](https://www.youtube.com/watch?v=qGou8F2asNw) - * [Replacing SQL with Kotlin's 'dataframe' on the Las Vegas Strip by Andrew Goldberg (video)](https://www.youtube.com/watch?v=sDZWiu9nnuU) - * [How to use KotlinDL, Kotlin DataFrame library and Kandy together (video)](https://www.youtube.com/watch?v=4IYBVdyP_8s) -4. **Join the Kotlin Data Science community:** - * ![Slack](https://kotlinlang.org/docs/images/slack.svg)Slack: - get an [invite](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up?_ga=2.148899126.1808346675.1686912551-1605045648.1686912543&_gl=1*64xzk9*_ga*MTYwNTA0NTY0OC4xNjg2OTEyNTQz*_ga_9J976DJZ68*MTY4NjkxMjU0Mi4xLjEuMTY4NjkxNTE3OC41OC4wLjA.) - and join the #datascience, #science, #mathematics channels. - * ![StackOverflow](https://kotlinlang.org/docs/images/stackoverflow.svg)StackOverflow: - subscribe to the ["kotlin"](https://stackoverflow.com/questions/tagged/kotlin) - and ["kotlin-dataframe"](https://stackoverflow.com/questions/tagged/kotlin-dataframe) tags. -5. **Follow Kotlin for Data Science** on ![Twitter](https://kotlinlang.org/docs/images/twitter.svg)[Twitter](https://twitter.com/KotlinForData) - -If you encounter any difficulties or problems, -report an issue to our [issue tracker](https://github.com/Kotlin/dataframe/issues). - - - -Support for programming in Jupyter Notebook is one of Kotlin's key benefits. -It brings new visualization and presentation possibilities -while retaining the power and benefits of a strongly typed programming language. - -1. **[Set up your environment for development in Jupyter Notebook.](https://jupyter.org/install)** -2. **Create your first notebook running with the Kotlin kernel:** - * To start from scratch, create a [basic Kotlin notebook](gettingStartedJupyterNotebook.md). - * If you prefer more robust examples, you can download and run [these examples](https://github.com/Kotlin/dataframe/tree/master/examples/notebooks) locally. -3. **[Learn more about Kotlin kernel for Jupyter Notebook.](https://github.com/Kotlin/kotlin-jupyter)** -4. **Join the Kotlin Data Science community:** - * ![Slack](https://kotlinlang.org/docs/images/slack.svg)Slack: - get an [invite](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up?_ga=2.148899126.1808346675.1686912551-1605045648.1686912543&_gl=1*64xzk9*_ga*MTYwNTA0NTY0OC4xNjg2OTEyNTQz*_ga_9J976DJZ68*MTY4NjkxMjU0Mi4xLjEuMTY4NjkxNTE3OC41OC4wLjA.) - and join the #datascience, #science, #mathematics channels. - * ![StackOverflow](https://kotlinlang.org/docs/images/stackoverflow.svg)StackOverflow: - subscribe to the ["kotlin"](https://stackoverflow.com/questions/tagged/kotlin) - and ["kotlin-dataframe"](https://stackoverflow.com/questions/tagged/kotlin-dataframe) tags. -5. **Follow Kotlin for Data Science** on ![Twitter](https://kotlinlang.org/docs/images/twitter.svg)[Twitter](https://twitter.com/KotlinForData) - - - -With Datalore, you can use Kotlin in the browser straight out of the box, no installation required. - -You can also collaborate on Kotlin notebooks in real time, -get smart coding assistance when writing code, and share results as interactive or static reports. - -1. **[Create your first notebook in Datalore](https://www.jetbrains.com/datalore/features/notebooks/)** - * To start from scratch, create a [basic Kotlin notebook](gettingStartedDatalore.md). - * If you prefer more robust examples, you can download them from GitHub, upload them to Datalore, - and run [these examples](https://github.com/Kotlin/dataframe/tree/master/examples/notebooks) here. -2. **[Learn more about Kotlin kernel for Jupyter Notebook.](https://github.com/Kotlin/kotlin-jupyter)** -3. **Join the Kotlin Data Science community:** - * ![Slack](https://kotlinlang.org/docs/images/slack.svg)Slack: - get an [invite](https://surveys.jetbrains.com/s3/kotlin-slack-sign-up?_ga=2.148899126.1808346675.1686912551-1605045648.1686912543&_gl=1*64xzk9*_ga*MTYwNTA0NTY0OC4xNjg2OTEyNTQz*_ga_9J976DJZ68*MTY4NjkxMjU0Mi4xLjEuMTY4NjkxNTE3OC41OC4wLjA.) - and join the #datascience, #science, #mathematics channels. - * ![StackOverflow](https://kotlinlang.org/docs/images/stackoverflow.svg)StackOverflow: - subscribe to the ["kotlin"](https://stackoverflow.com/questions/tagged/kotlin) - and ["kotlin-dataframe"](https://stackoverflow.com/questions/tagged/kotlin-dataframe) tags. -4. **Follow Kotlin for Data Science** on ![Twitter](https://kotlinlang.org/docs/images/twitter.svg)[Twitter](https://twitter.com/KotlinForData) - - - diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStartedDatalore.md b/docs/StardustDocs/topics/gettingStarted/gettingStartedDatalore.md deleted file mode 100644 index 064b683e13..0000000000 --- a/docs/StardustDocs/topics/gettingStarted/gettingStartedDatalore.md +++ /dev/null @@ -1,13 +0,0 @@ -[//]: # (title: Get started with Kotlin DataFrame on Datalore) - -## Datalore - -To start with the Kotlin DataFrame library in Datalore, create a Kotlin notebook first: - -![Installation in Datalore](datalore-1.png) - -As the notebook you've created is a Jupyter Notebook, you can follow the instructions -in the [previous section](gettingStartedJupyterNotebook.md) to use the Kotlin DataFrame library. -The simplest way of doing this is shown in the screenshot: - -![Datalore notebook](datalore-2.png) diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStartedGradle.md b/docs/StardustDocs/topics/gettingStarted/gettingStartedGradle.md deleted file mode 100644 index b45364901b..0000000000 --- a/docs/StardustDocs/topics/gettingStarted/gettingStartedGradle.md +++ /dev/null @@ -1,150 +0,0 @@ -[//]: # (title: Get started with Kotlin DataFrame on Gradle) - -This page explains how to: -* Set up the Kotlin DataFrame library in an IntelliJ IDEA project with Gradle. -* Import and manipulate data. -* Export data. - -## Create a Kotlin project - -1. In IntelliJ IDEA, select **File** | **New** | **Project**. -2. In the panel on the left, select **New Project**. -3. Name the new project and change its location, if necessary. - - > Select the **Create Git repository** checkbox to place the new project under version control. You can enable this - > later at any time. - > - {type="tip"} - -4. From the **Language** list, select **Kotlin**. -5. Select the **Gradle** build system. -6. From the **JDK list**, select the [JDK](https://www.oracle.com/java/technologies/downloads/) that you want to use in - your project. The minimum supported version is JDK 8. - * If the JDK is installed on your computer, but not defined in the IDE, select **Add JDK** and specify the path to the - JDK home directory. - * If you don't have the necessary JDK on your computer, select **Download JDK**. -7. From the **Gradle DSL** list, select **Kotlin** or **Groovy**. -8. Select the **Add sample code** checkbox to create a file with a sample `"Hello World!"` application. -9. Click **Create**. - -You have successfully created a project with Gradle. - -### Update Gradle dependencies - -In your Gradle build file (`build.gradle.kts`), add the Kotlin DataFrame library as a dependency: - - - - -```kotlin -dependencies { - implementation("org.jetbrains.kotlinx:dataframe:%dataFrameVersion%") -} -``` - - - - - -```groovy -dependencies { - implementation 'org.jetbrains.kotlinx:dataframe:%dataFrameVersion%' -} -``` - - - - - -### Add imports - -In `src/main/kotlin/Main.kt`, add the following imports at the top of the file: - -```kotlin -import org.jetbrains.kotlinx.dataframe.DataFrame -import org.jetbrains.kotlinx.dataframe.io.* -import org.jetbrains.kotlinx.dataframe.api.* -``` - -## Import data - -Download the file `movies.csv` from [here](https://github.com/Kotlin/dataframe/blob/master/data/movies.csv) to the root directory of your project: - - -Delete the `println()` functions and comments from your main function in `Main.kt`. - -To import the movie sample data into a data frame and print it inside your main function in `Main.kt`, -add the following code: - -```kotlin - // Import your data to a data frame - var df = DataFrame.read("movies.csv") - - // Print your data frame - df.print() -``` - -## Manipulate data - -To print some information about your data frame and sort your data, add the following additional lines of code: - -```kotlin - // Print some information about the data frame - println(df.columnNames()) // Print column names - println(df.count()) // Print number of rows - - // Sort your data alphabetically by title - df = df.sortBy("title") - - // Filter your data so that only comedy films remain, and print - df = df.filter { "genres"().contains("Comedy") } - df.print() -``` - -## Export data - -To export the current version of your data frame in CSV format, add the following additional lines of code and run `Main.kt`. - -```kotlin - // Export your manipulated data to CSV format - df.writeCSV("movies-by-title.csv") -``` - - - - movieId title genres - 0 9b30aff7943f44579e92c261f3adc193 Women in Black (1997) Fantasy|Suspenseful|Comedy - 1 2a1ba1fc5caf492a80188e032995843e Bumblebee Movie (2007) Comedy|Jazz|Family|Animation - 2 f44ceb4771504342bb856d76c112d5a6 Magical School Boy and the Rock of Wi... Fantasy|Growing up|Magic - 3 43d02fb064514ff3bd30d1e3a7398357 Master of the Jewlery: The Company of... Fantasy|Magic|Suspenseful - 4 6aa0d26a483148998c250b9c80ddf550 Sun Conflicts: Part IV: A Novel Espai... Fantasy - 5 eace16e59ce24eff90bf8924eb6a926c The Outstanding Bulk (2008) Fantasy|Superhero|Family - 6 ae916bc4844a4bb7b42b70d9573d05cd In Automata (2014) Horror|Existential - 7 c1f0a868aeb44c5ea8d154ec3ca295ac Interplanetary (2014) Sci-fi|Futuristic - 8 9595b771f87f42a3b8dd07d91e7cb328 Woods Run (1994) Family|Drama - 9 aa9fc400e068443488b259ea0802a975 Anthropod-Dude (2002) Superhero|Fantasy|Family|Growing up - 10 22d20c2ba11d44cab83aceea39dc00bd The Chamber (2003) Comedy|Drama - 11 8cf4d0c1bd7b41fab6af9d92c892141f That Thing About an Iceberg (1997) Drama|History|Family|Romance - 12 c2f3e7588da84684a7d78d6bd8d8e1f4 Vehicles (2006) Animation|Family - 13 ce06175106af4105945f245161eac3c7 Playthings Tale (1995) Animation|Family - 14 ee28d7e69103485c83e10b8055ef15fb Metal Man 2 (2010) Fantasy|Superhero|Family - 15 c32bdeed466f4ec09de828bb4b6fc649 Surgeon Odd in the Omniverse of Crazy... Fantasy|Superhero|Family|Horror - 16 d4a325ab648a42c4a2d6f35dfabb387f Bad Dream on Pine Street (1984) Horror - 17 60ebe74947234ddcab49dea1a958faed The Shimmering (1980) Horror - 18 f24327f2b05147b197ca34bf13ae3524 Krubit: Societal Teachings for Do Man... Comedy - 19 2bb29b3a245e434fa80542e711fd2cee This is No Movie (1950) (no genres listed) -[movieId, title, genres] -20 -movieId title genres -0 2a1ba1fc5caf492a80188e032995843e Bumblebee Movie (2007) Comedy|Jazz|Family|Animation -1 f24327f2b05147b197ca34bf13ae3524 Krubit: Societal Teachings for Do Man... Comedy -2 22d20c2ba11d44cab83aceea39dc00bd The Chamber (2003) Comedy|Drama -3 9b30aff7943f44579e92c261f3adc193 Women in Black (1997) Fantasy|Suspenseful|Comedy - - -Congratulations! You have successfully used the Kotlin DataFrame library to import, manipulate and export data. - -## Next steps -* Learn more about how to [import and export data](io.md) -* Learn about our different [access APIs](apiLevels.md) -* Explore the many different [operations that you can perform](operations.md) diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStartedGradleAdvanced.md b/docs/StardustDocs/topics/gettingStarted/gettingStartedGradleAdvanced.md deleted file mode 100644 index 4239ec9f25..0000000000 --- a/docs/StardustDocs/topics/gettingStarted/gettingStartedGradleAdvanced.md +++ /dev/null @@ -1,234 +0,0 @@ -[//]: # (title: Get started with Kotlin DataFrame on Gradle with custom configuration) - -## Gradle - -The Kotlin DataFrame library is published to Maven Central, -so you can add the following line to your Kotlin DSL -buildscript to depend on it: - -### General configuration - - - - -```kotlin -plugins { - id("org.jetbrains.kotlinx.dataframe") version "%dataFrameVersion%" -} - -dependencies { - implementation("org.jetbrains.kotlinx:dataframe:%dataFrameVersion%") -} - -// Below only applies to Android projects -android { - defaultConfig { - minSdk = 26 // Android O+ - } - compileOptions { - sourceCompatibility = JavaVersion.VERSION_1_8 - targetCompatibility = JavaVersion.VERSION_1_8 - } - kotlinOptions { - jvmTarget = "1.8" - } - packaging { - resources { - pickFirsts += listOf( - "META-INF/AL2.0", - "META-INF/LGPL2.1", - "META-INF/ASL-2.0.txt", - "META-INF/LICENSE.md", - "META-INF/NOTICE.md", - "META-INF/LGPL-3.0.txt", - "META-INF/thirdparty-LICENSE", - ) - excludes += listOf( - "META-INF/kotlin-jupyter-libraries/libraries.json", - "META-INF/{INDEX.LIST,DEPENDENCIES}", - "{draftv3,draftv4}/schema", - "arrow-git.properties", - ) - } - } -} -tasks.withType { - kotlinOptions.jvmTarget = "1.8" -} -``` - - - - - -```groovy -plugins { - id "org.jetbrains.kotlinx.dataframe" version "%dataFrameVersion%" -} - -dependencies { - implementation 'org.jetbrains.kotlinx:dataframe:%dataFrameVersion%' -} - -// Below only applies to Android projects -android { - defaultConfig { - minSdk 26 // Android O+ - } - compileOptions { - sourceCompatibility JavaVersion.VERSION_1_8 - targetCompatibility JavaVersion.VERSION_1_8 - } - kotlinOptions { - jvmTarget = "1.8" - } - packaging { - resources { - pickFirsts += [ - "META-INF/AL2.0", - "META-INF/LGPL2.1", - "META-INF/ASL-2.0.txt", - "META-INF/LICENSE.md", - "META-INF/NOTICE.md", - "META-INF/LGPL-3.0.txt", - ] - excludes += [ - "META-INF/kotlin-jupyter-libraries/libraries.json", - "META-INF/{INDEX.LIST,DEPENDENCIES}", - "{draftv3,draftv4}/schema", - "arrow-git.properties", - ] - } - } -} -tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).configureEach { - kotlinOptions.jvmTarget = "1.8" -} -``` - - - - - -Note that it's better to use the same version for a library and plugin to avoid unpredictable errors. -After plugin configuration, you can try it out, for [example](schemasGradle.md#annotation-processing). - -### Custom configuration - -If you want to avoid adding unnecessary dependencies, you can choose from the following artifacts: - - - - -```kotlin -dependencies { - // Artifact containing all APIs and implementations - implementation("org.jetbrains.kotlinx:dataframe-core:%dataFrameVersion%") - // Optional formats support - implementation("org.jetbrains.kotlinx:dataframe-json:%dataFrameVersion%") - implementation("org.jetbrains.kotlinx:dataframe-csv:%dataFrameVersion%") - implementation("org.jetbrains.kotlinx:dataframe-excel:%dataFrameVersion%") - implementation("org.jetbrains.kotlinx:dataframe-jdbc:%dataFrameVersion%") - implementation("org.jetbrains.kotlinx:dataframe-arrow:%dataFrameVersion%") - - // experimental - implementation("org.jetbrains.kotlinx:dataframe-openapi:%dataFrameVersion%") - - // experimental - // This artifact is only needed to directly call functions that generate @DataSchema code from OpenAPI specifications - // It's used by Gradle and KSP plugins internally. - // Your project needs dataframe-openapi to use generated code - implementation("org.jetbrains.kotlinx:dataframe-openapi-generator:%dataFrameVersion%") -} -``` - - - - - -```groovy -dependencies { - // Artifact containing all APIs and implementations - implementation 'org.jetbrains.kotlinx:dataframe-core:%dataFrameVersion%' - // Optional formats support - implementation 'org.jetbrains.kotlinx:dataframe-json:%dataFrameVersion%' - implementation 'org.jetbrains.kotlinx:dataframe-csv:%dataFrameVersion%' - implementation 'org.jetbrains.kotlinx:dataframe-excel:%dataFrameVersion%' - implementation 'org.jetbrains.kotlinx:dataframe-jdbc:%dataFrameVersion%' - implementation 'org.jetbrains.kotlinx:dataframe-arrow:%dataFrameVersion%' - - // experimental - implementation 'org.jetbrains.kotlinx:dataframe-openapi:%dataFrameVersion%' - - // experimental - // This artifact is only needed to directly call functions that generate @DataSchema code from OpenAPI specifications - // It's used by Gradle and KSP plugins internally. - // Your project needs dataframe-openapi to use generated code - implementation 'org.jetbrains.kotlinx:dataframe-openapi-generator:%dataFrameVersion%' -} -``` - - - - - - -`dataframe-json` is included with `dataframe-csv` and `dataframe-excel` by default. This is to interact with -JSON structures inside CSV and Excel files. If you don't need this functionality, you can exclude it like: -```kts -implementation("org.jetbrains.kotlinx:dataframe-csv:%dataFrameVersion%") { - exclude("org.jetbrains.kotlinx", "dataframe-json") -} -``` - - -#### Linter configuration - -We provide a Gradle plugin that generates interfaces with your data. -If you're using any sort of linter, it might complain about them generated sources. - -Use a configuration similar to this to prevent your linter from complaining about the -formatting of the generated sources. - - - - -```kotlin -// Exclusions for `kotlinter`, if you use it: -tasks.withType { - exclude { - it.name.endsWith(".Generated.kt") - } - exclude { - it.name.endsWith("\$Extensions.kt") - } -} -``` - - - - - -```groovy -// Exclusions for `kotlinter`, if you use it: -tasks.withType(org.jmailen.gradle.kotlinter.tasks.LintTask).all { - exclude { - it.name.endsWith(".Generated.kt") - } - exclude { - it.name.endsWith("\$Extensions.kt") - } -} -``` - - - - -```editorconfig -[{**/*.Generated.kt,**/*$Extensions.kt}] -ktlint = disabled -``` - - - - diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStartedJupyterNotebook.md b/docs/StardustDocs/topics/gettingStarted/gettingStartedJupyterNotebook.md deleted file mode 100644 index 998afd7ec5..0000000000 --- a/docs/StardustDocs/topics/gettingStarted/gettingStartedJupyterNotebook.md +++ /dev/null @@ -1,45 +0,0 @@ -[//]: # (title: Get started with Kotlin DataFrame on Jupyter Notebook) - -## Jupyter Notebook - -You can use the Kotlin DataFrame library in Jupyter Notebook and in Jupyter Lab. -To start, -install the latest version of [Kotlin kernel](https://github.com/Kotlin/kotlin-jupyter#installation) -and start your favorite Jupyter client from -the command line, for example: - -```shell -jupyter notebook -``` - -In the notebook, you only have to write a single line to start using the Kotlin DataFrame library: - -```text -%use dataframe -``` - -In this case, the version bundled with the kernel will be used. -If you want always to use the latest version, add another magic before `%use dataframe`: - -```text -%useLatestDescriptors -%use dataframe -``` - -If you want to use a specific version of the Kotlin DataFrame library, you can specify it in brackets: - -```text -%use dataframe(%dataFrameVersion%) -``` - -After loading, all essential types will be already imported, so you can start using the Kotlin DataFrame library. Enjoy! - -```kotlin -val df = DataFrame.read("https://raw.githubusercontent.com/Kotlin/dataframe/master/data/jetbrains_repositories.csv") -df // the last expression in the cell is displayed -``` - -When the previous cell with variable declaration is executed, `DataFrame` provides a data schema API based on data: -```kotlin -df.filter { stargazers_count > 50 } -``` diff --git a/docs/StardustDocs/topics/guides/Guides-And-Examples.md b/docs/StardustDocs/topics/guides/Guides-And-Examples.md index c54efe821d..c01207a1be 100644 --- a/docs/StardustDocs/topics/guides/Guides-And-Examples.md +++ b/docs/StardustDocs/topics/guides/Guides-And-Examples.md @@ -86,7 +86,7 @@ Explore our extensive collection of practical examples and real-world analytics — a practical example of working with deeply nested, hierarchical DataFrames using GitHub data. * [Netflix Example](https://github.com/Kotlin/dataframe/blob/master/examples/notebooks/netflix/netflix.ipynb) - — explore TV shows and movies from Netflix with the powerful Kotlin Dataframe API and beautiful + — explore TV shows and movies from Netflix with the powerful Kotlin DataFrame API and beautiful [Kandy](https://kotlin.github.io/kandy/) visualizations. * [Top-12 German Companies Financial Analyze](https://github.com/Kotlin/dataframe/blob/master/examples/notebooks/top_12_german_companies) @@ -96,7 +96,7 @@ Explore our extensive collection of practical examples and real-world analytics — basic Kotlin DataFrame operations on data from [movielens](https://movielens.org/). * [YouTube Example](https://github.com/Kotlin/dataframe/blob/master/examples/notebooks/youtube/Youtube.ipynb) - — explore YouTube videos with YouTube REST API and Kotlin Dataframe. + — explore YouTube videos with YouTube REST API and Kotlin DataFrame. * [IMDb SQL Database Example](https://github.com/zaleslaw/KotlinDataFrame-SQL-Examples/blob/master/notebooks/imdb.ipynb) — analyze IMDb data stored in MariaDB using Kotlin DataFrame diff --git a/docs/StardustDocs/topics/guides/quickstart.md b/docs/StardustDocs/topics/guides/quickstart.md index 06de611939..3b39381d35 100644 --- a/docs/StardustDocs/topics/guides/quickstart.md +++ b/docs/StardustDocs/topics/guides/quickstart.md @@ -15,7 +15,7 @@ Learn the basics of Kotlin DataFrame: reading data, applying transformations, an This guide shows how to quickly get started with **Kotlin DataFrame**: you'll learn how to load data, perform basic transformations, and build a simple plot using Kandy. -We recommend [starting with **Kotlin Notebook**](gettingStartedKotlinNotebook.md) for the best beginner experience — +We recommend [starting with **Kotlin Notebook**](SetupKotlinNotebook.md) for the best beginner experience — everything works out of the box, including interactivity and rich DataFrame and plots rendering. You can instantly see the results of each operation: view the contents of your DataFrames after every transformation, @@ -36,7 +36,7 @@ To start working with Kotlin DataFrame in a notebook, run the cell with the next ``` This will load all necessary DataFrame dependencies (of the latest stable version) and all imports, as well as DataFrame -rendering. Learn more [here](gettingStartedKotlinNotebook.md#integrate-kotlin-dataframe). +rendering. Learn more [here](SetupKotlinNotebook.md#integrate-kotlin-dataframe). ## Read DataFrame diff --git a/docs/StardustDocs/topics/schemas/DataSchemaGenerationMethods.md b/docs/StardustDocs/topics/schemas/DataSchemaGenerationMethods.md index 7c3755954b..d84c7ce7dc 100644 --- a/docs/StardustDocs/topics/schemas/DataSchemaGenerationMethods.md +++ b/docs/StardustDocs/topics/schemas/DataSchemaGenerationMethods.md @@ -47,7 +47,7 @@ Useful when you want to: * `extensionProperties`: `Boolean` – Whether to generate [extension properties](extensionPropertiesApi.md) in addition to `interface` declarations. Useful if you don't use the [compiler plugin](Compiler-Plugin.md), otherwise they are not needed; - the [compiler plugin](Compiler-Plugin.md), [notebooks](gettingStartedKotlinNotebook.md), + the [compiler plugin](Compiler-Plugin.md), [notebooks](SetupKotlinNotebook.md), and older [Gradle/KSP plugin](schemasGradle.md) generate them automatically. Default: `false`. * `visibility`: `MarkerVisibility` – Visibility modifier for the generated declarations. @@ -122,7 +122,7 @@ in cases where the schema cannot be inferred automatically from the source. * `extensionProperties`: `Boolean` – Whether to generate [extension properties](extensionPropertiesApi.md) in addition to `interface` declarations. Useful if you don't use the [compiler plugin](Compiler-Plugin.md), otherwise they are not needed; - the [compiler plugin](Compiler-Plugin.md), [notebooks](gettingStartedKotlinNotebook.md), + the [compiler plugin](Compiler-Plugin.md), [notebooks](SetupKotlinNotebook.md), and older [Gradle/KSP plugin](schemasGradle.md) generate them automatically. Default: `false`. * `visibility`: `MarkerVisibility` – Visibility modifier for the generated declarations. diff --git a/docs/StardustDocs/topics/schemas/schemas.md b/docs/StardustDocs/topics/schemas/schemas.md index 09981123f5..4a2328c500 100644 --- a/docs/StardustDocs/topics/schemas/schemas.md +++ b/docs/StardustDocs/topics/schemas/schemas.md @@ -20,7 +20,7 @@ This DataFrame consists of two columns: - `age` of type `Int` - `height` of type `Double` -
name
+
@@ -134,7 +134,7 @@ that modify the structure of the [`DataFrame`](DataFrame.md). > Extension properties generation was deprecated from the Gradle plugin in favor of the Compiler Plugin. > {style="warning"} -* When using Kotlin DataFrame inside [Kotlin Notebook](gettingStartedKotlinNotebook.md), +* When using Kotlin DataFrame inside [Kotlin Notebook](SetupKotlinNotebook.md), the schema and extension properties are generated automatically after each cell execution for all `DataFrame` variables declared in that cell. See [extension properties example in Kotlin Notebook](extensionPropertiesApi.md#example). diff --git a/docs/StardustDocs/topics/gettingStarted/Modules.md b/docs/StardustDocs/topics/setup/Modules.md similarity index 92% rename from docs/StardustDocs/topics/gettingStarted/Modules.md rename to docs/StardustDocs/topics/setup/Modules.md index 3225208ac0..fe4822268a 100644 --- a/docs/StardustDocs/topics/gettingStarted/Modules.md +++ b/docs/StardustDocs/topics/setup/Modules.md @@ -420,8 +420,8 @@ Provides functionality to support auto-generated data schemas from OpenAPI 3.0.0 This module is a companion to [`dataframe-openapi-generator`](#dataframe-openapi-generator): - `dataframe-openapi-generator` is used internally by the Gradle plugin and Jupyter integration - to generate data schemas from OpenAPI specs. - In the Gradle plugin, it powers the `dataschemas {}` DSL and the `@file:ImportDataSchema()` annotation. + to generate data schemas from OpenAPI specs. + In the Gradle plugin, it powers the `dataschemas {}` DSL and the `@file:ImportDataSchema()` annotation. In Jupyter, it enables the `importDataSchema()` function. - `dataframe-openapi` must be added as a dependency to the user project in order to use those generated data schemas. @@ -454,11 +454,11 @@ dependencies { #### `dataframe-openapi-generator` -Provides the logic and tooling necessary to import OpenAPI 3.0.0 specifications -as auto-generated data schemas for Kotlin DataFrame. +Provides the logic and tooling necessary to import OpenAPI 3.0.0 specifications +as auto-generated data schemas for Kotlin DataFrame. This module works in conjunction with [`dataframe-openapi`](#dataframe-openapi): -- `dataframe-openapi-generator` is used internally by the Gradle plugin and Jupyter integration +- `dataframe-openapi-generator` is used internally by the Gradle plugin and Jupyter integration to generate data schemas from OpenAPI specifications. - In Gradle, it enables the `dataschemas {}` DSL and the `@file:ImportDataSchema()` annotation. - In Jupyter, it powers the `importDataSchema()` function. @@ -497,27 +497,61 @@ dependencies { #### `kotlin.plugin.dataframe` — Kotlin DataFrame Compiler Plugin {id="kotlin.plugin.dataframe"} -The Kotlin DataFrame compiler plugin enables support for [extension properties](extensionPropertiesApi.md) -in Gradle projects, allowing you to work with dataframes in a name- and type-safe manner. +[Kotlin DataFrame Compiler Plugin](Compiler-Plugin.md) enables automatic generation of +[extension properties](extensionPropertiesApi.md) and updates [data schemas](schemas.md) +on-the-fly in Gradle projects, making development with Kotlin DataFrame faster, +more convenient, and fully type- and name-safe. + +> Requires Kotlin 2.2.20-Beta1 or higher. +> { style = "note" } + +To enable the plugin in your Gradle project, add it to the `plugins` section: + + + + +```kotlin +plugins { + kotlin("plugin.dataframe") version "2.2.20-Beta1" +} +``` + + + + -See the [Compiler Plugin setup guide](Compiler-Plugin.md#setup) for installation -and usage instructions for Gradle projects. +```groovy +plugins { + id 'org.jetbrains.kotlin.plugin.dataframe' version '2.2.20-Beta1' +} +``` + + + + +Due to [this issue](https://youtrack.jetbrains.com/issue/KT-66735), incremental compilation must be disabled for now. +Add the following line to your `gradle.properties` file: + +```properties +kotlin.incremental=false +``` Published as a Kotlin official plugin. [Source code is available in the Kotlin repository](https://github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe). #### `kotlinx.dataframe` – Gradle Plugin {id="kotlinx.dataframe"} -> The current Gradle plugin is **under consideration for deprecation** and may be officially marked as deprecated in future releases. +> The current Gradle plugin is **under consideration for deprecation** +> and may be officially marked as deprecated in future releases. > > At the moment, **[data schema generation is handled via dedicated methods](DataSchemaGenerationMethods.md)** instead of relying on the plugin. {style="warning"} -The Gradle plugin allows generating [data schemas](schemas.md) from samples of data +The Gradle plugin allows generating [data schemas](schemas.md) from samples of data (of supported formats) like JSON, CSV, Excel files, or URLs, as well as from data fetched from SQL databases using Gradle. -See the [Gradle Plugin Reference](Gradle-Plugin.md) for installation +See the [Gradle Plugin Reference](Gradle-Plugin.md) for installation and usage instructions in Gradle projects. > By default, the Gradle plugin also applies the [KSP plugin](#ksp-plugin). @@ -552,8 +586,8 @@ plugins { > At the moment, **[data schema generation is handled via dedicated methods](DataSchemaGenerationMethods.md)** instead of relying on the plugin. {style="warning"} -The Gradle plugin allows generating [data schemas](schemas.md) from samples of data -(of supported formats) like JSON, CSV, Excel files, or URLs, as well as from data fetched from SQL databases +The Gradle plugin allows generating [data schemas](schemas.md) from samples of data +(of supported formats) like JSON, CSV, Excel files, or URLs, as well as from data fetched from SQL databases using Kotlin Symbol Processing (KSP). This is useful for projects where you prefer or require schema generation at the source level. diff --git a/docs/StardustDocs/topics/setup/Setup.md b/docs/StardustDocs/topics/setup/Setup.md new file mode 100644 index 0000000000..1c4932b47e --- /dev/null +++ b/docs/StardustDocs/topics/setup/Setup.md @@ -0,0 +1,36 @@ +[//]: # (title: Setup Kotlin DataFrame) + + +Install Kotlin DataFrame in your preferred environment — Kotlin Notebook, Android, Jupyter, or Datalore +— and start working with data right away. + + + +Install Kotlin DataFrame in your preferred environment — Kotlin Notebook, Gradle, Android, Jupyter, or Datalore +— and start working with data right away. + + + +Step-by-step setup guide for using Kotlin DataFrame in different environments like Kotlin Notebook, Android, and Jupyter. + + +Kotlin DataFrame is supported in multiple environments — choose the one that suits you best and follow the setup steps. + +- [**Setup in Kotlin Notebook**](SetupKotlinNotebook.md) — one-line Kotlin DataFrame setup for powerful interactive +[Kotlin notebooks in IntelliJ IDEA and Android Studio](https://kotlinlang.org/docs/kotlin-notebook-overview.html). +No configuration required, full IDE support. +Perfect for learning Kotlin DataFrame basics and experimenting, and a great professional tool +for [data analysis](https://kotlinlang.org/docs/data-analysis-overview.html). + +- [**Setup in Gradle**](SetupGradle.md) — add Kotlin DataFrame to your Gradle project for convenient data handling +in your Kotlin applications. + +- [**Setup on Android**](SetupAndroid.md) — use Kotlin DataFrame in your Android apps to simplify local data processing +and transformation. + +- [**Setup on Datalore**](SetupDatalore.md) — run Kotlin DataFrame in +[JetBrains Datalore](https://www.jetbrains.com/datalore/) notebooks, +great for sharing and remote collaboration. + +- [**Setup in Jupyter**](SetupJupyter.md) — integrate Kotlin DataFrame +into interactive notebooks using the Kotlin Jupyter kernel. diff --git a/docs/StardustDocs/topics/setup/SetupAndroid.md b/docs/StardustDocs/topics/setup/SetupAndroid.md new file mode 100644 index 0000000000..b21534b3c2 --- /dev/null +++ b/docs/StardustDocs/topics/setup/SetupAndroid.md @@ -0,0 +1,172 @@ +[//]: # (title: Setup Kotlin DataFrame on Android) + + +Integrate Kotlin DataFrame into your Android app using the standard JVM dependency and simple Gradle configuration. + + + +Set up Kotlin DataFrame in Android — configure it easily using Gradle and start working with structured data. + + + +How to use Kotlin DataFrame in your Android project with Gradle setup and compiler plugin support. + + +> See an [Android project example](https://github.com/Kotlin/dataframe/tree/master/examples/android-example). + +Kotlin DataFrame doesn't provide a dedicated Android artifact yet, +but you can add the Kotlin DataFrame JVM dependency to your Android project with minimal configuration: + + + + +```kotlin +dependencies { + implementation("org.jetbrains.kotlinx:dataframe:%dataFrameVersion%") +} + +android { + // Requires Android 0+, i.e. SDK version 26 or higher. + defaultConfig { + minSdk = 26 + } + // Requires Java 8 or higher + compileOptions { + sourceCompatibility = JavaVersion.VERSION_1_8 + targetCompatibility = JavaVersion.VERSION_1_8 + } + kotlinOptions { + jvmTarget = "1.8" + } + packaging { + resources { + pickFirsts += listOf( + "META-INF/AL2.0", + "META-INF/LGPL2.1", + "META-INF/ASL-2.0.txt", + "META-INF/LICENSE.md", + "META-INF/NOTICE.md", + "META-INF/LGPL-3.0.txt", + "META-INF/thirdparty-LICENSE", + ) + excludes += listOf( + "META-INF/kotlin-jupyter-libraries/libraries.json", + "META-INF/{INDEX.LIST,DEPENDENCIES}", + "{draftv3,draftv4}/schema", + "arrow-git.properties", + ) + } + } +} +tasks.withType { + kotlinOptions.jvmTarget = "1.8" +} +``` + + + + + +```groovy +dependencies { + implementation 'org.jetbrains.kotlinx:dataframe:%dataFrameVersion%' +} + +android { + // Requires Android 0+, i.e. SDK version 26 or higher. + defaultConfig { + minSdk 26 + } + // Requires Java 8 or higher + compileOptions { + sourceCompatibility JavaVersion.VERSION_1_8 + targetCompatibility JavaVersion.VERSION_1_8 + } + kotlinOptions { + jvmTarget = "1.8" + } + packaging { + resources { + pickFirsts += [ + "META-INF/AL2.0", + "META-INF/LGPL2.1", + "META-INF/ASL-2.0.txt", + "META-INF/LICENSE.md", + "META-INF/NOTICE.md", + "META-INF/LGPL-3.0.txt", + "META-INF/thirdparty-LICENSE", + ] + excludes += [ + "META-INF/kotlin-jupyter-libraries/libraries.json", + "META-INF/{INDEX.LIST,DEPENDENCIES}", + "{draftv3,draftv4}/schema", + "arrow-git.properties", + ] + } + } +} +tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).configureEach { + kotlinOptions.jvmTarget = "1.8" +} +``` + + + + +This setup adds the [general Kotlin DataFrame dependency](Modules.md#dataframe-general), +which includes the [core API and implementation](Modules.md#dataframe-core) +as well as all [IO modules](Modules.md#io-modules) +(excluding [experimental ones](Modules.md#experimental-modules)). +For flexible configuration, see [Custom configuration](SetupCustomGradle.md). + +## Kotlin DataFrame Compiler Plugin + +[Kotlin DataFrame Compiler Plugin](Compiler-Plugin.md) enables automatic generation +of [extension properties](extensionPropertiesApi.md) and updates [data schemas](schemas.md) +on-the-fly in Android projects, making development with Kotlin DataFrame +faster, more convenient, and fully type- and name-safe. + +> Requires Kotlin 2.2.20-Beta1 or higher. +> { style = "note" } + +To enable the plugin in your Gradle project, add it to the `plugins` section: + + + + +```kotlin +plugins { + kotlin("plugin.dataframe") version "2.2.20-Beta1" +} +``` + + + + + +```groovy +plugins { + id 'org.jetbrains.kotlin.plugin.dataframe' version '2.2.20-Beta1' +} +``` + + + + +Due to [this issue](https://youtrack.jetbrains.com/issue/KT-66735), incremental compilation must be disabled for now. +Add the following line to your `gradle.properties` file: + +```properties +kotlin.incremental=false +``` + +## Next Steps + +* Once Kotlin DataFrame is set up in your Android project, continue with the [](quickstart.md) +to learn the basics of working with DataFrames. +* Explore [detailed guides and real-world examples](Guides-And-Examples.md) +to see how Kotlin DataFrame helps in different data tasks. +* Check out the +[Android project example](https://github.com/Kotlin/dataframe/tree/master/examples/android-example) +and more [IDEA examples on GitHub](https://github.com/Kotlin/dataframe/tree/master/examples/idea-examples). +* Learn more about the [compiler plugin](Compiler-Plugin.md). diff --git a/docs/StardustDocs/topics/setup/SetupCustomGradle.md b/docs/StardustDocs/topics/setup/SetupCustomGradle.md new file mode 100644 index 0000000000..af9522bebb --- /dev/null +++ b/docs/StardustDocs/topics/setup/SetupCustomGradle.md @@ -0,0 +1,71 @@ +# Custom Gradle Configuration + + +Add Kotlin DataFrame to your Gradle project and configure only the modules you need for data processing and IO. + + + +Modular setup for Kotlin DataFrame — include just the dependencies required for your use case. + + + +How to configure Kotlin DataFrame in Gradle using only the relevant modules for your project. + + + +Kotlin DataFrame is composed of multiple [modules](Modules.md), +allowing you to include only the functionality you need. + +To use Kotlin DataFrame in a [Gradle project](SetupGradle.md) — including [Android](SetupAndroid.md) — +you can configure your Gradle buildscript (`build.gradle.kts` or `build.gradle`) with selected dependencies: + + + + +```kotlin +dependencies { + // Core API and runtime + implementation("org.jetbrains.kotlinx:dataframe-core:%dataFrameVersion%") + + // Optional IO format support + implementation("org.jetbrains.kotlinx:dataframe-json:%dataFrameVersion%") + implementation("org.jetbrains.kotlinx:dataframe-csv:%dataFrameVersion%") + implementation("org.jetbrains.kotlinx:dataframe-excel:%dataFrameVersion%") + implementation("org.jetbrains.kotlinx:dataframe-jdbc:%dataFrameVersion%") + implementation("org.jetbrains.kotlinx:dataframe-arrow:%dataFrameVersion%") + + // Experimental features + implementation("org.jetbrains.kotlinx:dataframe-geo:%dataFrameVersion%") + implementation("org.jetbrains.kotlinx:dataframe-openapi:%dataFrameVersion%") + + // Only needed if you generate @DataSchema from OpenAPI specs + implementation("org.jetbrains.kotlinx:dataframe-openapi-generator:%dataFrameVersion%") +} +``` + + + + +```groovy +dependencies { + // Core API and runtime + implementation 'org.jetbrains.kotlinx:dataframe-core:%dataFrameVersion%' + + // Optional IO format support + implementation 'org.jetbrains.kotlinx:dataframe-json:%dataFrameVersion%' + implementation 'org.jetbrains.kotlinx:dataframe-csv:%dataFrameVersion%' + implementation 'org.jetbrains.kotlinx:dataframe-excel:%dataFrameVersion%' + implementation 'org.jetbrains.kotlinx:dataframe-jdbc:%dataFrameVersion%' + implementation 'org.jetbrains.kotlinx:dataframe-arrow:%dataFrameVersion%' + + // Experimental features + implementation 'org.jetbrains.kotlinx:dataframe-geo:%dataFrameVersion%' + implementation 'org.jetbrains.kotlinx:dataframe-openapi:%dataFrameVersion%' + + // Only needed if you generate @DataSchema from OpenAPI specs + implementation 'org.jetbrains.kotlinx:dataframe-openapi-generator:%dataFrameVersion%' +} +``` + + + diff --git a/docs/StardustDocs/topics/setup/SetupDatalore.md b/docs/StardustDocs/topics/setup/SetupDatalore.md new file mode 100644 index 0000000000..00f062b521 --- /dev/null +++ b/docs/StardustDocs/topics/setup/SetupDatalore.md @@ -0,0 +1,39 @@ +[//]: # (title: Setup Kotlin DataFrame on Datalore) + + +Set up Kotlin DataFrame on Datalore — +a cloud-based platform with Kotlin notebook support for collaborative data analysis. + + + +Set up Kotlin DataFrame on Datalore — +a cloud-based platform with Kotlin notebook support for collaborative data analysis. + + + +Set up Kotlin DataFrame on Datalore — +a cloud-based platform with Kotlin notebook support for collaborative data analysis. + + +[**Datalore**](https://www.jetbrains.com/datalore/) is a free data analysis platform +with [Kotlin Jupyter notebooks](https://github.com/Kotlin/kotlin-jupyter) support. +It's perfect for collaboration and sharing. + +To get started with Kotlin DataFrame on Datalore, first create a Kotlin notebook: + +Installation on Datalore + +To use Kotlin DataFrame in the notebook with the Kotlin Jupyter Kernel, +follow the same steps as described in the [Setup in Kotlin Notebook](SetupKotlinNotebook.md). +The easiest way is shown in this screenshot: + +Datalore notebook + +## Next Steps + +* Once you’ve successfully set up Kotlin DataFrame on Datalore, +you can move on to the [](quickstart.md) to learn the basics of working +with Kotlin DataFrame inside a notebook. +* For more advanced use cases, explore our collection of +[detailed guides and real-world examples](Guides-And-Examples.md), +showcasing how Kotlin DataFrame can help with a variety of data tasks. diff --git a/docs/StardustDocs/topics/setup/SetupGradle.md b/docs/StardustDocs/topics/setup/SetupGradle.md new file mode 100644 index 0000000000..eefa779135 --- /dev/null +++ b/docs/StardustDocs/topics/setup/SetupGradle.md @@ -0,0 +1,138 @@ +# Setup Kotlin DataFrame in Gradle + + +Set up Kotlin DataFrame in your Gradle project, configure dependencies, and start using the API with full IDE support. + + + +Learn how to add Kotlin DataFrame to your Gradle project. + + + +Guide for integrating Kotlin DataFrame in a Gradle-based project, with setup instructions and example code. + + +Kotlin DataFrame can be added as a usual Gradle dependency +to your Kotlin project (for now only Kotlin/JVM is supported). + +## Create a Kotlin project + +1. In IntelliJ IDEA, select **File** | **New** | **Project**. +2. In the panel on the left, select **New Project**. +3. Name the new project and change its location, if necessary. + + > Select the **Create Git repository** checkbox to place the new project under version control. + > You can enable this later at any time. + > {type="tip"} + +4. From the **Language** list, select **Kotlin**. +5. Select the **Gradle** build system. +6. From the **JDK list**, select the [JDK](https://www.oracle.com/java/technologies/downloads/) +that you want to use in your project. The minimum supported version is JDK 8. + * If the JDK is installed on your computer, but not defined in the IDE, select **Add JDK** + and specify the path to the JDK home directory. + * If you don't have the necessary JDK on your computer, select **Download JDK**. +7. From the **Gradle DSL** list, select **Kotlin** or **Groovy**. +8. Select the **Add sample code** checkbox to create a file with a sample `"Hello World!"` application. +9. Click **Create**. + +You have successfully created a project with Gradle. + +## Add Kotlin DataFrame Gradle dependency + +In your Gradle build file (`build.gradle` or `build.gradle.kts`), add the Kotlin DataFrame library as a dependency: + + + + +```kotlin +dependencies { + implementation("org.jetbrains.kotlinx:dataframe:%dataFrameVersion%") +} +``` + + + + + +```groovy +dependencies { + implementation 'org.jetbrains.kotlinx:dataframe:%dataFrameVersion%' +} +``` + + + + +This will add [general Kotlin DataFrame dependency](Modules.md#dataframe-general), +i.e., [core API and implementation](Modules.md#dataframe-core) as well as all +[IO modules](Modules.md#io-modules) (excluding [experimental ones](Modules.md#experimental-modules)). +For flexible dependencies configuration see [Custom configuration](SetupCustomGradle.md). + +## Hello World + +Let’s create your first [`DataFrame`](DataFrame.md) in the notebook — a simple "Hello, World!" style example: + +```kotlin +import org.jetbrains.kotlinx.dataframe.api.dataFrameOf + +fun main() { + val df = dataFrameOf( + "name" to listOf("Alice", "Bob"), + "age" to listOf(25, 30) + ) + + println(df) +} +``` + +## Kotlin DataFrame Compiler Plugin + +[Kotlin DataFrame Compiler Plugin](Compiler-Plugin.md) enables automatic generation of +[extension properties](extensionPropertiesApi.md) and updates [data schemas](schemas.md) +on-the-fly in Gradle projects, making development with Kotlin DataFrame faster, +more convenient, and fully type- and name-safe. + +> Requires Kotlin 2.2.20-Beta1 or higher. +> { style = "note" } + +To enable the plugin in your Gradle project, add it to the `plugins` section: + + + + +```kotlin +plugins { + kotlin("plugin.dataframe") version "2.2.20-Beta1" +} +``` + + + + + +```groovy +plugins { + id 'org.jetbrains.kotlin.plugin.dataframe' version '2.2.20-Beta1' +} +``` + + + + +Due to [this issue](https://youtrack.jetbrains.com/issue/KT-66735), incremental compilation must be disabled for now. +Add the following line to your `gradle.properties` file: + +```properties +kotlin.incremental=false +``` + +## Next Steps + +* Once you’ve set up Kotlin DataFrame in your Gradle project, continue with the [](quickstart.md) + to learn the basics of working with Kotlin DataFrame. +* Explore [detailed guides and real-world examples](Guides-And-Examples.md) + to see how Kotlin DataFrame helps with different data tasks. +* Check out various + [IDEA examples using Kotlin DataFrame on GitHub](https://github.com/Kotlin/dataframe/tree/master/examples/idea-examples). +* Learn more about the [compiler plugin](Compiler-Plugin.md). diff --git a/docs/StardustDocs/topics/setup/SetupJupyter.md b/docs/StardustDocs/topics/setup/SetupJupyter.md new file mode 100644 index 0000000000..f3b4033ee0 --- /dev/null +++ b/docs/StardustDocs/topics/setup/SetupJupyter.md @@ -0,0 +1,28 @@ +[//]: # (title: Setup Kotlin DataFrame in Jupyter) + + +Run Kotlin DataFrame inside Jupyter notebooks using the Kotlin Jupyter kernel — ideal for interactive data exploration and analysis. + + + +Use Kotlin DataFrame in Jupyter with the Kotlin kernel — an easy way to explore, clean, and analyze data interactively. + + + +How to set up Kotlin DataFrame in Jupyter using the Kotlin Jupyter kernel for interactive data work. + + +[Kotlin Jupyter kernel](https://github.com/Kotlin/kotlin-jupyter) allows you to run Kotlin code +in interactive [Jupyter notebooks](https://jupyter.org). +Kotlin DataFrame is fully integrated and works seamlessly within these environments. + +To use Kotlin DataFrame in a Jupyter notebook with the Kotlin Jupyter Kernel, +follow the same steps as described in the [Setup in Kotlin Notebook](SetupKotlinNotebook.md). + +## Next Steps + +* Once you’ve successfully set up Kotlin DataFrame in Jupyter, you can move on to the [](quickstart.md) +to learn the basics of working with Kotlin DataFrame inside a notebook. +* For more advanced use cases, explore our collection of +[detailed guides and real-world examples](Guides-And-Examples.md), +showcasing how Kotlin DataFrame can help with a variety of data tasks. diff --git a/docs/StardustDocs/topics/gettingStarted/gettingStartedKotlinNotebook.md b/docs/StardustDocs/topics/setup/SetupKotlinNotebook.md similarity index 80% rename from docs/StardustDocs/topics/gettingStarted/gettingStartedKotlinNotebook.md rename to docs/StardustDocs/topics/setup/SetupKotlinNotebook.md index 3cf016aa19..18258948cc 100644 --- a/docs/StardustDocs/topics/gettingStarted/gettingStartedKotlinNotebook.md +++ b/docs/StardustDocs/topics/setup/SetupKotlinNotebook.md @@ -1,4 +1,4 @@ -# Get started with Kotlin DataFrame in Kotlin Notebook +# Setup Kotlin DataFrame in Kotlin Notebook Use Kotlin DataFrame directly in Kotlin Notebook — write code, explore results, and refine your analysis step by step in a live environment. @@ -14,22 +14,23 @@ Follow a step-by-step introduction to Kotlin DataFrame in Kotlin Notebook: setup -[Kotlin Notebook](https://kotlinlang.org/docs/kotlin-notebook-overview.html) is an interactive environment -integrated into [IntelliJ IDEA](https://www.jetbrains.com/idea/), designed for fast, iterative, -and visual data exploration with Kotlin. +[**Kotlin Notebook**](https://kotlinlang.org/docs/kotlin-notebook-overview.html) is an interactive environment +integrated into [IntelliJ IDEA](https://www.jetbrains.com/idea/) (and can be easily added in +[Android Studio](https://developer.android.com/studio)), designed for fast, iterative, +and visual data exploration with Kotlin. -The Kotlin Notebook plugin transforms IntelliJ IDEA into a powerful data science workspace, -combining Kotlin’s strong language features with live code execution, -interactive data exploration, and rich visualizations +The Kotlin Notebook plugin transforms IntelliJ IDEA into a powerful data science workspace, +combining Kotlin’s strong language features with live code execution, +interactive data exploration, and rich visualizations. It’s perfect for working with Kotlin DataFrame — letting you write code, view results instantly, and refine your analysis step by step. -## Create Kotlin Notebook +## Create a Kotlin notebook -Before version 2025.1, Kotlin Notebook is unavailable in IntelliJ IDEA Community Edition +Before version 2025.1, Kotlin Notebook is unavailable in IntelliJ IDEA Community Edition and not bundled in IntelliJ IDEA Ultimate Edition by default. @@ -78,7 +79,7 @@ Or use the latest stable version of Kotlin DataFrame ## Hello World -Let’s create your first Kotlin DataFrame in the notebook — a simple "Hello, World!" style example: +Let’s create your first [`DataFrame`](DataFrame.md) in the notebook — a simple "Hello, World!" style example: ```kotlin val df = dataFrameOf( @@ -99,13 +100,13 @@ You will see the content of this `DataFrame` rendered as an interactive table di ## Next Steps -* Once you’ve successfully set up Kotlin DataFrame in Kotlin Notebook, +* Once you’ve successfully set up Kotlin DataFrame in Kotlin Notebook, you can move on to the [](quickstart.md) - which walks you through the basics of working with DataFrames inside a notebook. +which walks you through the basics of working with Kotlin DataFrame inside a notebook. -* For more advanced use cases, explore our collection of -[detailed guides and real-world examples](Guides-And-Examples.md), +* For more advanced use cases, explore our collection of +[detailed guides and real-world examples](Guides-And-Examples.md), showcasing how Kotlin DataFrame can help with a variety of data tasks. -* Discover powerful [](Kotlin-DataFrame-Features-in-Kotlin-Notebook.md)that +* Discover powerful [](Kotlin-DataFrame-Features-in-Kotlin-Notebook.md)that make exploring and understanding your data easier and more effective.
name