Skip to content

Plugin Chapter update #1542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 4, 2024
Merged

Plugin Chapter update #1542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
27d6ddb
Plugin Chapter update
NotTheDr01ds Sep 4, 2024
c299b9a
Cargo installation location
NotTheDr01ds Sep 4, 2024
ba9e8a2
More tweaks
NotTheDr01ds Sep 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 10 additions & 19 deletions book/dataframes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,10 @@
# Dataframes

::: warning

To use dataframes you'll need to install [cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html) and then install `nu_plugin_polars` with commands:

```nu no-run
# Install the `polars` nushell plugin
> cargo install nu_plugin_polars

# Add the plugin's commands to your plugin registry file:
> plugin add ~/.cargo/bin/nu_plugin_polars
```

After installation, you will need to restart the nushell instance. If everything is successful,
you should be able to see command completions for `polars`. For example, you should be able to execute
`polars into-df -h`.
::: warning Important!
This feature requires the Polars plugin. See the the
[Plugins Chapter](plugins.md) to learn how to install it.

To test that this plugin is properly installed, run `help polars`.
:::

As we have seen so far, Nushell makes working with data its main priority.
Expand All @@ -34,10 +23,12 @@ Arrow](https://arrow.apache.org/) specification, and uses
extremely [fast columnar operations](https://h2oai.github.io/db-benchmark/).

You may be wondering now how fast this combo could be, and how could it make
working with data easier and more reliable. For this reason, let's start this
page by presenting benchmarks on common operations that are done when
working with data easier and more reliable. For this reason, we'll start this
chapter by presenting benchmarks on common operations that are done when
processing data.

[[toc]]

## Benchmark Comparisons

For this little benchmark exercise we will be comparing native Nushell
Expand Down Expand Up @@ -1261,5 +1252,5 @@ However, the future of these dataframes is still very experimental. New
commands and tools that take advantage of these commands will be added as they
mature.

Keep visiting this book in order to check the new things happening to
dataframes and how they can help you process data faster and efficiently.
Check this chapter, as well as our [Blog](/blog/), regularly to learn about new
dataframes features and how they can help you process data faster and efficiently.
156 changes: 106 additions & 50 deletions book/plugins.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,90 +2,118 @@

Nu can be extended using plugins. Plugins behave much like Nu's built-in commands, with the added benefit that they can be added separately from Nu itself.

Nu plugins are executables; Nu launches them as needed and communicates with them over [stdin and stdout](https://en.wikipedia.org/wiki/Standard_streams) or [local sockets](https://en.wikipedia.org/wiki/Inter-process_communication). Nu plugins can use either [JSON](https://www.json.org/) or [MessagePack](https://msgpack.org/) as their communication encoding.
::: warning Important
Plugins communicate with Nushell using the `nu-plugin` protocol. This protocol is versioned, and plugins must use the same `nu-plugin` version provided by Nushell.

## Downloading and Installing a Plugin
When updating Nushell, please make sure to also update any plugins that you have registered.
:::

::: warning
[[toc]]

## Installing Plugins

### Core Plugins

Please note that plugin installation methods are still under heavy development and that the following workflow will be refined before the release of 1.0. The nupm official package manager should simplify installation in the future when it becomes ready for general use.
Nushell ships with a set of officially maintained plugins which includes:

- `polars`: Extremely fast columnar operations using DataFrames via the [Polars Library](https://github.com/pola-rs/polars). See the [DataFrames Chapter](dataframes.html) for more details.
- `formats`: Support for several additional data formats - EML, ICS, INI, plist, and VCF.
- `gstat`: Returns information on the status of a Git repository as Nushell structured data.
- `query`: Support for querying SQL, XML, JSON, HTML (via selector), and WebPage Metadata
- `inc`: Increment a value or version (e.g., semver). This plugin acts as both an end-user plugin as well as a simple developer example of how to create a plugin.

Nushell also ships with several plugins that serve as examples or tools for plugin developers. These include `nu_plugin_example`, `nu_plugin_custom_values`, and `nu_plugin_stress_internals`.

Core plugins are typically distributed with the Nushell release and should already be installed in the same directory as the Nushell executable. If this is the case on your system, core plugins should be using correct `nu-plugin` protocol version. If your package management system installs them separately, please make sure to update the core plugins whenever Nushell itself is updated.

::: tip Installing using Cargo
For example, when installing or upgrading Nushell directly from crates.io using `cargo install nu`, the corresponding core plugins for that version may also be installed or updated using `cargo install nu_plugin_<plugin_name>`.
:::

To install a plugin on your system, you first need to make sure that the plugin uses the same version of Nu as your system.
### Third-party Plugins

```nu
> version
```
You can find third-party plugins on crates.io, online Git repositories, [`awesome-nu`](https://github.com/nushell/awesome-nu/blob/main/plugin_details.md), and other sources. As with any third-party code you run on your system, please make sure you trust its source.

Find plugins that have the exact same Nushell version either on crates.io, online git repositories or [`awesome-nu`](https://github.com/nushell/awesome-nu/blob/main/plugin_details.md). You can find which version the plugin uses by checking the Cargo.toml file.
To install a third-party plugin on your system, you first need to make sure the plugin uses the same version of Nu as your system:

- Confirm your Nushell version with the `version` command
- Confirm the version the plugin requires by checking its `Cargo.toml` file

To install a plugin by name from crates.io, run:

```nu
> cargo install plugin_name
cargo install nu_plugin_<plugin_name>
```

If you chose to download the git repository instead, run this when inside the cloned repository:
When installing from a repository (e.g., GitHub), run the following from inside the cloned repository:

```nu
> cargo install --path .
cargo install --path .
```

This will create a binary file that can be used to add the plugin.

Keep in mind that when installing using crates.io, the binary can be saved in different locations depending on how your system is set up. A typical location is in the users's home directory under .cargo/bin.
::: tip Cargo installation location
By default, binaries installed with `cargo install` are placed in your home directory under `.cargo/bin`.
However, this can change depending on how your system is configured.
:::

## Adding a Plugin
## Registering Plugins

To add a plugin to the plugin registry file, call the [`plugin add`](/commands/docs/plugin_add.md) command to tell Nu where to find it.

Please note that the plugin name needs to start with `nu_plugin_`, Nu uses the name prefix to detect plugins.
::: tip Note
The plugin file name must start with `nu_plugin_`, Nu uses this filename prefix to identify plugins.
:::

Linux+macOS:
- Linux and macOS:

```nu
> plugin add ./my_plugins/nu_plugin_cool
```
```nu
plugin add ./my_plugins/nu_plugin_cool
```

Windows:
- Windows:

```nu
> plugin add .\my_plugins\nu_plugin_cool.exe
```
```nu
plugin add .\my_plugins\nu_plugin_cool.exe
```

When [`plugin add`](/commands/docs/plugin_add.md) is called, Nu runs the plugin binary and communicates via the [plugin protocol](/contributor-book/plugin_protocol_reference.md) in order to ensure compatibility and to get a list of all of the commands it supports. These are then saved to the plugin registry file (`$nu.plugin-path`) which acts as a cache.
When [`plugin add`](/commands/docs/plugin_add.md) is called, Nu:

Once added, the next time `nu` is started, the plugin should show up in [`plugin list`](/commands/docs/plugin_list.md) with all of its commands are available in scope:
- Runs the plugin binary
- Communicates via the [plugin protocol](/contributor-book/plugin_protocol_reference.md) in order to ensure compatibility and to get a list of all of the commands it supports
- This plugin information is then saved to the plugin registry file (`$nu.plugin-path`), which acts as a cache

```nu
> help commands | where command_type == "plugin"
```
Once added to the registry, the next time `nu` is started, the plugin will be available in that session.

You can also immediately reload a plugin in the current session by calling `plugin use`:
You can also immediately reload a plugin in the current session by calling `plugin use`. In this case, the name of the plugin (rather than the filename) is used without the `nu_plugin` prefix:

```nu
> plugin use cool
plugin use cool
```

It is not necessary to add `plugin use` to your config file. All previously added plugins are automatically loaded at startup.
It is not necessary to add `plugin use` statements to your config file. All previously added plugins are automatically loaded at startup.

Note that `plugin use` is a parser keyword, so when evaluating a script, it will be evaluated first. This means that while you can execute `plugin add` and then `plugin use` at the REPL on separate lines, you can't do this in a single script. If you need to run `nu` with a specific plugin or set of plugins without preparing a cache file, you can pass the `--plugins` option to `nu` with a list of plugin executable files:
::: tip Note
`plugin use` is a parser keyword, so when evaluating a script, it will be evaluated first. This means that while you can execute `plugin add` and then `plugin use` at the REPL on separate lines, you can't do this in a single script. If you need to run `nu` with a specific plugin or set of plugins without preparing a cache file, you can pass the `--plugins` option to `nu` with a list of plugin executable files:

```nu
> nu --plugins '[./my_plugins/nu_plugin_cool]'
nu --plugins '[./my_plugins/nu_plugin_cool]'
```

### Updating a Plugin
:::

### Updating Plugins

When updating a plugin, it is important to run `plugin add` again just as above to load the new signatures from the plugin and allow Nu to rewrite them to the plugin file (`$nu.plugin-path`). You can then `plugin use` to get the updated signatures within the current session.

## Managing Plugins

To view the list of plugins you have installed:
Installed plugins are displayed using [`plugin list`](/commands/docs/plugin_list.md):

```nu
> plugin list
plugin list
# =>
╭───┬─────────┬────────────┬─────────┬───────────────────────┬───────┬───────────────────────────────╮
│ # │ name │ is_running │ pid │ filename │ shell │ commands │
├───┼─────────┼────────────┼─────────┼───────────────────────┼───────┼───────────────────────────────┤
Expand All @@ -105,16 +133,36 @@ To view the list of plugins you have installed:
╰───┴─────────┴────────────┴─────────┴───────────────────────┴───────┴───────────────────────────────╯
```

All of the commands from installed plugins are available in the current scope:

```nu
scope commands | where type == "plugin"
```

### Plugin Lifecycle

Plugins stay running while they are in use, and are automatically stopped by default after a period of time of inactivity. This behavior is managed by the [plugin garbage collector](#plugin-garbage-collector). To manually stop a plugin, call `plugin stop` with its name:

For example, run the `gstat` command from the corresponding plugin, then check its `is_running` status:

```nu
> plugin stop gstat
gstat
# => gstat output
plugin list | where name == gstat | select name is_running
# =>
╭───┬───────┬────────────╮
│ # │ name │ is_running │
├───┼───────┼────────────┤
│ 0 │ gstat │ true │
╰───┴───────┴────────────╯
```

If we check `plugin list` again, we can see that it is no longer running:
Now stop the plugin manually, and we can see that it is no longer running:

```nu
> plugin list | where name == gstat | select name is_running
plugin stop gstat
plugin list | where name == gstat | select name is_running
# =>
╭───┬───────┬────────────╮
│ # │ name │ is_running │
├───┼───────┼────────────┤
Expand All @@ -124,7 +172,7 @@ If we check `plugin list` again, we can see that it is no longer running:

### Plugin Garbage Collector

Nu comes with a plugin garbage collector, which automatically stops plugins that are not actively in use after a period of time (by default, 10 seconds). This behavior is fully configurable:
As mentioned above, Nu comes with a plugin garbage collector which automatically stops plugins that are not actively in use after a period of time (by default, 10 seconds). This behavior is fully configurable:

```nu
$env.config.plugin_gc = {
Expand All @@ -149,39 +197,47 @@ $env.config.plugin_gc = {
}
```

For more information on exactly under what circumstances a plugin is considered to be active, see [the relevant section in the contributor book](/contributor-book/plugins.html#plugin-garbage-collection).
For information on when a plugin is considered to be active, see [the relevant section in the contributor book](/contributor-book/plugins.html#plugin-garbage-collection).

## Removing a Plugin
## Removing Plugins

To remove a plugin, call `plugin rm` with the name of the plugin you want to remove. For example, if you previously added the plugin `~/.cargo/bin/nu_plugin_gstat`, its name would be `gstat`. To remove it:
To remove a plugin, call `plugin rm <plugin_name>`. Note that this is the plugin name, rather than the filename. For example, if you previously added the plugin `~/.cargo/bin/nu_plugin_gstat`, its name would be `gstat`. To remove it:

```nu
plugin rm gstat
```

You can check the name of a plugin by running `plugin list`.
You can confirm the name of a plugin by running `plugin list`.

## Examples
Running `plugin rm` removes the plugin from the registry so that it will not be loaded the next time Nushell starts. However, any commands created by the plugin remain in scope until the current Nushell session ends.

## For Plugin Developers

Nu plugins are executables; Nu launches them as needed and communicates with them over [stdin and stdout](https://en.wikipedia.org/wiki/Standard_streams) or [local sockets](https://en.wikipedia.org/wiki/Inter-process_communication). Nu plugins can use either [JSON](https://www.json.org/) or [MessagePack](https://msgpack.org/) as their communication encoding.

### Examples

Nu's main repo contains example plugins that are useful for learning how the plugin protocol works:

- [Rust](https://github.com/nushell/nushell/tree/main/crates/nu_plugin_example)
- [Python](https://github.com/nushell/nushell/blob/main/crates/nu_plugin_python)

## Debugging
### Debugging

The simplest way to debug a plugin is to print to stderr; plugins' standard error streams are redirected through Nu and displayed to the user.

### Tracing
#### Tracing

The Nu plugin protocol message stream may be captured for diagnostic purposes using [trace_nu_plugin](https://crates.io/crates/trace_nu_plugin/).

**WARNING: trace output will accumulate for as long as the plugin is installed with the trace wrapper. Large files are possible. Be sure to remove the plugin with `plugin rm` when finished tracing, and reinstall without the trace wrapper.**
::: warning
Trace output will accumulate for as long as the plugin is installed with the trace wrapper. Large files are possible. Be sure to remove the plugin with `plugin rm` when finished tracing, and reinstall without the trace wrapper.\*\*
:::

## Help
### Developer Help

Nu's plugin documentation is a work in progress. If you're unsure about something, the #plugins channel on [the Nu Discord](https://discord.gg/NtAbbGn) is a great place to ask questions!

## More details
### More details

The [plugin chapter in the contributor book](/contributor-book/plugins.md) offers more details on the intricacies of how plugins work from a software developer point of view.