Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
d57b994
add unix socket support
jmcphers Jun 23, 2025
a386eee
make api more generic
jmcphers Jun 24, 2025
2fa2911
handle domain sockets
jmcphers Jun 24, 2025
c1638d7
rustfmt
jmcphers Jun 24, 2025
27c5e64
Merge remote-tracking branch 'origin/main' into feature/unix-domain-s…
jmcphers Jun 24, 2025
66f0f7c
initial impl of named pipes for windows
jmcphers Jun 24, 2025
a9a9ebc
write socket details to connection file
jmcphers Jun 24, 2025
74413da
use tcp transport for old test
jmcphers Jun 24, 2025
58ce905
rustfmt
jmcphers Jun 24, 2025
f19c65c
continued work on named pipe tests
jmcphers Jun 24, 2025
4cf0815
fix named pipe tests
jmcphers Jun 24, 2025
1cdd4fb
basic create/list over named pipe
jmcphers Jun 24, 2025
454c991
rustfmt
jmcphers Jun 24, 2025
6396c83
impl basic get/list over named pipes
jmcphers Jun 25, 2025
41cabde
use hyper pattern for named pipe
jmcphers Jun 25, 2025
a60b53d
add tests for unix domain socket / kernel io
jmcphers Jun 25, 2025
d40f0dc
add test for creating kernel session over named pipes
jmcphers Jun 25, 2025
604c159
add test for named pipe kernel communication
jmcphers Jun 25, 2025
e7c33a9
update README
jmcphers Jun 25, 2025
94cd088
update claude instructions
jmcphers Jun 25, 2025
92f120f
support ws+unix urls
jmcphers Jun 25, 2025
ce7f03c
allow client to send an upgrade header if it wants
jmcphers Jun 25, 2025
7457c3d
include socket path in kernel session info
jmcphers Jun 25, 2025
d3c0aff
clean up domain sockets on exit
jmcphers Jun 25, 2025
1b7f86f
fix unused import on windows
jmcphers Jun 25, 2025
d422eee
Merge branch 'feature/unix-domain-sockets' of github.com:posit-dev/ka…
jmcphers Jun 25, 2025
14d3bb1
cleanup and refactor integration tests
jmcphers Jun 25, 2025
efc29ef
fix websocket test (phase 1)
jmcphers Jun 26, 2025
44aa7f5
use domain socket for domain socket tests
jmcphers Jun 26, 2025
d0dd7a7
don't wait for kernel info reply
jmcphers Jun 26, 2025
b5c7a44
send a websocket handshake in the tests
jmcphers Jun 26, 2025
b0e6f41
fix compilation on windows
jmcphers Jun 26, 2025
fb2b87b
fix tests on windows
jmcphers Jun 26, 2025
204a4ac
clean up some duplicated code in server startup
jmcphers Jun 26, 2025
19235b6
reduce duplication in test setup
jmcphers Jun 26, 2025
b36456f
use session id to derive socket path
jmcphers Jun 26, 2025
f731fca
fix compile on windows
jmcphers Jun 26, 2025
4e44b86
support "websockets over named pipes" (tm)
jmcphers Jun 26, 2025
5f0dc50
don't exceed unix socket length
jmcphers Jun 26, 2025
6ebb55c
update readme
jmcphers Jun 26, 2025
751843e
no need for cfg(windows) in windows-only module
jmcphers Jun 27, 2025
f7059d5
allow configuring the socket directory
jmcphers Jun 27, 2025
953b133
clean up main socket on shutdown
jmcphers Jun 27, 2025
cc2de7b
validate transport arguments
jmcphers Jun 30, 2025
b61febe
refactor transport to an async crate
jmcphers Jun 30, 2025
aaa09fa
validate unix domain socket length
jmcphers Jun 30, 2025
cc3ef2c
minor formatting/cleanup
jmcphers Jul 1, 2025
af5e7e5
only create a release when building on main
jmcphers Jul 1, 2025
679d5e4
fix compile issues on windows
jmcphers Jul 1, 2025
190e46a
remove unused standalone named pipe generator
jmcphers Jul 1, 2025
604ef68
Merge remote-tracking branch 'origin/main' into feature/unix-domain-s…
jmcphers Jul 1, 2025
4d0f03b
use more specific imports for tests
jmcphers Jul 1, 2025
865e909
try `.exe` when finding kernel path on windows
jmcphers Jul 1, 2025
ffde9e3
windows is slow, okay?
jmcphers Jul 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ jobs:

do_release:
name: Trigger a new release
if: ${{ needs.check_release.outputs.EXISTING_RELEASE == 'false' }}
if: ${{ needs.check_release.outputs.EXISTING_RELEASE == 'false' && github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
needs: [check_release]
steps:
Expand All @@ -125,7 +125,7 @@ jobs:
build_macos:
name: Build macOS
runs-on: [self-hosted-staging, macos, arm64]
needs: [do_release, get_version]
needs: [get_version]
timeout-minutes: 40

env:
Expand Down Expand Up @@ -209,7 +209,7 @@ jobs:
name: Build Windows
runs-on: windows-latest
timeout-minutes: 40
needs: [do_release, get_version]
needs: [get_version]

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -254,7 +254,7 @@ jobs:
build_linux:
name: "Build Linux"
uses: ./.github/workflows/release-linux.yml
needs: [do_release, get_version]
needs: [get_version]
secrets: inherit
with:
version: ${{ needs.get_version.outputs.KALLICHORE_VERSION }}
Expand Down
75 changes: 74 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,4 +99,77 @@ Frontend ↔ HTTP/WebSocket ↔ Kallichore ↔ ZeroMQ ↔ Jupyter Kernels

Use `RUST_LOG` environment variable for detailed debugging:
- `RUST_LOG=trace` for maximum verbosity
- `RUST_LOG=kallichore=debug` for library-specific logging
- `RUST_LOG=kallichore=debug` for library-specific logging

## Transport Types

Kallichore supports three different transport mechanisms for client connections:

### TCP (Default)
- **Platform**: All platforms (macOS, Windows, Linux)
- **Usage**: `--transport tcp` or omit (default)
- **Description**: Standard TCP sockets on 127.0.0.1
- **Best for**: General purpose, cross-platform compatibility

### Unix Domain Sockets
- **Platform**: Unix-like systems only (macOS, Linux)
- **Usage**: `--transport socket`
- **Description**: Unix domain sockets for inter-process communication with WebSocket protocol support
- **Protocol**: WebSocket over Unix domain sockets (supports `ws+unix:` URLs)
- **Best for**: High performance local communication, reduced overhead
- **Note**: Not supported on Windows

### Named Pipes
- **Platform**: Windows only
- **Usage**: `--transport named-pipe`
- **Description**: Windows named pipes for inter-process communication
- **Best for**: Windows-native IPC, integration with Windows applications
- **Note**: Not supported on Unix systems

Example usage:
```bash
# TCP (default)
./target/debug/kcserver --port 8080

# Unix domain socket with WebSocket protocol (macOS/Linux)
./target/debug/kcserver --transport socket

# Named pipe (Windows)
./target/debug/kcserver --transport named-pipe
```

### WebSocket Protocol Support

**Unix Domain Sockets**: Starting with this version, Unix domain socket kernel client sessions use the WebSocket protocol instead of raw JSON. This enables clients to connect using `ws+unix:` URLs and provides a consistent WebSocket interface across all transport types.

**Client Connection Examples**:
- TCP WebSocket: `ws://localhost:8080/sessions/{session_id}/channels`
- Unix Socket WebSocket: `ws+unix:/path/to/socket:/sessions/{session_id}/channels` (conceptual - actual connection via domain socket path)

## Testing

### Running Tests

```bash
# Run all tests
cargo test

# Run with verbose output
cargo test -- --nocapture
```

### Test Types

- **Unit tests**: Located within `src/` files using `#[cfg(test)]` modules
- **Integration tests**: Located in `crates/kcserver/tests/`
- `integration_test.rs`: General TCP and WebSocket functionality
- `named_pipe_test.rs`: Windows named pipe specific tests (Windows only)
- **Platform-specific tests**: Automatically disabled on unsupported platforms

### Test Environment

Tests automatically:
- Start temporary server instances
- Create isolated temporary directories
- Handle platform-specific transport mechanisms
- Clean up resources after completion
43 changes: 42 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

115 changes: 115 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,121 @@ To make changes to the API, edit the `kallichore.json` file and then run the `sc
> [!IMPORTANT]
> Because we have custom behavior attached to some of the endpoints, we have some manual edits applied to the generated code. These are generally fenced with `--- Start Kallichore ---` and `--- End Kallichore ---` comments. Be sure to reapply these edits (or just revert changes that delete them) if you regenerate the API.

## Connection Methods

Kallichore supports multiple transport mechanisms for client connections. The server can be configured to use TCP, Unix domain sockets, or Windows named pipes.

### TCP (Default)

TCP is the default transport method that works across all platforms. In TCP mode, the supervisor listens on a local TCP port and accepts RPCs over HTTP. This is suitable for remote connections and works on all operating systems.

Connections to individual kernels are made with WebSockets, which are established over the HTTP connection.

#### Starting the server with TCP

```bash
# Use default port (random)
./target/debug/kcserver

# Use specific port
./target/debug/kcserver --port 8080

# Create a connection file with TCP
./target/debug/kcserver --connection-file connection.json --transport tcp
```

#### Example TCP connection file

When using `--connection-file`, the server writes connection details to the specified file:

```json
{
"port": 54321,
"base_path": "http://127.0.0.1:54321",
"transport": "tcp",
"server_path": "/path/to/kcserver",
"server_pid": 12345,
"bearer_token": "your-auth-token",
"log_path": "/path/to/logfile.log"
}
```

### Unix Domain Sockets (Unix/Linux/macOS)

Unix domain sockets provide high-performance IPC for local connections on Unix-like systems. They offer better security than TCP, since they use filesystem permissions. However, they are not available on Windows and are not suitable for remote connections.

When using domain sockets, the server listens on a specific socket file instead of a TCP port. When connecting to individual kernels, each kernel gets a dedicated Unix socket for communication.

#### Starting the server with Unix sockets

```bash
# Use connection file with socket (default on Unix when using --connection-file)
./target/debug/kcserver --connection-file connection.json

# Explicitly specify socket transport
./target/debug/kcserver --connection-file connection.json --transport socket

# Use specific socket path
./target/debug/kcserver --unix-socket /tmp/kallichore.sock
```

#### Example Unix socket connection file

```json
{
"socket_path": "/tmp/kallichore-12345.sock",
"transport": "socket",
"server_path": "/path/to/kcserver",
"server_pid": 12345,
"bearer_token": "your-auth-token",
"log_path": "/path/to/logfile.log"
}
```

### Named Pipes (Windows)

Named pipes are the Windows equivalent of Unix domain sockets, providing efficient local IPC on Windows systems. They are not available on Unix-like systems and are only used for local connections.

When using named pipes, the server listens on a named pipe path instead of a TCP port or Unix socket. Each kernel session also gets its own named pipe for communication.

#### Starting the server with named pipes

```bash
# Use connection file with named pipe (default on Windows when using --connection-file)
kcserver.exe --connection-file connection.json

# Explicitly specify named pipe transport
kcserver.exe --connection-file connection.json --transport named-pipe
```

#### Example named pipe connection file

```json
{
"named_pipe": "\\\\.\\pipe\\kallichore-12345",
"transport": "named-pipe",
"server_path": "C:\\path\\to\\kcserver.exe",
"server_pid": 12345,
"bearer_token": "your-auth-token",
"log_path": "C:\\path\\to\\logfile.log"
}
```

### Transport Selection Rules

The server automatically selects the appropriate transport based on:

1. **Explicit `--transport` flag**: Overrides all other settings
2. **Connection file mode**: Defaults to `socket` on Unix, `named-pipe` on Windows, `tcp` elsewhere
3. **Direct mode**: Uses `tcp` when no connection file is specified

### Security Considerations

- **TCP**: Binds to `127.0.0.1` (localhost only) by default for security
- **Unix Sockets**: Use filesystem permissions for access control
- **Named Pipes**: Use Windows security descriptors for access control
- **Authentication**: All transports support bearer token authentication when enabled

## Repository Structure

```
Expand Down
6 changes: 3 additions & 3 deletions crates/kallichore_api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ To see how to make this your own, look here:
[README]((https://openapi-generator.tech))

- API version: 1.0.0
- Build date: 2025-06-09T10:30:16.634931-07:00[America/Los_Angeles]
- Build date: 2025-06-25T12:12:05.755509-07:00[America/Los_Angeles]
- Generator version: 7.13.0

For more information, please visit [https://posit.co](https://posit.co)
Expand Down Expand Up @@ -91,7 +91,7 @@ cargo run --example client GetServerConfiguration
cargo run --example client ListSessions
cargo run --example client ServerStatus
cargo run --example client ShutdownServer
cargo run --example client ChannelsWebsocket
cargo run --example client ChannelsUpgrade
cargo run --example client ConnectionInfo
cargo run --example client DeleteSession
cargo run --example client GetSession
Expand Down Expand Up @@ -142,7 +142,7 @@ Method | HTTP request | Description
[**set-server-configuration**](docs/default_api.md#set-server-configuration) | **POST** /server_configuration | Change the server configuration
[**shutdown-server**](docs/default_api.md#shutdown-server) | **POST** /shutdown | Shut down all sessions and the server itself
[**adopt-session**](docs/default_api.md#adopt-session) | **PUT** /sessions/{session_id}/adopt | Adopt an existing session
[**channels-websocket**](docs/default_api.md#channels-websocket) | **GET** /sessions/{session_id}/channels | Upgrade to a WebSocket for channel communication
[**channels-upgrade**](docs/default_api.md#channels-upgrade) | **GET** /sessions/{session_id}/channels | Upgrade to a WebSocket or domain socket for channel communication
[**connection-info**](docs/default_api.md#connection-info) | **GET** /sessions/{session_id}/connection_info | Get Jupyter connection information for the session
[**delete-session**](docs/default_api.md#delete-session) | **DELETE** /sessions/{session_id} | Delete session
[**get-session**](docs/default_api.md#get-session) | **GET** /sessions/{session_id} | Get session details
Expand Down
Loading
Loading