Skip to content

Commit 257df1c

Browse files
xuanyuankingxuanyuanking
authored
Merge pull request #1 from sjrusso8/source
### Initial Merge Notice This is the initial merge from the original project into the Apache repository. - **Apache Spark PMC Acceptance Vote:** [https://lists.apache.org/thread/mj3ggbsqhjr2lblffmyn1rq5zcfckcrn](https://lists.apache.org/thread/mj3ggbsqhjr2lblffmyn1rq5zcfckcrn) - **IP Clearance File:** [https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/spark-connect-rust.xml](https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/spark-connect-rust.xml) All four contributors have ICLAs on file: - **Steve Russo** – Filed 2025-06-25 - **Stephen Carman** – Filed 2025-06-25 - **Alexander Brassel** – Filed 2025-07-03 - **Irfan Ghat** – Filed 2025-07-04
2 parents 84db605 + 379e775 commit 257df1c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+23599
-1
lines changed

‎.github/pull_request_template.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
<!--
2+
~ Licensed to the Apache Software Foundation (ASF) under one
3+
~ or more contributor license agreements. See the NOTICE file
4+
~ distributed with this work for additional information
5+
~ regarding copyright ownership. The ASF licenses this file
6+
~ to you under the Apache License, Version 2.0 (the
7+
~ "License"); you may not use this file except in compliance
8+
~ with the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing,
13+
~ software distributed under the License is distributed on an
14+
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
~ KIND, either express or implied. See the License for the
16+
~ specific language governing permissions and limitations
17+
~ under the License.
18+
-->
19+
20+
# Description
21+
22+
The description of the main changes of your pull request
23+
24+
## Related Issue(s)
25+
<!---
26+
For example:
27+
28+
- closes #106
29+
--->
30+
31+
## Documentation
32+
33+
<!---
34+
Share links to useful documentation
35+
--->

‎.github/workflows/build.yml

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
ensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: build
19+
20+
on:
21+
push:
22+
branches: [main, "v*"]
23+
pull_request:
24+
branches: [main, "v*"]
25+
26+
jobs:
27+
format:
28+
runs-on: ubuntu-latest
29+
steps:
30+
- uses: actions/checkout@v4
31+
with:
32+
submodules: "true"
33+
34+
- name: install protoc
35+
uses: arduino/setup-protoc@v2
36+
with:
37+
version: 23.x
38+
39+
- name: Install minimal stable with clippy and rustfmt
40+
uses: actions-rs/toolchain@v1
41+
with:
42+
profile: default
43+
toolchain: stable
44+
override: true
45+
46+
- name: Format
47+
run: cargo fmt -- --check
48+
49+
build:
50+
runs-on: ubuntu-latest
51+
52+
steps:
53+
- uses: actions/checkout@v4
54+
with:
55+
submodules: "true"
56+
57+
- name: install protoc
58+
uses: arduino/setup-protoc@v2
59+
with:
60+
version: 23.x
61+
62+
- name: install minimal stable with clippy and rustfmt
63+
uses: actions-rs/toolchain@v1
64+
with:
65+
profile: default
66+
toolchain: stable
67+
override: true
68+
69+
- uses: Swatinem/rust-cache@v2
70+
71+
- name: build and lint with clippy
72+
run: cargo clippy
73+
74+
- name: Check docs
75+
run: cargo doc
76+
77+
- name: Check no default features (except rustls)
78+
run: cargo check
79+
80+
integration_test:
81+
name: integration tests
82+
runs-on: ubuntu-latest
83+
env:
84+
CARGO_INCREMENTAL: 0
85+
# Disable full debug symbol generation to speed up CI build and keep memory down
86+
# <https://doc.rust-lang.org/cargo/reference/profiles.html>
87+
RUSTFLAGS: "-C debuginfo=line-tables-only"
88+
# https://github.com/rust-lang/cargo/issues/10280
89+
CARGO_NET_GIT_FETCH_WITH_CLI: "true"
90+
RUST_BACKTRACE: "1"
91+
92+
steps:
93+
- uses: actions/checkout@v4
94+
with:
95+
submodules: "true"
96+
97+
- name: install protoc
98+
uses: arduino/setup-protoc@v2
99+
with:
100+
version: 23.x
101+
102+
- name: install minimal stable with clippy and rustfmt
103+
uses: actions-rs/toolchain@v1
104+
with:
105+
profile: default
106+
toolchain: stable
107+
override: true
108+
109+
- uses: Swatinem/rust-cache@v2
110+
111+
- name: Start emulated services
112+
run: docker compose up -d
113+
114+
- name: Run tests
115+
run: cargo test -p spark-connect-rs --features polars,datafusion

‎.github/workflows/release.yml

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: release to cargo
19+
20+
on:
21+
push:
22+
tags: ["v*"]
23+
24+
jobs:
25+
validate-release-tag:
26+
name: Validate git tag
27+
runs-on: ubuntu-20.04
28+
steps:
29+
- uses: actions/checkout@v4
30+
- name: compare git tag with cargo metadata
31+
run: |
32+
PUSHED_TAG=${GITHUB_REF##*/}
33+
CURR_VER=$( grep version Cargo.toml | head -n 1 | awk '{print $3}' | tr -d '"' )
34+
if [[ "${PUSHED_TAG}" != "v${CURR_VER}" ]]; then
35+
echo "Cargo metadata has version set to ${CURR_VER}, but got pushed tag ${PUSHED_TAG}."
36+
exit 1
37+
fi
38+
working-directory: ./crates
39+
40+
release-crate:
41+
needs: validate-release-tag
42+
name: Release crate
43+
runs-on: ubuntu-20.04
44+
steps:
45+
- uses: actions/checkout@v4
46+
47+
- uses: actions-rs/toolchain@v1
48+
with:
49+
profile: minimal
50+
toolchain: stable
51+
override: true
52+
53+
- name: install protoc
54+
uses: arduino/setup-protoc@v2
55+
with:
56+
version: 23.x
57+
58+
- name: cargo publish rust
59+
uses: actions-rs/cargo@v1
60+
env:
61+
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
62+
with:
63+
command: publish
64+
args: --token "${CARGO_REGISTRY_TOKEN}" --package spark-connect-rs --manifest-path ./Cargo.toml

‎.gitignore

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
/target
19+
20+
.vscode
21+
*.ipynb
22+
23+
/spark-warehouse
24+
/artifacts

‎.pre-commit-config.yaml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
repos:
19+
- repo: https://github.com/pre-commit/pre-commit-hooks
20+
rev: v4.4.0
21+
hooks:
22+
- id: check-byte-order-marker
23+
- id: check-case-conflict
24+
- id: check-merge-conflict
25+
- id: check-symlinks
26+
- id: check-yaml
27+
- id: end-of-file-fixer
28+
- id: mixed-line-ending
29+
- id: trailing-whitespace
30+
- repo: https://github.com/doublify/pre-commit-rust
31+
rev: v1.0
32+
hooks:
33+
- id: fmt
34+
- id: cargo-check
35+
- id: clippy

0 commit comments

Comments
 (0)