Skip to content

add a Makefile to vendor/ #2218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions vendor/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Makefile for generating vendor packages

.PHONY: all clean fb303 hive-metastore

all: fb303 hive-metastore

# FB303 Thrift client generation
fb303:
rm -f /tmp/fb303.thrift
rm -rf fb303
curl -s https://raw.githubusercontent.com/apache/thrift/master/contrib/fb303/if/fb303.thrift > /tmp/fb303.thrift
rm -rf /tmp/gen-py/
thrift -gen py -o /tmp/ /tmp/fb303.thrift
mv /tmp/gen-py/fb303 fb303

# Hive Metastore Thrift definition generation
hive-metastore:
rm -rf /tmp/hive
mkdir -p /tmp/hive/share/fb303/if/
curl -s https://raw.githubusercontent.com/apache/thrift/master/contrib/fb303/if/fb303.thrift > /tmp/hive/share/fb303/if/fb303.thrift
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While at it, should we pass in the Hive version explicitly, instead of using master?

Copy link
Contributor Author

@kevinjqliu kevinjqliu Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was thinking about that too.
the fb303.thrift here is in the apache/thrift repo, which has its own versioning. Its currently on v0.22.0. See https://github.com/apache/thrift/tags

hive_metastore.thrift below is in the apache/hive repo which uses hive version. Its currently on 4.0.1. See https://github.com/apache/hive/tags

Its hard to figure out which version we should pick here for each. This might be left to the person upgrading the vendor/

curl -s https://raw.githubusercontent.com/apache/hive/master/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift > /tmp/hive/hive_metastore.thrift
thrift -gen py -o /tmp/hive /tmp/hive/hive_metastore.thrift
rm -rf hive_metastore
mv /tmp/hive/gen-py/hive_metastore hive_metastore
40 changes: 27 additions & 13 deletions vendor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,42 @@

Some packages we want to maintain in the repository itself, because there is no good 3rd party alternative.

## FB303 Thrift client
## Quick Setup

Generate all vendor packages:

```bash
make all
```

Generate individual packages:

```bash
make fb303 # FB303 Thrift client only
make hive-metastore # Hive Metastore Thrift definitions only
```

## Packages

### FB303 Thrift client

fb303 is a base Thrift service and a common set of functionality for querying stats, options, and other information from a service.

**Generate with Make:**
```bash
rm -f /tmp/fb303.thrift
rm -rf fb303
curl -s https://raw.githubusercontent.com/apache/thrift/master/contrib/fb303/if/fb303.thrift > /tmp/fb303.thrift
rm -rf /tmp/gen-py/
thrift -gen py -o /tmp/ /tmp/fb303.thrift
mv /tmp/gen-py/fb303 fb303
make fb303
```

# Hive Metastore Thrift definition

The thrift definition require the fb303 service as a dependency

**Generate with Make:**
```bash
rm -rf /tmp/hive
mkdir -p /tmp/hive/share/fb303/if/
curl -s https://raw.githubusercontent.com/apache/thrift/master/contrib/fb303/if/fb303.thrift > /tmp/hive/share/fb303/if/fb303.thrift
curl -s https://raw.githubusercontent.com/apache/hive/master/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift > /tmp/hive/hive_metastore.thrift
thrift -gen py -o /tmp/hive /tmp/hive/hive_metastore.thrift
mv /tmp/hive/gen-py/hive_metastore hive_metastore
make hive-metastore
```

## Requirements

- Apache Thrift compiler (`thrift`)
- `curl` for downloading Thrift definitions