Skip to content

Add REST catalog support in docs (#1) #4031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 21, 2025
Merged

Conversation

somratdutta
Copy link
Contributor

@somratdutta somratdutta commented Jul 7, 2025

Summary

This PR adds comprehensive documentation for ClickHouse's REST Catalog integration.

Checklist

@somratdutta somratdutta requested review from a team as code owners July 7, 2025 03:01
@somratdutta somratdutta requested a review from BentsiLeviav July 7, 2025 03:01
Copy link

vercel bot commented Jul 7, 2025

@somratdutta is attempting to deploy a commit to the ClickHouse Team on Vercel.

A member of the Team first needs to authorize it.

Copy link

vercel bot commented Jul 8, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
clickhouse-docs ✅ Ready (Inspect) Visit Preview Jul 21, 2025 8:01am
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
clickhouse-docs-ru ⬜️ Ignored (Inspect) Visit Preview Jul 21, 2025 8:01am
clickhouse-docs-zh ⬜️ Ignored (Inspect) Visit Preview Jul 21, 2025 8:01am

@Blargian
Copy link
Member

Blargian commented Jul 8, 2025

@somratdutta would you mind to merge main into your branch? The error the vercel deployment is failing on is unrelated to your changes and was fixed yesterday.

@somratdutta
Copy link
Contributor Author

Sure @Blargian.
I would do it, thanks for reviewing my PR.

Copy link
Member

@Blargian Blargian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@somratdutta firstly, thank you so much for this contribution, I am really excited to see it! I've left a few comments. I was unable to get this working following these steps. Let's put together an example which can be easily reproduced.

Comment on lines 50 to 68
clickhouse:
image: clickhouse/clickhouse-server:main
container_name: clickhouse
user: '0:0' # Ensures root permissions
networks:
iceberg_net:
ports:
- "8123:8123"
- "9002:9000"
volumes:
- ./clickhouse:/var/lib/clickhouse
- ./clickhouse/data_import:/var/lib/clickhouse/data_import # Mount dataset folder
networks:
- iceberg_net
environment:
- CLICKHOUSE_DB=default
- CLICKHOUSE_USER=default
- CLICKHOUSE_DO_NOT_CHOWN=1
- CLICKHOUSE_PASSWORD=
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
clickhouse:
image: clickhouse/clickhouse-server:main
container_name: clickhouse
user: '0:0' # Ensures root permissions
networks:
iceberg_net:
ports:
- "8123:8123"
- "9002:9000"
volumes:
- ./clickhouse:/var/lib/clickhouse
- ./clickhouse/data_import:/var/lib/clickhouse/data_import # Mount dataset folder
networks:
- iceberg_net
environment:
- CLICKHOUSE_DB=default
- CLICKHOUSE_USER=default
- CLICKHOUSE_DO_NOT_CHOWN=1
- CLICKHOUSE_PASSWORD=
clickhouse:
image: clickhouse/clickhouse-server:25.5.6
container_name: clickhouse
user: '0:0' # Ensures root permissions
networks:
iceberg_net:
ports:
- "8123:8123"
- "9002:9000"
volumes:
- ./clickhouse:/var/lib/clickhouse
- ./clickhouse/data_import:/var/lib/clickhouse/data_import # Mount dataset folder
networks:
- iceberg_net
environment:
- CLICKHOUSE_DB=default
- CLICKHOUSE_USER=default
- CLICKHOUSE_DO_NOT_CHOWN=1
- CLICKHOUSE_PASSWORD=

Comment on lines +101 to +103
┌─name──────────┐
│ default.taxis │
└───────────────┘
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I unfortunately don't get this when I try to run the steps. I'm getting back:

SHOW TABLES IN demo

Query id: 4411372a-a71c-44e9-b27b-146af2048670

Ok.

0 rows in set. Elapsed: 0.047 sec.

demo is however created:

SHOW DATABASES

Query id: 70f26176-08cd-4e5e-b788-44ce1adf10eb

   ┌─name───────────────┐
1. │ INFORMATION_SCHEMA │
2. │ default            │
3. │ demo               │
4. │ information_schema │
5. │ system             │
   └────────────────────┘

Can you confirm you were able to get this working locally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mentioned the reason below.

@Blargian Blargian self-assigned this Jul 9, 2025
@somratdutta
Copy link
Contributor Author

somratdutta commented Jul 14, 2025

@somratdutta firstly, thank you so much for this contribution, I am really excited to see it! I've left a few comments. I was unable to get this working following these steps. Let's put together an example which can be easily reproduced.

Thanks @Blargian for getting back fast on this.
I just realised that REST catalog support is available once after this PR is merged.
Our team has been working extensively to improve Iceberg support in Clickhouse.
This PR will add feature that resolves the issue you're facing where the tables aren't listed.
Internally, we have built Clickhouse and used that image to test it, alternatively in the meanwhile you can use a binary from the PR to check for yourself.

Hoping this clears things up.

- Change allow_experimental_database_rest_catalog to allow_experimental_database_iceberg
- This is the correct flag name for enabling Iceberg database integration
- Fix description to remove S3 buckets reference (not relevant for this guide)
- Fix docker-compose YAML network configuration to avoid duplication
- Add step-by-step setup instructions for better clarity
- Add troubleshooting guidance for users who don't see expected tables
- Include note about sample data loading requirements

Addresses feedback from PR ClickHouse#4031 review comments
- Step 1: Create new folder and docker-compose.yml from Databricks repo
- Step 2: Create docker-compose.override.yml with ClickHouse configuration
- Step 3: Run docker compose up
- Step 4: Wait for services and check logs

This addresses Blargian's specific feedback in PR ClickHouse#4031 for clearer setup instructions.
@Blargian
Copy link
Member

@somratdutta firstly, thank you so much for this contribution, I am really excited to see it! I've left a few comments. I was unable to get this working following these steps. Let's put together an example which can be easily reproduced.

Thanks @Blargian for getting back fast on this. I just realised that REST catalog support is available once after this PR is merged. Our team has been working extensively to improve Iceberg support in Clickhouse. This PR will add feature that resolves the issue you're facing where the tables aren't listed. Internally, we have built Clickhouse and used that image to test it, alternatively in the meanwhile you can use a binary from the PR to check for yourself.

Hoping this clears things up.

@somratdutta Thanks for clarifying, and sorry for the confusion on my part! Perfect, I'll keep an eye out for when that PR gets merged and we can merge this in shortly after.

@somratdutta
Copy link
Contributor Author

Hi @Blargian .
This PR is merged, can you take a look at this now?
Let me know if you have any doubts. 😄

@Blargian
Copy link
Member

@somratdutta would you mind to merge main into your branch? Should fix the vercel deploy failure

@somratdutta somratdutta requested a review from a team as a code owner July 21, 2025 06:16
@workflow-authentication-public

Trademark License Addendum

Merging of this pull request is temporarily blocked as it potentially contains a contribution containing a trademark. git diffPlease
read and agree to the Trademark License Addendum below to
unblock merging of this pull request.

Click to see Trademark License Addendum

This Trademark License Addendum ("Addendum") shall, if You have opted
in by replying to the comment that references this Addendum that you
have read and agree to theContributor License Agreement Addendum,
supplement the terms of the Individual Contributor License Agreement
between You and the Company ("Agreement"). Capitalized terms not
defined herein shall have the meanings ascribed to them in the
Agreement.

  1. Grant of Trademark License. Subject to the terms and conditions
    of this Addendum, You grant to the Company a revocable, worldwide,
    non-exclusive, non-sublicensable (except for contractors or agents
    acting on the Company's behalf, for whose compliance with this
    Addendum Company agrees to be responsible), royalty-free, and non-transferable
    right to display the Partner Trademarks, solely for the purpose of
    marketing and promoting your Contribution (i) on the Company's website
    and in any Company in-product integrations page; and (ii) in marketing, sales,
    and product materials for Company products. "Partner Trademarks" mean Your
    employer's name and any employer brand features (e.g., logo) You submit now or
    in the future to the Company in connection with your Contributions.
  2. Legal authority. You represent that you are legally entitled to
    grant the above license. If your employer(s) has rights to
    intellectual property in the Partner Trademarks, you represent that
    you have received permission to grant the above license on behalf
    of that employer, or that your employer has executed a separate
    agreement with the Company concerning the subject matter of this
    Addendum.
  3. Conditions. The license in Section 1 is subject to the following
    conditions:
    i. The Company shall use the Partner Trademarks in accordance with
    any reasonable trademark usage guidelines You provide;
    ii. You may revoke this license at any time upon thirty (30) days'
    written notice to the Company, after which the Company shall use
    commercially reasonable efforts to cease all further public
    use of the Partner Trademarks (but may maintain uses in archived
    web pages, changelogs, and previously distributed materials).
    iii. The Company acknowledges and agrees that it does not own the
    Partner Trademarks and that all goodwill derived from the use
    of the Partner Trademarks inures solely to benefit of the
    Partner Trademarks' owner(s).
    iv. The Company shall use the Partner Trademarks in a professional
    manner consistent with industry standards and shall not use
    them in any way that would reasonably be expected to diminish
    their value or harm the reputation of the Partner Trademarks'
    owner(s). The Company's use of Partner Trademarks shall not
    imply endorsement, sponsorship, or affiliation beyond the
    existence of the Contribution in the Company's integration program.
    v. The Company will not use the Partner Trademarks in connection
    with search engine rankings, ad word purchases, or as part of a
    trade name, business name, or Internet domain name.

To unblock this PR, reply with exactly:

I agree to the Trademark License Addendum
CLA-SIGNATURE: somratdutta

@somratdutta
Copy link
Contributor Author

Sure @Blargian !

@somratdutta
Copy link
Contributor Author

Trademark License Addendum

Merging of this pull request is temporarily blocked as it potentially contains a contribution containing a trademark. git diffPlease read and agree to the Trademark License Addendum below to unblock merging of this pull request.

Click to see Trademark License Addendum
To unblock this PR, reply with exactly:

I agree to the Trademark License Addendum
CLA-SIGNATURE: somratdutta

I agree to the Trademark License Addendum
CLA-SIGNATURE: somratdutta

@workflow-authentication-public

Trademark license agreement confirmed ✅

The trademark license agreement has been confirmed for @somratdutta.

Status: Confirmed
Date: 2025-07-21T07:36:27.287Z
Method: Self-signed agreement via comment

This PR is now unblocked and can proceed with normal review.

@Blargian Blargian merged commit 1f10253 into ClickHouse:main Jul 21, 2025
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants