chore!: replace direct download of nyc_taxi with hvsampledata #1462

Azaya89 · 2025-09-26T13:26:36Z

This PR replaces the direct download of the large nyc_taxi data with the version from hvsampledata.

Testing with an editable install of hvsampledata

import hvsampledata as hvs
import datashader as ds

df = hvs.nyc_taxi_remote("pandas", engine_kwargs={"columns": ['dropoff_x', 'dropoff_y']})
print(df.head())

agg = ds.Canvas().points(df, 'dropoff_x', 'dropoff_y')
print(agg.shape)

   dropoff_x  dropoff_y
0 -8234835.5  4975627.0
1 -8237020.5  4976875.0
2 -8232279.0  4986477.0
3 -8238124.0  4971127.0
4 -8238107.5  4974457.0
(600, 600)

Depends on hvsampledata PR being merged first for CI to pass

hoxbro

You should remove pyct + setuptools from pyproject.toml + pixi.toml

datashader/__init__.py

hoxbro · 2025-09-26T13:35:51Z

examples/getting_started/1_Introduction.ipynb

+    "import hvsampledata as hvs\n",
    "\n",
-    "df = pd.read_csv('../data/nyc_taxi.csv', usecols=['dropoff_x', 'dropoff_y'])\n",
+    "df = hvs.nyc_taxi(\"pandas\", engine_kwargs={\"columns\": ['dropoff_x', 'dropoff_y']})\n",


Suggested change

"df = hvs.nyc_taxi(\"pandas\", engine_kwargs={\"columns\": ['dropoff_x', 'dropoff_y']})\n",

"df = hvs.nyc_taxi_remote(\"pandas\", engine_kwargs={\"columns\": ['dropoff_x', 'dropoff_y']})\n",

Also, mention that this is the first time this cell is run.

hoxbro · 2025-09-26T13:37:29Z

pixi.toml

 setuptools = "*" # distutils for pyct
 toolz = "*"
 xarray = "*"
+fastparquet = "*"


Please check that this has not indirectly pinned anything like pandas.

scripts/download_data.py

codspeed-hq · 2025-09-26T13:51:29Z

CodSpeed Performance Report

Merging #1462 will improve performances by 41.46%

_{Comparing hvsampledata (a916499) with main (a5ae17c)}

Summary

⚡ 1 improvement
✅ 42 untouched

Benchmarks breakdown

	Mode	Benchmark	`BASE`	`HEAD`	Change
⚡	Instrumentation	`test_layout[forceatlas2_layout]`	72.2 ms	51 ms	+41.46%

maximlt

I think this should be first deprecated before being removed https://holoviz.org/about/heps/hep2.html.

hoxbro · 2025-10-09T10:29:26Z

I think this should be first deprecated before being removed holoviz.org/about/heps/hep2.html.

Are you okay with not having pyct as a required dependency? but still having the entrypoint and explaining that:

the entrypoint is deprecated, but users need to install pyct for this to work and,
maybe guide people to use hvsampledata if they want to download data?

This is also a little bit weird, because we don't break running code, but "only" the CLI.

maximlt · 2025-10-10T07:46:11Z

The CLI is part of public interface so I think it deserves the same treatment as the API. I don't see an urgent reason to immediately remove pyct from the dependencies. I'm in favor of simply adding deprecation warnings, and removing the whole thing in a few releases.

hoxbro · 2025-10-15T10:42:00Z

datashader/__init__.py

 from . import transfer_functions as tf                   # noqa (API import)
 from . import data_libraries                             # noqa (API import)

+with suppress(ImportError):


This is not the right place it should be in main.py.

maximlt · 2025-10-15T15:13:00Z

pyproject.toml

    'numpy',
    'pandas',
    'param',
-    'pyct',


I don't think pyct should be removed yet from the dependencies.

It's not removed totally. Just moved to optional dependencies.

With these changes if I install datashader and run datashader example it breaks. So that's a breaking change. The goal is to deprecate it so people get a chance to stop using datashader example. Then pyct can be removed entirely from the dependencies.

codecov · 2025-10-15T16:39:55Z

Codecov Report

❌ Patch coverage is 22.22222% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.31%. Comparing base (a5ae17c) to head (a916499).

Files with missing lines	Patch %	Lines
datashader/__main__.py	0.00%	5 Missing ⚠️
datashader/__init__.py	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1462      +/-   ##
==========================================
- Coverage   88.34%   88.31%   -0.03%     
==========================================
  Files          96       96              
  Lines       18932    18936       +4     
==========================================
- Hits        16725    16724       -1     
- Misses       2207     2212       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Azaya89 · 2025-10-15T17:37:26Z

I added a temporary patch to allow the CI to pass pending a new hvsampledata release but otherwise I think it's ready for review now.

hoxbro · 2025-10-16T06:44:34Z

scripts/download_data.py

+    gds.get_path("geoda health")
+
+
+with suppress(ImportError):


hvs.nyc_taxi_remote("download-only") # return None or path

I think we should add something like this in hvsampledata to avoid all of this boilerplate. This will likely also make it so we can move pyarrow back in pixi.toml

What is the goal of this code?

The goal of the script is to pre-download the data, so it doesn't fail in CI tests later on, and to avoid download bars in examples.

The goal of the code was likely to download it, without the latest hvsampledata.

hoxbro · 2025-10-16T06:45:38Z

pyproject.toml

    # 2025-09
    "ignore:Signature .* for <class 'numpy.longdouble'>.*:UserWarning",
+    # 2025-10
+    "ignore:The 'pyct' package bundled as a datashader dependency is deprecated since version 0.19 and will be removed in version 0.20.*" # https://github.com/holoviz/datashader/pull/1462


I think we should remove this filtering. Running datashader in Python (and not CLI) should never show this warning.

replace direct download of nyc_data with hvsampledata

b0f4c04

Azaya89 requested a review from Copilot September 26, 2025 13:26

This comment was marked as outdated.

Sign in to view

Azaya89 self-assigned this Sep 26, 2025

hoxbro reviewed Sep 26, 2025

View reviewed changes

Azaya89 added 3 commits September 26, 2025 18:35

remove pyct completely

015d39a

use pyarrow instead

39ec30e

download data in script and update example notebook

c54c98d

hoxbro changed the title ~~chore: replace direct download of nyc_taxi with hvsampledata~~ chore!: replace direct download of nyc_taxi with hvsampledata Sep 27, 2025

maximlt reviewed Oct 9, 2025

View reviewed changes

Azaya89 added 3 commits October 14, 2025 16:38

Merge branch 'main' into hvsampledata

3ff0ebf

Revert "remove pyct completely"

17d6402

add deprecation warning for pyct

83e277e

hoxbro reviewed Oct 15, 2025

View reviewed changes

Azaya89 added 3 commits October 15, 2025 12:04

temp workaround for CI

ecfbfab

move deprecation warning to main.py

c48447c

second try

0f657bd

maximlt reviewed Oct 15, 2025

View reviewed changes

third try for example notebook

8ba61da

Try GH install of hvsampledata for CI instead

192dd62

Azaya89 marked this pull request as ready for review October 15, 2025 17:32

Azaya89 requested review from Copilot, hoxbro and maximlt October 15, 2025 17:37

This comment was marked as spam.

Sign in to view

holoviz deleted a comment from Copilot AI Oct 15, 2025

hoxbro reviewed Oct 16, 2025

View reviewed changes

remove filtering

a916499

hoxbro added this to the v0.19.0 milestone Nov 13, 2025

	"df = hvs.nyc_taxi(\"pandas\", engine_kwargs={\"columns\": ['dropoff_x', 'dropoff_y']})\n",
	"df = hvs.nyc_taxi_remote(\"pandas\", engine_kwargs={\"columns\": ['dropoff_x', 'dropoff_y']})\n",

Uh oh!

chore!: replace direct download of nyc_taxi with hvsampledata #1462

Are you sure you want to change the base?

chore!: replace direct download of nyc_taxi with hvsampledata #1462

Uh oh!

Conversation

Azaya89 commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

hoxbro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codspeed-hq bot commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #1462 will improve performances by 41.46%

Summary

Benchmarks breakdown

Uh oh!

maximlt left a comment

Choose a reason for hiding this comment

Uh oh!

hoxbro commented Oct 9, 2025

Uh oh!

maximlt commented Oct 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Azaya89 commented Oct 15, 2025

Uh oh!

This comment was marked as spam.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Azaya89 commented Sep 26, 2025 •

edited

Loading

codspeed-hq bot commented Sep 26, 2025 •

edited

Loading

codecov bot commented Oct 15, 2025 •

edited

Loading