Support Polars as parameter of ReactiveESM component

[MarcSkovMadsen]

panel==1.5.3

I would like to add support for Polars DataFrame to GraphicWalker.object. Its unclear to me whether this is supported and how. I guess special care is taken about pandas dataframe serialization? Please support it.

import panel as pn
import param

from panel.custom import JSComponent
import polars as pl
import pandas as pd
pn.extension()

class GraphicWalker(JSComponent):

    object = param.ClassSelector(class_=(pd.DataFrame, pl.DataFrame))

    _esm = '''
    export function render({ model, el }) {
      console.log(model.object)
      el.innerHTML = `<h1>Hello World</h1>`
    }
    '''

GraphicWalker(object=pl.DataFrame({"x": [1,2,3]})).servable()

TypeError: the truth value of a DataFrame is ambiguous

Hint: to check if a DataFrame contains any values, use is_empty().

Traceback (most recent call last):
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/io/handlers.py", line 405, in run
    exec(self._code, module.__dict__)
  File "/home/jovyan/repos/private/panel-graphic-walker/script.py", line 20, in <module>
    GraphicWalker(object=pl.DataFrame({"x": [1,2,3]})).servable()
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/viewable.py", line 399, in servable
    self.server_doc(title=title, location=location) # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/viewable.py", line 1006, in server_doc
    model = self.get_root(doc)
            ^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/viewable.py", line 678, in get_root
    root = self._get_model(doc, comm=comm)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/custom.py", line 436, in _get_model
    model = self._bokeh_model(**self._get_properties(doc))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/panel/custom.py", line 390, in _get_properties
    'data': self._data_model(**{p: v for p, v in data_props.items() if p not in ignored}),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/model/data_model.py", line 49, in __init__
    super().__init__(*args, **kwargs)
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/model/model.py", line 119, in __init__
    super().__init__(**kwargs)
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/core/has_props.py", line 304, in __init__
    setattr(self, name, value)
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/core/has_props.py", line 336, in __setattr__
    return super().__setattr__(name, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/core/property/descriptors.py", line 332, in __set__
    self._set(obj, old, value, setter=setter)
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/bokeh/core/property/descriptors.py", line 598, in _set
    if self.property.matches(value, old) and (hint is None):
  File "/home/jovyan/repos/private/panel-graphic-walker/.venv/lib/python3.11/site-packages/polars/dataframe/frame.py", line 1114, in __bool__
    raise TypeError(msg)
TypeError: the truth value of a DataFrame is ambiguous

Hint: to check if a DataFrame contains any values, use `is_empty()`.

[MarcSkovMadsen]

Solution

To support a custom DataFrame like parameter you need to update panel.io.datamodel.PARAM_MAPPING:

import panel as pn
import param

from panel.custom import JSComponent
import polars as pl
import pandas as pd
from panel_gwalker._tabular_data import TabularData
from panel.io.datamodel import PARAM_MAPPING, bp

pn.extension()

_VALID_CLASSES = (
    "<class 'pandas.core.frame.DataFrame'>",
    "<class 'polars.dataframe.frame.DataFrame'>",
)


class TabularData(param.Parameter):
    def _validate(self, val):
        super()._validate(val=val)
        try:
            if str(val.__class__) in _VALID_CLASSES:
                return
        except:
            pass

        msg=f"A value of type '{type(val)}' is not valid"
        raise ValueError(msg)

def _column_datasource_from_polars_df(df):
    df = df.to_pandas()
    return ColumnDataSource._data_from_df(df)

PARAM_MAPPING.update({
    TabularData: lambda p, kwargs: (
        bp.ColumnData(bp.Any, bp.Seq(bp.Any), **kwargs),
        [(bp.PandasDataFrame, _column_datasource_from_polars_df)],
    ),
})

class GraphicWalker(JSComponent):

    object = param.DataFrame()

    _esm = '''
    export function render({ model, el }) {
      console.log(model.object)
      el.innerHTML = JSON.stringify(model.object)
    }
    '''

GraphicWalker(object=pd.DataFrame({"x": [1,2,3]})).servable()

[MarcSkovMadsen]

I can see that bp.PandasDataFrame is defined as below

class PandasDataFrame(Property["DataFrame"]):
    """ Accept Pandas DataFrame values.

    This property only exists to support type validation, e.g. for "accepts"
    clauses. It is not serializable itself, and is not useful to add to
    Bokeh models directly.

    """

    def validate(self, value: Any, detail: bool = True) -> None:
        super().validate(value, detail)

        import pandas as pd
        if isinstance(value, pd.DataFrame):
            return

        msg = "" if not detail else f"expected Pandas DataFrame, got {value!r}"
        raise ValueError(msg)

in https://github.com/bokeh/bokeh/blob/e0ac1c790f768f460ef1b7ea764d353281035915/src/bokeh/core/property/pd.py#L46.

Someday Bokeh should probably support a more general DataFrame property like narwhals.typing.IntoFrameT. See https://narwhals-dev.github.io/narwhals/basics/dataframe/.

Any plans about more general support for dataframes @mattpap?

[mattpap]

There's an ongoing discussion regarding this subject in bokeh/bokeh#13780.

[philippjfr]

I will apply a temporary fix that will convert polars DataFrames to pandas for now.

[MarcSkovMadsen]

I will apply a temporary fix that will convert polars DataFrames to pandas for now.

Would it be an idea to finalize the proof of concept in https://github.com/panel-extensions/panel-graphic-walker/blob/89d6bac66d119d6ad1f75dfe27ec984d18d895ed/src/panel_gwalker/_tabular_data.py#L1 first. To me it looks very close to the right solution.

Then apply in panel?

What is missing is usage of narwhals to enable any tabular/ dataframe like data source. Not just polars.

[philippjfr]

Started in Param: holoviz/param#975