what 'kind' of func_factory to use for multiple argument grouped non-aggregating function?

[ftobin]

Consider I'd like to create a function that just multiplies two column inputs.
Which 'kind' is appropriate? apply, apply_df, and transform all produce errors.

import datar.datasets
mtcars = datar.datasets.mtcars

@datar.core.factory.func_factory({"x", "w"}, kind="transform")
def mymult(x, w):  # Ideally I could say w=1
    return x*w

mtcars >> group_by(f.cyl) >> mutate(x = mymult(f.disp, f.hp))

...
ValueError: Incompatible value to recycle.  # with apply_df
ValueError: operands could not be broadcast together with shapes (7,) (3,2)   # with transform or apply

Edit: put the import statements in and changed the column inputs it's different than the group_by col.

[pwwang]

Sorry for the late response. I was off for a couple of days.

This should be apply_df for sure. So the solution will be:

from datar import f
from datar.core.factory import func_factory
from datar.core.tibble import TibbleGrouped
from datar.dplyr import group_by, mutate
from datar.datasets import mtcars

# register the generic function
mymult = func_factory({"x", "w"}, kind="apply_df", func=lambda x, w: ...)

# .register() provides post hook
# The index of the result is problematic for mutate()
@mymult.register(TibbleGrouped, post=lambda out, x, w: out.reset_index(drop=True))
def mymult(x, w):  # Ideally I could say w=1
    return x * w

mtcars >> group_by(f.cyl) >> mutate(x = mymult(f.disp, f.hp))

In the next major release, the confusion should be eliminated. Kinds agg and transform will be only applied on the first argument and return the expected results (agg returns an aggregated value and transform the same dimension as input). With kinds agg and transform, recycling other arguments to the first arguments will not be allowed. If you want to do so, apply is your choice. (apply_df won't exist anymore).

Maybe we should separate kinds agg, transform and apply. The idea is that agg and transform are for optimized functions (numpy ufuncs, or pandas optimized functions, for example, SeriesGroupBy.agg('mean')), otherwise, there is no need to distinguish them. apply should take care of all other cases.

[ftobin]

I appreciate the response! It seems like the this part of the interface is still fairly unstable. Writing vectorized multiple-argument functions that are group-unaware was commonplace for me in tidyverse-land, and I'd love to be able to get the point where I can get to that point with datar with ease!

If defaulted arguments are supported (like in R) that would be extra special :-)

[pwwang]

If defaulted arguments are supported

I suppose you mean:

def mymult(x, w=1):
  ...

If so, this should be already supported, but w is broadcasted into the same dimension of x. Say if x is a Series of length 3, then w, inside the function, should be Series([1, 1, 1]).

Yes, this series of API is still on its way to perfection. Your input is important!

[ftobin]

Yes, that's what I meant, thank you! I was trying to put it in the post hook which didn't work.