probability of improvement

[alwalid-somru]

Hi All,

I was Previously using Version 0.13.2, and in that version I was able to Calculate the probability of improvement when using The Desirability Objective even when i have multiple targets.

And now i am trying to update to Version 0.14. and given the same set up, i am not able to calculate it when there are multiple targets, even when using "as_pre_transformation = True"

Is this behavior no longer supported in v0.14, or am I missing something in the setup?

Here is a couple of google colab docs to demonstrate it:
V 0.13.2
V 0.14

[AdrianSosic]

Hi @alwalid-somru, thank you for reporting the issue 👏🏼 Seems like a check is incorrectly done in the acquisition function builder for the special case of desirability objectives. I have already a hunch of what might be going wrong and will have a look. However, pushing a fix release will probably have to wait until next week, because tomorrow is a bank holiday. I'll ping you here as soon as a branch is ready, then you can at least install from there and don't have to wait until it's fully integrated.

Regardless of the issue, please note that you can extract acquisition values much easier these days. Both Campaign and Recommender classes now have corresponding convenience methods, avoiding the need to manually code any logic.
I've posted a minimal example in the corresponding issue that both demonstrates the simplified logic and replicates the bug.

[AdrianSosic]

@alwalid-somru: I've drafted a fix which is on the fix/acquisition_builder branch. Can you do me a favor an test if this solves your problem, by installing via:

pip install git+https://github.com/emdgroup/baybe.git@fix/acquisition_builder

[alwalid-somru]

@AdrianSosic Yes, that solved my issue! When as_pre_transformation is set to True, I can extract the acquisition values.

However, your example did not have as_pre_transformation set to False, so I just want to confirm: is it correctly failing to extract it in that case?

Also, thank you for the tip about the convenience methods. it simplifies things quite a bit.

Thanks again!

[AdrianSosic]

Glad we found the cause. So let me then prepare the proper hotfix PR and ship it once reviewed 👌🏼

Re as_pre_transformation: Exactly right, evaluating PI (and other single-output acquisition functions) is only possible on a DesirabilityObjective when the flag is set to True. The reason is: these acquisition functions are only defined for one-dimensional distributions, i.e. you also wouldn't be able to evaluate them on a ParetoObjective, for example. The desirability objective is a bit special because its output is either one-dimensional or multi-dimensional, depending on the flag. When the flag is False, each target is described by its own surrogate model, while when the flag is True, the modeling process happens after applying the desirability transform (hence the name: as_pre_transform), resulting in one single quantity to be modeled. Note, however, that the involved architecture (and its capabilities) will be different in both cases. For more details, have look at our user guide

[AdrianSosic]

Hi @alwalid-somru. Just wanted to inform you that the fix has just been integrated into main and will be part of the upcoming release 0.14.2, which is expected for the next days. Took a little longer than expected since the proper way to fix it required more machinery than anticipated, but I'm glad this issue has been identified, so thanks again for reporting! 👏🏼