More clarity in autofocus output

[Manusreekumar]

The current output of the package is a list of animals along with their probability score. It would be great to improve the output so as to see just the predicted set of labels based on a certain probability cut off, instead of user trying to read through the decimal points in an attempt to understand the exact label.

[gsganden]

I don't think I agree with removing probability information from the output, especially given that our main user is quite statistically savvy.

[Manusreekumar]

@gsganden I would not want the probability scores to be removed as well. Instead I would like the output to be in a format that would help user to zero in on the labels predicted by the tool. Something like

{ imageLabel : {"human", "dog"}, {list of labels with probability scores} }

or

{list of labels with probability scores sorted in descending order }

[gsganden]

I'd be fine with sorting the outputs.

We could also add a field of predicted labels, but I would want to work with @mfidino to choose appropriate thresholds or an appropriate metric (e.g. an F-beta score) to use to tune the thresholds, and it would not be my top priority right now.

[DigitalPhilosopher]

I would suggest the user should be able to pass a threshold to the endpoint via a POST parameter. That way the user could specify his own threshold but is able to get all results without adding the parameter.

[gsganden]

I'm open to pushback, but I think I'd prefer just to return the probabilities and leave it to the user to apply a threshold on the client side. That's not hard to do, and it keeps the interface simple.

[DigitalPhilosopher]

Maybe an example would be better, to clarify what I meant:

So keeping the normal POST route:

{
    file: beaver.jpeg
}

would return:

{
    "beaver": 0.6,
    "bird": 0.1,
    "cat": 0.1,
    "chipmunk": 0.1,
    ...
}

and only if the user specifies a threshold parameter on the POST like this:

{
    file: beaver.jpeg,
    threshold: 0.5
}

then the endpoint predict returns

{
    "beaver": 0.6,
    ...
}

This would keep the expected behavior of returning predicted probabilities to the correct classes and enables the user to specify his own threshold if needed.

If this is acceptable, I would like to work on this, after the PR #121 is reviewed. Since this PR changed the code that needs to be changed for this task as well.

[gsganden]

What does the ellipsis signify here?

{
    "beaver": 0.6,
    ...
}

Do you mean this?

{
    "beaver": 0.6,
}

[DigitalPhilosopher]

In my understanding the ML algorithm could predict two different animals in one picture, right? Since the probability is calculated for each animal. So if a picture has for example a beaver and a racoon bird in it, the probabilities for both animals surpass the threshold and we want to return both probabilities.

An example could be:

POST {
    file: picture_with_racoon_and_bird.jpeg,
    threshold: 0.7
}

than returns:

{
    "racoon": 0.8,
    "bird": 0.9
}

So the ellipsis signifies, that there could potentially be other animals surpassing the threshold.

[gsganden]

Gotcha. I would rather not implement that filtering on the server side. The raw JSON responses are primarily for machine consumption rather than direct human consumption. (It's an Application PROGRAMMING Interface.) Filtering can be done on the client side as desired.