Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions docs/source/how_to/how_to_benchmarking.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,18 @@
"cell_type": "markdown",
"id": "10",
"metadata": {},
"source": [
":::{note}\n",
"\n",
"For details on using other plotting backends, see [How to change the plotting backend](how_to_change_plotting_backend.ipynb).\n",
"\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "11",
"metadata": {},
"source": [
"The x axis shows runtime per problem. The y axis shows the share of problems each algorithm solved within that runtime. Thus, higher and further to the left values are desirable. Higher means more problems were solved and further to the left means, the algorithm found the solutions earlier. \n",
"\n",
Expand All @@ -150,7 +162,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "11",
"id": "12",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -167,7 +179,7 @@
},
{
"cell_type": "markdown",
"id": "12",
"id": "13",
"metadata": {},
"source": [
"## 4b. Convergence plots\n",
Expand All @@ -178,7 +190,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "13",
"id": "14",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -194,7 +206,7 @@
},
{
"cell_type": "markdown",
"id": "14",
"id": "15",
"metadata": {},
"source": [
"The further to the left and the lower the curve of an algorithm, the better that algorithm performed.\n",
Expand All @@ -205,7 +217,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "15",
"id": "16",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -223,7 +235,7 @@
},
{
"cell_type": "markdown",
"id": "16",
"id": "17",
"metadata": {},
"source": [
"## 5a. Convergence report\n",
Expand All @@ -235,7 +247,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "17",
"id": "18",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -251,7 +263,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "18",
"id": "19",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -260,18 +272,18 @@
},
{
"cell_type": "markdown",
"id": "19",
"id": "20",
"metadata": {},
"source": [
"## 5b. Rank report\n",
"## 5b. Rank report\n",
"\n",
"The **Rank Report** shows the ranks of the algorithms for each problem; where 0 means the algorithm was the fastest on a given benchmark problem, 1 means it was the second fastest and so on. If an algorithm did not converge on a problem, the value is \"failed\". If an algorithm did encounter an error during optimization, the value is \"error\"."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "20",
"id": "21",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -288,7 +300,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "21",
"id": "22",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -297,18 +309,18 @@
},
{
"cell_type": "markdown",
"id": "22",
"id": "23",
"metadata": {},
"source": [
"## 5b. Traceback report\n",
"## 5b. Traceback report\n",
"\n",
"The **Traceback Report** shows the tracebacks returned by the optimizers if they encountered an error during optimization. The resulting ```pd.DataFrame``` is empty if none of the optimizers terminated with an error, as in the example below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23",
"id": "24",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -318,7 +330,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "24",
"id": "25",
"metadata": {},
"outputs": [],
"source": [
Expand Down
69 changes: 53 additions & 16 deletions src/optimagic/visualization/backends.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ def __call__(
height: int | None,
width: int | None,
legend_properties: dict[str, Any] | None,
margin_properties: dict[str, Any] | None,
horizontal_line: float | None,
) -> Any: ...


Expand All @@ -36,12 +38,32 @@ def _line_plot_plotly(
height: int | None,
width: int | None,
legend_properties: dict[str, Any] | None,
margin_properties: dict[str, Any] | None,
horizontal_line: float | None,
) -> go.Figure:
if template is None:
template = "simple_white"

fig = go.Figure()

fig.update_layout(
title=title,
xaxis_title=xlabel.format(linebreak="<br>") if xlabel else None,
yaxis_title=ylabel,
template=template,
height=height,
width=width,
legend=legend_properties,
margin=margin_properties,
)

if horizontal_line is not None:
fig.add_hline(
y=horizontal_line,
line_width=fig.layout.yaxis.linewidth or 1,
opacity=1.0,
)

for line in lines:
trace = go.Scatter(
x=line.x,
Expand All @@ -53,18 +75,6 @@ def _line_plot_plotly(
)
fig.add_trace(trace)

fig.update_layout(
title=title,
xaxis_title=xlabel,
yaxis_title=ylabel,
template=template,
height=height,
width=width,
)

if legend_properties:
fig.update_layout(legend=legend_properties)

return fig


Expand All @@ -78,6 +88,8 @@ def _line_plot_matplotlib(
height: int | None,
width: int | None,
legend_properties: dict[str, Any] | None,
margin_properties: dict[str, Any] | None,
horizontal_line: float | None,
) -> "plt.Axes":
import matplotlib.pyplot as plt

Expand All @@ -93,7 +105,17 @@ def _line_plot_matplotlib(
template = "default"

with plt.style.context(template):
fig, ax = plt.subplots(figsize=(width, height) if width and height else None)
px = 1 / plt.rcParams["figure.dpi"] # pixel in inches
fig, ax = plt.subplots(
figsize=(width * px, height * px) if width and height else None
)

if horizontal_line is not None:
ax.axhline(
y=horizontal_line,
color=ax.spines["left"].get_edgecolor() or "gray",
linewidth=ax.spines["left"].get_linewidth() or 1.0,
)

for line in lines:
ax.plot(
Expand All @@ -103,9 +125,17 @@ def _line_plot_matplotlib(
color=line.color,
)

ax.set(title=title, xlabel=xlabel, ylabel=ylabel)
if legend_properties:
ax.legend(**legend_properties)
ax.set(
title=title,
xlabel=xlabel.format(linebreak="\n") if xlabel else None,
ylabel=ylabel,
)

if legend_properties is None:
legend_properties = {}
ax.legend(**legend_properties)

fig.tight_layout()

return ax

Expand All @@ -129,6 +159,8 @@ def line_plot(
height: int | None = None,
width: int | None = None,
legend_properties: dict[str, Any] | None = None,
margin_properties: dict[str, Any] | None = None,
horizontal_line: float | None = None,
) -> Any:
"""Create a line plot corresponding to the specified backend.

Expand All @@ -144,6 +176,9 @@ def line_plot(
height: Height of the plot (in pixels).
width: Width of the plot (in pixels).
legend_properties: Backend-specific properties for the legend.
margin_properties: Backend-specific properties for the plot margins.
horizontal_line: If provided, a horizontal line is drawn at the specified
y-value.

Returns:
A figure object corresponding to the specified backend.
Expand Down Expand Up @@ -178,6 +213,8 @@ def line_plot(
height=height,
width=width,
legend_properties=legend_properties,
margin_properties=margin_properties,
horizontal_line=horizontal_line,
)

return fig
Loading
Loading