Skip to content

Commit bc69b38

Browse files
knoepfelknoepfel
committed
Flesh out section on windows
Some tweaks to the resources section
1 parent 3df1493 commit bc69b38

File tree

11 files changed

+137
-29
lines changed

11 files changed

+137
-29
lines changed

‎doc/appendices/requirements.rst‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -300,11 +300,7 @@ Conceptual requirements
300300

301301
The framework shall enable the specification of resources required by the program.
302302

303-
.. todo::
304-
305-
Find a different reference for this than the subsystem design.
306-
307-
- See :numref:`ch_subsystem_design/configuration:Program resource specification`
303+
- See :numref:`ch_conceptual_design/resources:Resources`
308304

309305
.. req:: Specification of user-defined resources
310306
:collapse:

‎doc/ch_conceptual_design/hofs/observers.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Phlex, however, supports observers as physicists rely on the ability to induce s
1616

1717
Unlike filters and predicates, observers (by definition) are allowed to be the most downstream algorithms of the graph.
1818

19-
Operator signature
19+
Operator Signature
2020
^^^^^^^^^^^^^^^^^^
2121

2222
.. table::
@@ -28,7 +28,7 @@ Operator signature
2828
| :math:`f` | :cpp:`void function_name(P1, Pn..., Rm...) [quals];` |
2929
+--------------+------------------------------------------------------+
3030

31-
Registration interface
31+
Registration Interface
3232
^^^^^^^^^^^^^^^^^^^^^^
3333

3434
The below shows how the :cpp:`histogram_hits` operator in :numref:`workflow` would be registered in C++ [#ffilter]_.

‎doc/ch_conceptual_design/hofs/partitioned_folds.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ This "combined" value is then returned by :math:`\textit{sum\_energy}` as the up
9494
The function :math:`\textit{sum\_energy}` is repeatedly invoked to update the accumulator with good-hits data product.
9595
Once all :cpp:`"GoodHits"` data products in `Spill` :math:`j` have been processed by :math:`\textit{sum\_energy}`, the accumulator's value becomes the fold result for that `Spill`.
9696

97-
Operator signatures
97+
Operator Signatures
9898
^^^^^^^^^^^^^^^^^^^
9999

100100
.. table::
@@ -118,7 +118,7 @@ The fold's :cpp:`result_type` must model the created data-product type described
118118
A fold algorithm may also create multiple data products by using a :cpp:`result_type` of :cpp:`std::tuple<T1, ..., Tn>` where each of the types :cpp:`T1, ..., Tn` models a created data-product type.
119119

120120

121-
Registration interface
121+
Registration Interface
122122
^^^^^^^^^^^^^^^^^^^^^^
123123

124124
The :math:`\textit{fold(sum\_energies)}` node in :numref:`workflow` would be represented in C++ as:

‎doc/ch_conceptual_design/hofs/partitioned_unfolds.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ The generator would thus return a pair with an advanced iterator and a :cpp:`"Wa
7575

7676
The choice of the next type :math:`N` thus depends on the use case and is not prescribed by Phlex.
7777

78-
Operator signatures
78+
Operator Signatures
7979
^^^^^^^^^^^^^^^^^^^
8080

8181
.. table::

‎doc/ch_conceptual_design/hofs/predicates.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ It is not until the predicate results are used by the filter that an input seque
1919

2020
Phlex will schedule a predicate HOF for execution only if it is included in a predicate expression (see :numref:`ch_conceptual_design/hofs/filters:Filtering`).
2121

22-
Operator signature
22+
Operator Signature
2323
^^^^^^^^^^^^^^^^^^
2424

2525
.. table::
@@ -32,7 +32,7 @@ Operator signature
3232
+--------------+------------------------------------------------------+
3333

3434

35-
Registration interface
35+
Registration Interface
3636
^^^^^^^^^^^^^^^^^^^^^^
3737

3838
The workflow in :numref:`workflow` demonstrates a use of a predicate in the :math:`filter(high\_energy)` node, where the predicate is :cpp:`high\_energy` that operates on each :cpp:`GoodHits` object.

‎doc/ch_conceptual_design/hofs/transforms.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Specifically, the algorithm :math:`f` is applied to each element of the input se
1818
where :math:`b_i = f\ a_i`.
1919
Note that the index set of the output sequence is the same as the index set of the input sequence.
2020

21-
Operator signature
21+
Operator Signature
2222
^^^^^^^^^^^^^^^^^^
2323

2424
.. table::
@@ -33,7 +33,7 @@ Operator signature
3333
The :cpp:`return_type` must model the created data-product type described in :numref:`ch_conceptual_design/algorithms:Return Types`.
3434
An algorithm may also create multiple data products by returning a :cpp:`std::tuple<T1, ..., Tn>` where each of the types :cpp:`T1, ..., Tn` models a created data-product type.
3535

36-
Registration interface
36+
Registration Interface
3737
^^^^^^^^^^^^^^^^^^^^^^
3838

3939
To illustrate the different ways a transform's algorithm can be registered with Phlex, we use the following classes and functions, which are presumably defined in some experiment libraries.

‎doc/ch_conceptual_design/hofs/windows.rst‎

Lines changed: 111 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,114 @@
22
Windows
33
-------
44

5-
+----------------------------------+---------------------------------------------------+------------------------+
6-
| **Window** | Operators | Output sequence length |
7-
+==================================+===================================================+========================+
8-
| :math:`y = \window{f}{label}\ x` | :math:`f: X \times \textrm{Opt}(X) \rightarrow Y` | :math:`|y| = |x|` |
9-
| +---------------------------------------------------+ |
10-
| | :math:`label: \one \rightarrow L` | |
11-
+----------------------------------+---------------------------------------------------+------------------------+
5+
+---------------------------------------+---------------------------------------------------------+------------------------+
6+
| **Window** | Operators | Output sequence length |
7+
+=======================================+=========================================================+========================+
8+
| :math:`y = \window{f}{adj}{label}\ x` | :math:`f': X \times \opt{X} \rightarrow Y` | :math:`|y| = |x|` |
9+
| +---------------------------------------------------------+ |
10+
| | :math:`adj: \iset{x} \times \iset{x} \rightarrow \bool` | |
11+
| +---------------------------------------------------------+ |
12+
| | :math:`label: \one \rightarrow L` | |
13+
+---------------------------------------+---------------------------------------------------------+------------------------+
14+
15+
One of the unique capabilities of Phlex is to execute an algorithm on data products that belong to adjacent data-product sets (see :numref:`ch_conceptual_design/registration:Data Products from Adjacent Data-Product Sets`).
16+
The workflow in :numref:`workflow` shows a such a node :math:`\textit{window(make\_tracks)}`, which is presented with pairs of :cpp:`"GoodHits"` data products, with each data product in the pair belonging to adjacent `APA`\ s.
17+
It is the user-provided :math:`adj` function which determines whether two data-product sets are adjacent.
18+
19+
For simplicity, imagine that each `APA` identifier (i.e. member of the set :math:`\iset{\text{APA}}`) can be represented as an integer.
20+
A straightforward :math:`adj` implementation might be to group the :cpp:`"GoodHits"` data products from `APA`\ s with consecutive numbers:
21+
22+
.. only:: html
23+
24+
.. math::
25+
26+
\isequence{hs}{\text{APA}} = [\ \rlap{\overbracket{\phantom{hs_1 \ \ hs_2}}^{a}\ \ \overbracket{\phantom{hs_3\ \ hs_4}}^{c}}hs_1, \underbracket{hs_2,\ hs_3}_{b},\ hs_4,\ \dots,\ \underbracket{hs_{n-1},\ hs_n}_{m}\ ]
27+
28+
.. only:: latex
29+
30+
.. math::
31+
32+
\isequence{hs}{\text{APA}} = [\ \rlap{$\overbracket{\phantom{hs_1 \ \ hs_2}}^{a}\ \ \overbracket{\phantom{hs_3\ \ hs_4}}^{c}$}hs_1, \underbracket{hs_2,\ hs_3}_{b},\ hs_4,\ \dots,\ \underbracket{hs_{n-1},\ hs_n}_{m}\ ]
33+
34+
The data products corresponding to windows :math:`a` through :math:`m` are grouped into pairs and presented to the :cpp:`make_tracks` algorithm, which has the signature :math:`\text{Hits} \times \text{Hits} \rightarrow \text{Tracks}`.
35+
There are, at most, :math:`n-1` unique pairs that can be presented to the function :cpp:`make_tracks` such that:
36+
37+
.. math::
38+
39+
\left[ts_i\right]_{i \in \iset{\text{APA}}'} = \left[make\_tracks(hs_i, hs_{i+1})\right]_{i \in \iset{\text{APA}}'} = \window{make\_tracks}{adj}{\text{APA}} \isequence{hs}{\text{APA}}
40+
41+
where the index set :math:`\iset{\text{APA}}'` is :math:`\iset{\text{APA}}` without the last identifier :math:`n` [#flast]_.
42+
In this example, the identifier of the first :math:`hs` object in the pair is used to identify the tracks collection :math:`ts`.
43+
But Phlex does not mandate this choice, and a different data category could be specified by the :math:`label` operator for the data products of the output sequence.
44+
45+
Operator Signatures
46+
^^^^^^^^^^^^^^^^^^^
47+
48+
One limitation of the above formulation is that index sets of the input and output sequences are not the same.
49+
To address this infelicity, the function signature of :cpp:`make_tracks` can be adjusted such that the second argument receives an optional type:
50+
51+
.. math::
52+
53+
make\_tracks': \text{Hits} \times \opt{\text{Hits}} \rightarrow \text{Tracks}
54+
55+
thus permitting symmetry between the input and output data-product sequences:
56+
57+
.. math::
58+
59+
\isequence{ts}{\text{APA}} &= \left[make\_tracks'(hs_i, hs_{i+1})\right]_{i \in \iset{\text{APA}}'} \boldsymbol{+}\ \left[make\_tracks'(hs_n, ())\right] \\
60+
&= \window{make\_tracks'}{adj}{\text{APA}} \isequence{hs}{\text{APA}}
61+
62+
where :math:`\boldsymbol{+}` is the list-concatenation operator, and :math:`()` is the unit element.
63+
Phlex supports the function signature whose second argument is an optional type :math:`\opt{X}`; the prime following the function name (e.g. :math:`f'`) is a reminder that the second argument is an optional type.
64+
65+
.. table::
66+
:widths: 12 88
67+
68+
+---------------+------------------------------------------------------------------------------------+
69+
| **Operator** | **Allowed signature** |
70+
+===============+====================================================================================+
71+
| :math:`f'` | :cpp:`return_type function_name(P1, P2*, Rm...) [quals];` |
72+
+---------------+------------------------------------------------------------------------------------+
73+
| :math:`adj` | :cpp:`bool function_name(<P1 identifier>, <P2 identifier>) [quals];` |
74+
+---------------+------------------------------------------------------------------------------------+
75+
| :math:`label` | *Name of data-set category of output data products* |
76+
+---------------+------------------------------------------------------------------------------------+
77+
78+
The :cpp:`return_type` must model the created data-product type described in :numref:`ch_conceptual_design/algorithms:Return Types`.
79+
The algorithm :math:`f'` may also create multiple data products by returning a :cpp:`std::tuple<T1, ..., Tn>` where each of the types :cpp:`T1, ..., Tn` models a created data-product type.
80+
81+
The second argument :cpp:`P2*` indicates that an optional type is passed to the algorithm.
82+
It is permitted to use resources (i.e. :cpp:`Rm...`) in the function :math:`f'`.
83+
The data-product set identifers of :cpp:`P1` and :cpp:`P2` are used to determine whether two data-products reside in adjacent data-product sets.
84+
85+
Registration Interface
86+
^^^^^^^^^^^^^^^^^^^^^^
87+
88+
The :math:`\textit{window(make\_tracks)}` node in :numref:`workflow` would be represented in C++ as:
89+
90+
.. code:: c++
91+
92+
class hits { ... };
93+
class tracks { ... };
94+
tracks make_tracks(tracks const& ts, tracks const* next_ts) { ... }
95+
bool are_adjacent(phlex::id const& left, phlex::id const& right) { ... }
96+
97+
PHLEX_REGISTER_ALGORITHMS(config)
98+
{
99+
products("GoodTracks") =
100+
window(
101+
"track_maker", // <= Node name for framework
102+
make_tracks, // <= Window algorithm (f')
103+
are_adjacent // <= Adjacency criterion
104+
"APA", // <= Output data-product category
105+
concurrency::unlimited // <= Allowed concurrency
106+
)
107+
.sequence("GoodHits"_in("APA"));
108+
}
109+
110+
Note that the second input parameter for :cpp:`make_tracks` is an optional type.
111+
The type :cpp:`phlex::id` is a putative type that enables the comparison of data-product set identifiers for establishing adjacency.
112+
113+
.. rubric:: Footnotes
114+
115+
.. [#flast] The expression :math:`f(hs_n, hs_{n+1})` is ill-formed as there are only :math:`n` elements in the set :math:`\iset{\text{APA}}`.

‎doc/ch_conceptual_design/resources.rst‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ Neither of these examples contain mutable state.
2121
Resources (unlike data products) may have mutable state accessible to the algorithm (e.g. a histogram instance that could be shared across multiple algorithms).
2222
For resources that are thread-unsafe, the framework ensures that two algorithms are not interacting with the resource at the same time.
2323
The framework is responsible for efficiently scheduling algorithms based, in part, upon the availability of resources :need:`DUNE 50`.
24+
To facilitate efficient scheduling of work, the resources needed by the program are specified via configuration or in the algorithm-registration code :need:`DUNE 47` [#fdetails]_.
2425

2526
Examples of resources include:
2627

@@ -38,12 +39,11 @@ Limited Resources
3839
Some resources are used to indicate that an algorithm requires sole use of some program entity.
3940
One example of such an entity is a thread-unsafe library, where the framework must ensure that only one algorithm is interacting with that library at any time :need:`DUNE 45`, :need:`DUNE 145`.
4041
A second example of a limited resource is a fixed number of GPUs present on a particular platform, where Phlex must ensure that each algorithm requiring the use of a GPU has sole access to the GPU it is running on for the duration of the algorithm's execution.
41-
A third example of a limited resource could be an algorithm's declaration that it requires spawning some number of threads for it execution (rather than using the framework's task-based execution model).
42+
A third example of a limited resource could be an algorithm's declaration that it requires spawning some number of threads for its execution (rather than using the framework's task-based execution model).
4243
Such an algorithm could declare the need for the reservation of some number of threads by requiing that number of thread resources :need:`DUNE 152`.
4344
The framework would then ensure that only as many threads as the configuration has provided can be used by algoirthms at any one time.
4445
These resources are called *limited resources*, because the framework is responsible for limiting the access to the resource to one algorithm at a time.
4546

46-
4747
An algorithm to be used by Phlex indicates that it requires a limited resource by requiring an argument that denotes such a resource.
4848

4949
GPUs
@@ -64,7 +64,7 @@ To ensure reproducible data and to ensure thread-safe access to stateful random-
6464

6565
DUNE has similar requirements on reproducibility of random numbers in a concurrent context :need:`DUNE 36`.
6666
However, instead of working around the limitations of stateful random-number engines, Phlex supports a random-number generation technique specifically designed to reproduce random-numbers in a concurrent program.
67-
Counter-based random number generators (CBRNGs) [Wiki-CBRNG]_ each contain one internal counter.
67+
Counter-based random number generators (CBRNGs) [Wiki-CBRNG]_ provide such capabilities, and Phlex will provide an interface algorithm authors can use to take advantage of them.
6868

6969
User-defined Resources
7070
----------------------
@@ -73,6 +73,10 @@ While Phlex will provide some commonly-used types to represent resources, it wil
7373
Such resource types have no dependency on Phlex, so that a user algorithm employing such a resource does not thereby incur any dependency on the framework.
7474

7575

76+
.. rubric:: Footnotes
77+
78+
.. [#fdetails] Details for specifying program resources are described in :numref:`ch_subsystem_design/configuration:Program resource specification`.
79+
7680
.. only:: html
7781

7882
.. rubric:: References

‎doc/ch_preliminaries/functional_programming.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,9 @@ Phlex will likely support other higher order functions as well.
131131
| | +-------------------------------------------------------------+ |
132132
| | | :math:`label: \one \rightarrow L` | |
133133
+-----------------------------------------------------------------------------------+----------------------------------------+-------------------------------------------------------------+---------------------------+
134-
| :ref:`Window <ch_conceptual_design/hofs/windows:Windows>` | :math:`y = \window{f}{label}\ x` | :math:`f: X \times \opt{X} \rightarrow Y` | :math:`|y| = |x|` |
134+
| :ref:`Window <ch_conceptual_design/hofs/windows:Windows>` | :math:`y = \window{f}{adj}{label}\ x` | :math:`f: X \times \opt{X} \rightarrow Y` | :math:`|y| = |x|` |
135+
| | +-------------------------------------------------------------+ |
136+
| | | :math:`adj: \iset{x} \times \iset{x} \rightarrow \bool` | |
135137
| | +-------------------------------------------------------------+ |
136138
| | | :math:`label: \one \rightarrow L` | |
137139
+-----------------------------------------------------------------------------------+----------------------------------------+-------------------------------------------------------------+---------------------------+

‎doc/ch_preliminaries/types.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ The above functions are thus represented in function notation as:
5151
This notation will be used as we discuss the operators required by Phlex's higher-order functions.
5252

5353
The single element of the set :math:`\one` can also be used to represent the value :cpp:`nullptr` for C++ pointers (see :numref:`ch_preliminaries/types:Representing optional types`).
54+
When necessary we will refer to that single element as the *unit element*, or simply the open-closed parentheses :cpp:`()`.
5455

5556
---------------------------
5657
Representing optional types

0 commit comments

Comments
 (0)