Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
358979e
changed a few names and added space for readability, otherwise nothin…
ElieOaks Feb 1, 2019
50288de
Added newlines for readability, changed some variable names
ulfsigvardsson Feb 3, 2019
53910b6
Lifted method calls out of loop conditions to minimize overhead
ulfsigvardsson Feb 5, 2019
1b5be83
Added a script to generate a list of all probes available in a progra…
pkrll Feb 13, 2019
2b7f341
Added folder
pkrll Feb 14, 2019
545d0d3
Added dtrace script with all probes
pkrll Feb 14, 2019
b568200
Added quiet option
pkrll Feb 14, 2019
b3ca0e1
counting calls
ElieOaks Feb 20, 2019
73c3e75
added future block information
ElieOaks Feb 24, 2019
3bb301c
Merge branch 'probing-tools' of https://github.com/pkrll/encore into …
ElieOaks Feb 24, 2019
8a72325
massa saker om futures - ingen chaining än dock
ElieOaks Feb 24, 2019
ee5bee6
chaining
ElieOaks Feb 25, 2019
84ef51d
actor steal and scheduling count
ulfsigvardsson Feb 25, 2019
4931c66
Merge branch 'probing-tools' of https://github.com/pkrll/encore into ulf
ulfsigvardsson Feb 25, 2019
ffe88ac
prettified the output
ulfsigvardsson Feb 25, 2019
b8fae71
added diagnostics and cpu structs
ulfsigvardsson Mar 1, 2019
1ba10fa
lite mer på future delen
ElieOaks Mar 2, 2019
829d5d1
probing core jumps, steals and time spent in functions
ulfsigvardsson Mar 5, 2019
591df58
tracing core jumps, steals and time spent in functions
ulfsigvardsson Mar 5, 2019
96f2587
Merge branch 'ulf' of https://github.com/pkrll/encore into ulf
ulfsigvardsson Mar 5, 2019
bb833f4
Changed 'Main' into '' in entry/return probes
ulfsigvardsson Mar 5, 2019
4b788e3
Merge pull request #2 from pkrll/ulf
pkrll Mar 6, 2019
9dbe02b
dtrade script now gathers info on futures
pkrll Mar 6, 2019
d46b1d3
anpassar formateringen efter dtrace vertyget
ElieOaks Mar 6, 2019
4f194f4
compatible scheduler inforamtion as with dtrace
ElieOaks Mar 6, 2019
40e40d7
Changes:
pkrll Mar 6, 2019
8a22369
Changed approach to time method calls + changed timestamp to vtimestamp
pkrll Mar 6, 2019
a51e45c
Fixed so that only information on methods called by the user is gathered
pkrll Mar 6, 2019
99a3c8a
Added future id to future-get list, removed one of the future block d…
pkrll Mar 7, 2019
98d425c
Merge pull request #3 from pkrll/dtrace-futures
ulfsigvardsson Mar 7, 2019
ac5932c
Removed unnecessary struct fields
pkrll Mar 7, 2019
b12be19
Added XML-output
pkrll Mar 8, 2019
af290dc
Fixed tag
pkrll Mar 8, 2019
439103f
Started on parser
pkrll Mar 8, 2019
ad19d37
Added missing id-tag to actor
pkrll Mar 9, 2019
3a08d9a
Fixed some parsing issues
pkrll Mar 9, 2019
eac9ba8
Renamed tag work-steal-success-count
pkrll Mar 9, 2019
c351d51
Implemented parsing of tags work-steal-success and work-steal-success…
pkrll Mar 9, 2019
69681a0
Implemented parsing of tags work-steal-failure and work-steal-failure…
pkrll Mar 9, 2019
2e8c37d
Implemented actor-stolen + changed all counts and durations to ints
pkrll Mar 9, 2019
b0764aa
Implemented parsing for tag methods
pkrll Mar 9, 2019
145991d
Removed debug code
pkrll Mar 9, 2019
0180392
Changed tags for counts
pkrll Mar 9, 2019
dbcc605
Implemented parsing of count tag
pkrll Mar 9, 2019
d9d7192
Implemented simple outputting of parsed result + added new class for …
pkrll Mar 9, 2019
621d449
Added documentation
pkrll Mar 9, 2019
388c333
Added actorid to method class
pkrll Mar 9, 2019
9f5973e
Info gathered on methods are now outputted as well
pkrll Mar 9, 2019
1500bc3
Added option to output either XML or human readable format + fixed is…
pkrll Mar 9, 2019
2b3a224
Changed FutureGets class to contain actor id and future id, instead o…
pkrll Mar 9, 2019
c3bb5bd
Future gets information now outputted
pkrll Mar 9, 2019
5790580
Added some documentation
pkrll Mar 9, 2019
367a5c8
Changed how future blocks are displayed
pkrll Mar 10, 2019
676f4a4
Merge pull request #4 from pkrll/dtrace-futures
ulfsigvardsson Mar 11, 2019
d25a60a
Merge pull request #5 from pkrll/dtrace-parser
ulfsigvardsson Mar 11, 2019
e9b97eb
put the two different stp together: futures and schedulers
ElieOaks Mar 13, 2019
bbc7abd
Changed structure of XML output and fixed issue with incorrect timest…
pkrll Mar 14, 2019
c012dfa
Added number of times an actor calls get on the same future
pkrll Mar 14, 2019
333a5d2
Implemented parsing of future chainings, added count to future gets a…
pkrll Mar 14, 2019
e56a1da
Added documentation
pkrll Mar 14, 2019
9590237
Removed catch-all probe
pkrll Mar 14, 2019
db5d556
an XML version and a none XML version, XML version is almost complete
ElieOaks Mar 21, 2019
a0614cf
XML version is working
ElieOaks Mar 21, 2019
a6943be
sample outputs for consideration
ElieOaks Mar 21, 2019
ebd2ff4
Merge pull request #6 from pkrll/dtrace-parser-futurechain
ElieOaks Mar 22, 2019
e19c56d
fixed a bug in future-block-lifetime
ElieOaks Mar 22, 2019
b1fe5a3
fixed a bug in block-lifetime
ElieOaks Mar 22, 2019
0649af1
Moved parser
pkrll Mar 22, 2019
8c7af4e
nothing points to pony_ctx_t anymore, but always to actor
ElieOaks Mar 22, 2019
f40b9da
Delete twoWhatTobiasAskedFor.stp
ElieOaks Mar 22, 2019
3bc199f
Delete output.txt
ElieOaks Mar 22, 2019
236b583
Delete output.XML
ElieOaks Mar 22, 2019
92838f0
Delete countingCalls.stp
ElieOaks Mar 22, 2019
dc40119
Delete .gitkeep
ElieOaks Mar 22, 2019
c51df5e
changed some names, and changed some tings that were still different …
ElieOaks Mar 22, 2019
e6659d0
added output files for comparison
ElieOaks Mar 22, 2019
c9250b4
some outputs to pass through the parser
ElieOaks Mar 22, 2019
744c786
Merge pull request #8 from pkrll/systemTap_joy
pkrll Mar 24, 2019
2337d7e
Fixed issue with empty tags
pkrll Mar 25, 2019
e8b4343
Removed some files, cleaned up dtrace.d
pkrll Mar 25, 2019
b859c3a
Merge pull request #9 from pkrll/cleanup
ElieOaks Mar 25, 2019
0155a4d
Creates a README
ElieOaks Mar 26, 2019
91970af
Update README
pkrll Mar 26, 2019
da377bf
Update README
pkrll Mar 26, 2019
c38c827
Update README
pkrll Mar 26, 2019
1c2150e
Update README
pkrll Mar 26, 2019
b773acf
Update README
pkrll Mar 26, 2019
da4ac15
Update README
pkrll Mar 26, 2019
25af44f
Merge pull request #10 from pkrll/misc/README
ElieOaks Mar 26, 2019
68d3c48
Merge remote-tracking branch 'upstream/development' into development
pkrll Mar 26, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 172 additions & 0 deletions probing-tools/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Probing toolset for Encore

These probing tools allows the user to gain insight into how many actors and futures are created, how work stealing occurs, succeeds or fails, and how many messages are sent and more, in a running Encore program.

## Table of contents

* [Installation](#installation)
* [Prerequisites](#prerequisites)
* [Setup](#setup)
* [MacOS 10.11+](#macos-1011)
* [Parser](#parser)
* [Usage](#usage)
* [Dtrace](#dtrace)
* [SystemTap](#systemtap)
* [Parsing](#parsing)
* [Further work](#further-work)
* [Authors](#authors)

## Installation

The toolset works only on MacOS, BSD or Linux systems.

### Prerequisites

* DTrace / SystemTap
* NodeJS (**above version 8**)

### Setup

In order for the probing tools to function, the Encore compiler must first be built with DTrace probes enabled, which can be done with the following command:

```bash
$ make config=release use=dtrace
```

#### MacOS 10.11+

Systems running MacOS 10.11 or above, must first disable **System Integrity Protection** (SIP) for DTrace to work. This can be done by booting into recovery mode, and running the following commands in the Terminal:

```bash
$ csrutil clear
$ csrutil enable --without dtrace
```

If it still does not work, you may disable SIP entirely, with the command ``csrutil disable`` (but do this at your own risk).

#### Parser

To setup the parser, just run ``npm install`` in the subdirectory ``probing-tools/parser``.

## Usage

Although the two scripts give the same output, depending on whether you have DTrace or SystemTap on your system, the usage will differ somewhat.



### DTrace

To probe a program using DTrace, run the following command:

```
$ dtrace -o ../output.xml -Cs dtrace.d -c ./[target binary] XML
```

This assumes that the working directory is ``probing-tools/dtrace``, and will produce an XML-file in the parent directory with the output from the DTrace script.

**Note:** If you want the output to be in a human readable format, you can ommit the ``-o`` flag, and replace the ``XML`` argument with an arbitrary argument (you can not leave it empty, though).

**Note:** If you need to pass an argument to the Encore program, you may do that by enclosing the target binary and the arguments in quotation marks, like so:

```
$ dtrace -o ../output.xml -Cs dtrace.d -c "./[target binary] someArgument" XML
```

### SystemTap

To probe the program using SystemTap, run the following command:

```
$ stap systemTapXML.stp -c [target binary] -o ../output.XML
```

This assumes that the working directory is ``probing-tools/systemtap``, and will produce an XML-file in the parent directory with the output from the SystemTap script.

**Note:** If you want the output to be in a human readable format, you may use the script ``systemTap.stp`` instead.

### Parsing

A simple parser is available to parse the XML output produced by the DTrace / SystemTap scripts.

To parse the XML output, simply run the following command:

```
node parser/index.js /path/to/output.xml
```

This assumes that the working directory ``probing-tools``, and will give an organized overview of the data gathered:

```bash
┌───────────────────────┬────────┐
│ (index) │ Values │
├───────────────────────┼────────┤
│ core-switches │ 2 │
│ work-steal-failure │ 2 │
│ work-steal-successful │ 3 │
│ work-steal-attempt │ 5 │
│ actor-msg-send │ 27 │
└───────────────────────┴────────┘
--------------------------- ACTORS ---------------------------
┌────────────┬──────────────┬──────────────┬──────────────────────┬─────────────────────┐
│ (index) │ id │ numberOfGets │ numberOfTimesBlocked │ numberOfTimesStolen │
├────────────┼──────────────┼──────────────┼──────────────────────┼─────────────────────┤
│ 4430339584 │ '4430339584' │ 0 │ 0 │ 1 │
│ 4933481472 │ '4933481472' │ 0 │ 0 │ 2 │
└────────────┴──────────────┴──────────────┴──────────────────────┴─────────────────────┘
--------------------------- Schedulers ---------------------------
┌────────────┬──────────────┬──────────────────┬──────────────┐
│ (index) │ id │ successfulSteals │ failedSteals │
├────────────┼──────────────┼──────────────────┼──────────────┤
│ 4430338432 │ '4430338432' │ 1 │ 1 │
│ 4430338048 │ '4430338048' │ 2 │ 1 │
└────────────┴──────────────┴──────────────────┴──────────────┘
--------------------------- Work steal success ---------------------------
┌─────────┬──────────────┬──────────────┬───────┐
│ (index) │ scheduler │ victim │ count │
├─────────┼──────────────┼──────────────┼───────┤
│ 0 │ '4430338432' │ '4430338048' │ 1 │
│ 1 │ '4430338048' │ '4430338432' │ 2 │
└─────────┴──────────────┴──────────────┴───────┘
--------------------------- Work steal failure ---------------------------
┌─────────┬──────────────┬──────────────┬───────┐
│ (index) │ scheduler │ victim │ count │
├─────────┼──────────────┼──────────────┼───────┤
│ 0 │ '4430338048' │ '0' │ 1 │
│ 1 │ '4430338432' │ '4430338048' │ 1 │
└─────────┴──────────────┴──────────────┴───────┘
```

## Further work

While the parser allows for an automated analysis of an Encore program, currently there is no script that does anything fancy with the data.

The data will be collected and accessible through a few properties of the parser, as seen below:

```js
// index.js
const parser = new Parser(nodes);
parser.start(); // Start parsing

const counts = parser.counts; // Holds a bunch of enumerations
const futures = parser.futures; // Holds info on all futures created
const futureGets = parser.futureGets; // Holds info on all future gets
const futureBlocks = parser.blocks; // Holds info on all blocks made
const actors = parser.actors; // Holds info on all actors
const schedulers = parser.schedulers; // Holds info on all schedulers
const workStealSuccess = parser.workStealSuccess; // Holds info on all successful work steals
const workStealFailure = parser.workStealFailure; // Holds info on all failed work steals
```

For example, to get the ratio between number of messages sent and number of futures created, you can do the following:

```js
const counts = parser.counts;
const ratio = parser.counts['actor-msg-send'] / parser.counts['future-create'];

console.log(ratio);
```


## Authors

The toolset was created by [Ardalan Samimi](https://github.com/pkrll), [Ulf Sigvardsson](https://github.com/ulfsigvardsson) and [Joy van den Eijkhof](https://github.com/elieoaks).
Loading