Interest in Additional LIbrary Features

[cwfitzgerald]

There are a couple of features that I am going to be developing for my personal project on a fork of this library, and I want to know if you would be interested in me upstreaming any/all of the features I'm planning on making. I'm not quite sure what you want the scope of this project to be. All of these would add on to the functionality, not replace it.

• A type safe interface built on std::function.
• A shared mutex.
• A separate threadpool for IO blocking operations with api to call it.
• Optional instrumentation of the fiber pool.

[martty]

Great!

I have already started working on a type safe interface (eg. lambda-as-a-task). I'll put it up in a PR by the end of the week. (I also made exploratory progress on coroutines and futures).

[cwfitzgerald]

@martty This is meant as an internal implementation, so it isn't fully polished, but I have a working interface here: https://github.com/BVE-Reborn/fiber-tasking-library/tree/type-safe-interface. If you have a better idea for the interface, I might rewrite this perfect forwarding wrapper on top of your interface.

[RichieSams]

That sounds great!

For the blocking IO threadpool, is that like in the GDC talk, where they had a few threads to handle blocking IO, so-as to not tie up the main workers?

What kind of instrumentation are you thinking?

[cwfitzgerald]

I feel bad for the poor souls watching this repo 😛

Yeah my plans are similar to the talk. I would have a pool of say... thread_count * 4 (by default) threads to do blocking IO in. There would be a call similar to my thread-safe-interface which would send off the data to be processed and have it wait on a counter that would be decremented by the worker thread.

The instrumentation would be similar to the talk as well. Not sure exactly what it would entail, but information about timings, task dependency stacks etc. I haven't really thought about it at this point.

[RichieSams]

That sounds good to me.

For the instrumentation, it would be really cool to have something like Remotery / Microprofile that understood fibers and task waiting / resuming. Perhaps we could add optional callback registering? So user / third party code would be notified when tasks are added, suspended, resumed, etc.

[cwfitzgerald]

Seems feasible. I'll take a look at those librarys and see what I can do.

[martty]

@cwfitzgerald Please check #69 for a preliminary implementation of the type safe interface.

[vblanco20-1]

@RichieSams ive used remotery a bit with this lib in my project https://github.com/vblanco20-1/Rasterizer . There are huge flaws with remotery, as it can "choke" really, really hard. Even like that, i just added the logging inside the task function. It ends up looking like this, almost perfect scaling!!. (one frame of the rasterizer)

To capture this image, i had to add a throttle to my rendering of 1 second beetween frames (rendering takes 20 ms), becouse if i let it go at full speed, the sheer amount of events would destroy Remotery internal memory.
Another issue with remotery is that this library completely breaks it on the "wait for counter" function, due to the fiber switching.

What you comment with the callback sounds great maybe i could help with that. It could work great if you can just send the "name" of the task in the AddTasks/RunTasks function.
taskScheduler->AddTasks(numTasks, tasks, &taskEnd->counter, "rasterize_tile");