Improvements & Additional Features
This is a collection of idea for improvements and additional features. Feel free to contribute.
Currently, there is no load balancing. You can add more than one transcode slave, but currently only one of them is used. However, real-time load-based balancing has been added in the development version and is currently being tested. Here is a screenshot showing a PMS master node (top) and two transcoder slave nodes--one with a single CPU core and the other with a dual code CPU. All three nodes are running in a VM. There are two transcode (encode) jobs running which have automatically been balanced across the two transcode slave nodes. Notice the load difference between the master and the two slaves.
Other possible load balancing options are discussed below.
As transcode requests come in, we simply pick the next slave from the list. This method is trivial to implement but not necessarily the most efficient. Consider that not all transcode requests are the same--some are direct copy (change of container), which are relatively cheap, whereas some requests involve re-encoding, which is expensive.
If each host has a maximum number of "transcode slots" available, we can send transcode jobs to the same slave until it has been fully saturated with work before moving on to the next host. This actually suffers from similar limitations as the round-robin approach.
Before deciding which host to delegate a transcode request to, we first check the current (and historical) load for slave and pick the one with the lowest load. This should lead to better utilization of each transcode slave.
Each transcode request includes a path to the file that is to be transcoded. Technically, this is done via the -i "/path/to/video.mkv" flag sent to the transcoder. This requires that each media path on the master be exported via a network-share to each slave. This results in increased complexity of transcode slave configurations. If a new path is added to the master then every slave must also be updated to have access to that directory.
One way to get around this issue is to remap the file path to a PMS HTTP URL. We can easily do this by parsing the -progressurl flag sent to the transcoder, extract the session ID and then query the master via the /status/sessions URL where we can get the HTTP URL to the file.
For example, consider the flag passed to the transcoder
-progressurl http://127.0.0.1:32400/video/:/transcode/session/3yk96eap8cp/progress
The session ID is 3yk96eap8cp. Before delegating a transcode request to a slave, on the master we can query http://127.0.0.1:32400/status/sessions, parse the XML, looking for the session ID 3yk96eap8cp, and get a URL like
http://127.0.0.1:32400/library/parts/18392/file.mkv
We can then swap the input from
-i "/path/to/video.mkv"
to
-i "http://127.0.0.1:32400/library/parts/18392/file.mkv"
The requested file can then be streamed directly from the master to the slave without having to deal with network shares for the media files.