Workaround for issue #711 - relative paths with Windows path separators cause TemplateNotFound error

[ernstki]

I guess this is the first thread about Windows in the Jinja GitHub discussions, weird!

This pertains to issue #711, which was closed by a project member in 2017. It seems unlikely to be resolved without a PR from a volunteer. Here's a workaround that (crudely) avoids the jinja2.exceptions.TemplateNotFound you'd otherwise get on Windows platforms if you feed a path generated by os.path.join to a Jinja FileSystemLoader.

Background

I'm using Jinja's FileSystemLoader for a project as a quick-and-dirty static site generator, where I'm iterating over the entire directory structure under the template directory and reproducing it with the processed templates in a parallel, destination directory. This project must work cross-platform.

One issue that came up is, as stated in the docs “/” is used as the path separator, even on Windows, which creates difficulties if you're using os.path.join or os.path.relpath.

Although modern Windows seems to accept either forward or backslashes, the os.path utilities insert the platform's native path separator: "\". However, FileSystemLoader.get_source calls split_template_path which only splits on /s, thus it crashes with a traceback similar to the following:

  File "C:\Users\me\src\myproject\lib\templates.py", line 84, in process_templates
    t = env.get_template(os.path.join('templates', template))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\src\myproject\venv\Lib\site-packages\jinja2\environment.py", line 1016, in get_template
    return self._load_template(name, globals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\src\myproject\venv\Lib\site-packages\jinja2\environment.py", line 975, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\src\myproject\venv\Lib\site-packages\jinja2\loaders.py", line 126, in load
    source, filename, uptodate = self.get_source(environment, name)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\src\myproject\venv\Lib\site-packages\jinja2\loaders.py", line 197, in get_source
    pieces = split_template_path(template)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\src\myproject\venv\Lib\site-packages\jinja2\loaders.py", line 36, in split_template_path
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: templates\data/README.txt

I understand why—there are ModuleLoaders and PackageLoaders, and someone had to pick a separator¹—but in this particular context, the behavior defies the principle of least astonishment. Importing posixpath explicitly as @ThiefMaster mentioned here is one possible workaround, with the downside being that that requires modifying code far removed from the actual template processing.

Workaround

Here's a crude workaround that reconciles the path separators right at the nexus. A more robust solution might split and then re-join using posixpath.join, but this was sufficient for my purposes, so I didn't think it through.

from jinja import Environment, FileSystemLoader

TEMPLATE_DIR = 'templates'

class BackslashFriendlyFileSystemLoader(FileSystemLoader):
    """
    Work around pallets/jinja issue #711, which pertains only to Windows.

    Jinja "template paths are not necessarily filesystem paths and […] use
    forward slashes," so just do a crude subsitution and hope for the best

    ref. https://github.com/pallets/jinja/issues/711#issuecomment-300070379
    and  https://github.com/pallets/jinja/blob/3.1.6/src/jinja2/loaders.py#L194
    """
    def get_source(self, environment, template):
        template = template.replace('\\', '/')
        return super().get_source(environment, template)

env = Environment(loader=BackSlashFriendlyFileSystemLoader(TEMPLATE_DIR))

Hope that helps somebody.

Footnotes

1. Also to avoid "drive:" or UNC segments breaking out of the search directory as mentioned here ↩