Jinja2 global/rendering contexts

[DanPartelly]

@ipspace How are those used in Netlab? i.e lifetimes/ access ?

Can we register custom Python functions as Jinja 2 filter/functions?

[ipspace]

@ipspace How are those used in Netlab? i.e lifetimes/ access ?

Tell me more...

Can we register custom Python functions as Jinja 2 filter/functions?

In theory, yes. In practice, I'd prefer if you'd do as much preprocessing as possible in Python code and then have simple templates.

Let's step back and start with what exactly are you trying to solve?

[DanPartelly]

Nothing right now.

Ive looked at some templates and they got pretty hairy. Then I realized that writting rendering support function in Python cold potentially improve legibility as you write less Jinja and more Python. The generic problem is complexity in template.

[jbemmel]

Hairy templates can be a sign that the underlying data model isn't fully capturing the compressed intent

#3654 is a recent example. Ideally, the Jinja2 templates should express only the specific keywords and order/formatting of arguments per platform, possibly guarded by simple logic.

Before writing Python filters, check to see if the data model can be enhanced locally ( e.g. by precalculating _condition such that the template can do {% if _condition %} as opposed to {% if this, that and the other but unless X ... %}

[DanPartelly]

Many of our templates are what I would call hairy. While I certainly agree with you on data model changes to keep template logic simple Jinja2 suffers from:

• it's almost incomprehensible by design. It is not a piece of SW designed for legibility. Go down 3 levels in a for loop, and you have to spend too many mental cycles to understand that crap. Not a matter of skill, but economy. One solution would be proper manual indent, or comments of any if/for/whatever end statement.
• templates should not be guarding against invalid state. yet we are not there.
• a language, even a template one, should be designed for legibility
• no automatic formater I can find can make sense of that mess. If you don't indent it manually with patience, good luck of even of a computer making sense of it.
• white space handling. Mess with crap like decorating with "-" , or just blast the data out from Python properly formatted.
• We already rely on third-party filters in some templates. Ansible's ippaddr . Which should be a no-no. It;s better to register our filters than rely on an enormous 3rd party dependency which some day may be off.
• So if we already do it in some templates, why not build ours ? A good set of orthogonal Python filters/functions exposed to ninja-jinja rendering context would do wonders for legibility,
• Contrast it with Golang templates. Less capability, you are almost forced to write functions to support the templating, but the legibility is way higher. (conscious design choice )
• No one asks anyone to build those functions. Just expose the mechanisms to register them to the rendering. Global context, from core or plugins.

Do we need custom Python to augment Jinja? No. Would it improve legibility of everything? Yes. One order of magnitude. In time.

[ipspace]

Many of our templates are what I would call hairy.

Quite often caused by someone trying to be way too smart and using a long pipeline of obscure functions instead of a "for if../if loop.first" construct ;)

• it's almost incomprehensible by design. It is not a piece of SW designed for legibility. Go down 3 levels in a for loop, and you have to spend too many mental cycles to understand that crap.

I worked with at least four "templating languages" (including XSLT), and they all suck in one way or another. I also had the privilege of having to deal with PHP code interspersed with verbatim text, and I wouldn't want to repeat that experience. I don't think there's a good way of solving the problem.

Not a matter of skill, but economy. One solution would be proper manual indent

Are you saying we don't do that=

or comments of any if/for/whatever end statement.

Go for it.

• no automatic formater I can find can make sense of that mess. If you don't indent it manually with patience, good luck of even of a computer making sense of it.

Arista has Jinja2 linter, and IIRC the only reason we're not using it is that it has hard-coded 4-space indents. I'm pretty sure @carlbuchmann could be persuaded to make that configurable ;)

• white space handling. Mess with crap like decorating with "-" , or just blast the data out from Python properly formatted.

There is no good way of handling whitespace. Whichever way you go, someone will complain.

• We already rely on third-party filters in some templates. Ansible's ippaddr . Which should be a no-no. It;s better to register our filters than rely on an enormous 3rd party dependency which some day may be off.

Would you PRETTY PLEASE look at the code and a bit of history before complaining? We reimplemented a few Ansible filters when we split from Ansible, and have no dependency on Ansible apart from deploying device configs. The only reason they are still called ipaddr (and the like) is because people are used to those filters, and the only reason we still support "ansible.utils.ipaddr" and "ansible.netcommon.ipaddr" is to keep older templates working.

I simply refuse to believe that breaking stuff that works for purity reasons (SR Linux style) makes sense.

• So if we already do it in some templates, why not build ours ? A good set of orthogonal Python filters/functions exposed to ninja-jinja rendering context would do wonders for legibility,

If you have a specific example of what you'd like to have, let's start with that. Generic complaining will not get us far.

• Contrast it with Golang templates. Less capability, you are almost forced to write functions to support the templating, but the legibility is way higher. (conscious design choice )

As I'm maintaining a few sites built with Hugo, I'm pretty familiar with how well that works. Let's just say that it looks like you haven't tried to solve a complex-enough problem with Golang templates yet ;))

• No one asks anyone to build those functions. Just expose the mechanisms to register them to the rendering. Global context, from core or plugins.

That functionality is already there. Just add your filters to utils/filters/BUILTIN_FILTERS (or add a helper function if you can't manipulate the dict from outside the module).

To wrap this up: if you have a specific challenge, let's talk about it, and we'll figure out a way to solve it. If you have a specific idea you'd like to see implemented, let's do it (for example, if you need a helper function to register filters). Generic complaining and comparing Jinja2 to Golang templates is bikeshedding and just wastes everyone's mental energy. No tool is perfect, and the grass is always greener on the other side of the fence (until you get there and realize you've been too optimistic).