fypp syntax highlighting support

[ivan-pi]

Would it be possible to add support for fypp (https://fypp.readthedocs.io/en/stable/fypp.html)?

I imagine two modes, one where fypp directives (lines beginning with #, $ or @) are highlighted as comments, and a second mode which would add coloring also to the preprocessor directives. Which mode to use could be set in a JSON settings file of the package.

As a side note, what are the advantages/differences of this package compared to the older https://packagecontrol.io/packages/Fortran? I've been using the previous one, but after re-installing Sublime thought I'd give this package a go instead.

Also perhaps of interest, @cdslaborg has published a list of Fortran keywords in response to a thread at Fortran Discourse. These can be useful to cross-check for syntax highlighting.

[eirik-kjonstad]

Thanks for opening the issue!

I haven't used fypp myself, but it seems like it will be relatively straight-forward to add support for it. I will look into it soon. Pull requests are also welcome, if you instead want to try adding support yourself.

The main difference between this package and the older one is that this package has improved highlighting for features introduced in modern Fortran standards (especially 03, 08, 18). The older package also includes support for fixed-format Fortran, which is not specifically supported in this package (yet).

Thanks for the keywords link. I will check whether we have missed some keywords in the syntax.

[ivan-pi]

Since fypp is essentially just a Python module, it's easy to drop it directly into the Sublime packages folder. Together with a Sublime build script, it makes it easy to generate generic Fortran code directly from Sublime. An advanced example of using fypp is the generic tensor contraction.

Are there any good resources I can follow on Sublime syntax highlighting? I've never used or written a regex before.

I imagine I'd start with something similar to

  fppCommands: (define|undef|ifdef|ifndef|if|elif|else|endif|include|error|warning|line-number|line)

and the block:

  preprocessing:
    - match: '(?i){{firstOnLine}}(\#{{fppCommands}})'
      captures:
        1: keyword.control.directive.fortran

Thanks for developing this package.

[eirik-kjonstad]

Yes, I've used these two extensively (for syntax and for selecting scopes):

https://www.sublimetext.com/docs/syntax.html
https://www.sublimetext.com/docs/scope_naming.html

What scopes to use is sometimes conventional, so it might also be useful to check out some other syntaxes that are included with Sublime - see https://github.com/sublimehq/Packages.

I've also found it useful to use https://regex101.com/ to test out the regex on some examples. Would also recommend using PackageDev (https://github.com/SublimeText/PackageDev), which provides highlighting of the syntax code.

[ivan-pi]

Slowly I'm getting a hang of this:

%YAML 1.2
---
name: fypp
file_extensions: [fypp]
scope: source.fypp

variables:
  firstOnLine: '^\s*'
  fyppDirectives: (set|del|if|elif|else|endif|for|endfor|def|enddef|block|endblock|call|endcall|global|include|mute|endmute|stop|assert)
  fyppPredefinedFunctions: (defined|getvar|setvar|delvar|globalvar)

contexts:
  main:
    - include: fypp_comments
    - include: fypp_preproc

  fypp_comments:
    - match: '#!(?![$])'
      scope: punctuation.definition.comment.fypp
      push:
      - meta_scope: comment.line.fypp
      - match: \n
        pop: true

  fypp_preproc:
    - match: '(?i){{firstOnLine}}(\#:\s*{{fyppDirectives}})'
      captures:
        1: keyword.control.directive.fortran

At the moment I've hard coded the #: symbol following the fppDirectives. More generally, fypp will preprocess lines containing

1. control directives, beginning with #: or the inline form enclosed between#{ and }#
2. eval directives, beginning with $: or the inline form enclosed between ${ and $}
3. direct call directives, beginning with @:
4. comments, which begin with #!

I have the feeling that all the fypp constructs should be placed in a meta.preprocessor kind of scope, but I guess I can wait with this.

Another question is whether it makes sense to extend the modern-fortran syntax (or vice-versa), to support both fypp and Fortran syntaxes combined?

[eirik-kjonstad]

Great!

I prefer if fypp is added similarly as fpp, i.e., that the fypp syntax is incorporated into the modern-fortran syntax. If you want I can incorporate your code into a branch based on the modern-fortran syntax (as a starting point). Let me know.