Generating test cases at build time

[plainionist]

Hi,

i like the lean approach of TickSpec but i found that generating test cases at runtime as described here has quite some drawbacks as described in detail in this article and this article so i have prototyped "test case generation at build time" and successfully use it in one of my F# projects.

The code can be found here: https://github.com/plainionist/TickSpec.Build

I just wanted to share this with you and your community and ask whether you find this approach interesting and would consider integrating it into TickSpec (which would certainly require some improvements like C# support and support for other test frameworks like XUnit)

[mchaloupka]

Hi,

I have read the articles - you were trying to solve the problem of not being able to set breakpoints in the feature file. Do I understand it correctly?

There are two problems you are solving:

• Better unit test integration to have separate test cases
• Ability to set breakpoints in feature files

It is an interesting idea to generate files and compile them. We originally used an approach of generating dynamic assembly on the fly for individual methods. That allowed us to set breakpoints, but it worked only in .NET Framework and does not work in .NET Core.

I would still prefer it if we can achieve the breakpoints even for dynamic loading of files. However, providing any kind of solution is better than no solution which is the state at this moment. And if we add it, we can delete all the existing functionality for IL emitting, allow async steps, etc.

Would you mind creating a PR? Either add the functionality into the nUnit wiring (gets published as separate nuget) or if it does not have an overlap, then you can create a new project in the wiring folder. Ideally with a readme that describes what are the steps to use your approach. Ideally, if it can be with minimum touches - how to create a project using your nuget package, feature file and minimum configuration.

[plainionist]

hi,
thx for the feedback!

Breakpoints are one use case. Another one, and currently this would be the more important one from my perspective, is to be able to navigate from a "test explorer" (e.g. in Visual Studio) to the scenario in the feature file (gherkin file). without generating the test fixtures during build time AND including "#line" directives i don't see how this feature could work. and without such a feature it is quite annoying to investigate failed test cases because i would have to manually find the file in the solution explorer and then the scenario in a maybe bigger feature file.

Currently i am adding support for generating HTML files from gherkin files for documentation purpose to my project as will. these changes will probably cause breaking changes.
once completed i ll try to provide a PR

[plainionist]

Pull request created: #65
It is a first step. NuGet integration open. Pls have a look and lets discuss next steps