Where should I propose Assertions for Nested Exceptions? Here or Pester?

[alx9r]

Neither the "-Throw" assertion in the Pester project nor Assert-Throw currently support search of nested or aggregate exceptions. I expect to implement such an assertion. Where should I propose this addition?

[nohwnd]

Assert-Throw is only meant to encapsulate the use cases that everyone would likely find useful. Stuff like message, exceptiontype, fqeid. For anything else it outputs the exception object and you can drill in it in with other assertions. I also wrote a piece of code that enables combining the assertions in one failure somewhere. This I would like to make part of the framework, so multiple assertions in a test would not be a problem. We could provide another assertion that asserts on exception objects to make this drilling down easier, the difficult part is naming it. Assert-Throw, Assert-Error and Assert-Exception are all the same thing in layman’s eyes so I don’t want to overload them. Could you think about it a bit and propose some syntax so we can brainstorm over it further? I am looking for nice non-complex syntax and features that are useful to many people. I can for example imagine that something like -Level 1 that would assert on first InnerException would be useful in some cases, but how often would you need to go deeper, and how easy is to guess what the parameter does? So in that case it might be better to call it -InnerException, even though we lose the ability to go deeper (without introducing parameter scopes).

[alx9r]

For anything else it outputs the exception object and you can drill in it in with other assertions.

Oh I see. Would you support this sort of usage?

{ ThisThrowsANestedException } |
    Assert-Throw |
    Assert-Message 'some deeply nested error message' -AnyWhereIn 'Inner','Aggregate'

I'm not sure where Assert-Message would live, but let's just it existed somewhere. Is that sort of composition what you had in mind?

[nohwnd]

This is the kind of custom DSL, non-powershelly stuff that I would like to avoid, unless we have a really good solution that makes powershell-sense (dibz on that term). We probably need more realistic example for this. We should also keep in mind that we are testing one specific thing, so the syntax afaik only about navigation and encapsulating useful cases. but please correct me if I am wrong. Last time I saw aggregate exception was from EntityFramework, and threre I always have a piece of code that flattens the exception.

This is a bit of extreme scenario, where you get AggregateException in InnerException and you want to check that at least one of the exceptions has the given message. This is the most convoluted scenario I can see as realistic, but quite rare. This you can solve like this:

$exception = { ThisThrowsExceptionWithAggregatedExceptionInTheInnerException } | Assert-Throw
$exception.InnerException.InnerExceptions | 
    Assert-Any -FilterScript { 
        $_.Message -like '*message on some of the aggregate exceptions*'  }

I also think there will need to be a set of cmdlets for Test-* to make the collection assertions useful (and writing custom assertions easy), and then you could do:

$exception = { ThisThrowsExceptionWithAggregatedExceptionInTheInnerException } | Assert-Throw
$exception.InnerException.InnerExceptions | 
     Assert-Any -FilterScript { 
        Test-Exception -ExceptionMessage '*message on some of the aggregate exceptions*' `
             -Type ([ArgumentException]) }

---

As I said when I was responding from phone: I can imagine a simpler scenario where someone is wrapping assertions for you and you are tired of unwrapping them (like MethodInvocationFailed ), there you might want to do this:

{ ... } | Assert-Throw -Type ([ArgumentException]) -InnerException
# or
{ ... } | Assert-Throw -Type ([ArgumentException]) -Level 1 #or -Unwrap 1

But I also see this being useful only rarely.

[nohwnd]

And the composition is in #19 :)

[alx9r]

This is the kind of custom DSL, non-powershelly stuff that I would like to avoid

What aspect do you regard as "non-powershelly"?

but please correct me if I am wrong. Last time I saw aggregate exception was from EntityFramework, and threre I always have a piece of code that flattens the exception.

I encounter both InnerException and AggregateException daily. The former results naturally from providing adequate exception messages when an exception is thrown in a PowerShell pipeline. (example) The latter can occur anytime an exception is thrown on a different thread in .Net and bubbles back up to the call in PowerShell. AFAIK it's .Net's task parallel library that usually creates those AggregateExceptions.

We probably need more realistic example for this. We should also keep in mind that we are testing one specific thing, so the syntax afaik only about navigation and encapsulating useful cases.

One reason I'm interested in this capability is because I regularly need to test that an exception thrown was about one particular domain-specific thing, not some other domain-specific thing, or some unrelated machinery thing like a null reference exception. I'd say this is occurs more frequently for integration tests but I don't see why it wouldn't occur when testing a PowerShell unit that includes some non-trivial but side-effect free .Net object.

In concrete terms, the need for better exception matching most recently arose because I needed to assert which reason System.Net.Http.HttpClient was failing. (See Save-WebFile implementation and integration test.) That is, I needed to distinguish a failure during protocol negotiation or certificate validation failure from any other kind of failure. In that case the test didn't need to care from which branch of the Inner/AggregateException tree the message originated. It only needed to care whether a message mentioning "trust relationship" or "secure channel" was in one of the exceptions or not. In such scenarios being too specific about the provenance of the message makes the tests brittle to changes in .Net implementation, so matching on any message in the whole tree seemed to be a reasonable compromise.

[nohwnd]

What aspect do you regard as "non-powershelly"?

-AnyWhereIn 'Inner','Aggregate' I don't know what exactly you want this to do, but PowerShell cmdlets usually offer single words for parameters. And arrays mean that you want to include both, but from this syntax it's hard to say what will happen, and I don't see how we could expand on it when adding more use-cases.

AFAIK it's .Net's task parallel library that usually creates those AggregateExceptions.

In C# that rarely happens unless you call .Result on the task which I never do. There are other ways to avoid getting aggregate exceptions. But I understand that they are your daily bread.

For the case when we don't care about the origin of the exception I would sugguest a function much like the one you have in your code, but with whole exception objects, that would flatten the Exception tree into a collection, and then apply Assert-Any on it. Maybe we could call it Resolve-Exception.

It's too late I will need to review this again tomorrow.

[alx9r]

-AnyWhereIn 'Inner','Aggregate' I don't know what exactly you want this to do, but PowerShell cmdlets usually offer single words for parameters. And arrays mean that you want to include both, but from this syntax it's hard to say what will happen, and I don't see how we could expand on it when adding more use-cases.

I agree. I was more focused on whether piping from one Assert- to the next is what you had in mind. I didn't think -AnyWhereIn through.

In C# that rarely happens unless you call .Result on the task which I never do. There are other ways to avoid getting aggregate exceptions.

I hadn't considered this until now. I think I have some homework to here first.

[alx9r]

I don't have a compelling use case for assertions on AggregateException, so I'd like to drop that from discussion and resume this discussion for assertions for nested exceptions that arise when the InnerException property is populated.

I've experimented with some approaches to such an assertion. The best approach I've found so far is in #28 for discussion.