Skip to content

DataFactoryTestingFrameworkExpressionsEvaluator adds status property even if None is set. #164

@LeonardHd

Description

@LeonardHd

We have scenarios where certain Activity outputs (like status as described in #162 ) are not present on the Activity Outputs.

Currently neither functional tests nor activity tests can capture these scenarios because we add the status always to the activity results (see https://github.com/microsoft/data-factory-testing-framework/blob/main/src/data_factory_testing_framework/_pythonnet/data_factory_testing_framework_expressions_evaluator.py#L40).

The expression evaluator returns a error we can handle to mimic the behaviour:

>           raise Exception(f"Unknown error: {e}") from e
E           Exception: Unknown error: The template language expression 'contains(
E             createArray(
E               activity('Fail').status,
E               activity('Notebook').status
E             ),
E             'Failed'
E           )' cannot be evaluated because property 'status' doesn't exist, available properties are 'output, error'. Please see https://aka.ms/logicexpressions for usage details.
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionTextExpression.SelectObjectProperty(JObject root, JToken property, Boolean safeDereference)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionTextExpression.SelectProperty(JToken root, TemplateDereferenceExpression[] dereferences, TemplateExpressionEvaluationContext context, Boolean failOnError)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionTextExpression.EvaluateRootExpression(TemplateExpressionEvaluationContext context, Boolean failOnError, ISkippableEnumerator`1 parameterValues, Int32 parametersCount)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionExpression.EvaluateExpression(TemplateExpressionEvaluationContext context, Boolean failOnError)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionExpression.LazyParameterEnumerator.MoveNext(Int32 skip)
E              at Microsoft.Azure.Workflows.Templates.ISkippableEnumeratorExtensions.ToEnumerable[T](ISkippableEnumerator`1 enumerator)+MoveNext()
E              at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
E              at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateExpressionBuiltInFunctions.EvaluateFunction(String functionName, ISkippableEnumerator`1 parameters, Int32 parametersCount, Boolean preserveAnnotations)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateExpressionEvaluationHelper.EvaluateFunction(String functionName, ISkippableEnumerator`1 parameters, Int32 parametersCount)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionTextExpression.EvaluateRootExpression(TemplateExpressionEvaluationContext context, Boolean failOnError, ISkippableEnumerator`1 parameterValues, Int32 parametersCount)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionExpression.EvaluateExpression(TemplateExpressionEvaluationContext context, Boolean failOnError)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionExpression.LazyParameterEnumerator.MoveNext(Int32 skip)
E              at Microsoft.Azure.Workflows.Templates.ISkippableEnumeratorExtensions.ToEnumerable[T](ISkippableEnumerator`1 enumerator)+MoveNext()
E              at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
E              at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateExpressionBuiltInFunctions.EvaluateFunction(String functionName, ISkippableEnumerator`1 parameters, Int32 parametersCount, Boolean preserveAnnotations)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateExpressionEvaluationHelper.EvaluateFunction(String functionName, ISkippableEnumerator`1 parameters, Int32 parametersCount)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionTextExpression.EvaluateRootExpression(TemplateExpressionEvaluationContext context, Boolean failOnError, ISkippableEnumerator`1 parameterValues, Int32 parametersCount)
E              at Microsoft.Azure.Workflows.Templates.Expressions.TemplateFunctionExpression.EvaluateExpression(TemplateExpressionEvaluationContext context, Boolean failOnError)
E              at Microsoft.Azure.Workflows.Templates.Helpers.TemplateExpressionsHelper.EvaluateTemplateLanguageExpression(String expression, TemplateExpressionEvaluationContext evaluationContext)
E              at Microsoft.Azure.DataFactoryTestingFramework.Expressions.Evaluator.EvaluateExpression(String expression, String parametersJson, String variablesJson, String itemValueJson, String activityValuesJson)
E              at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
E              at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

Proposed solution:

Only add the add properties if they are set to anything other than None.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions