Replacement API for `IClass.getInstanceHashCodes()` and `IClass.getInstances(boolean)`

[marcphilipp]

In 7.10.1, the IClass.getInstanceHashCodes() method was deprecated. As discussed in junit-team/testng-engine#116 (comment), it is used by the JUnit Platform adapter (testng-engine) to append the instance index to the display name which is important to differentiate between different runs. Prior to removing the method, a replacement for this use case should be added.

[juherr]

As I understand, something like Integer ITestResult.getFactoryMethod().getInvocationIndex() should provided.

[marcphilipp]

Yes, that sounds sensible. 👍

[krmahadevan]

@marcphilipp - Can you please check if this would work for you ( This exists already within TestNG)

        ITestResult itr = Reporter.getCurrentTestResult();
        int index = itr.getMethod().getFactoryMethodParamsInfo().getIndex();

[marcphilipp]

Yes, that works on 7.5 and later (IParameterInfo.getIndex() was added in 5cffe02). 👍

Why is IParameterInfo in an internal package? Is it ok to use it anyway?

[krmahadevan]

🤦 - I didnt see the package information. Let me see what we can do to ensure that information is available. Worst case, we can expose that interface to the public package since it does have information that can be of relevance to the user.

@juherr WDYT ?

[marcphilipp]

I checked for other deprecations and found IClass.getInstances(boolean) which is used to compute the index of the test instance: https://github.com/junit-team/testng-engine/blob/dcd461e556f618685084ec6a6719c91a227a1298/src/main/java/org/junit/support/testng/engine/MethodDescriptor.java#L53-L64

Is there an existing replacement for that as well?

[marcphilipp]

WIP PR for replacing the call site of deprecated APIs: junit-team/testng-engine#121

[juherr]

@krmahadevan I think IParameterInfo can be moved to the public package.

@marcphilipp You don't need result.getTestClass().getInstances(true) if result.getMethod().getFactoryMethodParamsInfo().getIndex() provides the good index value, right?

[marcphilipp]

You're right. I found one case where that doesn't work, though:

import static org.testng.Assert.assertNotEquals;

import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class FactoryMethodTestCase {

	private final String a;
	private final String b;

	@Factory
	public static Object[] factoryData() {
		return new Object[] { new FactoryMethodTestCase("a", "b"), new FactoryMethodTestCase("c", "d") };
	}

	public FactoryMethodTestCase(String a, String b) {
		this.a = a;
		this.b = b;
	}

	@Test
	public void test() {
		assertNotEquals(a, b);
	}
}

For this test class, IParameterInfo.getIndex() always returns 0 even though it should return 1 for the second instance. Is that a known issue?

[juherr]

I didn't find any test that checked the feature.

Samples from https://github.com/testng-team/testng/tree/master/testng-core/src/test/java/test/factory/github1083 could be used.

[krmahadevan]

@marcphilipp

For this test class, IParameterInfo.getIndex() always returns 0 even though it should return 1 for the second instance. Is that a known issue?

There are two scenarios in which someone would use the @Factory annotation. We are seeing a value of 0 for the second scenario. I will debug and find out why is this the case.

Are you still looking for a replacement for getInstanceHashCode() ?

Scenario 1

A constructor that is annotated using the @Factory annotation and the @Factory is tied to a data provider.

Scenario-1-Sample

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(LoggingListener.class)
public class TestCaseSample {

    private final int ignored;


    @Factory(dataProvider = "getData")
    public TestCaseSample(int i) {
        this.ignored = i;
    }

    @Test
    public void testMethod() {
    }

    @DataProvider
    public static Object[][] getData() {
        return new Object[][]{{1}, {2}};
    }
}

Execution output

LF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Index = 0
Index = 1

===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

Scenario 2

A static method that is annotated using @Factory annotation.

Scenario-2-Sample

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(LoggingListener.class)
public class AnotherTestCaseSample {

    private final int ignored;


    public AnotherTestCaseSample(int i) {
        this.ignored = i;
    }

    @Test
    public void testMethod() {
    }

    @Factory
    public static Object[] getInstance() {
        return new AnotherTestCaseSample[]{
                new AnotherTestCaseSample(1),
                new AnotherTestCaseSample(2),
        };
    }

    @DataProvider
    public static Object[][] getData() {
        return new Object[][]{{1}, {2}};
    }
}

Execution output

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Index = 0
Index = 0

===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

[marcphilipp]

Are you still looking for a replacement for getInstanceHashCode() ?

Only if result.getMethod().getFactoryMethodParamsInfo().getIndex() doesn't work here.

[krmahadevan]

@marcphilipp - Can you please tell me how can I programmatically use the JUnit TestNG platform library and run things from within an IDE (without using the console launcher or the maven integration)

I am basically looking for something similar to

TestNG testng = new TestNG();
testng.run();

I am doing something like this

        TestNGTestEngine engine = new TestNGTestEngine();
        engine.discover()
        ExecutionRequest request = new ExecutionRequest();
        engine.execute(request);

but am not sure as to how do I pass in a proper implementation to the discover and the execute method.