Skip to content

Commit 48fb526

Browse files
committed
Fixed Flaky Tests Caused by JSON permutations
###Description Flaky Tests found using NonDex by running the commands - mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentComplicatedJsonObject mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentSimpleJsonArray mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentSimpleJsonObject The logged failure was- [ERROR] Failures: [ERROR] XMLTest.testIndentSimpleJsonObject:1193 expected:<...> <employee> <[married>true</married> <name>sonoo</name> <salary>56000</salary]> </employee> </Te...> but was:<...> <employee> <[name>sonoo</name> <salary>56000</salary> <married>true</married]> </employee> </Te...> The issue is the same for all three tests, so here I only show the failure message for the third test (to reduce the length of the error message). ### Investigation The tests fail with a comparison error while comparing an expected JSON String and the result from the value returned from XML.toString(). The toString function of XML makes no guarantees as to the iteration order of the attributes in the object. This makes the test outcome non-deterministic, and the test fails whenever the function returns a mismatch in order of the elements in the JSON String. To fix this, the expected and actual keys should be checked in a more deterministic way so that the assertions do not fail. ### Fix Expected and Actual values can be converted into JSONObject and the similar function can be used to compare these objects. As this function compares the values inside the JSONObjects without needing order, the test becomes deterministic and ensures that the flakiness from the test is removed. The PR does not introduce a breaking change.
1 parent 8353b9c commit 48fb526

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

src/test/java/org/json/junit/XMLTest.java

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1111,6 +1111,7 @@ public void testIndentComplicatedJsonObject(){
11111111
"}" ;
11121112
JSONObject jsonObject = new JSONObject(str);
11131113
String actualIndentedXmlString = XML.toString(jsonObject, 1);
1114+
JSONObject actualJsonObject = XML.toJSONObject(actualIndentedXmlString);
11141115
String expected = "<success>true</success>\n" +
11151116
"<response>\n" +
11161117
" <dateTimeISO>2022-10-05T00:00:00+03:00</dateTimeISO>\n" +
@@ -1170,7 +1171,8 @@ public void testIndentComplicatedJsonObject(){
11701171
" <timestamp>1664917200</timestamp>\n" +
11711172
"</response>\n" +
11721173
"<error>null</error>\n";
1173-
assertEquals(actualIndentedXmlString, expected);
1174+
JSONObject expectedJsonObject = XML.toJSONObject(expected);
1175+
assertTrue(expectedJsonObject.similar(actualJsonObject));
11741176

11751177

11761178
}
@@ -1183,14 +1185,16 @@ public void testIndentSimpleJsonObject(){
11831185
" }}";
11841186
JSONObject jsonObject = new JSONObject(str);
11851187
String actual = XML.toString(jsonObject, "Test", 2);
1188+
JSONObject actualJsonObject = XML.toJSONObject(actual);
11861189
String expected = "<Test>\n" +
11871190
" <employee>\n" +
11881191
" <name>sonoo</name>\n" +
11891192
" <salary>56000</salary>\n" +
11901193
" <married>true</married>\n" +
11911194
" </employee>\n" +
11921195
"</Test>\n";
1193-
assertEquals(actual, expected);
1196+
JSONObject expectedJsonObject = XML.toJSONObject(expected);
1197+
assertTrue(expectedJsonObject.similar(actualJsonObject));
11941198
}
11951199

11961200
@Test
@@ -1201,6 +1205,7 @@ public void testIndentSimpleJsonArray(){
12011205
"] ";
12021206
JSONArray jsonObject = new JSONArray(str);
12031207
String actual = XML.toString(jsonObject, 2);
1208+
JSONObject actualJsonObject = XML.toJSONObject(actual);
12041209
String expected = "<array>\n" +
12051210
" <name>Ram</name>\n" +
12061211
" <email>[email protected]</email>\n" +
@@ -1209,7 +1214,8 @@ public void testIndentSimpleJsonArray(){
12091214
" <name>Bob</name>\n" +
12101215
" <email>[email protected]</email>\n" +
12111216
"</array>\n";
1212-
assertEquals(actual, expected);
1217+
JSONObject expectedJsonObject = XML.toJSONObject(expected);
1218+
assertTrue(expectedJsonObject.similar(actualJsonObject));
12131219

12141220

12151221
}

0 commit comments

Comments
 (0)