test/py: print summary in test order

Use lists rather than sets to record the status of tests. This causes the test summary in the HTML file to be generated in the same order as the tests are (or would have been) run. This makes it easier to locate the first failed test. The log for this test might have interesting first clues re: interaction with the environment (e.g. hardware flashing, serial console, ...) and may help tracking down external issues. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>

test/py: print summary in test order
Use lists rather than sets to record the status of tests. This causes the test summary in the HTML file to be generated in the same order as the tests are (or would have been) run. This makes it easier to locate the first failed test. The log for this test might have interesting first clues re: interaction with the environment (e.g. hardware flashing, serial console, ...) and may help tracking down external issues. Signed-off-by: Stephen Warren <swarren@nvidia.com> Acked-by: Simon Glass <sjg@chromium.org>
Stephen Warren · Tom Rini
1 parent 1235c79182
Showing 1 changed file with 8 additions and 8 deletions Side-by-side Diff
test/py/conftest.py
@@ -312,12 +312,12 @@
     return console
  
 anchors = {}
-tests_not_run = set()
-tests_failed = set()
-tests_xpassed = set()
-tests_xfailed = set()
-tests_skipped = set()
-tests_passed = set()
+tests_not_run = []
+tests_failed = []
+tests_xpassed = []
+tests_xfailed = []
+tests_skipped = []
+tests_passed = []
  
 def pytest_itemcollected(item):
     """pytest hook: Called once for each test found during collection.
@@ -332,7 +332,7 @@
         Nothing.
     """
  
-    tests_not_run.add(item.name)
+    tests_not_run.append(item.name)
  
 def cleanup():
     """Clean up all global state.
@@ -493,7 +493,7 @@
     if failure_cleanup:
         console.drain_console()
  
-    test_list.add(item.name)
+    test_list.append(item.name)
     tests_not_run.remove(item.name)
  
     try:
...	...	@@ -312,12 +312,12 @@
312	312	return console
313	313
314	314	anchors = {}
315		-tests_not_run = set()
316		-tests_failed = set()
317		-tests_xpassed = set()
318		-tests_xfailed = set()
319		-tests_skipped = set()
320		-tests_passed = set()
	315	+tests_not_run = []
	316	+tests_failed = []
	317	+tests_xpassed = []
	318	+tests_xfailed = []
	319	+tests_skipped = []
	320	+tests_passed = []
321	321
322	322	def pytest_itemcollected(item):
323	323	"""pytest hook: Called once for each test found during collection.
...	...	@@ -332,7 +332,7 @@
332	332	Nothing.
333	333	"""
334	334
335		- tests_not_run.add(item.name)
	335	+ tests_not_run.append(item.name)
336	336
337	337	def cleanup():
338	338	"""Clean up all global state.
...	...	@@ -493,7 +493,7 @@
493	493	if failure_cleanup:
494	494	console.drain_console()
495	495
496		- test_list.add(item.name)
	496	+ test_list.append(item.name)
497	497	tests_not_run.remove(item.name)
498	498
499	499	try: