Question regarding Build and Results

[jptest]

Let's say I have a Project X.
Test Plan would be "PX Release 2.0"
In that release, I will have builds B1, B2, B3 etc.

If I have 200 TCs in the test plan, all show up as cases in B1.
Say I run 50 TCs against B1.

"General Test Plan Metrics" show the "Overall Build Status" for B1 as 200 TCs, 150 Not Run, 40 Passed, 10 Failed, 0 Blocked.

Now I have B2. All 200 cases show up in B2. Ok, i'll use some filter to tell me which ones have not been run yet so I can run them on B2 now.
But the "General Test Plan Metrics" show the "Overall Build Status" for B1 as 200 TCs, 150 Not Run, 40 Passed, 10 Failed, 0 Blocked.
1. How do I get it to show Overall Build Status for B2 (as well as B1 or just B2 as the latest)?
2. Will it show only those that are executed against B2? If yes, how do I get Overall progress for the Test Plan to indicate total execution across all builds?

I know "Results by Tester per Build" gives me status by build B1, B2, B3 etc. But then, for me to report project progress to my management, I need overall execution. If I test 50 (unique) cases on B1 and 25 (unique) on B2 and 10 (5 of which are unique) on B3, I want to be able to report out of total 200 cases, 120 not run, 65 Passed, 10Failed, 5 Blocked.

Is this not doable? What am I missing?


Thanks in advance for a response.

[jptest]

No reply! Was it too verbose and not easy to understand the query?

[Unhandled]

I think I understand your question. I have had same issue/question.

Scenario is:
Run half of test cases in build #1, run second half of test cases in build #2. How do you get a report/chart/table/etc. showing overall pass/fail status regardless of build.

I am pretty sure that in 'Charts' report, the pie chart reflects "cumulative" results
I know that in 'Test results matrix' there is the right-most column '[Last build]'. Here in this column it shows what you want, the most recent pass/fail status from any build.
Also, the 'Test Report' will show the most recent result.

There may be better ways I am not aware of. One thing to remember is that In traditional software testing environment, builds are generated infrequently and all test cases are run against each build. In faster paced environments (agile/xp) this is difficult to do and testing is staggered over several builds. From my experience, Testlink tends to be more suited to the traditional, infrequent build environment. Of course it can be used in fast paced build cycle environment too!

[fman]

>>> Run half of test cases in build #1, run second half of test cases in build #2. How do you get a report/chart/table/etc. showing overall pass/fail status regardless >>> of build.
Please go to mantis and add a detailed feature request -> use case, with some out put mock up

on 1.9.3 a NEW column will be added on test matrix LAST EXECUTION RESULT => last exec in ANY BUILD

[jptest]

Tried to search and did not find anything reported so filed a bug 4331 "Overall Status on General Test Plan Metrics report".
But than Julian found 3577 that requests the same "Copy "Metrics dashboard" results to "General Test Plan Metrics" "