TopCoder has created Testing Competitions to create testing scenarios and automated scripts to be used in conjunction with Assembly competitions or other applications needing testing. Through these competitions, participants will create high quality test code to ensure that the application in question meets the specified requirements and functionality.
TopCoder posts the competition details, specification deliverables, architecture deliverables and timeframe for active contests. Members interested in competing in the Testing Competition should read through the documentation and decide whether or not to register.
To register for a testing competition, click the name of a competition on the active contests page. This links to a description of the competition and buttons for registering. The competition description page also has links to the forum where you can find documents needed to compete.
Registration will remain open until the Registration deadline has passed. As soon as you register, you may begin working on the competition.
In test case/scenario competitions you expected to define a testing strategy for the application (QA plan). Both high level application test cases and detailed test scenarios must be included in the QA plan. Test scenarios are derived from requirements documentation and in many cases a prototype. Deliverables include details such as scenario setup, pre/post conditions to the scenario, step by step instructions to execute the scenario, whether the scenario can be automated or not, and expected results to determine success.
In automated script competitions, competitors are expected to deliver automated test scripts derived from a comprehensive test scenario inventory. Deliverables include coded automated test scripts written in automation tools such as JUnit or SwingUnit.
Assembly competitions may combine scenario and script creation with the application Assembly process where testers work in concert with developers during and after the Assembly phase of an application. The end result is test code that will be run against submissions in the corresponding Assembly Competition.
In all cases, testing scenarios and scripts map to specific use cases in the supplied Requirements Specifications and Activity diagrams for the application. These document serve as background and context for the tests.
Participants will be granted access to a forum for each Testing Competition. This forum will be monitored by TopCoder staff who will answer any and all technical questions in the forum.
Once a participant has submitted their documents/code, it will be automatically screened to enforce coding standards. Additional manual screening by a reviewer will be used to check the completeness of the testing deliverables. A reviewer will screen on a per submission basis.
At the end of the competition all of the test submissions that passed screening will be fully reviewed by the member review board.
The Testing competitions will be graded based on how well they meet the criteria set forth in the scorecard, including code coverage, number of test cases created, number of identified bugs, etc. Be sure to review the scorecard beforehand to see exactly what the criteria are.
Prizes will be assigned for first and second place per testing competition. The prize purse for each Testing Competition will be based on the complexity and number of test case items.
Post-Competition Support for Scenario Competition Winners
The following applies to competitions that separate the creation of test scenarios from the creation of automated scripts.
After you have won a test scenario competition, the testing is ready to move to the automated script development phase. The winner of the test scenario competition is responsible to support the corresponding scripting competition. So, when you see your testing scenario competition posted for automated script development, you should start monitoring the developers' forums again.
At this point, you are the main authority to answer the script developers' questions related to your scenario designs. Sometimes you will get simple questions from developers who miss a thing or two in the documentation. These are the easiest questions to answer - instead of copying the explanations from the documentation, you should refer developers to the answer in the requirements specification or in the UML documentation. This will help reviewers of the script development effort to understand what has really changed as the result of the forums interaction. However, most questions that you are going to get are really questions that point out things that you and the review board did not catch and you must be prepared to address these.