The first step to effectively outlining an application review process is the creation of a thorough evaluation method. Rankings and scorecards are two common reviewing types used by judges to express preferences and perceptions of quality in a collection of submissions. A program manager should consider the components of the program application and how each should be assessed.

A score-based evaluation allows you to set as many judging criteria as needed, then have the judges simply score each entry on a scale for each criteria. The question is how detailed you want to be with scoring. Should each application component receive a distinct score?

On the other hand, a ranking evaluation form with a simple drag-and-drop feature based on reviewer’s assessment makes it easy for judges to input their scores. They typically have the option to leave comments on the evaluation for feedback. This mode of reviewing applications has its advantages, and the following scenarios are typical settings when ranking review format is more beneficial to the program evaluation process.

1. Single criterion based review

By making clear your foundational principles and defining a clear set of criteria for evaluation applications, you can decide that different norms (e.g. innovation, feasibility, etc) are not relevant for your program. Instead, you can decide to focus on the overall order of submitted ideas according to their perceived qualities. In order words, your single criterion based order of applications rests on the terms of reviewer’s selecting the highest-quality ideas.

In such cases it is useful to leave the reviewers an option to provide additional comment for clarifying their decision. Such comments are typically visible for admins and other reviewers, and hidden from the applicant.

2. High volume of applications

A ranking system can be another great collaborative way to select program winners for a high volume of applications. Using this strategy, reviewers are assigned a set number of applications to read and rank in quality order. If reviewers are tasked with a high volume of applications, this method can help alleviate the pressure.

Once the review process has started, all your reviewers can see is application materials in one place, and rank them according to their quality as they go. No more toggling between tabs, drowning in email, or time-consuming analysis of each criterion used in the scorecard model of review.

3. Multiple rounds of review

For programs with high volume of applications that need to be thoroughly evaluated, first stage should focus on selecting the best applications for another round. It makes sense to use ranking method in the first round, and then scorecard review in the second round, when a detailed input is required from the reviewers.

Just as your programs are unique, a review process that is fine-tuned will contribute to your outstanding application and review process, so that both your applicants and your review team appreciate the results.

After all, as a program manager, you want to be sure your award program makes the most impact. By incorporating these best practices, your organization can feel confident that they have fulfilled their promise to continuously strive toward this goal.