How to Interpret Concept Screening Results: Averages, Segments, and Evaluation Reasons

If you have evaluated 10 ideas and need to pick 3 to validate in the next stage, your eye naturally goes to the top-ranked ones first. But what if an idea that sits in the middle overall drew a strong response from a specific group of consumers? If that group is the product’s target, it is hard to drop the idea just because its overall rank is low.

The purpose of idea screening is to select candidates for validation in the next stage. That means looking not only at rankings but also at the overall average and the response of each consumer group. Teams first select the ideas that perform well overall, then look for additional candidates for specific consumers. Examining the reasons behind each evaluation also helps clarify how to refine an idea and what to validate in the next concept test.

Concept Studio results can be used in the same order. In Concept Studio, purchase likelihood is the percentage of synthetic panel members that an AI researcher predicts would want to buy the product. Teams review the overall results through rankings and bands, then look at group-level responses and evaluation reasons to select and refine the ideas to validate next.

Prioritize ideas that beat the overall average

Start by comparing each candidate against the average of all ideas evaluated under the same purpose and conditions. When using purchase likelihood, check whether each idea’s purchase likelihood is higher than the average across the ideas evaluated alongside it. Rather than relying on a number that is merely a little higher, use a statistical test suited to the research design to confirm whether the difference is significant.

On this basis, ideas can be divided into three groups: significantly above average, no significant difference from average, and significantly below average. Ideas significantly above average become the priority candidates for validation in the next stage. In other words, you begin with the candidates that drew a relatively strong response among the ideas compared this time.

This average, however, is not an absolute threshold for launch success. It shifts whenever the set of ideas evaluated together changes, so it serves as a guide for deciding which of the current candidates to review first. Ideas with no significant difference from the average, or those below it, are not dropped right away. Instead, their group-level results get a second look.

[Figure] Ideas that drew a significantly stronger response than the average of the ideas evaluated together are selected first, and the rest can be reviewed further by group.

A low overall score can still hide a strong group-level candidate

Overall results combine the responses of many different consumers. An idea that one group loves may still have low overall purchase likelihood if other groups show little interest. That is why ideas not prioritized overall are also broken down by the groups that matter for planning, such as gender and age or product usage experience. Here too, you look for ideas that perform significantly better than the group’s own average.

Suppose you evaluated snack ideas, and a roasted sweet potato tteok (a Korean rice cake) came out significantly above the overall average, while a matcha chapssaltteok (a chewy glutinous rice cake) came out below it. Based on the overall results, you would prioritize the sweet potato tteok. But if a group-level look showed that the matcha chapssaltteok drew a significantly stronger response than average among consumers in their 20s, it could also be considered a candidate for that group.

Ideas with no significant difference from the overall average are reviewed the same way. An idea that did not stand out overall may still be worth prioritizing for a particular group. Rather than deciding what to keep and drop from the overall results in one pass, teams use group-level results to find additional candidates.

[Figure] Even when an idea has low overall purchase likelihood, it can be considered a candidate for a specific group of consumers if it draws a strong response from that group.

Check whether the responsive group matches the target consumer

A strong group-level response does not automatically make an idea a candidate. You also need to check whether that group matches the target consumer for the product being planned. If you are looking for a dessert for consumers in their 20s, the matcha chapssaltteok is worth reviewing. If you are looking for a snack the whole family can share, you need to check how other groups responded as well.

So rather than slicing the results into more and more groups in search of high numbers, it is better to decide in advance which key groups to examine based on the planning goal. Group-level comparisons also call for statistical tests suited to the sample size and comparison method. Ideas with low purchase likelihood both overall and within the target group can be deprioritized in this round, freeing you to focus on other candidates.

This process also separates candidates by why they were selected: those chosen for their strong overall response, and those chosen for a specific group of consumers. Add the evaluation reasons, and you can see which features drove a positive response and which parts need further refinement.

Use evaluation reasons to refine ideas and find the next questions to test

Idea screening is not only about selecting candidates. It is also a process for working out how to refine each idea and what to validate in the next concept test. The synthetic panel’s evaluation reasons show which features come across as appealing and which descriptions raise concerns. Based on this, you can revise how an idea is described and compare responses before and after the change.

Returning to the matcha chapssaltteok, suppose the synthetic panel responded positively, saying the slightly bitter matcha flavor seemed to pair well with the chewy texture, but also expressed concern that the cream might make it too sweet. You could then add a description such as “lightly sweetened cream,” or revise the idea to emphasize the combination of matcha flavor and chewy texture over the cream. Before building the actual product, you can explore which descriptions communicate its appeal better and ease concerns.

The revised idea can be re-evaluated by the synthetic panel alongside the original. For example, you might compare the original with a version that adds “lightly sweetened cream” while keeping other conditions, such as price and size, the same. Here you can set a hypothesis: “Describing the cream as lightly sweetened will ease concerns about sweetness and increase purchase likelihood among consumers in their 20s.” In the re-evaluation, check whether the expected change appears not only in purchase likelihood but also in the evaluation reasons.

Through revision and re-evaluation, you can define more precisely which ideas to carry into the next concept test. In the follow-up test, you can measure purchase intent for the revised version among real consumers and see whether “lightly sweetened cream” is an important factor in their choice. You can also check whether the combination of matcha flavor and chewy texture is seen as a key appeal. The evaluation reasons point to how an idea should be revised, and lead directly to the hypotheses and questions for the next evaluation.

[Figure] By refining an idea’s description to emphasize reasons to buy or reduce reasons not to, teams can create an additional version to review alongside the original.

Concept Studio follows the same flow to select and refine candidates

Concept Studio provides rankings, bands, and group-level results designed for this screening flow. Rankings show each idea’s relative position, while the Strong, Moderate, and Weak bands show its position relative to the average of the ideas evaluated together. Teams can review Strong ideas first, then look at responses by gender and age for Moderate and Weak ideas to find additional candidates.

Selected candidates can then be refined using the evaluation reasons and compared again. Along the way, it becomes clear which features to keep, which propositions to try changing, and what to confirm with real consumers. The results table is not the end point. It becomes the starting point for preparing the next concept test.

Good idea screening does not end with picking the top few ideas. It means prioritizing the ideas that perform well overall while also finding candidates for specific consumers. Add revision and re-evaluation based on evaluation reasons, and you can define which ideas you chose, who you chose them for, and what needs to be validated next.