Making the Right Choice: Comparing Research Methods for Banner Maker

For my research on the usability of the product configurator Banner Maker, I have identified three potential research methods that would allow me to conduct this investigation: usability testing, heuristic evaluation, and survey research. In this blog, I will discuss the strengths and limitations of each method.

Usability Testing

Usability testing is a research method that involves recruiting participants to perform tasks using a product or system to see how easily and effectively they can complete the tasks and identify any issues or problems they encounter (Dumas and Redish, 1999).

In the context of my research question, usability testing would involve creating a list of tasks that users might perform with a product configurator. The list of tasks could include customising the banner size or uploading artwork, and I could ask participants to complete these tasks using each configurator. The participants’ actions would be observed and recorded, and the data collected would be used to identify usability issues or problems.

There are several benefits to using usability testing for my research. By observing real users, I can identify specific issues with the product and understand how users interact with it – whereas other research methods rely on self-reported data, leading to biased results (Nielsen, 2019). An additional benefit of usability testing is that it can provide qualitative insights and quantitative metrics (Moran, 2019), making it a versatile method. Such metrics could include task success and time spent on the task (Moran, 2019).

However, there are some drawbacks that I am considering. Time consumption and resource intensiveness are two examples of drawbacks that can easily lead to high costs (Grozny, 2020). Costs include the recruitment of participants, user testing lab and time spent by the internal staff (Grozny, 2020). The cost aspect of this research method is essential to me because the research we conduct must be something Simply Signs can afford. The high price tag could be out of scope for smaller companies (Grozny, 2020).

To mitigate the costs,  I can reduce costs by using remote usability software instead of hiring a usability testing lab and limiting the number of participants.

Heuristic evaluation

Another research method that I am considering is heuristic evaluation. Heuristic evaluation is a process in which experts use established rules of thumb to assess the usability of user interfaces and identify issues (Interaction Design Foundation, 2019).

From my own experience, heuristic evaluations are excellent for small research budgets. Anyone with enough expertise in UI can pick out issues such as a bad UI structure, missing elements and bugs (Nielsen, 1995).

Unlike usability testing, a heuristic evaluation does not require recruiting participants or setting up a test environment, which can make it cost-effective and more efficient (Nielsen, 1995).

I will need an established set of heuristics to perform a heuristic evaluation. Nielsen’s 10 principles for interaction design are a reliable and widely accepted set of guidelines (Langmajer, 2020) that would be well-suited for conducting the heuristic evaluation.

They are flexible enough to be applied to a wide range of products, including product configurators. However, it is essential to note that heuristic evaluation can be subjective as it relies on the expertise of the evaluators (Nielsen, 1995). It is the best practice to use a team of at least three evaluators, who should have both UI and domain-level expertise (Interaction Design Foundation, 2019).

To conduct a heuristic evaluation for my research question, I will select a team of experts with experience in product configurators and UI design. I will provide them with the product configurators to be studied (Banner Maker and the competitors) and the heuristics to be used (Nielsen’s 10 principles).

The experts will then independently assess the usability of each configurator and identify any issues or problems. I can compare and analyse results to understand the strengths and weaknesses of each configurator in terms of usability.

Survey Research

The third research method I am considering is survey research. Surveys involve collecting data from a sample of people through self-reported questions (Zak, 2020).

In the context of my research question, survey research could include creating a survey that asks users about their experience using different product configurators. The survey could have questions about the user’s overall satisfaction with the product, ease of use, and any issues or problems they encountered.

One benefit of survey research is that it is relatively cheap and efficient to conduct (Anand, 2022). Surveys can be easily distributed online or through other methods, such as mail or phone, making it easy to reach many participants.

There are some limitations to consider. One limitation is that survey research relies on self-reported data, which can be biased (Anand, 2022). Furthermore, survey responses may not accurately reflect the user’s experience or behaviour (Anand, 2022).

To ensure the validity of the survey research, I will need to ensure that the questions I ask are not biased or leading (Anand, 2022).

Concluding thoughts

In conclusion, all three research methods have their strengths and limitations. Usability testing offers the ability to observe user behaviour, identify specific issues, and provide qualitative and quantitative data. However, it can be time-consuming and costly. Heuristic evaluation is cost-effective and efficient but subjective and relies on the evaluators’ expertise. Survey research is quick and affordable but relies on self-reported data and can be biased.

It may be helpful for me to use a combination of the outlined research methods, such as usability testing and heuristic evaluation, to triangulate the data and provide a more comprehensive understanding of user experiences with product configurators (Whitenton, 2021).

After considering the pros and cons of each method, usability testing is the best option for my research question. Its ability to provide excellent qualitative and quantitative data and observe user behaviour makes it the most comprehensive method for evaluating product configurators’ usability and user experience.

In the next blog, I will show you how I will plan the usability testing for the research on product configurators.