There are still doubts and disparagement about quantitative and qualitative tests. While some people say quantitative data is more reliable, other people prefer qualitative insights.
Most of these are due to misconceptions about what they do. Most of the developers, project managers, and QAs rely more on quantitative tests ignoring the “human experience”. But, are qualitative data insights better than quantitative information?
Let’s clear all superstitions about these tests while we get more in-depth on how they enhance users’ experience.
Quantitative Tests: The Most Reliable?
Some project managers and analysts prefer to rely on data alone. That is the reason why quantitative tests are more used than qualitative tests for research purposes. However, when you try to figure out what is on users’ minds you need qualitative tests for sure.
Makes perfect sense to assume that “2+2=4” is more reliable than “some even number”. However, abstract elements and subjective elements have value for analysts and marketers. It is for sure that it is easier to read the “2+2=4” kind of information.
For now, let’s focus on how quantitative tests can improve UX. That is the main reason you are reading this right now. It is better to start by looking at what type of test you can run, and later I will tell you how it will improve UX.
Analytics
Web analytics and App analytics should be the first place to look in the search for quantitative data. So analytics is the kind of information that tells how good the performance is.
With the proper lines of code inside your app or your website, you can have valuable information. This information will let you see what the average users’ behavior is.
For the sake of the users’ experience improvement, analytics deliver clear and ordered data.
By taking a look at it, analytics experts can have a deep insight into what a user might do. As you know, trends are nothing but the path followed by too many people.
So using the insight provided from analytics you can improve your website in many ways. You can take action to avoid the bounce rate or delete a redundant piece of the website.
Even simple data like the loading time of the website can tell you what to improve.
It is good to keep a history of the data obtained to see how good the improvement was. It is not necessary to engage in deep research of the data.
The best you can do is focus on a maximum of 5 data at the same time. The ideal is keeping track of one at a time, but sometimes you must speed the process.
Split Testing
This is better known as “A/B Testing” or also “Multivariate Testing” depending on the compared items. This test is mostly used for comparative purposes. However, it is more complex than just a comparative testing tool.
Gazing into the average user’s mind is sometimes a matter of gathering all possible information.
This type of testing is intended for the selection of the best-performed website or app. Nevertheless, it would be a pity to leave out of the selection all the things that worked well.
In that sense, after making the right evaluation, go to a deeply granular level and see what went right. Then, change the testing subjects and go and see how similar each result is. Remember that the goal is to find out which version works best for the user.
Even the smallest changes may throw different results because users “feel” the difference. They might not know how to tell what it is they liked more from one version. That is exactly the purpose of split testing, to select for them what they can not ask for lack of knowledge.
How many times do you think it is necessary to perform split testing? One? Two? Ten times? Every time you feel the need to compare different versions, you need to perform A/B testing. The main goal is to let users get what they want in the most accurate possible way.
Benchmarking
It is good to see your project growing, but from time to time it becomes handy to look outside. That is exactly what benchmarking testing is about: comparing your performance with others.
If your goal is to be on top of your business, compare your product useability with the top competitor’s product. Otherwise, your company will face problems that top companies have already solved. It makes sense, right? comparing your product with the best there is.
Regardless of how beneficial it may be, benchmarking takes more time than split testing. It is necessary to study competitors’ products with your product for about 90 days to get a good insight. Make sure to start this evaluation ASAP to make the most of the evaluation time.
From time to time it is good to see how well other companies (besides top companies) are doing. Then, do not compare their overall performance, but pinpoint what metrics are worth comparing. The number of visitors, conversions, and other important metrics.
In benchmarking, comparisons are key to success as well as knowing what to compare. Do it quickly, accurately, and as many times as you need.
Card Sorting
This is an inventive and resourceful method for users’ experience research. The method itself is an effective way to label and categorize information for people to understand it. In other words, the method reorganizes information to make it easier to digest for users.
Card sorting is not meant to solve all kinds of issues. It is meant to work under specific circumstances. The first circumstance is when you wish the design and the content to be almost intuitive for users.
On the other hand, the second circumstance is when you wish to find out how people understand information.
The goal is to make accurate assumptions about how people understand what you show. This is perfect to adapt the information to a certain demographic targeted audience.
At this point of the method, you might be wondering how many cards are good to perform card sorting. A hundred? Two hundred? Ten? Okay, the decision is entirely yours, but the usual recommendation is to use from 30 to 60 cards. These must be enough topics to sort.
Remember to ask the team for their opinion. You can go beyond words by putting images on some cards. Images are a real game-changer on this method. The ideal participants of card sorting should be the desired demographic, not just anybody.
By the end, you’ll have a stronger information architecture which leads to more engagement. There will be no better information than the one designed especially for their consumer.
What More?
There are other quantitative tests like Surveys or Tree Testing, but the goal is not to cover all. There should be an article later on this website responding to how to perform each test separately.
The strongest side of quantitative tests is how finite the data collected from each test is. Quantitative data is objective and clear and sometimes is more reliable than qualitative data. Speaking of which…
Qualitative Tests
While quantitative tests record data, qualitative tests record experiences and opinions. In other words, quantitative count items and qualitative data tell the story of how you find them. They are both important to get an optimized UX, but let us focus on stories for the moment.
Since the interface is the user’s environment, you can’t rely only on data to fully understand a customer. It is necessary to watch the user’s actions and reactions to your interface recording them and their impressions.
The following are the two qualitative tests that most companies apply to their test subject. All the information gathered there are more opinions than objective data. Nevertheless, both test types might give you a better insight into what people may want.
In-Person Moderated
The resources needed for this type of test are nothing special. You’ll need a camera and the prototype that will be about to be tested. However, gathering the information, moving the people, and gathering the infrastructure make the test more expensive.
This method of testing is preferred due to the number of observations all moderators can point to. All body language can give more sincere information than analytic data. Everything from a raised eyebrow to a deep breath is worth it for the sake of UX improvement.
Moderators are there not only for giving instructions but to ask questions whenever they feel they need to. When a confusing gesture confuses the moderator, he should ask the test subject’s thoughts. There should not be hesitation in asking what they feel or think.
In-person moderated user testing should be the best quality testing, but it does have a flaw. Since you must move the testing subjects to a facility with the minimum conditions the problem is gathering people.
When they can’t move to the testing location the test must be postponed. So during the pandemic, it is less likely to be the one of choice.
Aside from the physical limitations and the high costs, it is the best test in terms of “reading people’s minds”. An empathetic moderator is the best asset you have in this testing method to make the most of it.
Remote Moderated
This qualitative test solves the issue with the moving of the test subjects. Instead, the infrastructure must change to provide all needed elements to read persons. You’ll need a real-time connection tool like Skype or similar to see what test subjects do.
You also need screen recording tools like Camtasia Studio or ScreenFlow. It is very important to analyze the videos later for a better understanding of the user’s needs.
Additionally, you will need an editing tool for emphasizing a part of the recorded video for notes.
After setting all in place, the participant and the moderator must connect to the same online conversation. The moderator must make sure the two-way communication software is fully functional as well as the other tools.
During the test, the moderator can do the same questions as in the previous method. After the test, the moderator must label and save the recorded video for later analysis.
The not-so-bright side of this test is that body language reading is limited. Also, all control you have in the focus and sound clarity is lost in the natural environment of the testing subjects. Everything from a window to a doorbell will be a challenge for their focusing.
Even hiring a more complete system connection tool like validately.com or similar won’t help without the right focus. Paid tools like usertesting.com will help with the platform and communication issues, but not with the test environment.
Additionally: Heatmaps
A heatmap is a tool that follows the actions during the test to track most of the prototype usage. It records and analyzes the mouse movements and with the proper gear the eyes movement. But it does not only record movement.
The tool shows an image of where most of the action took place during the tests. It allows easier reading of the test and therefore a faster reading of the user’s intention. It is a better way to put together all the insight from the user’s experience in the website or app.
Mouse movement is better to understand what the user clicked and where he was about to click. By knowing where and what the user clicks you can see what he was intended to achieve. So whether he wanted to stay or leave, the heatmap will tell.
Eye tracking, on the other hand, shows how much time the user stares at a button or an image. Even by following the path of the eye movement, you can see where his attention went pixel by pixel.
In Summary
Just by gazing into people’s thoughts, you can adapt your content to their likes bringing a better user experience. So all you need is to decide which tool you would like to use and what demographics to target.
Regardless of the information that comes from the quantitative or qualitative test, try to think about how to help users. Also, try to be as empathetic as possible to be in their shoes before adjusting your website.
Users are the cornerstone of digital products. By improving their experience on your website your brand gains engagement.
Having loyal fans is key for a brand to grow and it is key to success. Now that you know how important it is to UX, go and improve your website the best you can.