USABILITY TESTING ON E-COMMERCE SHOPEE BASED ON MOBILE APPLICATION WITH COGNITIVE WALKTHROUGH METHOD

An e-commerce mobile apps can be said to have an easy level of use if the level of usability is high. In addition, a visual design interface can also improve usability. Shopee leads as the number one largest online shop site in Indonesia on iPrice’s 2020 research. This study aims to measure and provide recommendation for improvements in e-commerce based on the Shopee mobile application based on the results of the usability evaluation which is reviewed by Shopee users, effectiveness, efficiency and satisfaction. After conducting usability testing on e-commerce based on Shopee’s mobile apps with the cognitive walkthrough method and the System Usability Scale (SUS), the results of the research on aspects of effectiveness are 93%, the level of speed efficiency every second is 0,08 and 72% satisfaction. This show that the Shopee application has met the usability criteria of a mobile apps because it complies with the usability criteria of the ISO 9241-11 model.. To increase the usability level of e-commerce based apps, this study designed a recommendation on the Shopee e-commerce application that changes the layout of the pages and menus, and the others is simplifies the display.


Introduction
The growth of information technology causes changes in globalization, starting from one's perspective and even its use in everyday life which is practical, fast and sophisticated due to the merger between digital technology and the internet. E-commerce is a combination of buying, selling in which marketing transactions occur between goods and services using modern electronic systems (Wong, 2010). E-commerce is a way of shopping online that uses the internet so that it can be connected and accessed anytime anywhere. The workings of the e-commerce system seek to present detailed information to introduce a product so that users are sure of the product they want to buy. This e-commerce application is also strongly supported by a complex interface design feature. The application that users are interested in is the ease of use (user friendly) so that the user experience becomes more secure. Mobile apps are said to need an evaluation if the usability level is bad because it will affect the ease of use (Wedayanti, Wirdiani, & Purnawan, 2019).
Shopee is a marketplace that is widely used by the public. This marketplace provides a search feature to make it easier for users who want to search for goods and services, filters so that the information needed can be detailed about the seller's location to the desired price, and category features that can be used as a shopping catalog and much more. In the research conducted by iPrice 2020, Shopee leads as the number one largest online store site in Indonesia.
In accordance with the explanation above, the background of the author makes it a problem in the research. The reason for considering the use of this problem is because the researcher intends to share and present the results of usability values on the Shopee application.
The results of these values can be used as criteria and parameters for the level of satisfaction, convenience and provide changes to the application from its users. The changes that may be given in this research can be used as input and development material as well as considerations so that the Shopee application can develop better, especially in terms of usability as an online- based buying and selling application.
The purpose of this study is to measure usability testing on e-commerce based on Shopee's mobile apps using the cognitive walkthrough method and provide recommendations resulting from the results of the Usability evaluation.

Method
This research will focus on usability testing using Cognitive Walkthrough, distributing SUS questionnaires, and interviews. The research procedure can be seen in Figure 1.

Research Variables
The variables in this study include three aspects of usability testing, namely the aspects of effectiveness, efficiency and satisfaction. The respondent is someone chosen by the researcher as the research subject. In usability testing, the number of users to be used as respondents is five people. 13 Temukan cara untuk menandai dan menyimpan produk yang Anda inginkan.

Interview
After the respondent works on the task scenario that has been made by the evaluator, the respondent will be asked to answer questions through an interview session. The questions that the evaluator will give to the respondents during the interview process are: a. Is there a display of feature that annoys you during the process?
b. In your opinion, what difficulties did find?
c. Did the respondent find the control easly?

Usability Testing
Usability testing is used to look for usability problems that exist in e-commerce, measuring the extent to which the level of ease of users in completing tasks on the use of e-commerce, the extent to which the level of speed required by users to find the information needed in e-commerce, and the extent to which the level of user error.

Results
To determine the respondents' criteria in this study, the main target was used, namely users who had used Shopee. Then the criteria will be taken as research samples using purposive sampling technique where the sample will be determined from direct sampling by considering the main target criteria.

Task Scenario Completion Rate
This measurement will be carried out based on how many percentage of the task scenarios the respondent has successfully completed without experiencing errors in the stages.

Amount of Time to Complete Scenario Tasks
To complete the task scenario will require a time calculation where the amount of time represents the amount of time it takes the respondent to complete the task.

User Satisfaction
SUS is a method used to assess a product by measuring the level of usability. The level of satisfaction can be measured by filling out the System Usability Scale (SUS) questionnaire.

Discussion
The usability test results show that there are scenarios of tasks completed by respondents by making mistakes. Errors can be in the form of slips, unintentional actions, errors or omissions when the user tries to complete a given task scenario (Misfud, 2015). The results obtained from the measurement of % success is 93%. A system's effectiveness is said to be effective if its % To complete the task scenario will require a time calculation where the amount of time represents the amount of time it takes the respondent to complete the task (Raharjo, Kusuma, & Sukoco, 2016). The fastest amount of time to complete the task scenario is 413 seconds and the longest time is 1264 seconds. It is concluded that the average amount of time to complete the task scenario of the respondents is 861 seconds. The result of time based efficiency calculation is 0.08 goals/second. Then the respondent can complete the task scenario as much as 8% in every second. These results can be useful to see the speed level to find the information needed for the user (Farouqi, Aknuranda, & Herlambang, 2018).
SUS is a method used to assess a product by measuring the level of usability. The level of satisfaction can be measured by filling out the System Usability Scale (SUS) questionnaire (Wardani, Darmawiguna, & Sugihartini, 2019). This data is generated from the distribution of the SUS questionnaire to five respondents as a form of measuring user satisfaction on the Shopee e-commerce application. The results obtained from the calculation of the average SUS of 72. Then the results are represented in the appropriate category according to Jeff Sauro. The percentage generated from the SUS score of 72 falls into the category C grade scale with adjective ratings "good" while the acceptability ranges fall into the "acceptable" category.
A total of five problems that arise in this study on the Shopee e-commerce application are related to what has been experienced directly and the results of interviews by respondents.
The first usability problem (MA1) is that respondents are confused about finding products based on ratings so they scroll (scroll the screen) repeatedly and search manually. The second usability problem (MA2) is that respondents are confused when making sure the selected product is available. The third usability problem (MA3) is the slow loading time of the Shopee application.
Alfarizi Alif Attamimi, Kartika Gianina Tileng / Usability Testing on E-commerce Shopee Based on Mobile Application with Cognitive Walkthrough Method

10
The fourth usability problem (MA4) is that respondents feel the Shopee application is heavier and transfer transactions using Shopee are too complicated. The fifth usability problem (MA5) is that respondents feel that the pop up ads displayed when opening the Shopee application are too annoying.
The Shopee e-commerce application has an effectiveness value of 93% which is included in the effective category and the efficiency results obtained with a value of 0.08 goals/second still need improvement based on respondents' input obtained from usability testing techniques using cognitive walkthroughs and interviews. Some of the obstacles are that respondents do not really understand the contents of the "filter" feature which results in some respondents having to search manually and the Shopee application which is too heavy. The recommendations given are in the form of improving the layout of the Shopee application and simplifying the display.