ICACT20220377 Slide.16
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Thank you very much for listening.
If there are any questions, I'm happy to discuss them.
|
|
ICACT20220377 Slide.15
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In this research, we have conducted usability testing before the sprint planning for building three features in the development of Angkasa LMS. The features are product information, pricing information and buying package. The Angkasa LMS is a moodle based LMS on top of the cloud providing a pay as the go LMS services so that users can get affordable LMS services based on their LMS usage. The aim of the test is to get feedback for these three features before the sprint begun. The user participants yield some benefits: increasing acceptance in the sprint review, validating user requirement in the user stories, increasing of users’ understanding on the requirement and of users’ participation on the Scrum project. The usability testing functions as card to facilitate conversation between product owner and his/her users thus can make better user stories and acceptance criteria. Further work is required to implement more UX practices in Agile-based projects.
|
|
ICACT20220377 Slide.14
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The sprint review, as shown in Table 2, indicates feedbacks that can be used to improve the features. The feedbacks do not reject the acceptance criteria. We argue that because the A/B testing’s participants were the usability testing’s participant, their initial understanding of the built features makes the feedback in the Sprint Review advanced. With our approach, the users’ participation shows positive result and still accordance with the Scrum by adding users’ participation before the Scrum planning . Based on Patton, as our product development is a consumer commercial, the Angkasa LMS used two practices of UX in the Scrum project: using prototype as specification and increasing user validation by adding the usability testing in the beginning of the sprint.
|
|
ICACT20220377 Slide.13
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The results show that almost all participants in the A/B testing agree that product increments in the Sprint B is better than in the Sprint A. As stated by the testing’s participants that compared to the increment in the Sprint A, the increment in the Sprint B has more information needed for the potential users (for Product and Pricing information’s increment. For the task of buying package with only small changes in the Sprint B, only one participant opted for the Sprint A. Based on the results, the usability testing can increase acceptance rates for the user stories without changing the user stories. Thus, the benefits of the usability testing are to increase users ‘acceptance in the sprint review.
|
|
ICACT20220377 Slide.12
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Based on the results of usability testing in the Sprint B, we used to improve the three features.
Task of Product information: merge product information and main feature information into product information (5 participants cannot find the product information (they navigated to main feature information) and only 1 participant can find the product information)
Task of Pricing information: same design-only added monthly charge for the package price ( 2 participants successfully find the pricing information)
Task of buying package: added monthly charge for the package price in the process of buying package (only 2 participants tried the task and all of them did not successfully complete the task because some navigations and buttons were not set properly in the maze’s tools)
As the testing in the form of prototyping testing, so that the testing can be done rapidly and can get quick feedback. This mechanism is similar to build measure learn in the lean start- up
Therefore, the second benefit of the usability testing is to use validated learning from the usability testing’s feedback to implement user stories whose user stories were validated as the results of the usability testing. From the perspective of requirement engineering, the usability testing validated user’s needs n the early software development using prototyping testing approach.
|
|
ICACT20220377 Slide.11
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The usability testing was only in Sprint B, before the Sprint B’s Planning. There were three tasks of product information, pricing information and buying package for the usability testing.
Of seven participants, only one participant whose feedback was used to update the sprint backlog for the Sprint B. We considered the feedback from this participant and did not request follow-up feedback from other participants because this participant is the VP Digital Transformation Division in a leading educational foundation in Indonesia covering from basic to higher education. His main role includes transforming digital environment in all schools and universities in the educational foundation. Based on this updated sprint backlog, the developer built updated product, pricing and buying package features for the Sprint B.
The following pictures are the results of the A/B Testing for the Sprint A (https://angkasa.rg.telkomuniversity.ac.id/clone1) and the Sprint B (https://angkasa.rg.telkomuniversity.ac.id/clone2). From the pictures, we can summarize that Sprint B was selected almost by all participants and only one participant selected buying package feature in the Sprint A.
|
|
ICACT20220377 Slide.10
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The sprint A was started from the sprint planning and proceeded until the sprint review. The sprint B was started with a sprint backlog tested by usability testing and then followed by a sprint event until the sprint review. The sprint backlog for both sprints was the same because we wanted to compare and analyse the two sprints. The sprint backlog had three features as follows:
• Product information • Pricing information • Buying package
Both sprints have the same development team consists of two UI/UX designers and one senior developer. Therefore, we expected that the feature increments had the same product quality because of the same development team.
For the spring B, 1. after initial sprint backlog definition, the product owner conducted a usability testing for three features. For the usability testing, there were modification related to above features as follows: adding three sentences in Product Information, adding two information about Pricing, and adding a Buy Now button in the landing page. The aim of these additions is to test whether more information and direct navigation will ease potential customers to decide and buy the Angkasa LMS services.
This testing aims to rapidly analyse and validate user stories based on our testing participants’ input. To gather information as the input of usability testing, we used a maze (https://t.maze.co/60719301?guerilla=true). In this research, we had total 7 testing participants (see Table 1).
Then for both sprints, these following steps were conducted.
Sprint reviews were conducted for the feature increments.
The incremented features, and user feedback were collected, compared, and analysed.
2n the sprint reviews, we conducted A/B Testing to compare and evaluate the feature increment between Sprint A dan Sprint B. The A/B testing was considered as a sprint review because in this testing, the participants were also asked to give inputs and select the better increment of the three tasks between the sprint A dan sprint B.
Of seven participants in the usability testing, only five agreed to participate in the A/B testing. We asked the participants to compare and select the best features from Sprint A (https://angkasa.rg.telkomuniversity.ac.id/clone1) and Sprint B (https://angkasa.rg.telkomuniversity.ac.id/clone2).
|
|
ICACT20220377 Slide.09
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
For the purpose of this research, two identical sprints were compared (Sprint A dan Sprint B). The aim of this comparison is to observe the use of usability testing before the sprint planning. This scrum project was part of Angkasa LMS (Learning Management System) development. The Angkasa LMS is a pay as you go LMS on top of cloud. This scrum project has focused on the order services of The Angkasa LMS.
|
|
ICACT20220377 Slide.08
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Generally, in usability testing, the participants involved in the testing need to represent the actual users. For example, we may ask them to test some of the many tasks that users will be able to do with the software product that is being developed; this case is typically applied particularly in functionally rich software and complex software product. In general, the testing can be done by recording the participant's activity while performing the usability test (i.e., completing the given tasks). The testing result is then used as input to improve the software.
|
|
ICACT20220377 Slide.07
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
To help improve the usability of a product, it is common sense that usability testing needs to be conducted in a software development process. The test should not be done just once at the end of the development process; however, it should be conducted as early as possible in the software development phases; in this case, it could be done in the requirements analysis phase. The usability testing can also be seen as an activity to improve the process by which the products are designed and developed; in this case, the objective is to avoid repeating the same problems in developing a similar product.
|
|
ICACT20220377 Slide.06
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Here is the diagram that visualizes the scrum event, scrum team and scrum artefacts.
|
|
ICACT20220377 Slide.05
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Scrum is a framework that can be applied within an organization or team to generate value through adaptive solutions to complex problems. Scrum is pretty much applied to software development projects. In Scrum, we are familiar with the terms Scrum events, Scrum team and Scrum artifacts. Scrum events consist of: (1) the Sprint, (2) sprint planning, (3) daily scrum, (4) sprint review, and (5) sprint retrospective. Scrum team consists of]: (1) developer, (2) product owner and (3) scrum master. While scrum artifacts consist of: (1) product backlog, (2) sprint backlog and (3) increment.
|
|
ICACT20220377 Slide.04
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Therefore, this research aims to facilitate the confirmation of the writing of user stories by providing techniques to test the user stories. This approach can fasten the acceptance of product increment because the acceptance criteria have been initially tested by user in before the sprint planning. Therefore, the hypothesis of this research, it is expected in the sprint review, the user will be more eager to accept the finished features’ increment because she/he has tested the product backlog using the usability testing in product backlog refinement.
|
|
ICACT20220377 Slide.03
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Even though, in Scrum, product backlog is not formal user requirements, it is a placeholder for user requirements]. A user story usually is used to express the user requirement . Jeffries argues that user stories can be effectively written to confirm the acceptance of the user stories. However, there is no definitive technique that can fasten the confirmation process.
|
|
ICACT20220377 Slide.02
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
According to The Scrum Guide, there is no specific requirement engineering techniques used in a Scrum project.
Requirement in Scrum are captured in product backlog. However, there are no specific requirement engineering techniques used to form product backlog, so that the product backlog not always the results of validated requirement.
In a sprint, a sprint backlog is implemented into working features. The validation of the features is in the sprint review. However, there is possibility that the respective users don't accept the feature in the review. Therefore, if the user can be involved earlier in the forming of acceptance criteria, the acceptance of features in the sprint review can be increased. The initial user involvement in Scrum can only take place during the product backlog refinement
|
|
ICACT20220377 Slide.01
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Good afternoon everybody, My name is Dana Sulistiyo Kusumo, I will be the presenter for this session. I am a Telkom University lecturer and researcher. This paper is a research about Angkasa LMS, A Cloud-based LMS and I am also with Nungki Selviandro,Kusuma Ayu Laksitowening, Veronikha Effendy, Monterico Adrian from The School of Computing, Telkom University. The title of our paper is An Initial Usability Testing for Improving Acceptance Criteria in A Scrum Project: An Angkasa LMS case study
|
