ICACT20220197 Slide.20
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Thank you, if there are any question feel free to ask us.
|
|
ICACT20220197 Slide.19
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is our references for this paper.
|
|
ICACT20220197 Slide.18
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The conclusions from this research are :
(1) Process mining can be used to analyse the teaching/ learning activities carried out by lecturers and students during a semester
(2) Process mining can also be used to analyse the sequence of activities during the teaching/ learning process. We analysed the durations of lecturer activities and found that the durations ranged from a few days up to five months. We also analysed the first and last activities carried out by the lecturers and students. The most frequent first activities of the lecturers are the Assignment, Forum, and Quiz; while the most frequent first activities of the students are Forum, Resource, and URL.
(3) The findings of the study experiments can be used to improve understanding on how the process has been carried out and suggest improvements. Based on the findings of the most frequent activities carried out by lecturers and students, we can suggest the University to promote other functionalities to be used by the lecturers and students. For example, the lecturers should be introduced to the LMS functionalities other than assignments, forums, and quizzes.
This study can potentially be reproduced and further developed to implement process mining in a larger scope of study. Potential future work is to apply this approach to analyse and compare teaching/ learning process in several study program; to improve data processing approach by generating ontology mapping on the activities in the process; and to recommend improvements in the teaching/ learning process using the LMS.
|
|
ICACT20220197 Slide.17
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Based on the 1st question result, the first activity that students most accessed is Forum (25%) then followed by resource file (25%), quiz (19%), assignment (17%), H5P (10%) and URL (3%). Both result between the most first student activity describe in process model (figure 5 and figure 6) and first activity from questionnaire result (figure 8) is forum. So between first activity in process model and the evaluation is match.
The result from 2nd question about the activities that access at the end of semester is assignment (30%), then followed by quiz (27%), resource file (19%), H5P (10%), forum (8%), and URL (6%). From the result of questionnaire that assignment is the most activity was accessed by student at the end of the semester. The questionnaire result match to process model from student questionnaire.
|
|
ICACT20220197 Slide.16
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Evaluation was done through interview and questionnaire to lecturer and student. The evaluation was especially done by questionnaire to 34 students¡¯ respondent and interview to five lecturers. Questionnaire containing questions about what their activities at the beginning and the end of their class for each semester. Students can choose more than one activity for each question from the questionnaire. The lecturer respondents interviewed about whether the process model is compatible with their activities in the LMS and how they start and ending their semester.
From interview to five lecturers, they agree with the process model was developed use process mining. The lectures initiate their lecturing activity in LMS with forum. In their first post at the forum, they can inform the student how they run the class. For the example at their first post in the forum, the lecturer usually publishes their email or video conference link so the student can contact their class lecture at the time. And the two other lecture they use quiz or assignment at the beginning of the semester. Quiz or assignment containing pre-test. The pre-test is used to know the student knowledge about each topic. So, at the class meeting using video conference the lecture can initiate the material subject based on the student pre-test result.
At the end of semester, the lecturer agree that they used assignment and quiz to take final score at that semester. And forum used to know the student¡¯s opinion about student learning experience in that semester. The last forum is usually used to discuss about every topic in the class meet or the other topic.
|
|
ICACT20220197 Slide.15
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The deeper analysis for student was done by checking the first and the last activities done by students.
The top three most frequent first activities are Forum (38%), Resource (27%), and URL (20%).
This finding reflects the reality from questionnaire to students, where most students accessed Forum, Resource, or URL in the beginning of their learning activities in the LMS.
Other activities are infrequent (less than 10%), which are: Assignment, Quiz, H5P, Page, Chat, Glossary, and Book.
The top three most frequent last activities are Assignment (39%), Quiz (36%), and Forum (10%).
This finding reflects the reality where most students accessed Assignment, Quiz, or Forum in the end of their learning activities during the semester.
It also related to the fact that most lecturers give assignments or quizzes to assess student understanding in the end of semester, or forum to discuss further topics in the learning activities.
Other activities are infrequent (less than 10%), which are Resource, URL, Page, H5P, Folder, and Feedback.
|
|
ICACT20220197 Slide.14
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
More detailed analysis was done by examining the trace variant diagram. The figure shows the top 8 most frequent trace variants covers 488 out of 3,994 traces (12.22%). Each row in a trace variant diagram represents a trace variant. The rows in the trace variant diagram are ordered from the most frequent variant to the least frequent one. Each coloured shape represents an activity. The student process model show that the top four variants consist of one activity each. There are 186 students (4.66%) only used Forum, 74 students (1.85%) only used URL, 64 students (1.6%) only used Quiz, and 50 students (1.25%) only used Resource.
|
|
ICACT20220197 Slide.13
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In the student process model, there are fifteen activity and resource that did by student in one semester.
the top six student activities are resource, assignment, URL, forum, quiz, and H5P that percentage is more than one percent .
From the process model we can see, First student activity in the beginning of the semester is forum.
In forum activity at the beginning of semester there are post about class information like the contact of the lecturer, the conference link or other class information.
The most learning material that student accesses is resource, resource contain lecture slide, book file, and other course material that can student use in one semester.
The second popular activity is assignment. Assignment is using to collect the student homework and get test or exam.
The URL is the third popular activity. URL usually contain conference link or learning material from outside LMS.
In the end of semester, there are quiz or assignment that student did.
Quiz and assignment usually used to collect the grade of final exam for the semester.
|
|
ICACT20220197 Slide.12
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Analysis was done by checking the first and the last activities of lecturer traces.
The most frequent first activities are Forum (81%), Assignment (15%), or Quiz (4%).
This finding reflects the reality based on interview to the lecturers, they are started using the LMS during a semester by a forum, assignment, or quiz.
The most frequent last activities are Assignment (65%), Forum (23.1%), or Quiz (11.5%).
This finding also reflects the reality where lecturers are mostly used Assignment, Forum, or Quiz to end their teaching during the semester.
|
|
ICACT20220197 Slide.11
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
More detailed analysis for the lecturer data log also can be done by examining the dotted chart.
Dotted chart shown the most frequent activity is Assignment pointed by light blue dots.
This reflects the fact that most lecturers use LMS mostly for managing assignments for the students.
The dotted chart also shows duration ranges from a few days up to nearly 5 months.
This reflects the real situation where some lecturers have used LMS much more that the other lecturers.
|
|
ICACT20220197 Slide.10
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is lecture proses model.
The lecture process model also shows that lecturers performed several activities, with the three most frequent activities being Assignment, Quiz, and Forum that percentage is more than one percent.
It is also shown in the table that Chat and Data are rarely used by the lecturers during their teaching activities through the LMS.
This finding suggests potential recommendation to promote Chat and Data to be used more often by the lecturers.
Forum is the first activity that lecturer did at the beginning of the semester. Forum contain lecturer contact and class information. Usually lecture post about the information to their student using forum post.
The last activity in the semester is assignment. Assignment is use to collect student grade for the final exam.
|
|
ICACT20220197 Slide.09
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is the main part of the study and consisting of process discovery, conformance checking, dan analysis steps. Process discovery was done using the heuristics miner on ProM tools.
In ProM, we used the interactive Data-aware Heuristics Miner (iDHM) plugin that allows interactive settings including several options of graphs to present the resulted process models.
The process model discovered using heuristics miner was then checked for its conformance to the traces in the event log, and vice versa. Conformance checking was done by measuring fitness, precision, and generalization.
|
|
ICACT20220197 Slide.08
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In student data log there are case id, activity, and event name attribute.
For one activity there are more than one event name.
Example quiz have more than one event name like attempt viewed, attempt submitted and course module viewed.
That is called redundant.
To minimize the redundancy we must choose event name that exactly represent one activity.
Choosing process base on the table mapping.
Table mapping that contain activity attribute, exactly one sub activity attribute that represent the activity and the description attribute.
Example : quiz represent by attempt submitted.
This process make one quiz activity to one ¡°event_name¡± that is ¡°attempt_submitted¡±.
|
|
ICACT20220197 Slide.07
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Data pre-processing is carried out to ensure that the available event logs are ready to be processed with the heuristic miner algorithm. In this case data processing is to minimize redundant data log. There are two data processing step has done to lecturer and student data log.
1st Lecture data processing
The first step is filtering the attribute namely CRUD (create-read-update-delete) by selecting records that have C and U value. This step based on LMS behavior.
At the time the lecturer adds and changes learning material like activities or resources LMS will add record to CRUD attribute with C or U value.
R value for Read or Viewed the learning material. D value for Delete learning material. So we can ignore this both value
After filtering CRUD attribute
Then we did the merge subsequent events process assisted by Pro M tools using the merge subsequent events plugin to ensure that there are no redundant records.
2nd Student data processing
We develop table mapping to mapping event name attribute to activity attribute in student data log.
This table can filter activity value with their event name value that chosen from description each sub activity attribute.
This process also to minimize redundant student log data.
|
|
ICACT20220197 Slide.06
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is sample event log from extraction phase. There are nine attribute to use in process mining.
To use this event log to process mining we must mapping the event log attribute to correct process mining attribute. Import column mapping there are
• 'userId¡¯ tobe Case ID
• 'component' tobe Activity
• 'time' tobe Timestamp (Pattern: 'yyyy/MM/dd HH:mm:ss')
• 'Role' tobe Activity
• Other attributes: 'courseId', 'contextid', 'crud', 'target', 'eventName¡¯ that we use for further filtering.
|
|
ICACT20220197 Slide.05
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Event log obtained from moodle table that are logstore_standard_log, course_category, and user. We use mysql workbench to process that table. The initial, event log obtained is the event log of the entire study program from the university, then event log was grouped based on the existing study program. To get the event log for each study program, the logstore_standard_log table is matched with the course and course_category tables in the courseid attribute. After getting the event log for each study program, then separating the records based on the lecturer and student users, which is done by matching the logstore_standard_log table with the user table on the userid attribute.
After getting the separated event logs of student and lecturer, then filtering process is carried out based on component attributes. This filtering process sorts out activities and resources such as assignments, quizzes, forums, chat, active quizzes, and feedback. Resources in the form of resources, URL, H5P, folders, pages, glossaries, and books. The filtering process is done using the DISCO application from Fluxicon and ProM tools to analyse the event log using process mining.
|
|
ICACT20220197 Slide.04
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The scope of this research is the teaching and learning activities of lecturers and students in the study program 365 during the first semester of 2020/2021, as recorded in the CeLOE LMS.
The main research question is ¡°How is the learning process of students and lecturers at CeLOE LMS?¡± and is divided into:
1st question is How can we use process mining approach to analyse teaching and learning activities in an LMS?
2nd question is What is the most common sequence of activities in the teaching and learning process?
3rd question is What can we learn from analysing the teaching and learning activities of lecturers and students that have been carried out for one semester in the study program 365?
The research team consists of computer scientists as the process mining experts who are also the authors of this paper and the CeLOE team as the domain experts and the process owners.
|
|
ICACT20220197 Slide.03
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is our research methodology in this paper. There are five phase from planning until evaluation.
In Planning phase we develop the research question, scope of the research and building the research team. The main research question is How is the learning process of students and lecturers at CeLOE LMS.
Second phase is extraction, we extracted the CeLOE LMS event log obtained from the moodle table. We extract log lecturer and student at LMS CeLOE at 1st semester in 2020 until 2021. There are 206 two hundred six event from lecturer and 178.367 one hundred seventy eight thousand three hundred sixty seven event from student.
Third phase is data processing, we divided lecturers and student activities to know the both behavior activities in LMS. Data pre-processing is carried out to ensure that the available event logs are ready to be processed with the heuristic miner algorithm. We undertake a role-based data processing to separately analyse lecturer and student records.
Phase forth is Mining an analysis. This is the main part of the study and consisting of process discovery, conformance checking, dan analysis steps. Process discovery was done using the heuristics miner on Pro M tools. Heuristics miner was chosen in this study because this algorithm is one of the most used algorithms for process discovery that can deal with noise and can be used to express the main behaviour recorded in an event log. In Pro M, we used the interactive Data-aware Heuristics Miner plugin that allows interactive settings including several options of graphs to present the resulted process models.
And the last phase is evaluation. Evaluation was done through an intensive discussion with the CeLOE team and further interview with representative of lecturers and questionnaire students to verify the findings.
|
|
ICACT20220197 Slide.02
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In this modern era, every organization strives to support its business processes with computer-based systems. Universities are no exception; more and more are implementing computer-based learning management systems (LMS). LMS allows universities to manage learning activities carried out by lecturers and students, in the form of material delivery, assignments, quizzes, exams, etc. One of the advantages of implementing LMS in universities is the automatic creation of an event log that is recorded by the LMS. The event log records step-by-step of the LMS usage by lecturers and students in the learning process.
The event log is useful to support process analysis. Process analysis allows understanding of business processes by analysing each activity and the activity sequence, identification of the most common activity sequence carried out by lecturers and students during the learning process, identification of exceptional sequence in certain groups of lecturers and students, and their conformity to learning process implementation guidelines at the university.
Telkom University has its own learning management system called CeLOE which was developed based on the open-source Moodle application. In this study, process mining is used to analyse the event log of CeLOE LMS, an LMS that is used at Telkom University. The case study in this research is learning in the study program 365 during the first semester of 2020 until 2021.
There are 1,754,444 events from the moodle log table.
|
|
ICACT20220197 Slide.01
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Dear all, my name is Gede Agung Ary Wisudiawan from Telkom University Indonesia. For ICACT 2022, I have cooperated with my partners Dr. Angelina Prima Kurniati. Our presentation today is about How is the learning process of students and lecturers at CeLOE LMS? How we know the learning and teaching activities in our learning management system using process mining. So we know what common sequence of activities in the teaching and learning process in one semester in program study. Data that we use in this paper based on our university Learning Management System.
|
