An evaluation of the Aptus within selected primary schools in Samoa

Int'l Conf. Foundations of Computer Science | FCS'19 | 61 An evaluation of the Aptus within selected primary schools in Samoa I.Chan Mow, E.Temese, V.Chan, F.Lafaele, M.Tanielu, I.Sinclair, E.Mauai, M.Mose, T.Patu, J.Namulauulu 1 Computing Department, National University of Samoa, Samoa Abstract - This paper documents the second phase of a study, which evaluated the suitability of the Aptus as a technology within the Samoan education context. The Aptus, a device invented by the Commonwealth of Learning enables students to access teaching resources wirelessly without the need for Internet or electricity. The research explored the acceptance and suitability of using the Aptus to access e-resources within schools in Samoa. The findings of the trial indicated very positive perceptions of students and teachers in primary schools in terms of ease of use and usefulness of the Aptus. From the results of Phase 2, a strong recommendation is made to adopt the Aptus as a technology for providing access to quality educational resources within primary schools. The study also recommends the need for training of teachers on the use of access devices such as tablets as well as the Aptus and its applications. Keywords: Aptus, Technology acceptance 1 Introduction Information Communication Technologies are seen as potentially powerful enabling tools for expanding access to education and improving the quality of education [1][3][11][13][15][16]. The dramatic growth in Information Communication and Technologies (ICT) globally have brought significant changes in the twenty-first century, as well as affected the demands of modern societies. As ICT becomes increasingly important in our daily lives and in our educational system, there is a growing awareness and growing demand on educational institutions to integrate ICT to teach the skills and knowledge students need for the 21st century [2]. Such awareness of the benefits and transformative nature of ICT in education is now also slowly but increasingly evident within small Pacific island states such as Samoa [3]. In Samoa, the integration of computer and communications technology into the school system is still in its initial stages and implemented through a variety of projects such as Schoolnet, the One Laptop per child (OLPC)1 and the CSL E-rate2 project [4]. The most successful of these projects is SchoolNet. The Samoa SchoolNet is a project funded by the Asian Development Bank (ADB) aimed at introducing computer into schools and was piloted in 2007 and expanded in 2013 [5]. While the SchoolNet project has been widely regarded as a success and provided access to computers and resources in 42 secondary schools, actual access to quality resources is still quite limited even within SchoolNet schools where technology resources are woefully underutilized. SchoolNet schools have substantial teaching resources in electronic form but can be accessed only within the SchoolNet labs. Hence, one of the key issues hindering the full realization of the potential benefits of ICT in education in Samoa is the issue of access and affordability. Access to quality teaching and learning resources through the use of technology is limited due to the high cost of technology and even in schools with technology as in the case of SchoolNet schools, access is limited to within computer laboratories [3]. Access to resources using technology is even more limited in primary schools with very few schools having access to computer technology. For example, in 2014 there were 607 computers in primary schools with a student to computer ratio of 139 to 62 [5]. Of particular challenge is access to the Internet which potentially provides a wealth of quality resources to facilitate and improve the teaching and learning process [4][5][6]. Additionally, in recent times, students in primary and secondary schools have been banned from using mobile phones in schools. This ban had stemmed from students using social media on mobile phones to incite fighting between schools in public places. It is then hoped that the use of mobile phones to access quality educational resources can demonstrate to students the more positive and constructive use of mobile phones for educational benefit and gain. OLPC project failed due to lack of training support CSL E-rate project providing Internet to several primary schools had initial success but ran out of funding 1 2 ISBN: 1-60132-497-9, CSREA Press © 62 Int'l Conf. Foundations of Computer Science | FCS'19 | One innovative solution for such challenges is a recent innovation launched by the Commonwealth of Learning in 2014 (www.col.org/Aptus). The Aptus also referred to as “classroom without walls” is a device set which allows access to digital or electronic resources in the absence of electricity or the internet [7]. Developed through the Commonwealth of Learning (COL) the Aptus enables “reaching the unreached” through the use of appropriate and affordable technologies. The COL’s Aptus device consists of two components: a minipc which acts as a server with the capability of storing large quantities of e-resources, which is used in combination with a wireless router which allows multiple learners to access the eresources stored on the server. Learners can access the eresources using laptops, tablets or mobile phones. With the dramatic growth of mobile phone usage in recent times and the ready availability of low cost tablets, access to quality eresources through the use of the Aptus provides a low cost solution to accessing the benefits of ICT. The Aptus in combination with the increasing availability of open educational resources (OERs) makes access to valuable educational resources achievable. As such the Aptus device has been recommended as suitable for remote and underserved areas in under developed and developing countries and transforms the learning and teaching experience using low cost innovative technology. The current study trialed the use of the Aptus and explored the feasibility of using the Aptus within the context of NUS and schools in Samoa to provide and improve access to eresources. The initiative was a collaborative effort by NUS, Commonwealth of Learning (COL) and the Ministry of Education Sports and Culture (MESC) to provide innovative low cost solutions to chronic teacher shortages, limited access to computers particularly in non-Schoolnet schools as well as provisioning of offline access to e-resources. COL contributed 15 sets of the Aptus and MESC provided transport and access to schools for the trial. The study was implemented in 2 phases. The first phase trialed the Aptus within the National University of Samoa. In Phase 2, the Aptus was trialed in selected primary and secondary schools within the urban area. At the end of each phase, user acceptance of the Aptus was evaluated using user perceptions on the ease of use and usefulness of the Aptus device in the provisioning of access to electronic resources. This paper is based on Phase 2 and focuses on the trial conducted within two primary schools in the urban area: Moataa Primary school and Magiagi Primary school. The research question for this study is: How acceptable is the Aptus device as a technology for providing access to e-resources? What are user perceptions on the usefulness and ease of use of the Aptus device within the context of education in Samoa? Hence the goals of this research were to explore the acceptance and suitability of using the Aptus to access eresources within the context of education in Samoa. The success of the Aptus trials for providing access to quality resources is important. First, the Aptus can be used to effectively expand access to quality educational resources for most of the student population without access to the Internet and electricity. As well, the Aptus can increase flexibility of access due to its portability by allowing access from wherever – this is the idea of the “classroom without walls”. Additionally, success of the Aptus trial is important in that it ensures that “strategic and appropriate use of ICT in education provides expanded access to quality education empowering educated citizens to influence and shape the course of national development” [3]. Ultimately, the use of the Aptus to provide quality educational resources, will contribute to achieving Goal 4 of the Sustainable Development Goals (SDGs) which is to “Ensure inclusive and quality education for all and promote lifelong learning” [17] 2 Methodology The research is quantitative and exploratory in nature and utilized the Technology Acceptance Model to conduct a multisetting case study approach. Purposive sampling was used to select the participants for this trial. In the primary school trial, due to the limits on the number of users to less than 20, classes selected were small classes within the two primary schools. The selected sample consisted of 57students from Moataa Primary and 45 students from Magiagi Primary (Table 1). Students in the sample were in the age range of 8 to 10 years in Year 4 and Year 6 classes of primary education. School Moataa Magiagi Total Class level Year4 Year6 27 30 25 20 52 50 Total 57 45 102 Table 1. Composition of Sample from the Two Primary Schools ISBN: 1-60132-497-9, CSREA Press © Int'l Conf. Foundations of Computer Science | FCS'19 | 4 ANALYSIS The survey evaluated the effectiveness of the Aptus within an educational context based on two variables: i) the ease of use and ii) usefulness. In the primary school trial, the overall reliability of the instrument using alpha Cronbach index yielded 0.83. In addition to scores on individual items two aggregate scores were generated to measure positive attitudes for ease of use and usefulness. Ease of use aggregate score was calculated by summing responses for each of the questions on ease of useQuestions B1 to B9. This means that Ease of use aggregate could take values between 9 and 45. The nine items were entered into a factor analysis using a Principal Components procedure which indicated that a single factor resolution was possible. That is, all nine items loaded strongly upon one factor. SPSS Reliability analysis was then used to check on scale properties, when all nine items are summed to one scale. The internal reliability coefficient alpha was found to be0.78, with all items contributing strongly. Ease of use aggregate was created with mean of 40, SD of 7.1, kurtosis of 0.3 and skewness of -1.04. Similarly, a Usefulness aggregate score was calculated by summing responses for each of the questions on usefulnessQuestions C1 to C11 with the aggregate score taking values between 11 and 55. The eleven items were entered into a factor analysis using a Principal Components procedure which indicated that a single factor resolution was possible. That is, all eleven items loaded strongly upon one factor. SPSS Reliability analysis was then used to check on scale properties, when all eleven items are summed to one scale. The internal reliability coefficient alpha was found to be 0.93, with all items contributing strongly. Usefulness aggregate was created with mean of49 with SD of 4.9, kurtosis of 1.7 and skewness of -1.2. 5 RESULTS 5.1 Ease of Use 10 ISBN: 1-60132-497-9, CSREA Press © 51 101 Std. Deviation 39 1 Strongly agree N 0 Mean 1.I was able agree Possible range of values (Table 2) for responses on individual items was from 1 to 5 with natural midpoint of 3 representing neutrality. Actual results indicated very positive and all above average responses ranging from 3.6 to 4.7. neutral In both primary schools, an Aptus device was setup for each of the selected trial settings and was used for teaching of Math and English. The choice of Math and English as the content area for the study was due to the emphasis in recent times on literacy and numeracy. Tablets were distributed to each of the participants as mobile phones were not allowed in schools (see earlier mention). The Aptus device was installed for use within each of the classes. Before the trial training was conducted to familiarize tutors with the use of tablets and the Aptus. Training involved the following activities. For teachers activities included i) setting up of the Aptus device ii) recharging the router, iii)uploading course notes and creating assessments. For teachers and students activities trialed were: how to navigate the Aptus Home page, iii)downloading notes from the Aptus using OwnCloud a file sharing application, iv) uploading documents to the Aptus using OwnCloud and OwnCloud to access course notes, upload student assignments and conduct online quizzes. During training, teachers also discussed with the research team what activities were planned for the 4 weeks of the trial based on the 4 activities mentioned above. These activities formed their teaching plan for the 4 weeks of the trial. Upon completion of training, staff and students trialed the Aptus for 4 weeks using activities introduced during their training. Upon completion of the trial, staff and students completed a survey evaluating user perceptions on ease of use and usefulness of the Aptus device. The survey which had both English and Samoan versions were administered by the research team with the assistance of the teachers in the selected settings. In the primary school trial, to ensure students understood the questions in the survey, the survey took the form of an interview with the teachers and the research team taking the student through each of the items in the questionnaire. The questionnaire consisted of mostly Likert type questions. In the Likert type questions responses were typically i) strongly disagree ii) disagree iii) neutral iv) agree v) strongly agree. The survey questionnaire consisted of the following sections i) Section A: Demographics ii) Section B: 11 Likert type items and 5 Yes/No items on Perceived Ease of Use iii) Section C: 11 Likert type items on Perceived usefulness iv) Section D: 2 open ended questions on problems using the Aptus and suggestions for improvement. These items have been adapted from various questionnaires of studies based on the Technology Acceptance Model [10]. Strongly disagree PROCEDURES Item 3 63 4. 0.9 Int'l Conf. Foundations of Computer Science | FCS'19 | 61 101 4. 2 1.1 6 5 11 22 57 101 4. 2 1.2 5.2 2 19 53 8 18 101 3. 2 1.0 3 15 13 25 45 101 3. 9 1.2 2 11 8 21 59 101 4. 2 1.1 4. 1 1.3 Usefulness As for ease of use, possible range of values for responses on individual items was from 1 to 5 with natural midpoint of 3 representing neutrality. Actual results on usefulness indicated very positive and all above average responses ranging from 3.7 to 4.7 (Table 3). 4 5 14 29 10 9 17 18 56 40 101 101 3. 6 1.4 1.The resources downloaded from the Aptus server were useful for my teaching/learn ing 2.The online virtual classroom Moodle was useful in facilitating my teaching and learning 3.Using the Moodle classroom on the Aptus allows me to interact with other members of Std. Deviation 13 Mean 17 Strongly agree N 10 Agree 0 9.I find the 2 8 6 18 64 101 4. 1.0 Aptus easy to 4 use Table 2: Individual Item Responses for Ease of Use The possible range for the Ease of use aggregate was from 9 to 45, with 27 as the natural midpoint, representing neutrality. However, the actual mean was 40 (SD of 7.1), which indicated, on the overall, a high level of positive attitudes towards ease of use. The scores for Ease of use ranged from 21 to 50 with 89.7% of the scores lying above the natural midpoint of 27. In short, students indicated very high levels positive perceptions in terms of ease of use. Neutral 1 Item to access content on the Aptus server up to a distance of 20 metres of the Aptus 2.Connecting my device to the Aptus was easy to do 3.Downloadin g of resources from the Aptus server to my device was relatively easy 4.Multiple users could view videos simultaneousl y 5.Home Page was simple and selfexplanatory: navigation was simple and easy to use 6.My interaction with the Aptus is clear and understandabl e 7.I find it easy to get the Aptus to do what I want it to do 8.Interacting with the Aptus does not require much mental effort Strongly Disagree 64 0 1 12 15 73 4. 6 0.7 0 10 16 19 1 5 45 14 ISBN: 1-60132-497-9, CSREA Press © 50 30 101 95 95 4. 2 3. 7 1.1 1.0 Int'l Conf. Foundations of Computer Science | FCS'19 | my class 4.The content 0 3 15 26 51 95 4. management 3 system of the Aptus device (Wordpress or Drupal) was useful for downloading content. 5. Using the 2 2 1 14 75 95 4. Aptus makes 7 teaching and learning more interesting 6.Using the 1 5 7 17 71 101 4. Aptus has 5 enabled me to learn new things 7. Using the 0 1 4 21 69 95 4. Aptus will 7 improve my work 8. Using the 0 8 8 14 65 95 4. Aptus 4 provides me with access to quality teaching/learn ing resources 9,Using the 1 4 1 27 60 93 4. Aptus will 5 enhance my effectiveness 10.Using the 1 1 6 18 69 95 4. 6 Aptus will improve my productivity 11. I find the 2 0 4 16 72 94 4. Aptus a useful 7 tool in my work Table 3: Individual Item Responses for Usefulness 0.9 65 towards ease of use. The scores for usefulness aggregate ranged from 30 to 55 with 98.9 % of the scores lying above the natural midpoint of 33. In short, students indicated very high levels positive perceptions in terms of usefulness of the Aptus. 5.3 0.8 0.9 0.6 0.9 0.8 0.7 0.8 The possible range for the usefulness aggregate was from 11 to 55, with 33 as the natural midpoint, representing neutrality. However, the actual mean was 49 (SD of 4.9), which indicated, on the overall, a high level of positive attitudes Feedback from Tutors Responses from the 4 tutors who taught the Year 4 and Year 6 classes indicated that tutor perceptions of ease of use were not as positive as students with a mean of 40 for the ease of use aggregate as compared to student mean of 49. Primary school tutor responses were also not as positive as responses from the university tutors. Although the tutors found the Aptus easy to install, they were undecided (neutral) on the performance of the Aptus. One tutor found the Aptus to overheat, videos slow to upload, multiple users could not view videos simultaneously. Tutors were enthusiastic with the use of the Aptus for their teaching as evident from their comments. “Using the Aptus is very good. I see the students has increase their motivation and interest in attending the activities by using the tablets. They wouldn’t show this kind of interest if it was a normal lesson.” “I see a lot of advantages and good change when using the Aptus. It eliminates a lot of waste when using a lot of paper and there is a lot of dust when using the blackboard.” However, the tutors pointed to the need for more training in the use of the tablets as well as more training on the use of the various features of the Aptus for their teaching. They also recommended to extend the use of the Aptus to teaching other subjects (not just Math and English). “Yes!! I strongly recommend Aptus to be used in school and also to all subjects and not just a few and with that, teachers should also be well trained to discover all functions that can be used when delivering a lesson.” “Yes! There is absolutely not enough training for me (teacher) to use the Aptus. I wasn’t comfortable to use guide the students in something I don’t know much about” 5.4 Gender & Class level effects An independent samples t test on all individual items and aggregate items to determine any gender differences achieved significance on only one item: Using the Aptus has enabled me to learn new things (t =-2.55, df = 98, p = 0.012) with mean for males = 4.28 (sd = 1.1) and mean for females = 4.73 (sd = 0.5) (Table 5). ISBN: 1-60132-497-9, CSREA Press © Downloading Year4 of resources from the Aptus server to my device was relatively easy Year6 Home Page Year4 was simple and selfexplanatory: navigation was simple and easy to use Year6 My interaction Year4 with the Aptus is clear and understandable Year6 I find it easy to Year4 get the Aptus to do what I want it to do Year6 Using the Year4 Moodle classroom on the Aptus allows me to interact with other members of my class Year6 Using the Year4 Aptus will enhance my effectiveness Year6 Easeofuse Year4 52 4.7 Year6 46 37.4 7.6 0.6 4.81 p df Table 4: Survey Items where there is significant difference between Class levels 4 and 6 T Std. Deviation Mean N Class level Int'l Conf. Foundations of Computer Science | FCS'19 | Survey Item/Varia ble 66 An independent samples t test on all individual items and 99 0.00 aggregate items to determine any significant differences between Year 4 and Year 6 students achieved significance in the following items. On the overall Year 4 students showed more positive perceptions of their Aptus experience than Year 6 students. 49 52 3.7 4.4 1.4 1.0 4.44 6 99 0.00 The results of the survey indicated very positive perceptions 49 52 3.4 4.6 1.2 0.8 3.21 99 49 52 3.9 4.5 1.3 1.0 4.28 99 49 52 3.6 3.6 1.3 1.1 4.61 93 43 52 SUMMARY AND DISCUSSION 3.9 4.7 0.9 0.7 2.13 91 41 4.3 51 42.4 0.9 5.8 3.64 95 of staff and students to the usefulness and ease of use of the Aptus. A comparison to results of the Aptus trial at the university (NUS) in Phase 1, indicated that student perceptions at both primary and secondary level were more positive. Additionally, on the overall Year 4 students showed more positive perceptions of their Aptus experience than Year 6 students. There were significant gender differences in responses to 4 items with females on average showing more 0.00 positive perceptions than males in these 4 items. Interviews with tutors indicated the Aptus increased “. student motivation and interest than a normal lesson.”. Hence tutors strongly recommended the use of the Aptus and to extend its use in teaching other subjects not just Math and English (as in the 0.00 study). Such indications point to the viability of the Aptus as a technology to be introduced into the teaching and learning environment to improve access to quality educational resources. As with the assertion made by Dennis Pack of Kiribati in Ghosh [7], the Aptus can very well be the main technology used in our schools. However, tutors also strongly 0.00 recommended the need for sufficient training in both the use of the tablets and the use of the Aptus. Tutors indicated that this was needed so that they can be comfortable in using these devices in their teaching. From the results of Phase 2, a strong recommendation is made to adopt the Aptus as a technology for providing access to quality educational resources within the context of primary and secondary schools in Samoa. As mentioned earlier, the Aptus can be used by lecturers within classrooms so students 0.04 can access course notes and other quality teaching resources made available on the Aptus without the need to be inside a computer lab or to have Internet or electricity. Due to its portability, the Aptus can also be used to access educational resources anywhere thereby extending the concept of a 0.00 traditional classroom – this is the idea of the “classroom without walls” [7]. This would transform the learning ISBN: 1-60132-497-9, CSREA Press © Int'l Conf. Foundations of Computer Science | FCS'19 | environment in addition to providing much needed access to resources, thereby contributing to achieving Goal 4 of the Sustainable Development Goals (SDGs) which is to “Ensure inclusive and quality education for all and promote lifelong learning”. In Samoa, despite the pervasiveness of mobile technology in recent times, there has been little effort to utilize mobile phones within the educational context [12]. The Aptus trials are an example of how schools can tap into the potential use of mobile technology in education. As referred to previously, the use of mobile phones in schools have been banned due to students using social media on mobile phones to incite violence leading to fighting amongst schools in public places. Hence, it is hoped that with the Aptus demonstrating the positive and beneficial uses of the use of mobile phones in schools that this will encourage educational leaders to reconsider the current ban on the use of mobile phones during school time. To conclude we make the assertion that with the success of the Aptus trials, accessing e-resources on the Aptus through mobile phone and tablets by students will be a monumental breakthrough in achieving access to technology and the wealth of electronic resources. 7 [1] [2] [3] [4] [5] [6] 67 Concerns and Perspectives IGI Global Hershey. 193-210. New York, USA. [7] Ghosh, M. (2013). MOOC M4D: An Overview and Learners viewpoint on Autumn 2013 course. International Journal of Interactive Mobile Technologies, 8(1), 46-50. [8] Blalaji, V., & Cheng, R. (2014). Field trials of Aptus Prototype. Commonwealth of Learning. [9] Connections/EdTech News. (2014, November). Commonwealth of Learning, 19, 3. Retrieved April 22, 2016, from https://www.col.org [10] Cassim, K. M., & EyonoObono, S. (2011, October). On the factors affecting the adoption of ICT for the teaching of world problems. Proceedings of the World Congress on Engineering and Computer Science WCECS. 19-21. San Francisco, USA. [11] Tinio, & L, V. (2006). CT in Education – E-primer UNDPADPIP. New York:UNPD. [12] Watson, A., J.S, S., Chan Mow, I., & Logan, S. (2017). In Cullen,R, & Hassall G.(eds) E-government in Pacific Island States Springer. Mobile phones in Pacific Island Countries: the potential for mobile government (m-government). [13] Kanwar, A., Aguti, J., & Balasubramanian, K. (2014). How ICT can support Teacher Training. Retrieved January 31, 2016, from https://hdl.handle.net/11599/757 [14] Teo, T., Su Luan, W., & Sing, C. (2008). Educational Technology & Society. A cross-cultural examination of the intention to use technology between Singaporean and Malaysian pre-service teachers: an application of the Technology Acceptance Model (TAM), 11(4), 265-280. [15] Kanwar, A., Aguti, J., & Balasubramanian, K. (2014). How ICT can support Teacher Training. Retrieved January 31, 2016, from https://hdl.handle.net/11599/757 REFERENCES Daniel, J. (2007-2008). The Virtual University for the Small States of the Commonwealth. What is your vision for the future? In Open and Distance Learning in a Changing World. Selected speeches of Sir John Daniel and Colleagues. Buabeng-Andoh, C. (2012). Factors influencing teachers’ adoption and integration of information and communication technology into teaching. International Journal of Education and Development using Information and Communication Technology (IJEDICT), 8(1), 136-155. Chan Mow, I., Kruse Vaai, E., Thomson, I., Pasiale, K., & Taupo, A. (2017). In Cullen R., & Hassall G.(eds) Egovernment in Pacific Island States Springe. ICT in Education in Small Island Developing States of the Pacific. Vaa, R. (2015). A Baseline Study On Technology-Enabled Learning In The Commonwealth Pacific Island Countries: Report, Commonwealth of Learning, Canada. Pacific Region Infrastructure Report on Economic and Social Impact of ICT in the Pacific. (2015). Pacific Region Infrastructure Facility PRIF. Retrieved August 20, 2015, from https://www.theprif.org/.../document-library/121-prif-ict-studyreport-2015 Chan Mow, I. (2010). Technology Enabled Education in Samoa: Issues, Challenges, Strategies and Recommendations. In S Mukerji& P Tripathi (eds) Cases on Interactive Technology Environments and Transnational Collaboration: [16] Whelan, R. (2008, May). Use of ICT in education in the South Pacific: findings of the Pacific eLearning Observatory Distance Education. ProQuest Education Journals, 29(1), 53. [17] UN Sustainable Development Goals. (2016). Retrieved February 6, 2017, from www.un.org/sustainabledevelopment/sustainable-developmentgoals ISBN: 1-60132-497-9, CSREA Press ©