In recent years, the application of digital technologies for learning purposes is increasingly discussed as smartphones have become an integral part of students’ everyday life. These technologies are particularly promising in the so-called “transition-in” phase of the student lifecycle when first-year students start to develop a student identity and integrate into the university environment. At that stage, most premature dropouts are observed, presumably due to a lack of self-organization or self-responsibility. Considering this, a mobile app to tackle insufficient student experiences, support learning strategies, and foster self-organization in the “transition-in” phase was developed. The research at hand proposes a generalizable success model for mobile apps with a focus on first-year students, which is based on the IS success model (Delone and McLean in Inf Syst Res 3(1):60–95, 1992) and analyzes those factors that influence student satisfaction with such an app, the intention to reuse the app, and—foremost—students’ learning effectiveness. The results indicate that learning effectiveness is determined both by the perceived user satisfaction and users’ intention to reuse, which are particularly influenced by perceived enjoyment but also system and information quality. Finally, design principles are derived to develop similar mobile solutions.
Avoid common mistakes on your manuscript.
For some time now, the European labor market is facing a severe lack of skilled professionals (Peichl et al. 2022). In 2022 alone, 29% of companies in the European Union (EU) reported problems in finding suitable personnel, which is an all-time high considering the development in recent decades (Peichl et al. 2022). The situation is agitated in Germany, with 50% of enterprises seriously suffering from the shortage of specialists (ifo Institut 2022; Peichl et al. 2022). Consequently, more than 770,000 vacant positions for the entire economy will not be adequately occupied in 2023 (cf. Statista 2023). In this context, a high student dropout rate is seen as a serious problem in meeting the economy’s demand for qualified workers in the upcoming years (cf. Ahlers and Quispe Villalobos 2022; Behr et al. 2021; Heublein 2014). While in Germany, 14.7% of Bachelor students do not finish their studies, this number is even higher in other EU countries like the Netherlands (28.3%) or Italy (34.1%) (Behr et al. 2020; Schnepf 2014). For education policies, such student dropout rates imply not only inefficiently used resources for higher education but also high educational costs for students not achieving the aspired educational goals (Baars and Arnold 2014; Behr et al. 2021). At the same time, dissatisfaction and negative psychological long-term effects are observed for corresponding students, which may paralyze them when searching for alternative pathways to gain a foothold in the labor market (Behr et al. 2021; Ibrahim et al. 2013; Roso-Bas et al. 2016).
In terms of time, most student dropouts happen during the first year of studies (Isleib et al. 2019; Neugebauer et al. 2019; Opazo et al. 2021). According to the student lifecycle (Lizzio 2011), which describes the evolution from a prospective to a commencing, continuing, and finally graduating student, first-year students find themselves in the “transition-in” phase. In this phase, self-organized research-based learning (cf. Huber et al. 2009) or self-responsibility, which are essential for a successful transition from the highly-structured school environment into the university system, are often perceived as challenging (cf. Zehetmeier et al. 2014). Moreover, an inadequate student experience and psychological factors—such as inefficient learning strategies or insufficient intrinsic motivation—are identified as additional reasons for early dropout (cf. Blüthmann et al. 2011; Heinze 2018; Neugebauer et al. 2019). Therefore, “remedial support early in the curriculum” (Baars and Arnold 2014, p. 106) is necessary to reach students who are at risk of dropping out prematurely.
Parallel to this, universities also experience a change in students’ way of consuming and processing information, organizing their daily routines, socializing or communicating with one another (Musik and Bogner 2019), which is mainly triggered by technological progress (Cho et al. 2021; Gómez-Galán et al. 2020; Gupta et al. 2021; Youssef et al. 2021). Consequently, the impact of new technologies on students’ learning behaviors or interactions with lecturers is increasingly discussed in higher education (Ronzhina et al. 2021; Sultana 2020). By now, it is widely recognized that digital technologies may support behaviorist, constructivist, collaborative, situated, and informal/lifelong learning (e.g., Criollo-C et al. 2021; Goksu 2021; Gupta et al. 2021). Thus, in the recent past, special attention was given to learning management systems (LMS), which are “web-based software platforms that provide an interactive online learning environment and automate the administration, organization, delivery, and reporting of educational content and learner outcomes” (Turnbull et al. 2020, p. 1). The functionalities of today’s open-source (e.g., Moodle) or proprietary LMS solutions (e.g., Blackboard, WebCT of the University of Columbia) are diverse and range from course management to communication tools and progress tracking abilities amongst others (Al-Sharhan et al. 2020; Koh and Kan 2021). Although several studies have shown a positive effect of LMS usage on students’ learning performance (e.g., Leontyeva 2018; Msomi and Bansilal 2019; Oguguo et al. 2021), there are concerns that the new generation of “information consumers”—who are now entering the university system—will refrain from using LMS if these systems have not been optimized for mobile devices or just serve the provision of course materials (cf. Koh and Kan 2021; Turnbull et al. 2020).
As a consequence, higher education gradually focuses on the ubiquity and great acceptance of mobile phones (cf. Al-Bashayreh et al. 2022; Author self-citation 2; Beatson et al. 2020), which have become an integral part of students’ daily lives to establish and maintain social networks (Criollo-C et al. 2021; Diacopoulos and Crompton 2020; Goksu 2021). The COVID-19 pandemic even accelerated this development, as higher education was challenged to find alternative teaching options and students primarily interacted electronically with fellow students and instructors (e.g., Al-Bashayreh et al. 2022). Several studies show a positive effect of smartphone-based learning on student performance for various courses (e.g., accounting, psychology, etc.; cf. Beatson et al. 2020; Diliberto-Macaluso and Hughes 2016; Voshaar et al. 2023). In particular, research outlines the supportive impact of gamification on learning effectiveness (Pechenkina et al. 2017; Voshaar et al. 2023).
Against this backdrop, this research addresses the necessity for abovementioned “remedial support” (Baars and Arnold 2014, p. 106) at initial stages of the student lifecycle to prevent first-year students from dropping out early and considers their affinity towards mobile phones equally. So, we focus on designing a mobile app for first-year students who are just about to develop a student identity and integrate into the student world (Lizzio 2011; Matheson 2018; Msomi and Bansilal 2019). We claim that a mobile app may be a suitable solution to tackle insufficient student experiences and support learning strategies as well as self-organization in the “transition-in” phase of the student lifecycle. We built a corresponding mobile app using a Design Science Research (DSR) approach, and the results of a first evaluation at the University of Bremen (Germany) encouraged us to pursue the project and develop the app further (cf. Johannsen et al. 2021). Further, in this research, a success model for mobile apps for first-year students in the “transition-in” phase, which is based on the IS success model of Delone and McLean (2003), is proposed and those factors that influence student satisfaction with the app, the intention to reuse the app, and students’ learning effectiveness are analyzed. This prepares the ground for formulating design principles for mobile app development afterwards. Accordingly, we pose the following research questions:
The contributions of this research are threefold: First, a self-developed mobile learning app with the aim to support students in the “transition-in” phase by improving learning strategies and self-organization abilities as well as promoting the perceived student experience is introduced. Thereby, we contribute to the ongoing search for (technological) solutions to prevent early student dropouts. Second, factors positively affecting user satisfaction, intention to reuse the app, and students’ learning effectiveness are identified with the help of a success model for mobile apps and data collected in an introductory accounting course at the University of Bremen (Germany). Based on that, mobile app functionalities can be assessed more purposefully regarding their relevance for first-year students, which complements the existing body of knowledge regarding student app design (e.g., Almaiah et al. 2022; Laine and Lindberg 2020). Third, the findings are used to formulate design principles (cf. Gregor and Hevner 2013) for mobile apps to support students in the “transition-in” phase, which are largely missing for apps that focus on this particular stage of the student lifecycle yet. Other institutions may reference these propositions to create beneficial mobile solutions for first-year students who strive to adapt to the university environment.
The structure of this paper is as follows: Section 2 provides an overview of mobile apps for higher education, the student lifecycle, and a self-developed mobile learning app to tackle challenges in the “transition-in” phase. Section 3 introduces the research model and describes the data collection. Afterwards, the results are presented (Sect. 4) and discussed (Sect. 5). The paper concludes with a summary and an outlook.
The use of mobile apps in higher education teaching—such as “learning management applications”, “vodcasts and podcasts”, “language learning applications”, “game-based learning applications” or “collaborative learning applications” (Goundar and Kumar 2022)—is discussed lively in the literature (e.g., Beatson et al. 2020; Gupta et al. 2021; Liu and Guo 2017; Ronzhina et al. 2021; Voshaar et al. 2023). In the following, we summarize related work on mobile learning apps in terms of potentials and challenges, technical and organizational issues, associated theories, as well as future developments. This should give readers a better overview of this mature research area.
In general, the potential of mobile student apps to support learning effectiveness is well-analyzed for various classroom and course examples (cf. Castek and Beach 2013). For instance, Larkin (2015) evaluates apps to foster the building of mathematical knowledge, while Diliberto-Macaluso and Hughes (2016) show that mobile apps may help psychology students achieve their learning objectives. Hence, apps can help to develop students’ self-regulation and deep thinking abilities or support them in labeling, summarizing, and discovering new knowledge amongst others (cf. Diliberto-Macaluso and Hughes 2016; Larkin 2015). In medical education, the benefits of mobile apps to offer an “enjoyable learning experience” are pointed out by Morris et al. (2016) for a neuroanatomy course. Mohapatra et al. (2015) present an overview of apps that are judged to be beneficial for medical education in general, with a particular focus on their ability to manage information from one or more sources to foster communication and support effective time management. Steel (2012) focuses on language students in particular and discusses the potential of mobile apps for this group, e.g., in terms of vocabulary acquisition. An overview of corresponding apps for language students is given by Gangaiamaran and Pasupathi (2017). In accounting and management, Beatson et al. (2020) and Voshaar et al. (2023) find out that students’ behavioral engagement with the help of mobile apps and gamification elements is positively associated with exam results. Seow and Wong (2016) introduce the so-called “Accounting Challenge (ACE)” app, which helps to keep up students’ motivation in studying accounting through gamification as well.
On the contrary, there are challenges of using mobile apps for student education. As Goundar and Kumar (2022) point out, the literature to date has a strong focus on “solution papers”, which introduce fully developed mobile applications that are supposed to improve learning performance. However, a discussion as to what degree singular app functionalities affect students’ cognitive knowledge processing or an explication of the implications for learning theories often come up short (e.g., Damyanov and Tsankov 2018). Along these lines, Mehdipour and Zerehkafi (2013) provide technical as well as social and educational challenges for mobile learning scenarios. These include content security and copyright issues, accessibility and cost barriers for end-users, or the lack of a learning theory for the mobile age in general, to mention just a few (cf. Mehdipour and Zerehkafi 2013). Furthermore, digital technologies may not adequately reproduce the emotional side of interactive learning, so attention should be given to the right balance between digital and human educational interactions (Montiel et al. 2020). A classification scheme for mobile learning challenges according to “management and institutional challenges”, “design challenges”, “technical challenges”, “evaluation challenges”, and “cultural/social challenges” is introduced by Damyanov and Tsankov (2018). In summary, education institutions need to establish a clear mobile learning policy, offer pedagogical support, consider the hardware capabilities of mobile devices, provide a suitable technical infrastructure, and deal with the cultural differences concerning perceptions and attitudes towards digital technologies (cf. Damyanov and Tsankov 2018).
From a technical perspective, the requirements on mobile learning environments, the core functionalities of apps to assure their practicability for educational purposes, and engineering processes for app realization are particularly important. In this context, Zhu et al. (2015) propose a design framework for mobile augmented reality education in healthcare. Further, Clayton and Murphy (2016) analyze mobile apps’ peer-learning and -teaching capabilities for conducting collaborative video design projects. The establishment of a content delivery infrastructure for educational material and suggestions on integrating mobile apps is done by Khaddage et al. (2011). Vázquez-Cano (2014) focuses on the mandatory capabilities of smartphones to support distance learning, while Pechenkina et al. (2017) identify the potential of gamification elements to increase student engagement, retention, and achievement. Finally, Papanikolaou and Mavromoustakos (2006) introduce critical success factors for learning app engineering processes, while Kumar and Mohite (2018) suggest approaches for testing their usability.
From an organizational perspective, the factors for successfully adopting digital technologies in higher education institutions are discussed (e.g., Chuchu and Ndoro 2019). It is accentuated that mobile learning initiatives are not limited to purchasing and deploying digital technologies but require a holistic consideration of diverse factors related to people, technology, or pedagogy (Krotov 2015). Thereby, principal factors that may impact user satisfaction, the intention to use, and the actual usage of mobile applications in higher education are examined (e.g., Almaiah and Alismaiel 2019; Chuchu and Ndoro 2019). As an example, Almaiah and Alismaiel (2019) focus on Jordanian universities and analyze two apps—one that provides student services (e.g., a timetable) and another one enabling “open virtual classes”—in light of the abovementioned factors. Thereby, so-called “quality factors” and “individual factors” that have been adapted from Delone and McLean (1992) and Davis (1989) seem to have a positive effect. Besides, also the variable “intention to use” was examined for this specific student group (cf. Almaiah and Al Mulhem 2019). Further, Chuchu and Ndoro (2019) present indicators that the “perceived usefulness” and “perceived ease-of-use” of a mobile learning app are central factors in creating a positive attitude among the target group and in ensuring its acceptance. An overview of critical success factors for mobile learning in organizations is provided by Krotov (2015). This study integrates the perspectives “organization” (e.g., executive involvement), “people” (e.g., personal innovativeness), “pedagogy” (e.g., quality of content provided), and “technology” (e.g., quality of mobile system) to arrive at a list of success factors from a socio-technical perspective (cf. Krotov 2015).
Considering the complex process of establishing mobile education technologies in organizations, a pedagogical and educational requirements model was proposed by Sarrab et al. (2018), which supports when searching for a suitable solution to deliver content for mobile learning. Besides, the role of mobile apps in facilitating the inclusion of students with handicaps into the university environment is a subject of investigation. For instance, Ok et al. (2016) introduce an evaluation scheme to purposefully select apps for students with learning disabilities. Moreover, people with developmental disabilities can benefit enormously from mobile apps, which hold true for educational, communication, and leisure purposes, helping them connect with their environment (Stephenson and Limbrick 2015). In addition, Bravou and Drigas (2019) reflect the suitability of mobile devices and apps for students with sensory, physical, and cognitive disabilities. In this respect, a comprehensive literature review on digital technologies for people with learning or cognitive disabilities was performed by Williams and Shekhar (2019).
Researchers are also engaged in theory building (cf. Hevner and Chatterjee 2010) to guide the purposeful usage of mobile apps in higher education. However, a widely accepted mobile learning theory has not yet been established (cf. Bernacki et al. 2020; Curum and Khedo 2021). Therefore, Park (2011) refers to the transactional distance theory (cf. Moore 1991), which defines “distance” as a pedagogical concept, and combines this theory with applications of digital technologies to arrive at a “pedagogical framework of mobile learning”. The framework distinguishes between four types of mobile learning depending on whether a (1) high or (2) low transactional distance is given and (3) an individualized or (4) socialized activity is to be solved. Thereby, the transactional distance is defined as the psychological gap between the learner and the instructor, whereas the activity type (i.e., individualized or socialized) assesses the importance of social aspects for a particular learning environment (Park 2011). Another mobile learning framework was introduced by Motiwalla (2007), who proposes to integrate the concepts “mobile connectivity” and “e-learning” for being able to delineate application requirements for mobile learning. Furthermore, a meta-framework to guide the establishment of mobile learning frameworks can be found in Liu et al. (2008). This meta-framework is, for instance, referenced by Nordin et al. (2010) as a theoretical base to create a lifelong, continuing learning framework.
Future developments of mobile learning apps will essentially emphasize the integration of Artificial Intelligence (AI) with learning environments (cf. Alzahrani et al. 2021; Chong 2019; Diaz et al. 2015; Kabudi et al. 2021). The purpose is to improve students’ learning performance via personalization of learning, facilitate the evaluation of student knowledge, or systematically assess learner requirements (Kabudi et al. 2021). Besides, the use of virtual reality (VR) and augmented reality (AR) to progress students’ learning experiences is intensively discussed (e.g., Fradika and Surjono 2018; Nicolaidou et al. 2021). For instance, Nicolaidou et al. (2021) show that a VR learning environment can positively affect vocabulary acquisition and learners’ experience when studying foreign languages. Further, the readiness of students to adapt VR technology to achieve learning goals is high (Ismail and Hashim 2020). Moreover, the use of chatbots and conversational agents is also rising (Hwang and Chang 2021; Liu et al. 2020; Smutny and Schreiberova 2020). Chatbots can serve as efficient information retrieval tools for specific domains to facilitate learning (cf. Liu et al. 2020). In this context, various platform-specific chatbots for learning (e.g., for the Facebook Messenger platform) at different maturity levels have been developed in recent years (cf. Smutny and Schreiberova 2020). Having said that, chatbots for education are primarily found for language courses as well as the disciplines of “engineering” and “computers”, while topics like “arts” or “mathematics” are less accentuated (Hwang and Chang 2021). Thus, chatbots may not be suitable for all types of courses alike, especially in case students’ hands-on competencies (i.e., arts) or computations and problem-solving skills (i.e., mathematics) are to be promoted. While the effectiveness of chatbots for learning purposes is usually measured by pre-/post-test questionnaires, profound insights on chatbots’ impact on behavioral aspects of the student learning process are still elusive (Hwang and Chang 2021).
To conclude this overview, our study aims to analyze factors contributing to the success of a mobile app, which was designed to meet the needs of first-year students in the “transition-in” phase of the student lifecycle. A particular interest is in the ability to positively impact their learning effectiveness, user satisfaction, and intention to reuse the app. To the best of our knowledge, a corresponding study concerning this stage of the student lifecycle has not been done yet. We provide insights that can help establish a mobile learning theory in the “transition-in” phase.
Throughout university life, students experience an evolution of their “student identity”, which goes along with a shift of priorities and agendas (Lizzio 2011). As mentioned above, our research focuses on “commencing” students who are just about to become acquainted with the university system and have an increased interest in opportunities for social interaction, active engagement, and early formative feedback (Matheson 2018). Generally, various propositions regarding the development stages of students exist (cf. Burnett 2007; Morgan 2013) that primarily differ in their conception of student transition (Gale and Parker 2014). A widely acknowledged proposition for an integrative framework was introduced by Lizzio (2011), which is depicted in Fig. 1 and differentiates between four major stages. Whereas future students (“transition-towards”) are engaged in finding an appropriate study program and university, commencing students (“transition-in”) work on the integration into the student world (Lizzio 2011). In the “transition-through” phase, continuing students work on developing graduate attributes and seek challenges by authentic curricula and assessments (Lizzio 2011; Matheson 2018; Msomi and Bansilal 2019). Finally, the “transition-up, out & back” stage addresses students that are graduating or returning for postgraduate studies to further strengthen their skills for employability (Lizzio 2011; Matheson 2018).
Against this background, most premature dropouts are observed in the “transition-in” phase (Chen 2012; Isleib et al. 2019; Neugebauer et al. 2019). An empirical study focused on German higher education institutions (60 universities and universities of applied sciences) identified a lack of social and academic integration as a major reason for premature dropouts (Isleib et al. 2019). More specifically, differences in the perception of study requirements were observed for dropout and non-dropout first-year students. These observations are generally also confirmed for other countries (cf. Chen 2012; Kehm et al. 2019; Xenos et al. 2002; Zvoch 2006). In terms of academic integration, many first-year students obviously struggle with self-organizing their studies and balancing the time slots for attending courses, obtaining credits, and preparing or post-processing lectures (Schulmeister 2007). In such cases, the danger of not meeting the academic standard and the probability of an early dropout increases significantly (Kehm et al. 2019). Consequently, the importance of promoting student retention and student achievements has been identified as a crucial responsibility of higher education institutions at that point (Matheson 2018; Sheader and Richardson 2006).
In the “Student Adjustment Model” of Menzies and Baron (2014), the stages experienced by first-year students when entering the university system are specified more in-depth, which helps to gain a deeper understanding of the overall “transition-in” phase. Hence, upon arrival, a sense of excitement can be observed among first-year students, which comes to a halt after some weeks when first negative experiences in the new environment have been made—a phase called the “party’s over stage” (Menzies and Baron 2014). Now students need to realistically assess their capabilities, identify gaps (e.g., self-organization skills) and carefully reflect on the institutional requirements (Matheson 2018). Here, universities can support by providing informative student feedback and curricula that offer opportunities for social networking and learning, or teaching methods that foster active learning and encourage student engagement (Matheson 2018; Whittaker and Brown 2012). Afterwards, students are able to enter the so-called “healthy adjustment stage” (Menzies and Baron 2014).
Furthermore, the base competencies of first-year students have been a subject of investigation (cf. Krumrei-Mancuso et al. 2013; Zehetmeier et al. 2014), which provides valuable insights into the academic skills of young people that contemporaneously enter the university system. Whereas some studies focus on special types of competencies—like digital (e.g., Reddy et al. 2020) or leadership competencies (e.g., Smart et al. 2002)—a more extensive investigation at a German higher education institute, comprising 18 competency types in total, was performed by Zehetmeier et al. (2014). Concerning first-year students, deficiencies in self-organization, accurateness, perseverance, intrinsic motivation, or self-criticism were described (Zehetmeier et al. 2014). Based on these findings, universities should develop solutions that can account for diverse student backgrounds, tackle insufficient experiences, and support individual learning and self-organizational strategies to achieve academic success.
Considering this, we argue that a mobile app may be a suitable solution to tackle the abovementioned challenges (e.g., insufficient student experiences, lack of self-organization, etc.) in the “transition-in” phase of the student lifecycle. To provide a general overview, Table 1 gives a brief selection of campus apps that come to use at German universities. Of course, campus apps can be found internationally (e.g., UC San Diego mobile app) (e.g., Almaiah and Alismaiel 2019; Holotescu et al. 2018).
Thereby, the term “student experience” subsumes “all experiences of an individual student” while being in the “identity as a ‘student’” including all “facets of the university” (e.g., administrative processes, IT support etc.), which “contribute” to the “personal development” as a learner (Baird and Gordon 2009, p. 194).
To specify the meta requirements and arrive at design requirements for the app, (I) user stories, (II) market research, (III) user requirements, and (IV) user journeys were used (Schilling 2016). In this context, also second- and third-year undergraduates (N = 54), who are still well familiar with the challenges experienced at the beginning of their studies were surveyed. Finally, we came up with eight major design requirements classified into the categories “course attendance/reminders” (Fig. 2—DR 1-2), “support of study phases” (DR 3-5), and “technical requirements” (DR 6-8) to support students’ experience, learning strategies, and self-organization (cf. Fig. 2, Johannsen et al. 2021).
The architecture of the app consists of a front- and a back-end. The frontend was developed with the help of the IONIC Framework (https://ionicframework.com/), which works based on Angular (https://angular.io/) (Green and Seshadri 2013). Further, the back-end was realized via the Spring Framework (https://spring.io/) and the Spring Boot solution (cf. Walls 2016). Figure 3 shows exemplary screenshots. So, a new course is added to a student’s timetable (Screenshot 1), and sample functionalities for this course—derived from the design requirements—are shown, such as the conduction of quizzes (Screenshot 2) or the comparison with a peer group (Screenshot 3). The app can be classified as type 2 (i.e., high transactional distance and individualized mobile learning activity) in the “pedagogical framework of mobile learning” of Park (2011). Hence, this type allows a high degree of flexibility and portability, enabling students to integrate it flexibly into their mobile lifestyle (Park 2011).
With respect to the challenges in the “transition-in” phase (see Sect. 2.2) and the meta requirements, various functions that support students’ learning strategies, experience, and self-organization are offered by the app. For instance, the features of tracking learning time along with an overview of exam dates and events largely foster students’ self-organization abilities (cf. Zehetmeier et al. 2014). This is further supported by push notifications or newsfeeds about new learning content as well as a calendar function with reminders for lectures and important academic dates contributing to student experience (e.g., Staddon and Standish 2012; Trotter and Roberts 2006). Gamification is used to enrich students’ learning strategies (e.g., performance tests via quizzes), while they can work on exercises independent of time and place.
In the following, we present the research model, the hypotheses, and the data collection. We heavily rely on the IS success model of Delone and McLean (1992), which is a commonly referenced model to measure the success of information systems, and has been referenced in many studies in the field of technology-supported education (cf. Almaiah and Alismaiel 2019; Aparicio et al. 2017; Cidral et al. 2018; Dorobat 2014; Holsapple and Lee‐Post 2006; Huang et al. 2015; Kruger-Ross and Waters 2013; Wang et al. 2019b). Hence, it arguably is one of the most widely applied models in this field (Almaiah and Alismaiel 2019).
When it comes to user acceptance of technologies also the TAM (Technology Acceptance Model) approach is discussed in literature (e.g., Davis et al. 1989; Liu and Guo 2017; Mohammadi 2015). According to TAM, the factors influencing the acceptance and usage of technologies can be categorized into the clusters “external variables”, “perceived usefulness”, and “perceived ease of use” (Davis 1989; Davis et al. 1989). Nevertheless, the model is criticized since it mainly focuses on individuals’ perception of technology, while the context in a business, university, or organizational setting (e.g., policy, IT guidelines) is neglected (Ajibade 2018).
In this light, the IS success model is particularly suitable for our research for several reasons: First, its quality dimensions can be easily aligned with web-based applications (cf. Delone and McLean 2003; Efiloğlu Kurt 2019), which are dominant in e-learning environments to foster students’ learning activities (Freeze et al. 2010; Muhammad et al. 2020). Our mobile app (see Sect. 2.3) represents a corresponding solution to support student learning, whereby the querying of database information (e.g., timetables), the login-logics, or the provision of content (e.g., training questions) are enabled by a backend server, while the data is sent to the frontend by help of the JSON and HTTP standard. Further, the mobile app considered in this study can be allocated to the “communication and system phenomenon” (Freeze et al. 2010) of e-learning solutions, for which not only the quality of the system is of interest but also the communication with a “service provider”, who creates study-relevant content, provides advice, or resolves problems (cf. Aparicio et al. 2017). The IS success model explicitly covers these aspects by corresponding constructs and, thus, represents the base of our research model introduced hereafter, whereas the quality dimensions are adapted to the study context as proposed by Delone and McLean (1992).
Second, although the model has already been intensively used in education research for years, the focus of this study is on a mobile app that was designed for the “transition-in” phase in particular and represents an instance of a “type 2 app” according to the “pedagogical framework of mobile learning” of Park (2011). In literature, there is a lack of knowledge regarding the success factors for apps of this type with a special focus on first-year students. Considering this, using a widely established success model and quality dimensions is promising to prepare the ground for further developments of similar solutions. Hence, results attained in other studies with the help of the IS success model may not necessarily be confirmed for the type of mobile app investigated herein. Furthermore, the way to compare the impact of the IS success model across studies is paved and, hence, the relevance of singular dimensions of IS success for various types of apps directed at different stages of the student lifecycle may be assessed more profoundly in the next steps.
Third, for being able to derive design principles that allow the creation of similar instances of artifacts that belong to the identical class (cf. Kruse et al. 2016; Sein et al. 2011), the use of the widely accepted IS success model is promising. This is because its success dimensions have already been broadly recognized and therefore represent a solid base for defining verifiable and comprehensible design principles. These may be extended in future steps as soon as the knowledge about beneficial mobile app development for the focused application field evolves along with additional insights about beneficial success dimensions for learning effectiveness. The proposed research model, its variables, and our hypotheses are introduced in the following section.
According to Delone and McLean (1992), system quality is a central success factor for IS. The variable describes the desired characteristics of the information system to produce the required information (Urbach et al. 2009). Thereby, Wang et al. (2019b) have shown that the system quality of paid mobile learning apps has a positive impact on “user satisfaction” and the “intention to (re-)use”. Similar results for mobile learning apps at Jordanian universities were introduced by Almaiah and Alismaiel (2019). Though, a study on an e-learning system in Brazil was less clear about the beneficial role of system quality (cf. Cidral et al. 2018). Moreover, Aparicio et al. (2017) investigated “grit” as a determinant of “e-learning system success” and confirmed the supportive effect of system quality on “user satisfaction”. A positive effect on “user satisfaction” was also shown by Chiu et al. (2016) for a “cloud e-bookcase system” for libraries, whereas Huang et al. (2015) identified a positive impact on both, “intention to (re-)use” and “user satisfaction” for a mobile library service system. Considering literature and the preferences of business students concerning mobile applications (e.g., Kouser et al. 2014), an app’s ease of use (Wang et al. 2019b), its structuredness (Cidral et al. 2018; Urbach and Müller 2012), an easy-navigation (Kouser et al. 2014), and the ability to efficiently retrieve relevant information (Wang et al. 2019a) are highly appreciated by the target group in terms of system quality (Aparicio et al. 2017; Urbach et al. 2010). Hence, we hypothesize:
The construct “service quality” refers to the overall support for users offered by a service provider (Delone and McLean 2003). In terms of “e-learning”, Aparicio et al. (2017) emphasize the importance of the willingness and readiness of the support staff to resolve students’ difficulties at any time because this positively influences the intention to use the system. This positive effect was also confirmed in earlier studies (e.g., Chiu et al. 2016; Huang et al. 2015, among others). Generally, “service quality” may be interpreted from different angles and refer to concepts such as assurance, empathy, or flexibility—just to mention a few (Urbach and Müller 2012). In alignment with the propositions of Aparicio et al. (2017) and Urbach et al. (2010), we see the willingness of the service personnel to provide support upon request immediately, the personal attention offered to students, the timeliness of the service response as well as the competence and knowledge of the service personnel as central factors for the app’s success. Considering this, we claim:
Information quality addresses the system output or the information that is produced by a system (Delone and McLean 1992). According to Almarashdeh et al. (2010), information quality is the most crucial factor when determining the success of educational technology systems (Almaiah and Alismaiel 2019). Hence, the positive impact of information quality on the “intention to (re-)use” and “user satisfaction” is confirmed by manifold studies that focus on e-learning or mobile learning systems (e.g., Aparicio et al. 2017; Cidral et al. 2018; Wang et al. 2019b). However, there are also studies in which information quality played a subordinate role for the acceptance of a system (cf. Chiu et al. 2016). Once more, the construct “information quality” can be reflected from various perspectives such as data accuracy, adequacy, or completeness (cf. Klier 2008; Urbach and Müller 2012). For our app, we determine information quality based on the reliability and understandability of the information provided and its usefulness and relevance for the target group (cf. Aparicio et al. 2017; Urbach et al. 2010).
Davis et al. (1992) summarize enjoyment in the information systems context as “the extent to which the activity of using the computer is perceived to be enjoyable in its own right, apart from any performance consequences that may be anticipated” (p. 1113). Against this background, the construct of “perceived enjoyment” is increasingly getting attention when it comes to the measurement of IS success (cf. Kim et al. 2007; Wang et al. 2019b). Therefore, it is suggested that technology adoption is more likely in cases where users experience immediate pleasure or joy through mere use (Kim et al. 2007). Since the positive influence of perceived enjoyment on users’ attitudes is well examined in the mobile services and mobile commerce context (cf. Tseng and Lo 2011; Wang and Li 2012; Wang et al. 2019b), it is increasingly discussed in terms of e-learning technologies, as well (cf. Balog and Pribeanu 2010; Hussein 2018; Khalid 2014). Hence, we also assume a positive effect on “user satisfaction” and “intention to (re-)use”. In this regard, gamification elements may purposefully impact the hedonic motivation to engage with mobile apps and, as a positive side-effect, impact users’ perceived enjoyment (cf. Beatson et al. 2020; Pechenkina et al. 2017; Wang et al. 2019b).
Generally, gamification is seen as a means to overcome a lack of motivation among students to deal with study-related content (cf. Kiryakova et al. 2014). Thereby, principles such as “freedom to fail”, “rapid feedback”, “progression”, or “storytelling” play a decisive role for the successful application of gamification elements in learning environments (Stott and Neustaedter 2013). Hence, specific mechanisms that are traditionally used in game design (e.g., Laine and Lindberg 2020) to increase user engagement and hedonic motivation have found their way into modern pedagogy, although their purposeful selection should be made in regards to the target group (Stott and Neustaedter 2013). For the design of mobile education apps, corresponding principles need to be purposefully transferred to corresponding design requirements (cf. Herrington et al. 2009; Laine and Lindberg 2020). Section 2.3 presents the design requirements of our mobile app (see also Fig. 3), whereas these are taken up in Sect. 5.2 once again and reflected against the findings of the study.
In summary, we determine perceived enjoyment based on the fun and enjoyment experienced by app users (cf. Kim et al. 2007; Wang et al. 2019b) and the abilities of entertaining and playful features to enhance users’ learning experience and structure their learning efforts (cf. Suki and Suki 2007).
“Intention to use” is specified as users’ intent to perform a defined behavior (Davis 1989). The construct is acknowledged to be strongly associated with the acceptance of an information system (Almaiah and Alismaiel 2019) and it largely depends on the users’ attitude towards the system (Agrebi and Jallais 2015). However, there is a distinct difference between “intention to use” and actual “use”, because the former represents an attitude, whereas the latter concept describes a concrete behavior (Delone and McLean 2003). To resolve the closed-loop relationships between user satisfaction, intention to use, and use in the original IS success model (Wang et al. 2019b), “intention to reuse” is commonly proposed as a worthwhile measure (Delone and McLean 2003; Wang 2008). In line with the proposition of Wang (2008), “intention to reuse” thus represents the favorable student attitude towards our app in this study.
In addition, “perceived user satisfaction” helps to measure the successful interaction of users with the IS (Delone and McLean 1992). Generally, user satisfaction can be interpreted as “the extent to which users believe the information system available to them meets their information requirements” (Ives et al. 1983, p. 785). Thereby, perceived user satisfaction leads to an increasing “intention to reuse” in the post-use situation (Wang 2008).
Either way, the major purpose of mobile learning technologies is to increase knowledge acquisition (cf. Wang et al. 2019b) and, hence, improve learning outcomes (cf. Noesgaard and Ørngreen 2015). Generally, the beneficial individual or organizational impact of IS, which is supposed to be measured by the IS success model may occur in many ways (e.g., awareness/recall, competitive advantage, etc.) (cf. Delone and McLean 2003; Urbach and Müller 2012). Considering this, there is a lively discussion on how to operationalize the individual benefits of using e-learning technologies that cumulate in better knowledge acquisition and learning outcomes in the end (e.g., Chiu et al. 2016; Noesgaard and Ørngreen 2015; Wang et al. 2019b; Zhang et al. 2006). In that context, “learning effectiveness” (cf. Noesgaard and Ørngreen 2015) has become a commonly accepted measure to assess the success of technology-assisted learning for individuals (Smith et al. 2006; Wang et al. 2019b; Zhang et al. 2006). The variable builds on the recognition that effective learning asks for learners’ engagement, motivation, awareness, and an individualized learning process, which can be enabled by offering access to content randomly or repeatedly on demand for instance (Zhang et al. 2006). This, in turn, promotes learning skills (e.g., enhanced problem-solving or critical thinking abilities; Zhang et al. 2006) and leads to an improved understanding of study-related content, which can be recollected any time (cf. Chiu et al. 2016; Gable et al. 2008; Wang et al. 2019b). Hence, improved knowledge acquisition and learning outcomes emerge from a general point of view.
To properly address these considerations, the literature proposes to ask for students’ perceptions of learning performance, efficiency, motivation (cf. Liaw 2008), awareness, and recollection of study-related information (Gable et al. 2008) along with their understanding of the course content (cf. Chiu et al. 2016). Accordingly, these aspects determine the items of our questionnaire to assess the variable “learning effectiveness” (see Appendix).
As evident from the above explanations, a rather broad spectrum of factors (e.g., awareness, motivation, etc.) is required to describe “learning effectiveness” comprehensively. Nevertheless, the variable allows students to carefully reflect on the achieved individual (net) benefits (cf. Delone and McLean 2003) when using a mobile learning app to cope with the challenges of the “transition-in” phase. Therefore, the variable is used hereafter to measure students’ (net) benefits since we believe that other variables that have been proposed in the context of the IS success model (e.g., recall, job simplification, etc.) (cf. Urbach and Müller 2012) would not comply with the multidimensionality of first-year students’ learning success in the “transition-in” phase and may not be adequately transferred to our context.
We formulate the following hypotheses:
Figure 4 summarizes the proposed research model, variables, and hypotheses.
We developed a questionnaire based on the abovementioned established and validated scales from previous studies and modified them accordingly for the mobile learning context to test our hypotheses. As previously described, the constructs of “system quality”, “service quality”, and “information quality” were adapted from Aparicio et al. (2016) and Urbach et al. (2010) and are all measured by four underlying items. “Perceived enjoyment” consists of five items, three being adapted from Kim et al. (2007) and Wang et al. (2019b), and two used by Suki and Suki (2007). Three items were adapted from Wang et al. (2019b) and Wang (2008) and are complemented by one item each from Chiu et al. (2016) and Sun et al. (2008) to measure the construct “intention to reuse”. “Perceived user satisfaction” was measured by four underlying items used by Liaw (2008). Finally, we adopted three items from a previous study of e-learning effectiveness from Liaw (2008) and added one item each from Chiu et al. (2016) and Gable et al. (2008) in order to measure “learning effectiveness”.
Initially, we developed the survey in English, in accordance with prior research, and then translated it to German through a professional translation service in order to ensure a low-threshold participation opportunity and, thus, a high number of participants. Subsequently, a different professional translator translated it back into English to ensure conversion correspondence (Brislin 1970). As previously described, the constructs were unanimously measured with four or five items each. All items were assessed on a seven-point Likert scale (from 1 = “strongly disagree” to 7 = “strongly agree”). Table 4 in the Appendix presents the final survey consisting of the mentioned items used in our research model.
The mobile learning app was initially implemented in a mandatory introductory accounting course in the Winter semester 2020/21. Because of the COVID-19 pandemic, the social distancing requirements, and the sudden closures of university campuses, all lectures were held digitally in an asynchronous format via screencasts. Complementing the lectures, students could participate in synchronous, live tutorials and submit exercise sheets. Additionally, preparatory courses were also offered synchronously via Zoom. For our study, we invited all students who used the mobile learning app at some point in the Winter semester 2020/21 to participate in the online survey conducted during the final week of teaching (i.e., before the final exam) and administered on the university’s LMS. We did not offer any additional (e.g., monetary or extra course credit) incentives for participating, and the students were informed of the research purpose and their voluntary participation in the study. Even if they took part in the survey, they had the possibility to refuse to answer any question. Subsequently, one member of the research team, who was not involved with the empirical analysis, merged and pseudonymized the data from students’ questionnaires with data from several other sources, including students’ demographics being collected through another survey in the first week of the semester, students’ course attendance during the semester, and the academic performance data. More specifically, the students’ attendance at tutorials and workshops has been manually evaluated via Zoom participation protocols. Finally, the central examination office provided the student’s exam performance. In our analysis, we only use the final pseudonymized dataset, which does not allow identification of individual students.
The students were asked to answer the questionnaire according to their user experience throughout the semester. Thereby and due to the requirements of the IS success model, we were ex-ante limited to the population of 367 students who used the app during the semester and participated in the final exam to draw our sample. Our initial sample consists of 131 students who participated in our survey regarding their user experience. Out of the initial sample, we exclude 10 observations due to missing values in their survey responses and 1 without any variation in the responses. Hence, we received 120 usable responses, bringing our usable response rate to 91.60%. Further, we exclude 7 students because of missing values in their demographics, resulting in a final sample of 113 students who participated in the final exam Footnote 1 of the mandatory introductory accounting course and used the mobile learning app for learning purposes. Accordingly, our sample represents 30.79% of the underlying population that could be used for a study of this type. Footnote 2 The final sample comprises 63 female and 50 male students, with the overwhelming majority (94.69%) being 25 years old and younger. Table 2 presents the summarized descriptive statistics for the final sample of 113 students. Footnote 3
First, considering information quality, the option to self-track one’s course attendance and analyze this data against the course offerings during the semester (DR 1), the functionality to allow pop-up messages as reminders for lectures and academic events (DR 2) as well as the availability of training and exam-oriented exercises to control the learning process (DR 3) have proven useful to provide understandable, interesting, and reliable information to first-year students. Thus, regarding DR 1, the user may navigate to a site called “course overview” after login, showing the portfolio of courses the user has registered for. There, the app provides the option to confirm attendance for lectures. On a more detailed level, the user can create a time tracker for each course on demand, which allows for tracking attendance and self-study periods. Concerning DR 2, students are notified via push messages if an important event (e.g., lecture, exam registration deadline) in a course for which they have registered is about to take place. As a further option, students can export the course or event dates to their personal smartphone calendars. Via the site “exercises”, exam-oriented exercises are offered to check one’s personal learning process (DR 3). At this point, the training content also comprises learning videos and course scripts, which can be accessed at any time, enabling students to adjust the pace of learning at their own discretion.
The information quality offered to students with the help of the abovementioned functionalities (see Fig. 6) supports them in planning their participation in academic events and, thus, integrating into the daily student life (student factor “student experience”). Furthermore, they can improve their self-organization and adjust their learning strategies in accordance with the results that were scored for the exam-oriented exercises for instance (student factors “self-organization” and “learning strategies”).
To increase users’ perceived enjoyment, gamification elements like quizzes (DR 4) and competitions with the peer group are available (DR 5). This is meaningful since a lack of motivation and engagement to participate in the learning process has been identified as a major challenge in contemporaneous education (Hassan et al. 2021; Kiryakova et al. 2014). In order to reach a higher level of commitment and motivation among students, we propose the functionalities DR 4 and DR 5. Therefore, the game design mechanisms “freedom to fail” and “rapid feedback” (Stott and Neustaedter 2013; Thakur et al. 2020) were purposefully transferred to our learning app. We consider both mechanisms as helpful for first-year students to reduce mental barriers to interact with fellow students and teachers. The principles can be ideally addressed by quizzes, which give students direct feedback, even beyond the classroom, and disassemble psychological barriers to “fail” or to give wrong answers (cf. Alberti et al. 2019; Gordon et al. 2021). Furthermore, rewarding students’ efforts immediately (e.g., by credits) is a recognized way to increase motivation (Kiryakova et al. 2014). Hence, during app usage, students may collect credits through various actions (e.g., quizzes, attending courses, correctly solving exam-oriented exercises, etc.), determining their ranking in a playful competition with their peer group. Further, the training exercises can also be offered in the form of an interactive survey during the lecture (i.e., a “clicker” functionality) to test students’ current knowledge and support their “progression” in becoming familiar with the course content (cf. Stott and Neustaedter 2013).
Against this background, we propose the described functionalities to increase students’ perceived enjoyment (when dealing with subject-related content) as means to address the student factors “learning strategies” and “self-organization” (see Fig. 6 and Sect. 2.3).
Additionally, we suggest the design of the solution as a hybrid app (DR 6), which differentiates between a front- and backend (DR 7) and transfers data with the help of HTTP and JSON (DR 8) as a way to effectively produce the required information and, hence, contribute to system quality in our setting (cf. Urbach et al. 2009). The design as a hybrid app allows us to offer the app for both common mobile phone platforms (i.e., iOS and Android), whereby the development efforts were less than for corresponding native apps (e.g., Schilling 2016). The app’s architecture enables easy maintenance, further development, as well as the addition of further services. Finally, the data (e.g., exam-oriented exercises, quiz questions, etc.) are stored in a database, which strongly facilitates content management and the provision of new content once users log in as “lecturers”. Figure 7 provides our proposition for the architecture.
Based on our findings, we propose the following four design principles to facilitate the design of related instances of the artifact (cf. Kruse et al. 2016; Sein et al. 2011), namely mobile solutions to support students in the “transition-in” phase. Generally, design principles build on the knowledge that is gained when developing and using a specific instance of an artifact and are formulated when reflecting upon the results from a more generic perspective (Kruse et al. 2016). Concretely, we propose the following design principles:
These design principles have been formulated based on a concrete instance of an artifact, considering the findings of this study. They can purposefully complement existing design principles, which are directed at the creation of innovative learning environments (cf. Herrington et al. 2009), the design of mobile course material (cf. Ally 2005), or making use of gamification elements (cf. Laine and Lindberg 2020). However, our design principles are different from existing propositions in the field of mobile learning (e.g., Palalas and Wark 2017) since we focus on support during the “transition-in” phase and the corresponding challenges. As such, the suggestions may be referenced and consolidated with further design principles to cover all stages of the student lifecycle in future research, contributing to an even better understanding of mobile learning in higher education.
As mobile phones are widely available among students and the field of higher education becomes aware of the potential to use them for learning purposes through the development and usage of mobile learning apps, it is crucial that researchers and practitioners develop a better understanding of what makes learning apps successful and—equally important—how to measure their success in the first place. Considering this, our study contributes to research in the following ways.
First, the mobile app to support students in the “transition-in” phase was developed with the help of a DSR procedure (cf. Peffers et al. 2007), as outlined in Sect. 2.3. The app represents the artifact (i.e., outcome) of the Design Science (DS) effort and, hence, a “human-made object” (Goldkuhl and Karlsson 2020, p. 1241) to solve a practical problem (March and Smith 1995). However, besides the artifact itself also its contribution to theory should be clearly highlighted in DSR (Baskerville et al. 2018). This contribution to scientific knowledge is addressed by Hevner’s “rigor cycle” (Hevner 2007) and a mandatory element from the perspective of the “design theory school of thought” (Baskerville et al. 2018, p. 359). Thereby, the IT-artifact (i.e., the mobile app) of our DS effort was built in previous work (see Sect. 2.3) and can be seen as an instance of a “type 2 app” according to the “pedagogical framework of mobile learning” (Park 2011) for students of an accounting course. In this paper, the app’s contribution to the scientific knowledge base is analyzed by identifying the factors that impact a student’s learning effectiveness, user satisfaction, and intention to reuse the app and linking these to design requirements (see Sects. 4 and 5). Furthermore, we present design principles that offer DS researchers “actionable knowledge useful in building new versions of similar artifacts” (Kruse et al. 2022, p. 1236). Accordingly, these insights may extend the “pedagogical framework of mobile learning” (Park 2011) by laying the foundation for success factors and design propositions that determine the acceptance of mobile apps among students in the “transition-in” phase.
Additionally, this research supports previous findings on the employment of system success models in the field of mobile learning. In this respect, we confirm the model of Wang (2008), which suggests that user satisfaction has a direct as well as indirect impact on other net benefits (e.g., learning effectiveness) through the mediation of intention to reuse. Another implication that can be drawn from this result is that the perceived user satisfaction and the intention to reuse are prerequisites for students’ learning effectiveness. Following Wang et al. (2019b), who implemented perceived enjoyment in the context of fee-based mobile learning, we now use the perceived enjoyment in our proposed success model for free mobile learning apps. Therefore, this study benefits future research by providing a validated IS success model for free mobile learning apps by combining perceived enjoyment and learning effectiveness with the established IS success model.
However, deviating from the traditional IS success model, we found service quality to have no significant impact on perceived user satisfaction and intention to reuse (see Fig. 5). Since this can be explained, as shown in Sect. 5.1, it can be stated that service quality is relatively less important in the context of knowledge-oriented IS success, which is in line with previous research (e.g., Wang et al. 2019b; Wu and Wang 2006) as well as rather good news for higher education organizations which mainly expend their resources on teaching staff instead of administrative personnel, which could provide user support for such apps.
In summary, the empirical results emphasize the importance of extending the traditional IS success model by other dimensions like perceived enjoyment when assessing mobile learning app success. Accordingly, future research can rely on this multidimensional approach, compare it to existing models, or examine the influence of the included constructs on mobile learning system success.
This research also provides several implications and benefits for practice. First, the results show that an app, which was collaboratively developed with the target groups (i.e., students and educators) and adapted to the particular needs of first-year students, can positively influence students’ learning effectiveness. This highlights the value of the design and development procedure of the app following a DSR approach (cf. Peffers et al. 2007) with several iterative steps.
Second, according to the employed and validated model, learning effectiveness is considered a more effective measure of mobile learning app success than the other six variables. In this regard, learning effectiveness should develop if the model components of system quality, service quality, information quality, perceived enjoyment, intention to reuse, and perceived user satisfaction are appropriately managed. Thus, to support learning effectiveness, an implication for developers of mobile learning apps is to focus on high system quality, information quality, and, foremost, an enjoyable learning experience. To this end, design requirements and principles have been proposed that allow the creation of similar artifacts. More detailed, our results show clear evidence that the impact of students’ perceived enjoyment on user satisfaction, intention to reuse, and learning effectiveness is substantially greater than the total effect of system quality, service quality, and information quality. This calls for strongly emphasizing gamification elements for mobile apps in corresponding DSR efforts.
Third, the four components of system quality, service quality, information quality, and perceived enjoyment have both direct and indirect effects. They all directly influence students’ perceived user satisfaction and the intention to reuse. The perceived user satisfaction, in turn, affects the intention to reuse as well as the learning effectiveness. Therefore, the intention to reuse and learning effectiveness are also influenced indirectly. The findings of this study suggest perceived user satisfaction has the most significant impact on intention to reuse and learning effectiveness. Moreover, perceived user satisfaction has the most substantial direct and total effect on students' learning effectiveness (see Fig. 4). Thus, the importance of student’s satisfaction with the learning app in improving their learning effectiveness is emphasized. However, the findings also suggest that developers as well as educators must track changes in both the perceived user satisfaction and intention to reuse, as user satisfaction does not totally mediate the impact of intention to reuse on students’ learning effectiveness.
To summarize, this study helps practitioners like developers and educators to identify the factors that make mobile learning applications more successful. The empirical findings encourage developers to consider the constructs of system quality, information quality, perceived enjoyment, perceived user satisfaction, intention to reuse, and learning effectiveness when designing their products. Moreover, the importance of students’ enjoyment and satisfaction while using the app for their learning outcomes is emphasized. Besides the implications for developers, this aspect is also a fundamental implication for educators, as it requests and motivates them to deliver learning content entertainingly to help their students succeed. Together, both aspects could help reduce early dropout rates among students and, thus, contribute to fighting the current shortage of skilled workforce and help meet the economy’s demand for qualified workers in the next decades.
This study deals with the analysis of factors that contribute to the success of a mobile app to support first-year students in the “transition-in” phase in view of learning effectiveness, user satisfaction, and intention to reuse the app. Furthermore, the factors are linked to design requirements and the derived design principles. Our study covers one semester during the COVID-19 pandemic and thus includes only digital courses, which should be considered when interpreting the results. Nevertheless, if this circumstance has any impact, we expect it to be in favor of our results rather than against them. Presumably, the impact of our learning app would have been even stronger in the pre-pandemic period than in the online teaching period. This is because the app functionalities purposefully complement the attendance of face-to-face lectures. In a pure online semester, the “value” provided by the app, which is still positive for users, may be less than in the era of traditional teaching. Though, this proposition needs to be explored in more detail in future studies.
Besides providing several benefits for both research and practice, this study has some noteworthy limitations. First, the discussed findings and the implications drawn are limited to a specific context of an app adapted to first-year students’ particular needs in a mandatory introductory accounting course at the University of Bremen (Germany). However, in terms of the topics, structure, and practicalities, the course setting is similar to most foundational undergraduate courses in Continental European study programs in business and economics. Second, since we rely on self-reported data to examine the mobile learning app’s success, this may introduce the risk of common method and response bias. Having said that, as we assured participants of the confidentiality of their responses and offered no monetary rewards or other incentives for participation, we assume that the risk of systematically biased responses is minimal. Third, we employ a cross-sectional approach, which causes possible feedback links from learning effectiveness to perceived user satisfaction and the intention to reuse could not be considered in this study. Finally, our research model largely builds on the initial elements of the IS success model. This was done to receive design principles that are based on widely accepted elements for success, which may positively affect the general acceptance of such principles in the DSR community. Moreover, to the best of our knowledge, corresponding studies based on the IS success model have not been done for “type 2 apps” according to the “pedagogical framework of mobile learning” of Park (2011).
In future research, a longitudinal design to take these possible feedback links into account and, thus, enhance the understanding of the causality and interrelationships of the research elements in the context of mobile learning app success will be performed. Going forward, the app will be continuously developed further and is planned to be fully integrated into the entire undergraduate curriculum at our faculty. In this course, the integration of AI-based conversational agents to further improve students’ learning experience will be investigated more closely. Particularly their impact on students’ learning behavior is to be considered since the literature on accounting education as well as information systems lacks theoretical foundations in this respect. Moreover, the research model will be extended by additional elements in the next step to identify additional influencing factors that may positively affect student performance (e.g., base competencies or grit; cf. Aparicio et al. 2017; Zehetmeier et al. 2014).
In order to ensure the representativeness of our drawn sample, we conducted two-tailed t-tests for differences in means between the group of students included in the sample and the underlying population of app users. The results indicate that the characteristics are essentially similarly distributed. The only documented significant differences are in the share of students in Business Studies and Engineering and Management, in the share of (almost) always and (almost) never attending students, as well as in the share of students with a very good exam performance. However, the significance level is only slightly pronounced (p < 0.1) for most differences. We present a comparison between the sample and the underlying population with the conducted t-tests in Table 9 (Appendix).
The high proportion of never and rarely attending students is likely due to the conitions of COVID-19 induced online teaching. In order not to disadvantage any students, we provided the recordings of the zoom sessions of tutorials and workshops afterwards. However, we were not able to assess which students accessed the recordings. Moreover, the distribution of the exam performance in our sample, which documents a high level of insufficient performance and thus failure, is in line with both the exam performance of the whole population of the course and the distribution of the exam performance in previous cohorts.
Open Access funding enabled and organized by Projekt DEAL.
