|Computers, Materials & Continua |
Mobile Devices Interface Adaptivity Using Ontologies
1Department of Software Engineering, The Superior University, Lahore, 53700, Pakistan
2INP-ENIT, University of Toulouse, Tarbes, 65000, France
3Riphah School of Computing & Innovation, Faculty of Computing, Riphah International University Lahore Campus, Lahore, 54000, Pakistan
4Pattern Recognition and Machine Learning Lab, Department of Software, Gachon University, Seongnam, 13557, Korea
5Department of Computer Engineering, Gachon University, Seongnam, 13557, Korea
*Corresponding Author: T. Whangbo. Email: firstname.lastname@example.org
Received: 31 August 2021; Accepted: 13 October 2021
Abstract: Currently, many mobile devices provide various interaction styles and modes which create complexity in the usage of interfaces. The context offers the information base for the development of Adaptive user interface (AUI) frameworks to overcome the heterogeneity. For this purpose, the ontological modeling has been made for specific context and environment. This type of philosophy states to the relationship among elements (e.g., classes, relations, or capacities etc.) with understandable satisfied representation. The context mechanisms can be examined and understood by any machine or computational framework with these formal definitions expressed in Web ontology language (WOL)/Resource description frame work (RDF). The Protégé is used to create taxonomy in which system is framed based on four contexts such as user, device, task and environment. Some competency questions and use-cases are utilized for knowledge obtaining while the information is refined through the instances of concerned parts of context tree. The consistency of the model has been verified through the reasoning software while SPARQL querying ensured the data availability in the models for defined use-cases. The semantic context model is focused to bring in the usage of adaptive environment. This exploration has finished up with a versatile, scalable and semantically verified context learning system. This model can be mapped to individual User interface (UI) display through smart calculations for versatile UIs.
Keywords: User context; adaptive interfaces; human computer interaction
Currently mobile phone user interfaces provide adaptivity features options regardless of context of use. It is user's choice to select any adaptivity feature according to their choice. By definition, user interface adaptivity is required to read the context and provide respective interaction style and mode. There have been many studies and researches, mentioned in literature review, that suggest context-based user interface adaptively [1–3]. Context has been modelled in various scenarios and application domain focused on different context elements like user demography, devices and software capabilities etc. Domain context modelling is also provided by some researcher using semantic and ontological frameworks [4,5]. Context, in previous researches [6,7] are modelled with the consideration of exact requirements and very specific to the application domain. There is still need to study the elements of context influencing the usability aspects of mobile device. There may be a merging of distinct context models stating variety of context elements like users, tasks, device and environment [2,3]. The previous solutions also provide programmable libraries, overlays and patches to add adaptivity in current user interfaces. These user interface adaptivity solution are not aligned with any standard user interface development process model [5,7]. Thus, the AUI designing and development need to be enriched with ontological based context modelling. The adaptive user interface models are also needed to be formed in response to the context.
1.1 Usage of Context
The concept of context is related to user emotional and behavioral attention, time, place and situation, also the surrounding entities and users in the user's space. Additionally, user's environment that the user's computer knows about is another way of describing synonyms for the term context. To define information that could be considered as context, should be more precise during the development of adaptive-applications .
1.2 Mobile Contextual Model
As of now, there are a huge number of cell phone users accessible all around because of the quick adjustment of adaptive processing. Unreasonable ease of use of cell phones has massive information of user inclinations. User profiles, settings, circumstances and depiction of semantics give the premise to an interest driven customized data and administration co-ordinations. Every one of these portrayals depend on various leveled set of properties that describe a substance, instantiated as user profiles, circumstance profiles and administration profiles . Sensors accumulate setting estimations of all users naturally and at whatever point a user altogether changes his or her unique circumstance, the (new) circumstance is determined powerfully by the induction motor. At last, the administration coordinating gives all appropriate setting qualities to the user's circumstance and profile, gathered into classifications. This refresh the real arrangement of suggested administrations at the user's personal digital assistant. The majority of this is finished utilizing semantic principles characterized based on various classes e.g., gadget, user, undertaking and condition
1.3 Ontology for the Context of Use
User context ontology is used to define formally the concepts of usage context of user in detail. The context elements can be read and understood by any machine or computational system with formal definitions stated in OWL/RDFs . While defining context, it is referred as knowledge which is utilized to explain a particular functionality of an entity while that entity or object can be a user or a location . Furthermore, the context can be look-attentively as a specific type of knowledge that can be modeled as ontology. Context based ontology models permit to represent complex and formal semantics to context knowledge, which provisions the sharing and/or integration of context information . Even though, distinct ontologies have been developed with different perspectives and approaches, however, there is no mutual model available that can be used/reused for modeling context in applications . The ontologies have capacity to communicate information and the ability to represent entities to name the concepts in machine readable form. It can define these concepts in different classes and specific instances. Humans have concepts, based on knowledge cognition including system devices, senses, experiences and context . These concepts are abstract, vague, composite or real. Humans create mental model and expect system and device to behave accordingly. All automated system comprised of software and hardware provide a computational model of real-world concepts. Ontologies are used to express these concepts formally to define a computational model. Ontology driven information systems approach is being used to design the application in human's perspective .
Thus, exploiting the benefits of ontologies to formally state the usage context of adaptive user interfaces is established in literature previously. The ontology may be comprised of representational terms for the elements of the mobile usage context. This type of ontology represents association among objects with natural classification, for example relations, object constraints etc.
The research is organized into 5 Sections, next Section 2 provides the literature review and covers the second phase of research methodology, in which User context ontology (UCO) modeling is performed that leads towards Adaptive interface ontology (AIO). Section 3 adaptive interface ontology as well as demonstrates the research methodology in terms of ontology engineering for user context, story boarding, use case scenarios, vocabulary, taxonomy, semantic relations, constraints, verification through Reasoner and SPARQL queries also discusses the results generated through reasoning tools and querying the defined use case scenarios. Section 4, enhanced user center design model, the challenges to successful conduction of this research and the impacts including research questions are also described. while in Section 5 the intelligent adaptive interface model is defined. Finally, in Section 6, the conclusion and future work is presented precisely.
2 Literature Review
The adaptivity features analysis, in previous research, have proven the role of adaptive interfaces in the level of usability conditional to its harmonization with the usage context. A detailed context analysis is, thus, required to adapt the interface accordingly. Different situations exhibit variety of context patterns which are directly influencing on the usability of tasks to be performed at mobile devices by users. In this research, the user context modeled through knowledge engineering process to handle the contextual complexity faced by mobile phone interfaces. This research covers the second phase of research methodology, in which UCO  modeling is performed that leads towards adaptive interface ontology.
The major participants, including user, task, environment and device, are defined as base concepts of the usage context. The ontology, engineered here, adopted many context concepts from well matured and widely used ontologies presenting standard definitions. Here these formal definitions have joined with some additional concepts studied through adaptive features analysis to provide a comprehensive mobile device usage context. The UCO enables to capture all the information according to the context of user. It changes the user interface characteristics automatically according to the situation. The UCO is responsible to discover the dependencies and semantic relations amongst the classes and attributes. The target here is defined in this research by the competency questions that can define, store and retrieve user context with its semantic relations and constraints.
2.1 Usage of Context and Ontologies
The context is any data that can be utilized to portray the circumstance of a substance identified with association among application and user. It is difficult to check the vital parts of all circumstances that will change as per conditions. Some relevant data, for example, restriction, time, date and close-by individuals is expected to translate something in setting . To show the setting of user, people have diverse ways of life with their individual or aggregate needs. There is a need to improve client profile to accomplish the customized administrations with the capacity to adjust client's specific situation. The ordinary and basic data that in the long run prompts an assessment of user's inclinations is the present circumstance. For instance, client may be “driving a vehicle” and the setting changes constantly. In like manner, the circumstance “taking selfie” is the user setting where cell phone use can't change amid certain time interim . These kinds of circumstances may see user setting to adjust changeability of activities and presentations to accomplish the objectives of user which will make the interface increasingly adaptable.
The context can be estimated a specific sort of learning which can be displayed as an ontology. An ontology-based setting models are powerful methods for sharing and combination of setting data [19,20]. A client setting ontological model can be created based on client's qualities, for example, individual data, aptitudes, inclinations and setting to give customized administrations . For the most part, people comprehension and ideas are diverse for different circumstances, for example, at-home or in-kitchen. Computationally it is required to characterize these ideas through induction rules or semantic relations or classes on a philosophy. As of late, various association styles and methods of cell phones make complexity of interface communication [22,23]. Over the most recent couple of years the examination on user profiling and context has been supported for the improvement of adaptive frameworks that are to be utilized by heterogeneous clients . In such manner, ontological displaying on user profiles is by and large application explicit that has been made for explicit area or assignment. The adaptive interface request to be a keen arrangement that can adapt new settings, sense existing settings and alter itself as per any obscure setting. Formal ontology is a strategy to grow such setting knowledge-base. It has capacity to keep all the setting picture steady, institutionalized, shared and versatile .
Adaptive user interfaces (AUIs) are known for more than a decade for providing context fitting interfaces for mobile devices. It is desirable that interfaces may adapt their behavior when circumstances change rapidly according to context. This research provides the interface modeling over studied context as a subsequent ontology, AIO. In the third phase of this research plan, AIO is developed through the standard ontology engineering incorporating UCO instances. The interface model, presented through AIO, merges the context knowledge and depicts the reflective interface through context to interface mapping knowledge [25,26].
2.2 Adaptive Interface Context Modelling
This part of research discusses the methods to inculcate the user context to adaptive interface development. It includes the mapping of user context ontology to interface ontology resulting in a merged adaptive interface ontology. As per requirements of User centered design (UCD) process model the UCO specifies the user context, mapping functions provide the impact of user context on interface development and the resultant product is tested against the user requirements. The complete ontology engineering process for adaptive interface ontology is followed here. To conclude the research, complete path proposed in research methodology is verified and an enhanced UCD model is presented. The UCD model is enhanced by providing further depth in terms of context specification, adaptive interface modeling and verification using ontologies.
2.2.1 User Context Ontology (UCO)
Previously authors  has provided complete ontology engineering process, verification through logical reasoner and testing according to user requirements. The user context ontology provides a base classes and semantic relational structure for instantiation of any mobile usage context. The plea here is to read and formally describe the usage context containing relevant details about user, environment, device and task. Adaptation of a user interface style is dependent on the context knowledge presented by the UCO.
2.2.2 Adaptive Interface Ontology (AIO)
The adaptive interface development is modelled and verified by AIO that merges UCO with adaptive interface elements. It takes context as input and prescribe adaptive interface model using semantic relations over provided mapping functions. These mapping functions are executed over the context instances and available device interface elements. These mapping functions are built against the provided user requirements stating the specific interface style for a specific context.
2.2.3 User Context Ontology Mapping to Adaptive Interface
Rendering the previous use cases described in UCO engineering process for different context, respective interfaces styles are followed. In this research, the functions are made on individual context and corresponding interface style basis. In future the usage context can be categorized (based on combination of different dimension like user, task, environment and device) mapped to interface styles according to the respective categories.
3 Adaptive Interface Ontology Engineering
The adaptive interface ontology has been engineered considering its integrated existence with user context ontology. The aim of the engineering process is to design and develop device interface model well connected with the interaction context.
3.1 Competency Questions
The competency questions of under design ontology focuses on adaptive user interface modeling based on the context read through user context ontology. The questions are listed as follows:
CQ1: How usage of context can be associated with the Interface Elements?
CQ2: How context properties effect the mobile interface?
CQ3: How mobile interface model can be affected by its usage context?
3.2 Defining Context and Interface Parameters
Sample use cases for model verification are shown in following tables (Tabs. 1–5). The different contexts and their interface parameters are defined according to user requirements.
Tab. 1 presents the context of use case I: “A child is recording rain fall video by using mobile phone in the room window at night”. It shows the context of user “Child” where it comprises that Has Time: Night, Has Device: Mobile Phone, Has Interface: Interface One, Has Activity: Reading Rain Fall Video and Has Location: Room Window. According to context the behavior of interface is mapped on the parameters such as Flashlight: On, Mobile Data: On, Camera: On, Wifi: On, Brightness: medium and Background Image: Image 1.
“Use case II is defined for Interface Two as “An illiterate person is listening news by using mobile phone in TV lounge at night” in Tab. 2. The context of user “Illiterate Person” is given and the mapping values set the behavior of interface as per desires.
An office worker is doing video conferencing on mobile phone in office at noon” is a specified use case III shown in Tab. 3 It shows the context for user “Office Worker” where it includes specific context which mapped the values at Interface Three.
The mapping Tab. 4 is plotted the instances selected by the context for user “Student” in use case IV “A student is downloading lecture slides through mobile phone at class room in morning”. The Interface Four is drawn according to user's preferences such as Font Size: 14, Volume: 05, Wifi: On, Mobile Data: On and Brightness: Medium.
The interface taxonomy is described in Fig. 1 where condition properties give ordinary settings as a feature of entire context class. The properties are made by the example situation necessities. For example, interface one generated against context one which has medium brightness etc. Similarly, other interfaces are recognized according to their given context elements.
The Fig. 2 demonstrations the concept of adaptive interface ontology semantic relations. It characterized the relations according to user's preferences for specified context. Semantic relations show the properties, classes, subclasses and functions for device contextual interface. Here every idea has its semantic relations with the created taxonomy.
In Fig. 3 the given situation determines that “Interface One” for “Context One” contains object properties Has Background image: Image 1 and Has Brightness: Medium. The interface data properties are defined as is Flashlight On: true, is Mobile Data On: false, is Camera On: true and is Wifi On: true. The characterized requirements are connected on several users allowing them to be in association with obvious choice of qualities. The constraints are demonstrated that “is Wifi On: false, is Flashlight On: false and is Camera On: false”.
The constraints, “is Wifi On: false and is Flash Light On: false” for “Interface Two” of “Context Two” are shown in Fig. 4.
The Fig. 5 shows the constraints for interface three, “is Flash Light On: true, is Wifi On: false, is Message Status On: true and is Call Status On: true”.
The Fig. 6 shows object, data and negative properties for interface four in context four. The constraints for data properties in specified context are, “is Flash Light On: false, is Message Status On: false and object property is Has Flashlight: On”.
The constraints for interface five of stated context are described in Fig. 7. The constraints for data properties are, “is Flash Light On: false, is Camera On: false” and object properties are “Has Message Status: Off, Has Bluetooth: On and Has Camera: On”.
3.5 Model Verification
The ontological model of adaptive user interfaces is verified over its logical presentation and knowledge representation in specified interface cases.
3.5.1 Logical Verification Through Reasoner
Hermit and Pellet reasoning tools were used to verify the ontological model as well as defined instances. The reasoning tools verify the consistency of semantic relations and constraints of base classes and their individuals.
3.5.2 Experimentation Using SPARQL Queries
The SPARQL querying is used to verify the storage and retrieval of interface knowledge associated with the context [27–29]. The context is passed to knowledge-base as querying argument and respective interface details are inquired.
The Fig. 8 presents the query for the interface details extraction respective to context one. The context and associated interface are defined according to user requirements in UC-1 scenario presented in Tab. 1. The query provided interface model detailing all the properties need to be set for specified context. These details of properties include “is Flash Light” with value “on”, interface “brightness” is “Medium”.
Extraction interface using SPARQL query for context two is displayed in Fig. 9 The context and associated interface are defined according to user requirements in UC-2 scenario presented in Tab. 2 The query provided interface model detailing all the properties need to be set for specified context. These details of properties include “is Vibration” with value “on” and Volume is “0”.
The Fig. 10 shows the extraction interface for context three as in Tab. 3. The scenario in UC-3 is accessible. The interface extraction by query describing all the properties need to be set for specified context. These details of properties include “is Flashlight” with value “on” and Volume is “10”.
The Fig. 11 illustrates the extraction interface for the context of UC-4 according to user preferences as in Tab. 4. The query provided interface model detailing all the properties need to be set for specified context. These details of properties include “is Mobile Data” with value “on” and Font Size is “14”.
The Fig. 12 shows the extraction interface for context five. The scenario is presented in UC-5 for context and related interface according to user necessities as in Tab. 5. The interface extraction by query describing all the properties need to be set for specified context. These details of properties include “is Call Status” with value “on” and Message Status is “auto generated”.
3.5.3 Adaptive Interface Development
The formal ontological modelling has made context specifications, shared, standardized and machine-readable. In this research, it is combined with the device properties not only in logical models but also in the development environment. AIO is developed using Protégé software though the ontology engineering process. The competency questions frame the objective of the ontology while use-cases presents the scenarios of variety of user interface styles for different context. The ontology presents the mobile device interface adaptivity features semantically connected with the context elements drawn from UCO. AIO is also verified for its consistency by the reasoning software and various adaptive interface styles are verified for given context use-cases using SPARQL querying. The context to interface mapping is also simulated for an android based mobile device interface in a JSON fashion with combination of context and interface. This combination of usage context model and subsequent interface model is carried forward in next part to be used in UCD process model.
4 Enhanced User Center Design
This research conducted to enhance UCD. The context and adaptive interface ontological models combined in this enhanced UCD to represent the usage context and respective adaptive interface. The UCD model presents the need identification and context modelling at the initial phases while user requirements and solution design development are next phases. UCO is being used in the UCD for initial phases while AIO provide detailed solution design based on the context presented by UCO. This research also identifies the responses to the formulated research questions found in all research phases. At the end, future work in the current research directions is discussed.
4.1 User Centered Design Enhancements
In this research the questions are defined to deal with better usability and satisfaction of user with customized interfaces according to user's context. The challenges to successful conduction of this research and the impacts of research are also described.
Following are the research questions:
RQ1: How AUIs can map the context knowledge to User Interfaces?
RQ2: How adaptive mobile interfaces improve user efficiency and satisfaction as compared to non-adaptive interfaces?
RQ3: How context aware AUIs can be designed by UCD paradigm?
Adaptive interface needs user collaboration during its designing and development for better usability. Question wise conclusion of this research is presented in below given sections.
4.2 Context Knowledge Representation
RQ1: How AUIs can map the context knowledge to User Interfaces?
UCO is built to represent the context knowledge used for mobile device interface. The ontology is developed through the ontology engineering process and duly tested over its consistent formation and user requirements. The ontological representation of context provides context knowledge-base that can be reused to define known contexts in interface development phases like designing and testing. Being machine readable, it also facilitates the automated development and verification process of user context driven interfaces. AIO reads user context through the ontological model and create interface instance according to the user requirements. This research has discussed the AIO engineering and testing details. These context and interface ontologies are enriched with semantic relations and constraints with growth capacity. They provide a knowledge base for mobile usage context and respective interface styles. With every context read and developed interface style, its data instances will be grown. The deductive reasoning capabilities allows the knowledgebase to select interface styles for all known context patterns. The future research in this regard includes intelligent and efficient mechanism to handle massive data read by many of the mobile users switching in variety of contexts. It will require handling many issues related to big data analytics, communication patterns and individual's privacy.
4.3 Adaptive Interface Impact
RQ2: How adaptive mobile interfaces improve user efficiency and satisfaction as compared to non-adaptive interfaces?
The answer of this question shows the impact of effectiveness, efficiency and satisfaction for the environment of AUI. It is identified that usability issues of adaptivity still exist due to uniform adaptive features provided by smart phone vendors regardless of user ability and task context. Currently, it is more on user's choice to turn on or off any adaptive feature while performing specific task. The experimental results advised that the interface should be more equipped with adaptive features to increase the usability of smart phones. It is also suggested that user and task context should be studied or sensed for switching to any adaptive environment. Diverse user's contextual behavior can be used for many changes and functionalities that will help for future tasks. UCD has been found to supply relevant adaptive information for linking functions to their better contextual behavior.
4.4 Enhanced UCD
RQ3: How context aware AUIs can be designed by UCD paradigm?
Current section presents enhance UCD model incorporating the ontology driven modelling of user context to develop an adaptive user interface.
The phases of enhanced UCD process model are prescribed in Fig. 13. The engineering of UCO and AIO for mobile device interfaces in user's context. In the context aware system development for mobile adaptive user interfaces, more detailed investigations are required including scales and value ranges etc. Elicitation of user context is a precondition for analysis and modelling of the context. Modelling user context provides the classification of user-based groups, combinations and semantic relations. Overall UCO enables to capture all the information according to the context of user. It changes the user interface characteristics automatically according to the situation. The UCO discovers the dependencies and semantic relations amongst the classes and attributes. The adaptation is based on the user context which is influenced by goals and associated tasks. Interaction mode represents the information of parallel assistance for visualization and speech. It also deals with user response modes such as look and feel. AIO enables all the interfaces to change its behavior according to the user needs. It also describes the mapping functions of UCD with UCO to AIO which leads towards the experimentation and evaluation of results for the satisfaction of users. Refinement of UCO and AIO is made on the basis of mapping rules and evaluated results. All features of UCD is provided real benefits to the user experience. It focuses on iterative refinement to the usability of interface to fulfil the needs of users. In UCD, the prototypes are very useful to translate the user requirements into contextual experience. It is used to enhance the usability, satisfaction and optimization of adaptive mobile user interfaces.
5 Adaptive Interface as an Intelligent Interface
The Fig. 14 illustrates the architecture of this intelligent adaptive user interfaces, sensors are used to sense the context information such as location, direction, sunlight and time etc. This model provides the connectivity and data sharing with cloud for decision making. The model also presents the adaptation strategy that can be triggered from the device with the sensing of context change. The knowledge cloud can provide the suitable interaction style on the basis of context pattern matching. New context patterns can be added to the knowledgebase with user selection of interaction styles.
Building mobile context-aware systems is inherently complex and non-trivial task. It consists of several phases starting from acquisition of context, through modelling to execution of contextual models. Today, such systems are mostly implemented on mobile platforms, that introduces specific requirements, such as intelligibility, robustness, privacy, and efficiency. Over the last decade, along with the rapid development of mobile industry, many approaches were developed that unevenly support these requirements. This is mainly caused by the fact that current modelling and reasoning methods are not crafted to operate in mobile environments . Human activity is one of the most important pieces of context affecting an individual's information needs. Understanding the relationship between activities, time, location, and other contextual features can improve the quality of various intelligent systems, including contextual search engines, task managers, digital personal assistants, chat bots, and recommender systems .
Systems capable of changing their internal functioning must take decisions according to a number of variables. These decisions could be taken on different basis and could be the output of different reasoning processes.
7 Conclusion and Future Work
In future the devices may have strong communication with clouds in which the adaptive environment can be extended to handle the situation by using collective intelligence. For example, if some information is missing it can be acquired from other sources like mobile phones in surroundings. Data mining, context knowledgebase, ontology modelling and cloud computing may be involved to convert data into wisdom. At the end, the wisdom is transformed into adaptive user interface development for UX through UCD process model.
A rich knowledgebase can be developed and populated for the context experienced by various mobile device users, user preferences for the interaction styles and their combinations. Currently the competitors of mobile service industry win through the most usable devices and services that requires intense consumer behaviors analysis and respective design decisions.
Acknowledgement: We thank our families and colleagues who provided us with moral support.
Funding Statement: This research is supported by the Ministry of Culture, Sports and Tourism and Korean Creative Content Agency (Project No: 2020040243).
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
|This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.|