Assessment using Bloom’s taxonomy levels has evolved in a variety of contexts and uses. In the era of the COVID-19 pandemic, which necessitates use of online assessment, the need for teachers to use digital-based taxonomy skills or Bloom’s Digital Taxonomy (BDT) has increased even more. However, the existing studies on validity and reliability of BDT items are limited. To overcome this limitation, this study aims to test whether BDT has good psychometric characteristics as a teacher’s self-assessment tool using the Rasch model analysis and to investigate the pattern of BDT usage in teaching and learning. By using a quantitative online survey design, this study involves six levels of BDT, namely, Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating. The questionnaire was developed and validated by two experts prior to administration. A stratified random sampling technique was conducted on 774 secondary teachers from five geographical zones in Malaysia, and the Rasch model was analyzed using WINSTEPS 3.71 software. The performances of items improved by Rasch psychometric assessment including the application of BDT among teachers. The hierarchy level was also assessed through graphical analysis, including the Wright map and bubble chart, to demonstrate the powerful performance of the Rasch model analysis in investigating item quality and reliability. Overall, these empirically validated items using the Rasch model could advance the academic knowledge of BDT for future assessment and promote the Rasch calibration in an educational setting.
The term “taxonomy” originates from the Greek words “taxis” and “nomos,” which refer to “order” and “method,” respectively. This term may be referred to as an arrangement or a law in a specific order that is borrowed from biology, which allows certain classifications of the order. In the development of effective methods to perform mental operations, the notion of ordering is essential to classify these operations and skills and to determine the formation sequences in order to grow and solve certain problems [
In 2008, the era of Bloom’s taxonomy named the Bloom’s Digital Taxonomy (BDT) was introduced by Andrew Churches [
In recent years, there has been an increasing amount of literature on BDT, which has been published. Some studies have been interested in examining the BDT application to the teaching process. The related studies have shown that the role of teachers in BDT is of great significance when it is used in the classroom. In fact, previous studies have tended to criticize the original Bloom’s taxonomy because of its changes toward the digital.
In [
In [
In [
The adaptation of the revised taxonomy to a new generation of students and a general summary of how Bloom’s Original Taxonomy could evolve to Bloom’s Revised Taxonomy and initiate Bloom’s Digital Taxonomy were presented in [
Recent taxonomies of objectives and learning-objective strategies can be categorized in terms of the content types (e.g., facts, principles, procedures, concepts, and processes) and performance level (e.g., using and remembering). In [
In [
In Malaysia, only a few studies have discussed Bloom’s taxonomy from the digital-based aspect. Previous studies have reported that Bloom’s taxonomy has the potential to be applied to different fields, including vocational taxonomic proposals [
In [
As presented in this section, there have been many studies discussing BDT application to the teaching activities, and some of these studies considered the BDT usage in Malaysia. However, in order to measure the BDT, a very high quality of the measurement items is required. The related literature has emphasized the lack of psychometric assessment of items to measure BDT. This limitation can be overcome using modern assessment theory, such as the Rasch model, to ensure the reliability and validity of the measurement. Hence, the BDT practice application in the classrooms by teachers should be further studied. These findings could provide a useful reference in identifying the teachers’ ability to use each level of BDT maximally. In view of all mentioned, one may suggest an approach to examine the psychometric characteristics of BDT measurement items in more detail besides creating a new bloom. Hence, BDT discussion with efforts to provide empirical evidence for new psychometrics items or constructs will help researchers strengthen their future studies more meaningfully.
Conceptualization and operationalization used in this study entail BDT based on the definition presented by [
Level | Key term | Definition | |
---|---|---|---|
Level 1 (L1) | Remembering | Recalling and recognizing knowledge from memory to produce facts, definitions, and lists, as well as reciting or retrieving materials | |
Level 2 (L2) | Understanding | Developing meaning from different function types verbally or graphically | |
Level 3 (L3) | Applying | Utilizing or performing a procedure through execution or implementation, involving the use of learned materials through products, such as presentation, models, simulations, and interviews | |
Level 4 (L4) | Analyzing | Separating concepts or materials into parts to indicate the relation or interrelation between the parts relative to their overall purpose or structure. Mental actions denote the ability to differentiate, organize, attribute, and distinguish between components | |
Level 5 (L5) | Evaluating | Making decisions by checking and criticizing based on standards and criteria | |
Level 6 (L6) | Creating | Incorporating the elements for coherent or functional development and regrouping them into a new pattern through planning, generating, or producing |
No. | Remembering | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
1A | Bullet pointing | Students are able to produce listings in a digital format. | |
1B | Highlighting | Students are encouraged to select and highlight phrases and keywords as a recalling technique. | |
1C | Bookmarking or favoriting | Students are able to mark and organize resources, websites, and files for later use. | |
1D | Social networking | Students are able to establish networks among friends and partners by forging and creating associations between different individuals. | |
1E | Social bookmarking | Students are able to produce other tags and bookmarks (an online version of local bookmarking or favorites). | |
1F | Searching or “Googling” | Students are able to simply enter a phrase or keyword into the basic entry pane through search engines. |
No. | Understanding | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
2A | Advanced and Boolean searching | Students are able to produce, modify, and refine searches according to their needs. | |
2B | Blog journaling | Students are able to talk, write, or type a daily journal or a task-specific journal to understand the activity report. | |
2C | Categorizing and tagging | Students are able to organize, structure, and attribute online data and meta-tagging web pages (organizing and classifying websites, files, and materials into folders), as well as understanding the page content to be tagged. | |
2D | Commenting and annotating | Students are able to make comments and annotations on PDF files, web pages, and other tools, in addition to establishing understandings through comments on pages. | |
2E | Subscribing | Students are able to subscribe, read, and revisit the subscribe feeds for an in-depth understanding. |
No. | Applying | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
3A | Running and operating | Students are able to operate and manipulate hardware and applications in accomplishing a certain objective or goal. | |
3B | Playing | Students are able to operate or play a game whilst understanding the skill process, task, and application successfully. | |
3C | Uploading and sharing | Students are able to upload and share materials on sites such as Flickr. | |
3D | Hacking | Students are able to hack in simpler forms using a simple set of rules to achieve a certain objective or goal. | |
3E | Editing | Students are able to make editing with the most media (procedure or process employed by the editor). |
No. | Analyzing | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
4A | Mashing | Students are able to integrate or mash up a number of data sources into a single resource. | |
4B | Linking | Students are able to establish/develop associations across and outside web pages and documents. | |
4C | Reverse-engineering | Students are able to deconstruct to cracking without any associated negative implications. | |
4D | Cracking | Students are able to crack to comprehend and run the system or application to be cracked as well as analyzing and exploiting its strengths and weaknesses. |
No. | Evaluating | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
5A | Blog/Vlog commenting and reflecting | Students are able to comment and evaluate the material in context and reply to postings with constructive criticism and reflective practice. | |
5B | Posting | Students are able to comment on discussion boards, blogs, and threaded discussions. | |
5C | Moderating | Students are able to assess comments or postings from various viewpoints in terms of their value, worth, and suitability. | |
5D | Collaborating and networking | Students are able to effectively collaborate in assessing strengths, abilities, and contributions and have networking, such as engaging and communicating with concerned individuals through a partner network. | |
5E | Testing (Alpha and Beta) | Students are able to test the processes, applications, and procedures in developing tools by analyzing their purpose or process, correct functions, and their current functions. | |
5F | Validating | Students are able to affirm the accuracy of their information sources and make judgments by analyzing and evaluating the data sources. |
No. | Creating | I am doing this in my teaching and learning process to ensure that |
|
---|---|---|---|
6A | Programming | Students are able to create programs suitable to their needs and goals (applications, macros, multimedia applications, or games in systematic environments). | |
6B | Filming, animating, video casting, Podcasting, mixing, and remixing | Students are able to capture, create, mix, and remix content to produce unique products. | |
6C | Directing and producing | Students are able to view and understand the components to be melded into logical products (production or performance is an extremely highly-creative process in the creation of a product). | |
6D | Publishing | Students are able to publish not only text but also media or digital formats either from home computers or through the web, which requires an immense overview of the content to be published, as well as the process and products, such as video blogs, blogging, and also wikiing or mashups. |
In recent times, COVID-19 has been a major public health problem worldwide, including Malaysia, and it has been recording a large number of new cases exceeding a thousand cases daily. The break out of COVID-19 has affected many life aspects, including education. The recent increase in the number of COVID-positive cases has highlighted the need for transforming the teaching and education process from face-to-face methods to online education. The primary concern of this transformation is how to conduct lectures online since this is compulsory for all institutions in order to avoid the risk of further spreading of the COVID-19 virus. The need for online teaching and learning is high since all schools and learning institutions have been closed. However, there are many problems related to online education, such as problems of unstable Internet access, low student focus, incomplete equipment, and many others. Online learning also makes it difficult for some teachers to assess and test students’ achievements and knowledge. Namely, for cognitive assessment, teachers need to be adept at applying BDT since the learning process is conducted online. Thus, teachers should master and use BDT well in their teaching process. To measure the extent to which the BDT aspects are used among teachers, the evaluation of psychometric characteristics on the measurement items is necessary, and it is very important to ensure that the measurements are accurate, especially those involving the use of modern measurement theories.
Hence, this study aims to test whether BDT measurement items have good psychometric characteristics based on the teacher’s self-assessment using the Rasch model analysis, which represents modern measurement theory. The modification of assessment by Rasch model will be able to ensure that BDT items difficulty are match with the individual abilities. This study also examines the pattern of the BDT application in teaching from the teachers’ perspective.
This study adopts a quantitative approach using an online survey research design. The quantitative approach is used because it is suitable for a large number of respondents [
The measurement originally included 30 measurement items that involved 6 levels of BDT, including 6 items for Remembering (A1 to AF), 5 items for Understanding (2A to 2E), 5 items for Applying (3A to 3E), 4 items for Analyzing (4A to 4D), 6 items for Evaluating (5A to 5F), and 4 items for Creating (6A to 6D). These items were adapted according to the BDT definitions [
The data of this study entailed a dichotomous scale that was used to elicit a Yes or No answer [
The Rasch model assumes that each item comprises only a difficulty parameter and that all items have the same discriminatory index. This ensured that low-capable students could not obtain the correct answer to the items that they did not know by guessing [
In
The results considered several key parameters: (a) item fit and unidimensionality, (b) Wright map and a bubble chart, (c) mean measure by each level of BDT, and (d) reliability and separation index. The obtained results not only showed the quality characteristics of the psychometric items but also indicated the pattern of BDT usage in teaching and supervision from the teachers’ perspective. As explained in Section 2.3, there were 30 items assigned with 6 levels of BDT.
As shown in
Item | Score total | Measure | Standard error | MNSQ | PTMEA | ||
---|---|---|---|---|---|---|---|
Infit | Outfit | Correlation | Exp. | ||||
6A | 968 | 1.61 | 0.10 | 1.03 | 1.18 | 0.49 | 0.51 |
1E | 1012 | 1.20 | 0.09 | 1.04 | 1.13 | 0.50 | 0.53 |
5D | 1029 | 1.05 | 0.09 | 0.91 | 0.89 | 0.58 | 0.54 |
6C | 1029 | 1.05 | 0.09 | 0.91 | 0.89 | 0.58 | 0.54 |
6B | 1031 | 1.04 | 0.09 | 0.91 | 0.95 | 0.58 | 0.54 |
2C | 1040 | 0.96 | 0.09 | 1.00 | 0.95 | 0.54 | 0.54 |
6D | 1082 | 0.62 | 0.09 | 1.04 | 1.03 | 0.53 | 0.55 |
1C | 1083 | 0.61 | 0.09 | 0.82 | 0.79 | 0.64 | 0.55 |
2E | 1083 | 0.61 | 0.09 | 0.82 | 0.79 | 0.64 | 0.55 |
3A | 1095 | 0.52 | 0.09 | 0.98 | 0.98 | 0.56 | 0.55 |
2A | 1097 | 0.50 | 0.09 | 1.05 | 1.06 | 0.52 | 0.55 |
2B | 1117 | 0.35 | 0.09 | 0.97 | 1.08 | 0.55 | 0.55 |
2D | 1132 | 0.23 | 0.09 | 0.90 | 0.86 | 0.60 | 0.55 |
5E | 1132 | 0.23 | 0.09 | 0.90 | 0.86 | 0.60 | 0.55 |
3C | 1154 | 0.07 | 0.09 | 0.86 | 0.82 | 0.62 | 0.55 |
4D | 1184 | −0.16 | 0.09 | 1.06 | 1.11 | 0.50 | 0.54 |
4C | 1196 | −0.25 | 0.09 | 1.13 | 1.24 | 0.46 | 0.54 |
4A | 1225 | −0.46 | 0.09 | 1.12 | 1.17 | 0.47 | 0.53 |
3D | 1241 | −0.59 | 0.09 | 1.07 | 1.16 | 0.48 | 0.53 |
1D | 1288 | −0.95 | 0.09 | 1.04 | 1.08 | 0.48 | 0.51 |
3B | 1288 | −0.95 | 0.09 | 1.04 | 1.08 | 0.48 | 0.51 |
5F | 1288 | −0.95 | 0.09 | 1.04 | 1.08 | 0.48 | 0.51 |
1A | 1304 | −1.08 | 0.09 | 1.04 | 1.13 | 0.47 | 0.50 |
4B | 1304 | −1.08 | 0.09 | 1.04 | 1.13 | 0.47 | 0.50 |
5A | 1330 | −1.30 | 0.09 | 1.01 | 1.21 | 0.46 | 0.49 |
1B | 1345 | −1.44 | 0.09 | 1.01 | 1.09 | 0.47 | 0.48 |
5C | 1345 | −1.44 | 0.09 | 1.01 | 1.09 | 0.47 | 0.48 |
MEAN | 1163.8 | 0.00 | 0.09 | 0.99 | 1.03 | ||
S.D. | 114.5 | 0.90 | 0.00 | 0.08 | 0.13 |
Note: Mean square (MNSQ); Point-measure (PTMEA).
The highest measure value was that of item 6A (1.61 logits) and the lowest values corresponded to 5C and IB (−1.44 logits). Overall, activity 6A denoted “programming,” and 5C denoted “moderating.” The standard value of errors was in a range of 0.09–0.10 and complied with the recommended value [
Based on the MNSQ fit settings, the used range was from 0.77 to 1.30 [
The unidimensionality of items indicated that items did not have equality characteristics in the matter to be measured. Dimensionality can be defined as determining an instrument in one direction and one dimension or the force given to one dimension or attribute at once [
Wright Map or item-person map in this analysis denotes a figure that represents items by the item number and the performance of each person to effectively observe the ability of the measured scale items to match the respondents. The distribution of the measurement items according to BDT levels from the aspect of their usage by teachers is presented in
A total of 55.5% of the total items were above the average difficulty value, while 45% of the total items were below the average difficulty value. This distribution proved that the respondents found it difficult to perform item 6A (Programming), by which the students were able to create programs suitable to their needs and goals (applications, macros, multimedia applications, or games in systematic environments). Meanwhile, the most easily performed activities by the teachers were item IB (Highlighting), by which the students were encouraged to select and highlight phrases and keywords as a recalling technique, and item 5C (Moderating), by which the students were able to assess comments or postings from various viewpoints in terms of their value, worth, and suitability. The results indicated that
The bubble chart that graphically illustrates the measurement value and item compatibility [
The L1 analysis (Remembering) showed that item IE was the hardest item with 1.20 logits. This result showed that Social Bookmarking was the least applied activity by teachers in the teaching and learning process at the L1 level. Meanwhile, activity 1B (Highlighting) included the most performed items by teachers with −1.44 logits. The L2 analysis (Understanding) showed that item 2C was the hardest item with 0.96 logits. This result indicated that Categorizing and Tagging was the least applied activity by teachers in the teaching and learning process. In contrast, activity 2D (Commenting and Annotating) included the most performed items by teachers with 0.23 logits. The L3 analysis (Applying) showed that item 3A was the hardest item with 0.52 logits, indicating that Running and Operating was the least applied activity by teachers in the teaching and learning process. Meanwhile, activity 3B (Playing) included the most performed items by teachers with −0.95 logits. The L4 analysis (Analyzing) showed that item 4D was the hardest item with −0.16 logits; thus, Cracking was the least applied activity by teachers in the teaching and learning process at the L1 level. However, activity 4B (Linking) included the most performed items by teachers with −1.08 logits. The L5 analysis (Evaluating) showed that item 5D was the hardest item with 1.05 logits; Collaborating and Networking was the least applied activity by teachers in the teaching and learning process. Meanwhile, activity 5C (Moderating) included the most performed items by teachers with −1.44 logits. The L6 analysis (Creating) showed that item 6A was the hardest item with 1.61 logits, and Programming was the least applied activity by teachers in the teaching and learning process. Last, activity 6D (Publishing) included the most performed items by teachers with −1.44 logits 0.62 logits.
Based on the mean logit value results, the hardest level of BDT to be conducted by teachers in their teaching and learning was Level 6, which was Creating (+1.08), followed by Level 2 (+0.53), Level 3 (−0.24), Level 1 (−0.33), and Level 5 (−0.48), while the easiest level was Level 4, i.e., Analyzing. Based on the level of BDT, Level 6 (Creating) is the highest level of BDT. This level represents the hardest level from the aspect of activity implementation. Thus, the results presented in this study are logical because the logit value of 1.08 has indicated level L6 as the most difficult activity for teachers to implement. However, interestingly, the results showed that the easiest level to be implemented by the teachers was not L1 (remembering) as it was expected but L4 (analyzing). According to [
This result could be caused by the elements of Higher Order Thinking Skill (HOTS) instilled by teachers to the students. The Malaysian Education Development Plan 2013–2025 explains that national examinations and school-based assessments (PBS) have been revamped to gradually increase the percentage of questions that define high-level thinking skills. By 2016, high-level thinking questions included at least 40% of the questions in
The rating scale instrument quality criteria used in this study are based on the setting in [
The overall separation value of persons was
The current study aims to improve the measurement for BDT items through teachers’ self-assessment in teaching and learning, and the Rasch measurement model is proposed for the assessment of psychometric properties. The results show that a total of 27 measurement items can be used as an alternative for BDT measurement using the Rasch model. The results show that the Rasch model can more clearly demonstrate various item properties compared to the classical test theory. Moreover, this study indirectly shows to which extent teachers tend to apply each level of BDT in their teaching and learning practice and examines which BDT activities are the hardest and easiest to apply. However, certain limitations need to be considered in future works. First, the results presented in this study are applicable only to the Malaysian population, so the study should be expanded regarding both contexts and countries. Namely, it would be interesting to explore and compare more characteristics of item response for various levels of respondents’ ability through systematic comparisons. Second, this investigation has been limited to the teachers’ perspective, so future research is highly encouraged to introduce scale analysis to develop specific questionnaires from the perspective of students’ understanding of the BDT levels. In fact, this measurement construct can be tested for its validity using multivariate analysis, such as factor analysis or principal component analysis, to provide empirical evidence for future reference. Third, this study is limited to general definitions in each level, starting from L1 to L6, so further investigations can be performed to each specific activity for every level of BDT. The information can be useful for customizing digital teaching activities that suit both teacher and students’ abilities relative to the implementation of teaching and learning in class.
The author would like to express the utmost gratitude to the anonymous reviewers for their insightful comments and suggestions, which helped to improve the content, quality, and presentation of this research paper. The author would also like to honor and give high appreciation to the Faculty of Education, University Kebangsaan Malaysia (UKM).