Life Skills are considered a critical aspect in engineering practice. Not only are these skills needed in order for an individual to get a job, but also to still remain employable. Despite all the efforts that have been done in trying to improve Life Skills assessment, the impact of its implementation in Brunei Darussalam still remains elusive. This study aims to help students develop these skills by integrating feedback within their assessments. Two groups were compared and analyzed during 4 weeks activity to see the impact of this strategy. Using both quantitative and qualitative method, the study sampled 48 students and 2 teachers with a Mechanical Engineering background. Paired t-tests were performed to determine participant group’s achievement of Life Skills at the end of the activity. The results (calculated p=0.00) suggest there is a significant development in their Life Skills. However, in spite of the encouraging outcomes, there are still some major issues in its implementation. Qualitative analysis done with the instructors involved exposed feedback to being time consuming, labor intensive and it is dependent on the students to act upon it for improvement.
Key words: Life skills, employability skills, transversal skills, assessment, feedback assessment
1.1 Background of the study
Life Skills have been defined as ‘psychosocial abilities for adaptive and positive behavior that enable individuals to deal effectively with the demands and challenges of everyday life’ (UNICEF 2015). Over the years, it has commanded a central place amongst engineering practice all around the world by industry representatives (Connelly & Middleton 1996) and engineering educators (Shuman et al. 2005). These skills are needed not only to get a job, but also to remain employable. Although a Life Skills programme has been implemented in Brunei’s Vocational and Technical Education (VTE) system over the past 12 years, the impact of this programme remains elusive which questions its potency. Further research revealed several barriers present in the current Life Skills programme (Kahar 2006; Rosli 2012; Paryono 2014). Since its implementation in 2001, the policy guide on life skills has been regularly reviewed and updated to meet the needs of both students and employers. From an implementation perspective, it was observed that there is room for improvement when conducting assessment for life skills, and that feedback to the students has not been stressed adequately in the current practice and is often over-looked.
Feedback is a vital part of assessment in terms of closing the skills gaps between students and industrial standards (Ramaprasad 1983; Sadler 1989). Providing students with feedback can help them develop and nurture these skills according to the set standards. The lack of feedback results in students not having any room for improvement which indirectly affects their employment performance. The importance of giving effective feedback should be emphasized among the teachers under Vocational and Technical Education (VTE) to ensure consistency across assessors and the quality of certification for graduate skills. Since the Life Skills programme in Brunei Darussalam is taken as part of “integrative’ courses to be embedded in each module, feedback offers an alternative teaching and learning tool for students to develop their skills. Although research done on the role of feedback in engineering education is sparse, promising findings on the use of feedback on writing (Srichanyachon 2012; Taylor 2011; Mackiewicz 2004) and communication skills (Bjorklund, Parente, & Sathianathan 2002) has prompted this research.
1.2 Research Questions
Most research done on the assessment of Life Skills was conducted in western countries and in Australia mostly for Vocational Education and Training (VET) within industries but only a handful of studies were done in Vocational and Technical Institutions (VTI). Therefore, significant approaches between these two frames may differ. The main aim of this study is to investigate the impact of feedback in assessment of Life Skills in a Vocational institution in Brunei Darussalam. Hence, with this aim in mind, the researcher sought to answer the following research questions.
- Does integrating feedback into assessment improve the students’ learning in Life Skills?
- What are the perceptions of the teachers towards the use of feedback in assessment of Life Skills?
2 Literature Review
2.1 Defining Life Skills
With pressure from government and employer groups for change, the time for considering Life Skills in academic environments has come. These skills refer to non-technical skills needed to effectively function as an employee and work for the betterment of oneself and the organization (Bakar, Mohamed, & Hamzah 2013). In Brunei Darussalam, Life Skills are integrated through modules in which eight Life Skills are to be assessed. Figure 1 is a brief description of each component of Life Skills as stated in the curriculum.
|2. Planning and Organizing
|4. Working with others
|5. Problem solving
|6. Initiative and Enterprising
|7. Applying numeracy, design and technology skills
Figure 1: Description of the eight components of Life Skills
Feedback is defined as “information provided by an agent (e.g. teacher, peer, book, self, experience) regarding aspects of one’s performance or understanding” (Hattie & Timperley 2007). The extant literature is clear that feedback is arguably the most important aspect of assessment in its potential to affect future learning and student’s achievement (Black & William 1998; Gibbs & Simpson 2004; Hattie & Timperley 2007). Although an assessment of hundreds of meta-analyses from 180,000 studies showed that “the most powerful single moderator that enhances achievement is feedback” (Hattie 1999), this is not always the case. Kluger and DeNisi (1996) found that more than one third of the effects indicated negative impact on learning. Based on further research, Nicol and Macfarlane-Dick (2006) found that effectiveness of feedback is dependent on the construct of teachers’ feedback. According to Villiers (2013) in-depth review, there are seven principles of highly effective feedback as illustrated in Figure 2.
Feedback within assessment is an integral part of the learning process because it aims at to contribute to the progression of learning and hopefully will support students’ improvement in Life Skills. Feedback is one tool that can be used to help students master their difficulties. With this strategy, the researcher is able to target in which area the students are struggling with respect to eight skills set by BDTVEC’s (Brunei Darussalam Technical and Vocational Education Council) Guide (2012). The feedback intervention will be done according to the Seven Principles above to ensure the highest quality feedback given to the students.
2.3 Types of feedback
The aim of feedback is to increase student knowledge, skills, and understanding in some content area or general skill (e.g., communication, teamwork, problem solving), and there are multiple types of feedback that may be employed toward this end (e.g., response specific, goal directed, immediately delivered) (Shute 2007). It is therefore, the teacher’s responsibility to acknowledge the different types of feedback and to know when to use them. Gilbuena et al. (2015) has summarized them as shown in Table 1.
Table 1: Types of Feedback
|Type of feedback||Literature source||Description|
Hewson and Little 1998;
Black and William 1998
Acknowledges a correct response and may include praise.
Tells the recipient what must be corrected.
Direct request for action.
Provides suggestions to guide the recipient toward his/her own revisions. Generally includes leading question.
For the purpose of this study, feedback was considered to be directive whereby each intervention has unique learning outcomes that are tailored to each participant’s needs.
This study is designed to investigate the impact of using feedback in Life skills’ assessment in Vocational and Technical students in Brunei Darussalam using both quantitative and qualitative data. In this study, students’ achievement will be measured as quantitatively whilst the teacher’s perceptions will be analyzed qualitatively to tie the results together. The convenience samples of this research are two groups (48 students) of Mechanical Engineering Skills Certificate 2 level students. The two teachers, who were interviewees for this research, have had years of teaching experience. They were interviewed at the end of the study.
The alignment of research questions and data collection instruments is presented in Table 2.
Table 2: Outline of instruments in relation research questions
1. Does integrating feedback into assessment improve the students’ learning in Life Skills?
|Life Skills Grading Criteria Assessment|
2. What are the perceptions of teachers towards the use of feedback in assessment of Life Skills?
- Life Skills Grading Criteria Assessment. The Grading Criteria Assessment was adopted from the BDTVEC’s 2012 policy guide. All the eight Life Skills components were assessed based on their performance in the activity. Data from both groups were measured and compared using the before-and-after design. The ‘before’ observation is established before the intervention, and was compared with the ‘after’ observation at the end of the study (Kumar 2005). In this study, their first semester’s Life skills results are taken as baseline results.
- Semi-structured interview. With relation the second research question investigating the teachers´ perceptions towards integrating feedback into grading of Life Skills, the researcher conducted semi-structured interviews with the two teachers who assessed the students during the activity. The findings from these interviews were beneficial as it could be used to elicit respondents’ perceptions, as well as having different perspectives than the researcher. The interview was done at the end of the activity. It was recorded and transcribed to validate and assist the evidence on the printed document.
- Feedback strategy. The design of this study consists of two groups, which are the ‘Feedback’ group and the ‘Conventional’ group. This type of ‘before-and-after’ design is used to compare and determine whether the intervention has an effect on the achievement of the students. The conventional group on the other hand, is assessed traditionally as per usual where the teacher records and grades without guiding the students. Reflecting from the procedures done in several case studies reviewed by Juwah et al. (2004), the standard procedure in implementing feedback into assessment is a step-by-step procedure as shown in Figure 3.
It is crucial to dedicate the initial stage of learning towards briefing the students with what is expected from them. By introducing Life Skills at the early stage, it will enable them to know what the requirements are and goals they need to fill in order to do well in Life Skills.
The first part of the activity (practical session) requires the students to work in groups of four authentic learning was used to test their critical thinking skills in relating real-life environment with theoretical knowledge. The students were assessed using the rubric with respect to the eight Life Skills (see Figure 1 above) during the practical sessions. Each teacher assessed two groups of students simultaneously during the course of 4 weeks of activity.
Once the students have completed their lesson, the teachers administered a face-to-face feedback intervention for their respective groups. In order to ensure the reliability of the assessors, the researcher was present in assisting the two teachers when giving feedback to the students for consistency measures. The researcher focused on the areas within the eight Life Skills components that needed further improvement. Students provisional grades were shown to them according to the assessment criteria for the Feedback Group. Sadler (1989) argues that students must know: what good performance is (the concept of a goal or standard); how current performance relates to good performance; and how to act to close the gap between current and good performance.
4 Results and discussions
4.1 Quantitative data
Results of the students from their first semester were taken as baseline to compare the two groups. For both semesters, assessment of Life skills was done using the Rubric Evaluation provided by the BDTVEC’s 2012 policy guide. Analysis of the data for the Feedback group showed that the majority of the students improved their skills over the four weeks course of the activity as evidently shown in Table 3. It was observed by the assessors that the students were more aware of what was expected of them and knew they were being assessed throughout. This resulted in their putting in more effort to enhance and present their skills and this was reflected in their final and baseline results comparison.
Table 3: Individual Baseline and Final Life Skill’s results (both groups)
|Student Number||Life Skills (%)||Student Number||Life Skills (%)|
Most of the students in Feedback Group managed to improve the performance criteria and scored higher than they did in Semester 1. The overall mean score for the group is 67% However, this was not the case for students F11 and F20, their mean scores did not improve as much as others and only made a point difference from Semester 1 for the case of Student F20. This student in particular was monitored closely as it was noticed in the beginning that he needed more attention in developing these skills. It may be argued that Life Skills are leaning more towards habits, and therefore take time to change. The student may not be able to perform and give their best as there is a limited time of four weeks to do so. Life Skills is known to be enhanced and practiced over a period of time in order for it to be embedded into our everyday lives. When Student F20 was confronted during one of the feedback sessions, he admitted that he did not comprehend the teacher’s feedback as the criteria are too generalized.
On the contrary, for students who were assessed conventionally (without feedback) through their activity, they did not improve as much and remained stagnant. In addition, most of them scored within the passing mark margin of 50%, which is only considered as ‘Satisfactory’. Although at the beginning of the activity both groups were briefed with necessary knowledge on the eight Life Skills components, it could be argued that the Conventional group were not able to perform as well as Feedback group because of the different instructional design given, where the first group had feedback from the assessors themselves and able to practice them throughout the activity. For example, ‘Self-management’ skill, the students from Feedback group were advised and encouraged to record what they have done into their laboratory book as usually done in industry. However, the Conventional group was not given any directive nor facilitative feedback to remind them to do so and hence, they did not carry out the task professionally and this was reflected in their scores. Therefore, this result suggests that there is a difference in the scores between the two groups.
To further validate the difference between performance achievements of the two groups, the results were further analyzed using IBM Statistical Package for Social Sciences (SPSS) Version 21.0. Both mean scores and standard deviation of score distribution were calculated using paired t-test at which scores of two different variables with the same group of cases are compared. The null hypothesis used is there is no difference between the mean results of the students exposed to feedback given by the teacher and the students who were assessed as usual i.e. both groups should perform similarly. The descriptive result of the paired t-test was summarized in Table 4 below. Results from the paired t-test on the baseline means scores revealed that there is not much difference between the two groups (XA=51.5; XB= 50.8). This result suggests that the students have the same level of Life Skills and this has been the trend in Vocational institutions in Brunei Darussalam. During Semester 1, teachers assessed the students without guiding them on how to improve and were merely dependent on the student’s own effort. Paired t-test was used because both groups were matched since the beginning of the study, both coming from Skills Certificate 2 level and performed satisfactorily in Semester 1.
Table 4: Individual Baseline and Final Life Skill’s t-test results (both groups)
Results from the paired t-test on the baseline means scores as evidently shown in Table 4 revealed that there is not much difference between the two groups (XA=51.5; XB= 50.8). This result suggests that the students have the same level of Life Skills and this has been the trend in Vocational institutions in Brunei Darussalam. During Semester 1, teachers assessed the students without guiding them on how to improve and were merely dependent on the student’s own effort. Paired t-test was used because both groups were matched since the beginning of the study, both coming from Skills Certificate 2 level and performed satisfactorily in Semester 1.
Although the baseline results were almost similar for these two groups, their final results suggests otherwise. As seen from Table 4 above, there is a significant difference on their final Life Skills’ results obtained by the two groups (XA=66.9; XB= 50.9). The p value for Feedback group were calculated to be less than 0.05 (p=0.00) and hence validates the fact that there is a significant difference between their baseline and final results. In the case of Conventional group, the p value is more than 0.05 (p=0.857). Hence, with these data in hand, the null hypothesis can be rejected and it can be claimed that there is an improvement in the Feedback group compared to the Conventional group.
The qualitative data from the semi-structured interviews were conducted with the two teacher participants who were involved in assessing the students in order to find out their views and preferences towards the practice of giving feedback. This was beneficial as it would provide a valuable insight for this study in determining the limiting factors if feedback were to be integrated in the future. The following were some of the prominent findings noted by the teacher participants on the feedback intervention:
(a) Teacher’s competency on Life Skills
Mainly, the most common reason for preferring teacher feedback is that the teacher knows the standards that determine students’ grades. The comment made by T1 when responding to the question ‘In what ways do you think feedback can be used to enhance students’ Life Skills?’
“…Personally speaking, I think feedback is important for students to improve their skills. Through teacher feedback, it actually act as a guide on how they are expected to perform when they work in industries. Teachers like us have years of trainings and we have been working close with the industries. I don’t think the student knows how Life Skills is like in working environment because they haven’t gotten any experience yet. If we were to just let them be and just see them perform without feedback and giving our perspective on how they perform tasks. They will be stuck on whatever they know…they don’t know where to go”.
The comment above shows how important is the role of teacher’s feedback in improving students’ Life Skills. She deduced that teacher feedback would be useful if it were to be integrated because teachers have years of experience and a better grasp on what is the standard expected by industry – the students’ potential employers. It is however, important to take note that new teachers may not be as competent as senior staff members.
(b) Teacher feedback motivates students
Another main finding that was commented by T1 was the fact that teacher’s feedback motivates students in trying to improve their Life Skills. This is aligned with several studies done on feedback (Hattie 2009; Zacharias 2007; Srichanyachon 2012)
“Based from my experience in this study, I noticed that students are actually motivated to improve their skills. They feel as though it’s like a glimpse into their future profession. Through my feedback, they were able to act upon it as they see its importance and how it relates to them. As the end prize, they will have provisional grade for their effort during the feedback session and this then furthered their willingness to improve”
This is also supported by numerical analysis done where 92% of the students feel motivated from the teacher’s feedback. It was claimed by Gilbuena et al. (2015) that in such context, students are more likely to integrate feedback into their Life Skills because they will view these skills as an integral part of what an engineer does.
(c) Teacher control grades
It was found that students preferred teacher feedback for the sole reason that teachers have control over their grades (Zacharias 2007). In this study, the teachers detected that students behave differently when around the assessors, even when it’s not during class. She explained that the students were aware that their grades may deteriorate if they behave differently when in lesson and outside of class as Life Skills covers an overall attitude of a student. This is a concern as it questions whether the students only behave professionally in front of the assessors; feel ‘motivated’ to improve in order to improve their grades. This finding is indeed worrying as the students are more interested in their grades rather than seeing the true light of Life Skills in their everyday practice.
Issues and limitations
Research has shown that there are still a number of issues and limitations existing in the integration of feedback in assessment, which has also been pointed out by the interviewees. There are two main limitations that were experienced by these teachers. As expressed by T1 and T2 respectively:
(a) Teacher feedback is time consuming and labor intensive
“ .. I would say that giving feedback is time consuming because we have these sessions where we provide students with feedback during lunch breaks. I wouldn’t mind doing it but it’s also taking up the students’ time… in terms of giving feedback, we have to take time and analyze and assess each student with what they have done during the activity… and it’s a bit tedious. Maybe it’s okay if it’s a smaller group of students I guess…but definitely not for a whole class. Another thing is when we had more than 1 assessors, because I don’t think it’s possible to do that in a long run”
“I think feedback is one sided… teachers make most of the work to give feedback and assess them according to the grading criteria. They somehow have to correlate these two together which takes a lot time and effort…Honestly speaking, I do not think most teachers would want extra work”
(b) Students’ initiative to use feedback
Secondly, the teachers were concerned in terms of ensuring the students use the feedback given to them. It is possible that students do not pay attention to comments because they don’t make sense to them (Duncan 2007) or that students simply do not see its importance in their education. T2 expressed:
“Giving them feedback is good. But there is a limit to that because it’s like spoon-feeding. There are times when there’s no initiative from them and feedback that is when our feedback becomes useless, because in the end, it’s up to the students whether they will use it or otherwise. Even if they do, results still vary with students.”
With reference to the results to answer Research Question 1, the use of feedback in enhancing students’ Life Skills in Brunei Darussalam was achieved in this study. The independent t-test for their final result has shown that there is a significant difference (P < 0.05 (0.000)), which indicates an improvement in learning. Although they did not manage to achieve the highest industrial standard, their final results have given promising evidence that the feedback strategy aids the improvement of their scores. Perhaps over a longer period, they would be given more opportunity to improve their Life Skills beyond academic achievements.
In exploring the answer to Research Question 2, it gave an inside perspective as to some of the advantages, challenges and limitations in providing feedback to the students. Most of all, the assessors are in favour of the learning strategy. It was noted that feedback has a positive impact towards the student’s achievement as they have this cultural belief that teachers are more competent as they have more experience. This, therefore, reflects on their marked preference to have feedback embedded in the assessment of Life Skills. Interestingly, however, it was also shown, from the students’ view, that the students were aware that teachers control the grades. This is indeed worrying. Zacharias (2007) found teacher feedback contributes greatly to students’ emotional states mainly towards their motivation and attitudes. While too much feedback would make the students depressed, little feedback was considered more motivating because, for the students, it indicated fewer mistakes. The assessors have also pointed out that feedback is only effective if the students make the initiative to improve. Therefore, the construct of feedback plays an important role in determining the students’ attitude. Directive feedback was also said to be time consuming for both parties, therefore, it is crucial to connect both professional skills and technical skills together instead of having to sacrifice time. Nevertheless, the results from this study are limited in terms of time and opportunities. Hence, additional research is warranted.
It is widely recognized that feedback is a significant part of a learning cycle to improve performance. Life Skills on the other hand, is a collective of skills that needs to be embedded into students to ensure that they will become successful members of society especially in their working environment. In this study, this was attempted by integrating feedback in assessment of Life Skills. Overall review has shown positive impact in terms improvement in students’ achievement. Feedback on Life Skills helps students recognize how to incorporate and demonstrate these skills into their activity. In addition, these students have shown positive attitudes towards this strategy in their learning. Nevertheless, a few have shown otherwise due to unclear feedback provided. However, there are still limiting factors present in integrating feedback as noted by their assessors such as it being time consuming, labor intensive and it is dependent on the student’s willingness and initiative to improve. This study has shown a valuable insight towards the power of feedback and its implications on Life Skills.
Bakar, A. R., Mohamed, S., & Hamzah, R. (2013). An Assessment of Workplace Skills acquired by students of Vocational and Technical Education Institutions. In: International Education Studies, 6, 11, 15-20.
Bjorklund, S. A., Parente, J. M., & Sathianathan, D. (2002). Effects of faculty interaction and feedback on gains in student skills. In: Journal of Engineering Education, 93 (2), 153-160.
Brunei Darussalam Technical and Vocational Education Council (BDTVEC) (2012). BDTVEC Guide: Life Skills Implementation and General Teaching Practice, BDTVEC, Brunei Darussalam: Government printing department.
Black, P. & William, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5 (1), 7-74.
Connelly, J. D. & Middleton, J. C. R. (1996). Personal and professional skills for engineers: One industry’s perspective. In: Engineering Science and Education Journal, 5 (3), 139-142.
Duncan, N. (2007). Feed-forward: improving students use of tutor comments. In: Assessment & Evaluation in Higher Education, 32 (3), 271-283.
Gilbuena, D. M., Sherrett, B. U., Gummer, E. S., Champagne, A. B., & Koretsky, M. D. (2015). Feedback on professional skills as enculturation into communities of practice. In: Journal of Engineering Education, 104 (1), 7-34.
Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students’ learning. In: Learning and Teaching in Higher Education, 3-31.
Hattie, J. A. (2009). Visible learning: A synthesis of 800+ meta-analyses on achievement. Oxford: Routledge.
Hattie, J. A. & Timperley, H. (2007). The power of feedback. In: Review of Educational Research, 77 (1), 81-112.
Hattie, J. A. (1999). Influences on student learning. Inaugural lecture, University of Auckland, New Zealand.
Online: http://projectlearning.org/blog/wp-content/uploads/2014/02/Influences-on-Student-Learning-John-Hattie.pdf (retrieved 16.02.2017).
Juwah, C., Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D., & Smith, B. (2004). Enhancing student learning through effective formative feedback. The higher Education Academy Generic Centre.
Kahar, I. N. (2006). Common skills: a hidden curriculum policy in its implementation stage. Unpublished M. Teach Research Exercise, Universiti Brunei Darussalam, Sultan Hassanal Bolkiah Institute of Education, Bandar Seri Begawan, Brunei Darussalam.
Kluger, A. N. & DeNisi, A. (1996). The effects of feedback intervention on performance: a historical review, meta-analysis and a preliminary feedback intervention theory. In: Psychological Bulletin, 119 (2), 254-284.
Kumar, R. (2005). Research methodology (2nded.). Pansheel Enclave, New Delhi: SAGE publications.
Mackiewicz, J. (2004). The effects of tutor expertise in engineering writing: A linguistic analysis of writing tutors; comments. IEEE Transaction on Professional Communication, 47 (4), 316-328.
Nicol, D. J. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. In: Studies in Higher Education, 31 (2), 199-218.
Paryono (2014). Transferable skills in Technical and Vocational Education and Training (TVET) in Brunei Darussalam. In: TVET@Asia, issue 3, 1-15. Online: http://www.tvet-online.asia/issue3/paryono_tvet3.pdf (retrieved 09.01.2015).
Ramaprasad, A. (1983). On the definition of feedback. In: Systems Research and Behavioural Science, 28, 4-13.
Rosli, M. R. (2012). A study on the implementation of VTE common skills: A case study. Unpublished M.Teach Research Exercise, Universiti Brunei Darussalam, Sultan Hassanal Bolkiah Institute of Education, Bandar Seri Begawan, Brunei Darussalam.
Sadler, R. D. (1989). Formative assessment and the design of instructional systems. In: Instructional Science, 18, 145–165.
Shuman, L. J., Besterfield-Sacre, M., & McGourty, J. (2005). The ABET ‘Professional Skills’ – Can they be taught? Can they be assessed? In: Journal of Engineering Education, 94 (1), 41-55.
Shute, V. J. (2008). Focus on Formative Feedback. In: Review of Educational Research, 78, 153-189.
Taylor, S. S. (2011). “I really don’t know what he meant by that”: How well engineering students understand teachers’ comments on their writing? In: Technical Communication Quarterly, 20 (2), 139-166.
United Nations International Children’s Emergency Fund (UNICEF). Definition of Terms. Online: www.unicef.org/lifeskills/index_7308.html (retrieved 26.05.2015).
Villiers, R. D. (2013). 7 principles of highly effective mangerial feedback: Theory and practice in managerial development interventions. In: The international Journal of Management Education, 68-72.
Zacharias, N. (2007). Teacher and Student Attitudes toward Teacher Feedback. In: RELC Journal, 38 (1), 38-52.
Pg Mahmud, S. N. A. & Chin, W. K. (2017). A Study on the Use of Feedback to Improve Assessment of Life Skills in Mechanical Engineering Course. In: TVET@Asia, issue 8, 1-14. Online: http://www.tvet-online.asia/issue8/pg-mahmud_chin_tvet8.pdf (retrieved 21.02.2017).