SAJHRM SA Journal of Human Resource Management 1683-7584 2071-078X AOSIS SAJHRM-22-2449 10.4102/sajhrm.v22i0.2449 Original Research A disconnect between training evaluation theory and the practical realities of South African businesses https://orcid.org/0000-0001-7597-9709 Duffy Carren G. 1 School of Management Studies, Faculty of Commerce, University of Cape Town, Cape Town, South Africa Corresponding author: Carren Duffy, carren.duffy@uct.ac.za 29022024 2024 22 2449 28092023 23112023 © 2024. The Author 2024 Licensee: AOSIS. This work is licensed under the Creative Commons Attribution License. Orientation

This article emphasises the need to rethink training evaluation due to its lack of practice in South Africa.

 Research purpose

This research aimed to investigate the extent to which South African corporates engage in training evaluation.

 Motivation for the study

While theorists and training evaluation experts argue the importance of evaluating training, globally, training evaluation is primarily limited to collecting reaction-level data. Training evaluation is a staple in most training and human resource development textbooks and forms part of undergraduate and postgraduate curricula in the Human Resources Management domain, yet, little is published about training evaluation in South Africa and its practical relevance in modern-day corporate environments.

 Research approach/design and method

An explanatory sequential (mixed-methods) research design was utilised.

 Main findings

While there is a high commitment to training among South African corporates, it is not accompanied by a commitment to training evaluation. Training evaluation is not considered a priority business practice, especially in highly demanding and complex corporate environments. Additionally, with South Africa’s unique legislative frameworks, the motivation for providing training is sometimes distorted, causing a reluctance to determine training effectiveness.

 Practical/managerial implications

The findings suggest that a substantial shift in how training evaluation is theoretically conceptualised and practically applied is necessary for South Africa.

 Contribution/value add

The research indicates a need to explore creative and nuanced ways, perhaps utiliing established human resource analytics and metrics to assess the merit and worth of training.

 training training evaluation corporate training and development South Africa skills legislation Funding information The author discloses the receipt of a Development Grant from the University of Cape Town during the second data-collection phase. Introduction Orientation

There is likely a multitude of disciplines where what is suggested as best practice, in theory, does not translate into practical action in the business world. In the field of training, especially in training evaluation, for example, theorists and evaluation experts argue the importance of evaluating training as a critical business and training function. Yet, the practice around the globe has, for many years, been limited to collecting trainee satisfaction data, with little or no consideration of measurement of the value aspects of the training.

 Research purpose and objectives

Little is known about the state of training evaluation in South Africa. The last publication investigating these practices in South Africa dates back to 2004. Anecdotal evidence suggests a disconnect between training evaluation theory and its practical relevance in corporate settings. Corporate refers to business corporations operating as legal entities, typically managed by a board of directors. This article provides insight into the training evaluation practices of corporates in South Africa.

 Literature review Training evaluation defined

Training evaluation aims to establish the merit and worth of training and development programmes. Several definitions of training evaluation offered in the seminal literature (e.g. Basarab & Root, 2001; Casey, 2006; Phillips, 2006; Salas et al., 2012; Topno, 2012) provide somewhat different perspectives on the purposes of engaging in training evaluation. By synthesising these definitions, training evaluation can be viewed as having four fundamental purposes: ‘(1) to assess if a programme has achieved its intended objectives and participants are able to demonstrate enhanced skills and knowledge; (2) to assess whether transfer of learning to workplace tasks is taking place; (3) to assess a programme’s total value (worth) in terms of its financial benefit; and (4) to assess the overall quality and functioning of a programme and whether any improvements are required’ (Duffy, 2018, p. 52).

These four purposes are aligned with the five commonly cited explanations for organisations engaging in training evaluation (as cited in Duffy, 2018). These include: (1) to determine whether training objectives have been met (Topno, 2012; see also Casey, 2006; Farjad, 2012; Hall & Yoder, 2003; James & Roffe, 2000; Kumpikaitė, 2007), (2) to improve training design, quality, delivery, and training-related activities (Farjad 2012; see also Blanchard et al., 2000; Hashim, 2001; Hung, 2010; James & Roffe, 2000; Topno, 2012), (3) to justify the training budget allocated (level of accountability achieved and to secure larger budgets and more resources for training and development initiatives (Hung, 2010; see also Lien et al., 2007; Phillips & Phillips, 2001; Topno, 2012), (4) to provide information about the financial value of training via return on investment (ROI) (Phillips & Phillips, 2001; see also Hashim, 2001; Kumpikaitė, 2007; Lien et al., 2007; Topno, 2012), and (5) to help identify how an organisation can support and facilitate learning transfer by identifying what factors affect individual learning (Brinkerhoff & Dressler, 2002; see also Hung, 2010; Kraiger et al., 2004; Topno, 2012).

Training evaluation is critical in ensuring a learning organisation (an entity that prioritises growth through the transfer of learning). Cronbach (as cited in Torres et al., 2005, p. 2) argues that ‘evaluation is about learning … the focal point for learning to occur is [the] communication of knowledge generated by an evaluation’. Similarly, Brinkerhoff (2006) and Salas et al. (2012) argue that competitive organisations use the results of training evaluation to organise, plan, and implement effective training.

In some of the seminal training models, which continue to be taught in institutions of higher education undergraduate and postgraduate programmes worldwide, training evaluation is an integral step in the training cycle [see the Critical Events Model (Nadler, 1982); the Training Model (Camp et al., 1986 as cited in Erasmus Erasmus & van Dyk, 2003); and the High Impact Training Model (Sparhawk, 1994)] (see Duffy, 2018). Given its purposes and advantages, training evaluation is still positioned by academics and theorists as a training best practice.

Table 1 depicts some of the commonly prescribed human resource textbooks for tertiary education. The list was shortened to include only texts published in the last 7 years. As shown in the last column of Table 1, each publication contains a chapter dedicated to training evaluation.

Commonly prescribed textbooks published within the last 7 years with dedicated training evaluation chapter(s).

Title Author(s) Year Publisher Chapter
Employee training and development Noe, R. A 2022 (9th ed.) McGraw-Hill Education Chapter 6: Training Evaluation
Human resource development: Talent Development Werner, J. M 2022 (8th ed) Cengage Chapter 7: Evaluating workplace learning interventions
Design thinking for training and development Boller, S., & Fletcher, L 2020 ASTD Chapter 10: Evaluate
Strategic training and development Berkley, R. A., & Kaplan, D. M 2020 Sage Publications Chapter 5: Training Evaluation: Reaction and LearningChapter 6: Training Evaluation: Transfer and Results
Effective training: Systems, strategies, and practices Blanchard, P. N., & Thacker, J. W 2019 (6th ed.) Chicago Business Press Chapter 9: Evaluation of Training
Managing training and development in South Africa Du Plessis, M., Mda, T., & Nel, P. 2019 (8th ed.) Oxford University Press Chapter 8: Assessing learning and evaluating training and development
Training and development. Why training matters Carliner, S., & Driscoll, M 2019 Lakewood Media Group Chapter 7: About work processes and tools: Program evaluation
HR solutions for excellence in training and development Jain, S 2019 Society Publishing Chapter 7: Evaluation of training
Managing performance through training and development Saks, A. M., & Haccoun, R. R 2018 (8th ed.) Nelson Canada Chapter 10: Training evaluation
Practising learning and development in South African organisations Coetzee, M 2018 (3rd ed.) Juta and Company Chapter 8: Evaluating learning intervention effectiveness
ATD’s foundations of talent development. Launching, leveraging, and leading your organisation’s TD effort Biech, E 2018 ATD Press VII: Define and measure the impact of talent developmentChapter 29: Determine and demonstrate the organisational impact of talent developmentChapter 30: Evaluation methodsChapter 31: Getting started with evaluationChapter 32: The future of evaluation
The art and science of training Biech, E 2017 ATD Press Chapter 11: Why bother with assessment and evaluation?

Several textbooks have also been published in the last decade that aim to teach students and practitioners how to evaluate their training and development interventions. Table 2 lists the most recently published training evaluation texts.

The most recently published training evaluation textbooks.

Title Author(s) Date Publisher
10-Step evaluation for training and performance improvement Chyung, S. Y 2018 SAGE Publications
Evaluation basics McCain, D 2016 ATD Press
Kirkpatrick’s four levels of training evaluation Kirkpatrick, J. D., & Kirkpatrick W. A 2016 ATD Press
Handbook of training evaluation and measurement methods Phillips, J. J., & Phillips, P. P 2016 Routledge
Monitoring and evaluation training: A systematic approach Chaplowe, S. G., & Cousins, J. B 2016 Sage Publications
Learning and performance: A systemic model for analysing needs and evaluating training Hopkins, B 2016 Routledge
Real World Training Evaluation: Navigating Common Constraints for Exceptional Results Phillips, P. P., & Phillips, J. J 2015 Association for Talent Development
Complete training evaluation: The comprehensive guide to measuring return on investment Griffin, R 2014 Kogan Page
Training evaluation pocketbook Donovan, P., & Townsend 2014 Management Pocketbooks Ltd

With the need for and the importance of training evaluation academically emphasised, one would assume that it is a widespread practice. Yet, prior studies have determined that organisations globally are not sufficiently assessing their training endeavours (Griffon, 2014; see also Abernathy, 1999; Eseryel, 2002; Giangreco et al., 2010; Kumpikaitė, 2007; Nickols, 2005; Phillips et al., 2004). This is discussed in more detail in the following sections.

 Global training evaluation trends

This study included a literature review to locate peer-reviewed articles presenting audits or reviews that reported on the state of training evaluation in various countries from 2000 onwards. Only articles that reported on countrywide training evaluation statistics within a corporate context were included. Despite the thorough and exhaustive review, only four articles met these inclusion criteria (Al-Athari & Zairi, 2002; Blanchard et al., 2000; Meyer & Bushney, n.d.; Twitchell et al., 2000). Moreover, the articles all dated back more than 20 years, and only one had emanated from Africa, specifically South Africa. The articles are shown in Table 3, utilising Kirkpatrick’s (1959) Four-level Hierarchy and ROI as an additional dimension to indicate the frequency of training evaluation practice. Illustrating frequency according to these five levels is consistent with the reporting style of the articles located.

Training evaluation statistics from global surveys.

Country Frequency of reaction-level evaluation Frequency of learning-level evaluation Frequency of behaviour-level evaluation Frequency of results-level evaluation Frequency of ROI evaluation Reference
United States 74% 47% 31% 10% 23% Twitchell et al. (2000)Meyer and Bushney (n.d.)
Canada - - 37%†43%† 47%‡36%‡ - Blanchard et al. (2000)
Kuwait 85%§35%§ 73%11% 47%§40%§ 10%48% - Al-Athari and Zairi (2002)
South Africa 68% - - 25% 39% Meyer and Bushney (n.d.)

, managerial training;

, non-managerial training;

, government organisations;

, private organisations.

ROI, return on investment.

 Synthesis of global trends

Companies globally engage in some form of evaluation of their training and development programmes, but the results in Table 3 suggest that such efforts are weak and lack depth. Organisations relied heavily on trainees’ reaction and satisfaction data and less on obtaining data on potential behaviour change and related outcomes (Blanchard et al., 2000; see also Meyer & Bushney, n.d.; Twitchell et al., 2000). These findings support evaluation theorists who have argued that the most used training evaluation method is reaction-level data (Giangreco et al., 2010; see also Alliger et al., 1997; Blanchard et al., 2000; Eseryel, 2002; Foreman, 2008; Galloway, 2007; Hale, 2003; Hashim, 2001; Phillips et al., 2004).

Overall, the global trends in Table 3 indicate a lack of systematic training evaluation practices in surveyed countries in the early 2000s and that training evaluation was not practised as frequently as what academic theory proposes it should be. Without any recent data and sparse data on Africa, we are left to speculate regarding the quality of training provided in corporates worldwide and specifically in South Africa if training evaluation is not carried out properly. This study was thus based on the concern of limited and outdated information. It was aimed at investigating the extent to which South African corporates engage in training evaluation and whether this is an applied discipline in modern-day workplaces.

 Research design and methods Research approach and strategy

A mixed-methods approach was adopted for the research, specifically an explanatory sequential research design. This design is well-suited to the investigation because of training evaluation in South Africa being a vague study area (Burns & Burns, 2008). Given the absence of accessible published information on current training evaluation practices in the country, the quantitative Phase 1 was a crucial step in gathering comprehensive data on the topic (Burns & Burns, 2008). The qualitative Phase 2 enabled the researcher to interrogate the knowledge generated in Phase 1 (Creswell & Plano Clark, 2011).

 Method: Phase 1 Sampling and respondents

Non-probability purposive sampling was used to obtain respondents for Phase 1. Social media and personal networks were used to advertise the research. Approximately, 9000 South African Board of People Practices (SABPP) members were invited to participate. Because of the nature of the questions in the exploratory survey, the respondents were required to possess knowledge of their corporate training and development practices, including training evaluation. As such, only senior HR practitioners and specialists in the field of training and development were included. A total of 579 responses were collected, of which 310 (each representing a South African corporate) were usable. The corporates operated in all sectors and industries of the economy, with staff complements ranging from 9 to 160 000.

 Measurement instrument

Aspects of Blanchard et al.’s (2000) survey were considered when the survey for this research was developed because Blanchard et al.’s (2000) survey was used in the global trend studies (refer to Table 3). The customised survey included Section A, which elicited responses about the training expenditure of the corporate, the corporate’s commitment to training, and the reasons for commitment (or lack thereof) to training. Sections B and C were aimed at ascertaining the types of training the corporates offered, along with the training evaluation practices implemented.

 Research procedure

The survey was disseminated via the SABPP’s database and returned online using Qualtrics. The survey remained live for 4 weeks, with reminders sent every week.

 Statistical analysis

Once the data had been checked, formatted, and cleaned, descriptive statistics were performed in SPSS to analyse the data. The descriptive statistics and stacked bar charts enabled the researcher to determine and present patterns and trends in the data.

 Method: Phase 2 Research setting

The setting for Phase 2 also included South African corporates. This phase of the research was aimed at expanding on and explaining the results obtained in Phase 1. The researcher sought to develop an extensive understanding of how training evaluation was perceived in different South African corporations, documenting how these organisations utilised feedback and results from evaluations to guide learning and facilitate change.

 Research participants and sampling methods

The sampling frame for Phase 2 consisted of respondents in the exploratory survey (Phase 1) who had indicated an interest in continuing their participation in the research. Of the 310 respondents from Phase 1, only 64 respondents indicated their willingness to be contacted for participation in Phase 2. Therefore, the non-probability sampling strategy employed. Several methods were used to secure data providers for Phase 2, including emails and phone calls. Some contact information was, however, incorrect or no longer in service, and some respondents declined to participate. Of the 64 respondents who were initially interested in participating in Phase 2, the researcher was able to secure only nine data providers.

 Data-collection method

In Phase 2, the researcher conducted in-depth personal interviews with respondents who indicated that they were willing to continue participation in the study to gain further insights and explanations for the results obtained in Phase 1. The interviews were semi-structured to allow participants to freely express their views and insights while ensuring that all participants were asked the same questions and that the discussions remained relevant to the research topic (Adams, 2015).

 Data recording

The letter of consent conveyed to the participant that the interview would be recorded and specified that the researcher would hire a transcriber for transcription purposes.

 Strategies employed to ensure quality data and integrity

The researcher ensured the trustworthiness of the study by employing strategies to enhance credibility, transferability, and dependability (Shenton, 2004). Credibility was fostered through prolonged engagement with participants to establish rapport and obtain a deepened understanding of their perspectives and training experiences (Cypress, 2017). Transferability was addressed by providing a detailed account of the research context, methods, and participant characteristics (see Duffy, 2018), allowing for the assessment of the study’s applicability in different settings (Cypress, 2017). Dependability was assured through transparent documentation of the entire research process (see Duffy, 2018), facilitating the replication of the study by others (Cypress, 2017).

 Data analysis

The interview transcripts were imported into the data-management software Atlas.ti. The Notice Things, Collect Things, and Think about Things (NCT) model originally developed by Seidel, and further developed by Friese (2012), was used to code the data and conduct thematic analysis.

Reporting style

The quantitative results and qualitative findings were collated and both are reported as per the key findings of the study.

 Ethical considerations

Ethical approval for the research was sought and obtained from the University of Cape Town Commerce Faculty’s Ethics in Research Committee(REC 2014/03/17). There were no risks associated with either research phase, and participation was voluntary. Informed consent was obtained from all participants, and they were assured of anonymity. The interviews were recorded with the participants’ permission. The data were stored on a password-protected computer to which only the researcher had access. The reporting of the study findings does not contain any identifiers of industries, companies, or individuals.

 Results

Two key discoveries emanated from the research. The quantitative and qualitative data, firstly, showed a lack of training evaluation practices. Secondly, the qualitative data indicated a complete disregard for the importance of training evaluation. The interviewees ascribed this to training not being valued as a strategic imperative, as well as organisations providing training merely to ensure compliance with legislation. The further sections provide more detail.

Lack of training evaluation

In Phase 1, more than half of the respondents (62.9%) rated their organisation as highly committed to training. Respondents indicated that training expenditure ranged from R5000 to R468 million per annum, with a mean of R21898533.70 (x = R21898533.70; SD = R63196310.47). This high level of commitment is evident in the substantial financial investments South African corporates make each year to train their staff, as noticed by the interviewees, who indicated that approximately 10% of their organisations’ annual budget is allocated to training and development initiatives.

However, despite corporates’ commitment and training spend, there is a lack of training evaluation. For all categories of training provided, the corporates sampled rarely collect evaluation data. Figure 1 depicts the frequency of training evaluation data collection.

Frequency of evaluation data collected per level for each training category.

As shown in Figure 1, corporates mainly use data on trainees’ satisfaction with the development intervention, that is, reaction data. On average, reaction-level data were collected 32% of the time training programmes were run. This result indicates that only around one-third of corporates consistently collect what is arguably the quickest and easiest training evaluation data to gather.

The frequencies of data collection to evaluate learning (21%), behaviour (16%), and results (15%) were considerably lower. This trend is evident across all the types of training provided. Furthermore, as the level of training evaluation intensifies, the frequency of such evaluation decreases.

The corporates sampled collected ROI evaluation data least often – on average, 14% of the time. Figure 1 also shows that management and development training programmes showed the most frequent collection of evaluation data (on reactions, learning, behaviour, and ROI).

These results illustrate that little training evaluation is conducted in the South African corporates under study. Moreover, these efforts seem sub-standard compared with the training evaluation frequencies reported in the global surveys (outlined previously). Table 4 compares the training evaluation practises documented in this research with those reported in international research.

A comparison of training evaluation frequency statistics.

Level of evaluation The frequency range in global surveys The frequency of current research
Reactions 68% – 85% 32%
Learning 10% – 47% 21%
Behaviour 11% – 47% 16%
Results 10% – 48% 15%
ROI 8% – 9% 14%

Source: Duffy, C.G. (2018). The divide between academic and business practice: Exploring training evaluation practices and their perceived value in corporate South Africa (p. 159). Doctoral dissertation. Faculty of Commerce, Organisational Psychology, University of Cape Town, OpenUCT. Retrieved from http://hdl.handle.net/11427/29610

Note: The types of corporates, industries, and sample sizes differ between the research in the global surveys and the current research, which should be considered when reviewing these comparisons.

ROI, return on investment.

As seen in Table 4, the frequency of training evaluation practices in the South African sample is considerably lower than those reflected in the historically recorded global trends. However, the collection of ROI data suggests that South African corporates are collecting more data on this level than the countries sampled in the international surveys. Ultimately, the results confirm previous findings in that training evaluation is still not practised at the level, depth, and frequency of best practice (from a theoretical and academic viewpoint).

The same overarching trend of a lack of training evaluation emerged from the analysis of the qualitative data collected in Phase 2 of this study. Across the sample, participants indicated a lack of formal, standardised metrics and approaches to measuring training outcomes and impact. While it was reported that some corporates had training evaluation policies and ‘how-to’ manuals, there was neither compliance with nor implementation of these practices. The following quotations support this finding:

‘Um, so no, there is definitely no formal process. We don’t use any metric. If there is a problem, you [will] certainly hear about it.’ (Interviewee 2)

‘We don’t specifically drive Kirkpatrick’s levels and whatever [else] is around to measure and show the numbers … I mean …what we don’t do is integrate all the information back…so if we spent R120 million [Rand] a year on training … we don’t actually consolidate that into a view for the business at the end of the day to say this is how much you’ve invested in training and … this is the return on investment you [are] getting.’ (Interviewee 8)

‘My quick response to that question is that there’s no process in place to ensure people are measuring this.’ (Interviewee 4)

Several HR executives held negative opinions about the discipline when probed further about the lack of systematic training evaluation. These responses are discussed in more detail further in the text.

 Disregard for training evaluation

Six of the human resource executives interviewed did not perceive training evaluation as a worthwhile and strategic business practice. They explained that, because of the highly demanding and pressurised corporate environment, other business functions are given precedence. The interviewees observed limited prospects of this perception of training evaluation and its importance changing in the future. Some quotations are provided further to support this finding:

‘I think they would find it difficult to … bother really … the general feeling is that it’s a waste of money. Straight up.’ (Interviewee 2)

‘With all respect, I think it’s a lot of nonsense.’ (Interviewee 4)

‘So everything here is hunky-dory, and no evaluation is or will be done. So the justification of the business is we are making tons of money, so there’s no need for any change. The profitability levels are there. The Board of Directors only wants to see profits, nothing else.’ (Interviewee 7)

Several interesting reasons emerged, which create an understanding of the negativity and the lack of appetite for training evaluation. The interviewees observed the following: (1) Training is provided to employees mainly to ensure legislative compliance, (2) training is not viewed as a strategic imperative, and (3) the established theoretical approaches of training evaluation are deemed inappropriate for the contextual realities of the South African corporate environment (Duffy, 2018).

 Training for legislative compliance

Respondents emphasised legislative compliance as the primary driver of their commitment to and expenditure on training and development. In South Africa, training is legally mandated by the Skills Development Act (SDA) 97 of 1998 and the Skills Development Levies Act (SDLA) 9 of 1999. Additionally, training is indirectly mandated by broad-based black economic empowerment (B-BBEE) legislation.

Skills development and skills development levies act

The South African post-apartheid government initially implemented the SDA and the SDLA to increase the training provided to South African employees from previously disadvantaged groups. Not only do these two legal frameworks mandate training for employees (expenditure of at least 1% of the organisation’s annual payroll), but organisations that provide evidence of training are rewarded with rebates and funding. The HR executives attributed their corporates’ commitment to training to these legislative frameworks (Duffy, 2018).

 Broad-based black economic empowerment

According to the B-BBEE Amended Act 46 of 2013, South African organisations must attain equitable [racial] representation across all organisational levels and increase black ownership and management. Organisations that fulfil these obligations earn B-BBEE points and thereby achieve a higher B-BBEE rating. As a B-BBEE-rated employer, the enterprise: (1) can conduct business with the government and any organ of the state; (2) is provided with preferential procurement; (3) can apply for tenders, licences, and concessions; (4) can enter public–private partnerships; and (5) can purchase state-owned assets (B-BBEE Amendment Act No 46 of 2013) (Duffy, 2018). With these benefits, South African enterprises can ultimately gain a competitive advantage and grow the business. Of specific interest to this research is the fact that organisations can increase their B-BBEE status and earn more B-BBEE points with their training and development practices. The HR executives described this as a significant influencer in their organisations’ budget allocations for training and development. The following quotations support this finding:

‘A lot of training [we] do is basically for our, is because of our employment equity report, to say that we are spending x amount of money on training and development.” “So it’s not we’ve paid this much, this is how much we’re gaining, it’s just simply for a box exercise.” “So provided it’s done, to get our BEE scorecard points, that is the way [management] thinks.’ (Interviewee 6)

‘The newest challenge for us probably is ensuring that we comply [with] triple B, double E employment equity and making sure that we train enough. Because we get rated and if we are not rated sufficiently … [we] lose business … they say sorry, your rating is too bad in terms of what you’re investing in developing black skills.’ (Interviewee 8)

‘Our intention is to do it [training] for, I suppose, really for the BEE points …’ (Interviewee 2)

 The absence of a strategic training lens

Respondents explained that the lack of training evaluation was that training itself is not viewed as a strategic imperative. In the sampled corporates, training is provided based on reactive training needs assessments. The relationships between training, individual learning, and overall performance (both personal and organisational) are not made explicit, indicating a lack of alignment with organisational strategy. The training is used predominantly to improve task performance. There seems to be little consideration of the fact that training can aid organisational learning, innovation, knowledge generation and sharing, and thus positively impact the organisation’s competitive advantage. The following quotation supports this finding:

‘… management is not, um, it’s not interested in training … [m]anagement only needs to show results. The Board of Directors only want to see profits, nothing else. “The majority of the training that’s done is the mandatory training, and so they have to give us a competency certificate.” “That company assesses whether the person is competent and gives them the certificate.” [For other training] it’s really just an attendance course. They’ve got to be recertified every two years.’ (Interviewee 2)

 Misalignment between theory and contextual reality

The third and final reason for the lack of training evaluation is that HR executives view the theoretically positioned training evaluation practices as impossible to perform in their corporate environments. There is also an insufficient budget in the sampled corporates for evaluation purposes. They observed that they already have high workloads and that training evaluation would add to this burden. They further reported having inadequate time and a lack of training evaluation knowledge and competencies. The HR executives noticed that the methods, approaches, and models of training evaluations prescribed in theory are inappropriate for highly demanding corporate environments, revealing a disconnect between theory and contextual reality. The following quotations support this finding:

‘The other thing is resources when you have. I mean, our learning and development team was three people for an organisation of locally 1200 in seven regions and seven offices. So while there’s the right things to do, there’s also what’s kind of, um, what you’re able to. So for us, it was definitely a resource issue.’ (Interviewee 3)

‘Cost, we’re under huge cost pressure and to set up an admin that’s not necessarily going to add value.’ (Interviewee 7)

‘I mean if there’s no, there’s no mandate, there’s no mandate from any executive member, for me, to say for every training programme produced at ROI or ROE figure, um if there was, we’d be able to do it. But I mean, with such a lot of other priorities on your table, I’m not going to start doing these calculations for programmes if no one is going to use that figure. And why isn’t it being done? What is inhibiting us from actually going and measuring this and taking it back the full loop? The reality is, um, it is business pressure.’ (Interviewee 4)

 Discussion Outline of results and findings

In summary, this study found that training evaluation is not considered a practice that supports the strategic goals of South African corporates. There are insufficient resources – time, expertise, and budget – to carry out best practices in training evaluation, and it is not considered to add value other than improving task performance. Furthermore, the provision of training is for compliance reasons, as opposed to real investment in human capital development, which could explain why there is a lack of training evaluation and why South African corporates do not allocate additional time and money to ensuring proper evaluation of their training. The responses from the HR executives indicated that training is a routine operational function, one that they perform because they must. The aim is not to achieve organisational performance outcomes but rather to enhance the organisation’s B-BBEE status, that is, it is a ‘tick-box exercise’.

 Practical implications

If corporates in South Africa are training predominantly for compliance purposes, they will not realise the strategic advantages that training could provide. This missing strategic human resource development lens and a lack of acknowledgement that training can contribute to human resources being a strategic business partner indicating that the likelihood of training evaluation being practised is also weakened. Griffon (2014) explained that the failure of an organisation to engage in training evaluation carries the risk that training functions will continue to have a low status in the organisation. Therefore, without determining the contribution and impact that training has on learning and business outcomes, corporates will remain unable to recognise and realise the true short-term and long-term worth of training.

Berge (2008) argues that the evaluation of impact is required to clearly demonstrate the effects of training on business profits. Considering the vast amounts of money invested in training, it should be a strategic imperative that the training be evaluated to ensure high-quality training that yields tangible and measurable results. There is a vast body of literature on conducting training evaluations. Still, the HR practitioners in this study indicated that this practice is not valued and that it is simply not required by the corporates’ executive management.

Without a mandate to evaluate training, the practice relies on the perceptions held by individuals within the organisation. This indicates the need for a drive from executive management to engage in systematic and objective training evaluation in alignment with the business strategy. However, given the low regard for the training of the HR executives interviewed, there seems little hope of instilling an appreciation for training evaluation’s importance.

 Limitations and future research

A methodological limitation of the research is the sample size in both phases, although the sample sizes were the largest compared with those of the global surveys that were reviewed. Furthermore, the fact that the results from the survey were aggregated countered this limitation to some extent.

More research is needed to assess the state of training evaluation in other African countries. Similar audits could reveal whether the lack of training evaluation practices is unique to South Africa or whether the discipline is neglected across the continent. Future research could also investigate what supports and hinders training evaluation in various countries to determine similarities and differences across different contexts.

Finally, there is a need to investigate, on a larger scale, what kinds of monitoring data corporates consistently collect. If training evaluation methods and approaches are unfeasible, perhaps the effectiveness of training and development practices can be investigated in a nuanced way. One suggestion could be exploring how training and training evaluation data could be mapped onto current HR analytics and HR metrics.

 Conclusion

This research presents an opportunity to rethink training evaluation practice as a strategic imperative and to examine evaluation models fit for the South African context. The HR executives should also raise awareness of the short-term and long-term benefits and value of high-quality training, which can only be ensured by evaluating training and making the necessary adjustments and updates.

Griffon (2014) shares the sentiment about the necessity to alter training evaluation practices, asserting that the evolution of learning over the last three decades demands a shift. As such, the training evaluation models and methods currently in use are outdated and ill-aligned with modern workplaces’ current organisational learning and training environments. Other researchers have supported this sentiment, arguing that how we conduct evaluation needs to change (Giangreco et al., 2010; see also Abernathy, 1999; Berge, 2008). While these arguments have been ongoing, little progress has been made, particularly in Africa. Researchers are publishing evaluations of specific training and development interventions, but little research attention is paid to modernising training evaluation.

Typically, theory informs practice, but based on the findings of this study, researchers should allow context to inform theory. Researchers and corporates should co-ordinate efforts to gain optimum value from corporates’ substantial investment in training. The findings of this study indicate a need for new, less stringent training evaluation approaches that are cost-effective and easily implementable and that offer valuable and objective data so that corporates will value training as a strategic imperative that can greatly enhance organisational performance.

 Acknowledgements

This article is partially based on the author’s thesis of the degree of Doctor of Philosophy in Organisational Psychology at the University of Cape Town, South Africa, with supervisor Professor Suki Goodman, received April 2018, available here: https://open.uct.ac.za/server/api/core/bitstreams/6baaeb48-2efd-4719-a4c5-926c18f6eb47/content.

Competing interests

The author has declared that no competing interest exist.

 Author’s contributions

C.G.D. completed the data collection, analysis, and interpretation, conceptualised the manuscript, and was the lead writer.

 Data availability

The data that support the findings of this study are not openly available due to confidentiality and are available from the corresponding author, C.G.D., upon reasonable request.

 Disclaimer

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency, or that of the publisher. The authors are responsible for this article’s results, findings, and content.

 References Abernathy, D.J. (1999). Thinking outside the evaluation box. Training & Development, 53(2), 18–24. Adams, W.C. (2015). Conducting semi-structured interviews. In K.E. Newcomer, H.P. Hatry, & J.S. Wholey (Eds.), Handbook of practical program evaluation (pp. 492–505). John Wiley & Sons, Inc. Al-Athari, A., & Zairi, M. (2002). Training evaluation: An empirical study in Kuwait. Journal of European Industrial Training, 26(5), 241–251. https://doi.org/10.1108/03090590210424911 Alliger, G.M., Tannenbaum, S.I., Bennett, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the reactions among training criteria. Personnel Psychology, 50(2), 341–358. https://doi.org/10.1111/j.1744-6570.1997.tb00911.x Basarab, D.J., & Root, D.K. (2001). The training evaluation process. Kluwer Academic Publishers. Berge, Z.L. (2008). Why it is so hard to evaluate training in the workplace. Industrial and Commercial Training, 40(7), 390–395. https://doi.org/10.1108/00197850810912270 Berkley, R.A., & Kaplan, D.M. (2020). Strategic training and development Chapter 5: Training evaluation: Reaction and learning Chapter 6: Training evaluation: Transfer and results. Sage. Biech, E. (2017). The art and science of training Chapter 11: Why bother with assessment and evaluation? ATD Press. Biech, E. (2018). ATD’s foundations of talent development. Launching, leveraging, and leading your organisation’s TD effort VII: Define and measure the impact of talent development Chapter 29: Determine and demonstrate the organisational impact of talent development Chapter 30: Evaluation methods Chapter 31: Getting started with evaluation Chapter 32: The future of evaluation. ATD Press. Blanchard, P.N., & Thacker, J.W. (2019). Effective training: Systems, strategies, and practices Chapter 9: Evaluation of training (6th ed.). Chicago Business Press. Blanchard, P.N., Thacker, J.W., & Way, S.A. (2000). Training evaluation: Perspectives and evidence from Canada. International Journal of Training and Development, 4(4), 295–304. https://doi.org/10.1111/1468-2419.00115 Boller, S., & Fletcher, L. (2020). Design thinking for training and development Chapter 10: Evaluate. ASTD. Brinkerhoff, R.O. (2006). Increasing impact of training investments: An evaluation strategy for building organisational learning capability. Industrial and Commercial Training, 38(6), 302–307. https://doi.org/10.1108/00197850610685824 Brinkerhoff, R.O., & Dressler, D. (2002). Using evaluation to build organisational performance and learning capability: A strategy and a method. Performance Improvement, 41(6), 14–21. https://doi.org/10.1002/pfi.4140410605 Burns, R.P., & Burns, R. (2008). Business research methods and statistics using SPSS. Sage Publications. Carliner, S., & Driscoll, M. (2019). Training and development. Why training matters Chapter 7: About work processes and tools: Program evaluation. Lakewood Media Group. Casey, M.S. (2006). Problem-based inquiry: An experiential approach to training evaluation. Unpublished doctoral thesis, University of Akron. Chaplowe, S.G., & Cousins, J.B. (2016). Monitoring and evaluation training: A systematic approach. Sage. Chyung, S.Y. (2018). 10-Step evaluation for training and performance improvement. SAGE. Coetzee, M. (2018). Practising learning and development in South African organisations Chapter 8: Evaluating learning intervention effectiveness (3rd ed.). Juta and Company. Creswell, J.W., & Plano Clark, V.L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications. Cypress, B.S. (2017). Rigor or reliability and validity in qualitative research: Perspectives, strategies, reconceptualization, and recommendations. Dimensions of Critical Care Nursing, 36(4), 253–263. https://doi.org/10.1097/DCC.0000000000000253 Donovan, P., & Townsend, J. (2014). Training evaluation pocketbook management. Pocketbooks Ltd. Du Plessis, M., Mda, T., & Nel, P. (2019). Assessing learning and evaluating training and development. in B. Erasmus & P. Loedolff (Eds.), Managing training and development in South Africa (8th ed., Chapter 8). Oxford University Press. Duffy, C.G. (2018). The divide between academic and business practice: Exploring training evaluation practices and their perceived value in corporate South Africa (p. 159). Doctoral dissertation. Faculty of Commerce, Organisational Psychology, University of Cape Town, OpenUCT. Retrieved from http://hdl.handle.net/11427/29610 Erasmus, B.J., & van Dyk, P.S. (2003). Training management in South Africa (3rd ed.). Oxford University Press. Eseryel, D. (2002). Approaches to evaluation of training: Theory and practice. Educational Technology & Society, 5(2), 93–98. Farjad, S. (2012). The evaluation effectiveness of training courses in university by Kirkpatrick Model (Case study: Islamshahr University). Procedia-Social and Behavioral Sciences, 46, 2837–2841. https://doi.org/10.1016/j.sbspro.2012.05.573 Foreman, S.M. (2008). Kirkpatrick model: Training evaluation practices in the pharmaceutical industry. ProQuest. Friese, S. (2012). ATLAS.ti 7 user manual. ATLAS.ti Scientific Software Development GmbH. Galloway, D.L. (2007). Evaluating distance delivery and e-learning. Is Kirkpatrick’s model relevant? Performance Improvement, 44(4), 21–27. https://doi.org/10.1002/pfi.4140440407 Giangreco, A., Carugati, A., & Sebastiano, A. (2010). Are we doing the right thing? Food for thought on training evaluation and its context. Personnel Review, 35(2), 162–177. https://doi.org/10.1108/00483481011017390 Griffin, R. (2014). Complete training evaluation: The comprehensive guide to measuring return on investment. Kogan Page. Hale, R. (2003). How training can add real business value: Part 1. Industrial and Commercial Training, 35(1), 29–32. https://doi.org/10.1108/00197850310458225 Hall, J.D., & Yoder, J.A. (2003). Conducting evaluations to determine training effectiveness. Retrieved from http://www.iaea.org/inis/collection/NCLCollectionStore/_Public/34/062/34062611.pdf Hashim, J. (2001). Training evaluation: Clients’ roles. Journal of European Industrial Training, 25(7), 374–379. https://doi.org/10.1108/EUM0000000005838 Hopkins, B. (2016). Learning and performance: A systemic model for analysing needs and evaluating training. Routledge. Hung, T.K. (2010). An empirical study of the training evaluation decision-making model to measure training outcome. Social Behavior and Personality: An International Journal, 38(1), 87–101. https://doi.org/10.2224/sbp.2010.38.1.87 Jain, S. (2019). HR solutions for excellence in training and development Chapter 7: Evaluation of training. Society Publishing. James, C., & Roffe, I. (2000). The evaluation of goal and goal-free training innovation. Journal of European Industrial Training, 24(1), 12–20. https://doi.org/10.1108/03090590010308228 Kirkpatrick, D.L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13(11), 3–9. Kirkpatrick, J.D., & Kirkpatrick, W.A. (2016). Kirkpatrick’s four levels of training evaluation. ATD Press. Kraiger, K., McLinden, D., & Casper, W.J. (2004). Collaborative planning for training impact. Human Resource Management, 43(4), 337–351. https://doi.org/10.1002/hrm.20028 Kumpikaitė, V. (2007). Human resource training evaluation. Engineering Economics, 5(55), 29–36. Lien, B.Y.H., Hung, R.Y.Y., & McLean, G.N. (2007). Training evaluation based on cases of Taiwanese benchmarked high-tech companies. International Journal of Training and Development, 11(1), 35–48. https://doi.org/10.1111/j.1468-2419.2007.00268.x McCain, D. (2016). Evaluation basics. ATD Press. Meyer, M., & Bushney, M. (n.d.). The state of ROI training measurement in South Africa: Growth at levels 4 and 5. Retrieved from http://www.roionline.co.za/ROI-ROI%20State%202005.htm Nadler, L. (1982). Designing training programs: The Critical Events Model. Addison-Wesley Publications. Nickols, F.W. (2005). Why a stakeholder approach to evaluating training. Advances in Developing Human Resources, 7(1), 121–134. https://doi.org/10.1177/1523422304272175 Noe, R.A. (2022). Employee training and development Chapter 6: Training Evaluation (9th ed.). McGraw-Hill Education. Phillips, J.J. (2006). Handbook of training evaluation and measurement methods (3rd ed.). Gulf Publishing Company. Phillips, J.J., & Phillips, P.P. (2016). Handbook of training evaluation and measurement methods. Routledge. Phillips, J.J., Phillips, P.P., & Hodges, T.K. (2004). Making training evaluation work. ASTD Press. Phillips, P.P., & Phillips, J.J. (2001). Symposium on the evaluation of training. International Journal of Training and Development, 5(4), 240–247. https://doi.org/10.1111/1468-2419.00136 Phillips, P.P., & Phillips, J.J. (2015). Real world training evaluation: Navigating common constraints for exceptional results. Association for Talent Development. Republic of South Africa. (1998a). Employment Equity Actß No. 55 of 1998. Government Gazette. Republic of South Africa. (1998b). Skills Development Act No. 97 of 1998. Government Gazette. Republic of South Africa. (1999). Skills Development Levies Act No. 9 of 1999. Government Gazette. Republic of South Africa. (2013). Broad-Based Black Economic Empowerment Amended Act No. 46 of 2013. Government Gazette. Saks, A.M., & Haccoun, R.R. (2018). Managing performance through training and development Chapter 10: Training evaluation (8th ed.). Nelson Canada. Salas, E., Tannenbaum, S.I., Kraiger, K., & Smith-Jentsch, K.A. (2012). The science of training and development in organisations: What matters in practice. Psychological Science in the Public Interest, 13(2), 74–101. https://doi.org/10.1177/1529100612436661 Shenton, A.K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75. https://doi.org/10.3233/EFI-2004-22201 Sparhawk, S. (1994). Identifying targeted training needs. Richard Chang Associates, Inc. Topno, H. (2012). Evaluation of training and development: An analysis of various models. IOSR Journal of Business and Management, 5(2), 16–22. https://doi.org/10.9790/487X-0521622 Torres, R.T., Preskill, H., & Piontek, M.E. (2005). Evaluation strategies for communicating and reporting: Enhancing learning in organisations (2nd ed.). Sage Publications. Twitchell, S., Holton, E.F., & Trott, J.W. (2000). Technical training evaluation practices in the United States. Performance Improvement Quarterly, 13(3), 84–109. https://doi.org/10.1111/j.1937-8327.2000.tb00177.x Werner, J.M. (2022) Human resource development: Talent Development Chapter 7: Evaluating workplace learning interventions (8th ed.). Cengage.

How to cite this article: Duffy, C.G. (2024). A disconnect between training evaluation theory and the practical realities of South African businesses. SA Journal of Human Resource Management/SA Tydskrif vir Menslikehulpbronbestuur, 22(0), a2449. https://doi.org/10.4102/sajhrm.v22i0.2449