Original Research

A framework to assess compliance training effectiveness: The case of banks in South Africa

Marié D. Botha, Stephanus P. van der Merwe

Received: 27 Nov. 2024; Accepted: 08 Apr. 2025; Published: 29 May 2025

Copyright: © 2025. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Orientation: Compliance functions constitute a major cost factor for banks, with challenges in demonstrating training effectiveness.

Research purpose: This research intended to design and validate a measurement tool to assess perceived implementation of compliance training effectiveness factors. Validation was achieved using confirmatory factor analysis, and these factors were incorporated into a Compliance Training Effectiveness Framework.

Motivation for the study: There is limited research on compliance training and a lack of theoretical or conceptual frameworks. Compliance training costs can be substantial, including time away from duties. Evaluating training effectiveness is challenging without a framework, yet essential for demonstrating value add. Traditional metrics offer limited insight and fail to capture training’s impact on employee and departmental performance. This research addresses the need for a comprehensive training evaluation framework.

Research approach/design and method: A quantitative approach was used, employing descriptive statistical analysis on survey data. The research targeted all compliance officers in South African banks, using full population sampling. Respondents indicated agreement with statements related to application of training effectiveness factors.

Main findings: The tool was validated through confirmatory factor analysis. Most compliance officers applied the effectiveness factors, except for measuring ‘return on investment’.

Practical/managerial implications: The framework enables selection and measurement of training effectiveness factors, demonstrating value to stakeholders. Compliance officers should be trained on its application.

Contribution/value-add: The research revealed limited use of return on investment calculations, but widespread application of other measures. The framework consolidates training success models and remains relevant for South African banks, demonstrating both quantitative and qualitative value.

Keywords: banking; compliance; effectiveness; evaluation; framework; training.

Introduction

Orientation

Assessing compliance training’s effectiveness and impact, its effect on employee conduct and its overall influence on banks can be challenging (FinSMEs, 2023). Furthermore, compliance training is typically mandatory and often seen as a reluctant but necessary expense (DiJulius, 2022). Compliance training is crucial for preventing legal penalties and maintaining operational integrity but does not add to the profits of the organisation directly. The compliance costs for banks in South Africa are excessively high (Marx & Mynhardt, 2011). In a global cost of compliance survey conducted in 2023, addressing financial pressures while balancing compliance and competitive requirements was identified as a challenge (Thomson Reuters, 2023). This negative cost perception is worsened by control deficiencies, penalties and the challenge of proving the effectiveness of compliance efforts, including training (Pogson, 2019).

Banks in South Africa function under an extensive framework of both foundational and supplementary legislation (South African Reserve Bank, 2020). The Banks Act of 1995, mandates that banks maintain an independently operating compliance function (Marx & Mynhardt, 2011). The compliance function ensures that the bank complies with applicable laws, regulations and rules to maintain its integrity and reputation (Dayton, 2021). Compliance officers are responsible for various duties, including ensuring that employees and external parties are trained to comprehend regulatory updates and understand the significance of compliance (Haig, 2020). The Compliance Institute Southern Africa (CISA) is instrumental in advancing and upholding compliance standards in South Africa. As the independent professional body for the compliance profession, CISA establishes, sustains and advocates for best practice standards through a range of initiatives (Compliance Institute of Southern Africa, 2025).

Hence, the current study attempted to respond to the abovementioned challenges and address the complexity of demonstrating the importance of compliance training, in a context where costs are perceived as high, by designing a framework for assessing compliance training effectiveness.

Research purpose and objectives

The research purpose was to create and validate a measurement instrument that could assess the extent to which the researched factors of compliance training effectiveness are perceived to be implemented within the training process.

The research objective was to assess the perceived use of compliance training effectiveness factors during the training process of banks in South Africa.

Literature review

Existing literature was explored to identify factors of compliance training effectiveness. From the review of previous research findings, compliance-specific effectiveness factors and global best practices on compliance training were identified.

Corporate training environments and compliance training

According to Allied Market Research (2022), the corporate training industry is projected to grow by 8.0% annually, reaching a global value of $487.3 billion by 2031. Considering the time employees spend away from their duties to complete an average of 5 h of compliance training, the cost can amount to over $1.6 million annually for an organisation with 7500 employees (Rogers, 2016). This estimate excludes the costs associated with designing and developing the training material or purchasing it from external vendors.

Training is the process through which individuals acquire a particular skill (such as how to change the tyre on a vehicle) and subsequently putting the knowledge into practice (Barnes, 2014). Conversely, learning can occur at any place and at any time (Thoms & Burton, 2016). This study refers to compliance training because of the specific compliance skills and their applications taught. The mandatory nature of many compliance training interventions places a high premium and pertinence on the vast costs involved, the value-add of which is difficult to prove (FinSMEs, 2023). Therefore, this research sought to establish a framework for evaluating the successful execution (effectiveness) of compliance training in banks in South Africa.

Training effectiveness and evaluation models

The study addressed a paucity in research on compliance training and relevant theoretical or conceptual frameworks. A holistic compliance training effectiveness framework was developed for the banking industry in South Africa. Previous studies have revealed general corporate training effectiveness models, including the aspects a learning organisation should ascribe to as outlined and initially published by Garvin et al. (2008), the four tiers of training evaluation proposed by the renowned author, Kirkpatrick (1996), and the classical author, Phillips’ (1996) return on investment model, which was specifically adapted for compliance training by Marcheselli (2019). The Kirkpatrick and Philips training evaluation models have profoundly influenced global training evaluation methods over time (Mehale et al., 2021). For an organisation to be recognised as a learning organisation, its employees must be observed generating, acquiring and sharing insights, thereby helping the organisation adapt to uncertainties more swiftly than competitors (Garvin et al., 2019).

The compliance-specific effectiveness factors that emerged from the literature included obtaining business value by completing compliance training, identified by Young (2017); the efficacy and impact of compliance programmes, which incorporated a revised study from the Office of the Inspector General at the Department of Health and Human Services (Health Care Compliance Association, 2017); key effectiveness factors of compliance training, as defined by Dixon and Overton (2017); the essential features of a comprehensive communication and training programme for compliance (Rogers, 2022) and the elements that ensure compliance training is retained (Katz, 2014).

This section elaborates on the abovementioned training effectiveness and evaluation models.

A learning organisation’s key traits

Three key traits crucial to a learning organisation, as identified by Garvin et al. (2019), include concrete learning processes being in place, a leadership that reinforces learning and a learning environment that is supportive. These are discussed briefly further in the text.

Learning processes that are concrete

In a learning organisation, essential learning processes include experimenting with new work methods, systematically gathering information, conducting active workplace analysis, offering and valuing education and training and sharing information with those who need it to make important decisions (Garvin et al., 2019).

Learning reinforced by leaders

Leaders’ actions have a direct impact on the organisation’s learning processes (Garvin et al., 2019). By actively involving employees in conversations and discussions, they are encouraged to learn. To enhance learning, leaders must emphasise the significance of identifying problems, transferring knowledge and conducting reflective reviews. Allowing different perspectives enables employees to present novel ideas and options.

A supportive learning environment

Four essential features of learning environments are: Feeling psychologically safe, appreciating differences, encouraging the expression of receptiveness to innovative concepts and allocating moments for contemplation (Garvin et al., 2019).

Feeling psychologically safe highlights the fact that employees should not fear ostracism for not agreeing with peers or senior leaders, admitting faults, asking naive questions or presenting minority points of view. To foster psychological safety, employees need to feel comfortable expressing their opinions at work (Garvin et al., 2019). Appreciating differences results in learning when employees are cognisant of and understand the importance of contrasting viewpoints. Energy and motivation are boosted, new ideas are ignited and lack of direction and fatigue are avoided (Garvin et al., 2019). Encouraging the expression of openness to new and novel ideas fosters learning and the creation of innovative methods. Consequently, it is important to motivate employees to embrace challenges and take risks to venture into unknown areas (Garvin et al., 2019). When people are too busy or overstressed because of time limitations and planning demands, their capacity for analytical thinking and resourcefulness may be hindered. Pauses in action and providing moments for contemplation and time for reflection are necessary for prompting a thoughtful evaluation regarding the business’s procedures (Garvin et al., 2019).

Training evaluation’s four levels

The evaluation framework by the classical author Kirkpatrick (1996) emphasises the necessity of harmonising the assessment of training effectiveness and evaluation strategy with the broader strategies and objectives of the organisation.

Kirkpatrick and Kirkpatrick (2009) suggest four levels of training evaluation. Level one, which focuses on reaction, assesses how learners respond to the training intervention, including their feedback on topics covered, the effectiveness of the presenters, media used and content included in the training (Kirkpatrick, 1996). Level two focuses on whether learning took place, evaluating learners based on their acquisition of the intended knowledge, competence and attitude during the intervention, by way of assessments. Level three, which evaluates behaviour, requires learners to demonstrate the application of the knowledge they have acquired on the job. Thus, the question is asked whether behaviour has changed after the training session. Kirkpatrick (1996) describes change of behaviour as the application and transfer of training. The fourth level assesses how effectively the training intervention and consequent reinforcement contribute to the achievement of the business’s targeted goals. According to Kirkpatrick (1996), results may include higher sales or efficiency, enhanced earnings, reduced costs, reduced turnover of employees and improved quality delivered by employees.

Return on investment

The Kirkpatrick model was expanded on by translating the training outcomes into monetary terms and contrasting them with the expenses associated with the training. In essence, Phillips (1996) suggested assessing the financial benefits of training and learning interventions and thus calculating the return of monies spent on training. The financial impact of training is determined by evaluating the outcomes against the costs incurred for the training intervention. From a compliance perspective, Marcheselli (2019) recommends that managers evaluate their compliance training objectives and the advantages of meeting them.

In this context, Marcheselli (2019) recommended the following to improve compliance training:

• Focus on reducing training time, customer complaints, employee errors and time and effort to implement regulatory changes.
• Utilise existing data by measuring employee training time, analysing auditor and monitoring feedback and calculating the financial value of decreasing compliance risk exposures.
• Implement new approaches such as role-specific training to target relevant regulations, consequently shortening training duration, boosting productivity and lowering risk.

Deriving organisational benefits and business value from compliance training

Four characteristics of compliance training that are crucial in driving real business value were proposed by Young (2017). These include: providing training that is relevant to individual employees; using workplace and social media training that includes interactive digital training designed to create impactful experiences for learners; concentrating on knowledge, competence and decision-making processes that lead to lasting changes in behaviour and making compliance training a part of broader strategic learning, delivered and reinforced just-in-time, when needed.

Individually relevant training

Role-specific training should be based on everyday situations to enable the accurate implementation of behaviour, skills, judgement and knowledge. To master content, employees require ongoing participation and engagement. This should be reinforced with real-life case studies, examples, storytelling and coaching, all tailored to their specific needs. Greater customisation in training should be paired with flexible cost options (Young, 2017).

Using workplace and social media training

Many people now engage with content by using social media and specifically through video, which enhances retention as well as training through multisensory (using multimedia) participation that is designed to create impactful experiences for learners (Young, 2017). Gamification and augmented reality are increasingly used in training, making training enjoyable and effective by embedding behaviours and allowing practice in an environment that is non-critical but simulates the work environment. Collaboration tools and micro-action programmes, like a 30-day challenge including small, micro actions that need to be completed 30 days after training, help embed new habits and facilitate continuous training in a supportive environment (Young, 2017).

Concentrating on knowledge, competence and decision-making processes

Training on compliance should translate business values and regulations into practical, real-life behaviours, customised to the learners’ communication style, language and requirements (Young, 2017). Effective training involves the experiential learning cycle, where learners reflect, create, test and deeply understand the content through role play and group reflections (Kolb & Kolb, 2018). This approach ensures alignment with regulatory expectations and leads to changes in behaviour that last.

Making compliance training part of broader strategic learning

Training on compliance must align with organisational learning goals and involve close collaboration with professional bodies (such as CISA) and regulators to build trust and gain consumer confidence (Young, 2017). Effective training should focus on measuring its impact on individual performance, assessing behaviour, capability and culture beyond just completion rates. Compliance training needs to be delivered and reinforced just-in-time, when needed.

The effectiveness of compliance programmes

On 17 December 2017, in Washington DC, compliance experts from the Department of Health and Human Services’ Office of the Inspector General, together with employees, convened to deliberate on methodologies for evaluating compliance programmes (Health Care Compliance Association, 2017). The discussion included topics such as creating compliance awareness as well as training. This exercise, organised by the Health Care Compliance Association (2017), intended to explore various ways to measure different aspects of a compliance programme.

Assessment opportunities for compliance programmes were offered to institutions of various capacities, functional complexity, industries and resources. Four teams, each comprising 10 participants, concurrently deliberated two facets relating to compliance programmes. During four sessions, contributors had the chance to propose ideas on both ‘what to measure’ and ‘how to measure’. The current study employed several of the proposed measurement options, which are summarised into the following categories created by the author:

Analysis and planning for training

The following considerations are considered best practice in analysing the training need and planning for the training process:

• High-risk employees receive specific training, with formal assessments to evaluate risks and control weaknesses.
• An updated compliance training plan targets specific job families, addressing relevant topics, methods and the frequency of training, including new laws and continuing education.
• Systems are established to ensure employees receive training on new policies, procedures, laws and regulations, all underpinned by a compliance-specific training policy. Compliance themes are integrated into all organisational learning.
• Employees new to the bank are oriented regarding their compliance training requirements and obligations.
• Provisions are made for learners with special needs.
• Methods are in place to evaluate the impact of training.
• Training addresses vulnerabilities detected through appropriate risk assessments.
• Employee responses on training requirements are integrated into compliance content.

Implementation of training

• The most effective approach for implementing compliance training is utilised, taking into consideration the target audience’s preferences, size and geographical distribution of the learners.
• Responses to the training intervention are evaluated – the training should be regarded as beneficial for employees and adequate to meet the compliance obligations associated with their positions.

Evaluation after training and administration of compliance training

• Assessments are conducted post-training intervention to evaluate knowledge and application transfer.
• Training sessions are recorded, including the recording of test results (assessments) post-training.
• All compliance training records, including training materials and assessments, are maintained for archival and record-keeping purposes.
• Compliance and policy breaches are reviewed, and employees are re-trained where necessary.
• Reports of disclosures, such as direct interactions with compliance officers or subject matter experts, are monitored to assess whether the training increased employees’ responsiveness to specific vulnerabilities.
• The effectiveness of training is measured.

Compliance training effectiveness

Dixon and Overton (2017) stated that an internet-based survey of over 250 compliance, ethics and learning specialists took place in March 2017 and April 2017. Participants were asked to consider current training programmes and whether there are influences that enhance or hinder engagement of learners and the associated risk evaluation process of compliance. The survey results revealed the following considerations:

• Compliance training completion is measured by metrics such as completion rates, pass rates, training take-up rates and enrolment levels.
• Training cost, delivery time and study time are included as metrics of efficiency.
• It is important to work with leaders and champions within the business to encourage and inspire learners.
• It is crucial to have a transparent and effective communication strategy with stakeholders.
• Technological resources like adaptive training, which considers the learner’s current knowledge levels when assessing the content to be posed, can enhance the training experience.
• Not only should employees receive training on compliance but also third parties, contractors and outsourced service providers.
• Compliance training is influenced by frameworks such as the 70:20:10 model, where 70% of what is retained is grounded in workplace decisions, practical practice, experience and workplace skills (Nunaley, 2021), while 20% is recalled through peer learning, social learning, mentoring and coaching. Only 10% is retained from classical instruction.
• Models that support learning directly within the workflow (performance support resources) should be used to guide the training strategy.

The ideal attributes of a comprehensive compliance communication and training programme

The ideal attributes of a comprehensive training and communication programme for compliance, as outlined by Rogers (2022), encompass several key behaviour modification strategies. These strategies are particularly effective within the compliance context and include:

• Disrupting routines, also referred to as the interruption of patterns.
• Dedication and consistency in the application of compliance measures.
• Socially shaming and attaching a stigma to non-compliant behaviours.
• Repetition of compliance messages and training.
• Pursuing enjoyment and evading pain to motivate compliance.

These strategies aim to foster a culture of compliance by altering behaviours and reinforcing the importance of adherence to established guidelines.

Pattern interruption involves breaking behaviour sequences through inquiries or unforeseen occurrences (Rogers, 2022). Compliance training is often perceived as dull, repetitive and strict. However, engaging and interesting training can disrupt this pattern. Additionally, individuals tend to make larger commitments after smaller ones because of behavioural consistency and dedication (Rogers, 2022). Given that individuals generally resist change, when they have expressed an opinion or taken an action, they strive to remain consistent and avoid contradicting themselves. Therefore, incorporating commitments, such as learners acknowledging they have read and understood policies, into training can be effective. Socially shaming and social stigma refer to adverse societal repercussion arising from specific beliefs or behaviours. These actions deviate from norms, leading others to view the individual unfavourably (Rogers, 2022). The most effective leverage often arises when compliance training evokes anxieties related to social stigma, such as potential dismissals. Moreover, the significance of reiteration and repetition in compliance education and training is evident. Advertisers and promoters typically recognise that customers need to see product messages multiple times, ranging from 9 to 21, before making a choice to buy (Rogers, 2022). Microlearning, which involves brief snippets of information, can help reinforce previously taught material. Finally, as people generally strive to avoid discomfort and pain and pursue pleasure and enjoyment (Rogers, 2022), many programmes, including compliance training, highlight the negative outcomes of non-compliance using various approaches (Rogers, 2022).

Prominent figures in the compliance sector were interviewed and suggested that a holistic compliance communication programme should (Rogers, 2022):

• Contain continuous communication, including both training and regular reinforcement of key messages.
• Ensure low-bandwidth, seamless (non-intrusive) connections with interactions lasting between 30 s and 3 to 4 min, making it simple to incorporate into the work environment.
• Include a variety of media that is engaging and captivating.
• Be tailored to the learning preferences of diverse audiences, including kinaesthetic, auditory and visual styles.
• Track commitment and participation.
• Communicate instantly, ideally within moments of detecting a risk so that employees become aware of the risk.
• Reduce training exhaustion, change employee behaviour and improve retention by regularly refreshing and reiterating essential content.

The considerations that enhance compliance training retention

According to Katz (2014), stories could transform complex concepts into vivid and unforgettable narratives. Incorporating humour in training can maintain employee interest in otherwise dull subjects. However, it is important to exercise caution, as compliance itself is not amusing or humorous. Additionally, leveraging current news events can effectively convey and reinforce crucial compliance principles (Katz, 2014).

From the literature review, the 10 compliance training effectiveness models identified were incorporated into a measuring instrument to assess the application of perceived compliance training effectiveness within banks in South Africa. These models shared overlapping factors.

Research methodology

Research approach

A quantitative research approach was implemented using a survey design. Quantitative research employs statistical and visual analysis techniques (Saunders et al., 2019). This study approach explored the application of effectiveness measures of compliance training using data from a survey with close-ended questions. An empirical study was conducted, where participants indicated the extent to which they agreed or disagreed with statements on applying compliance training effectiveness in their banks. Following the empirical study, the Compliance Training Effectiveness Framework was created by consolidating the researched factors of compliance training effectiveness.

Research method

A single (mono) method approach was chosen, specifically quantitative research, using a research questionnaire in the form of a survey. A mono method was selected for the study to evaluate the current perceptions of compliance officers regarding their use of compliance training effectiveness measures at a specific point in time.

Research participants

The study population comprised compliance officers within banks in South Africa, which, according to communication from the BANKSETA’s skills planning and research department, amounted to 1232 employees as of 2020 (BankSeta, personal communication, 04 March 2021). The sampling method employed was full population sampling, including all compliance officers throughout the banking sector of South Africa. The compliance officers were selected using a purposive method of sampling, considering a uniformity factor (Saunders et al., 2019). Participants fell within the same occupation and perform similar duties, including the responsibility for compliance training (Lunday, 2010). The BANKSETA agreed that existing industry relationships could be used to access the email addresses of the employees responsible for skills development at the different banks. Compliance contacts and the employees responsible for skills development were used as entry points to respect and follow the participation approval processes of the banks.

Measuring instrument

The measuring instrument used to assess compliance training effectiveness within banks in South Africa was developed through the literature review. As discussed in the literature review section, the instrument included 10 corporate and compliance training effectiveness models.

The research instrument included the following 10 sections (Botha, 2024):

1. Four training evaluation tiers by the renowned author, Kirkpatrick (1996): reaction of learners tracked, training took place, behaviour changed and improvement of targeted business measures.
2. The return on investment model (Marcheselli, 2019; Phillips, 1996).
3. Requirements of a learning organisation (Garvin et al., 2019): an environment supportive of learning.
4. Requirements of a learning organisation (Garvin et al., 2019): learning processes that are concrete.
5. Requirements of a learning organisation (Garvin et al., 2019): learning that is reinforced by leaders.
6. Obtaining business value from training (Young, 2017), focusing on:
   • 6.1 Training that is relevant to the individual.
   • 6.2 Engaging learner experiences through social media, on-the-job training and multimedia-rich digital training.
   • 6.3 Emphasising knowledge, competence, behaviours and decision-making processes that foster enduring behavioural change.
   • 6.4 Making compliance training part of a broader strategic learning, delivered and reinforced just-in-time, when needed.
7. The effective impact of compliance training programmes, including those related to awareness creation and training (Health Care Compliance Association, 2017), with a focus on:
   • 7.1 Planning (preparing) for training.
   • 7.2 Implementation (deployment) of training.
   • 7.3 Evaluation (assessing the impact) and administration following training.
8. Key factors for successful compliance training (Dixon & Overton, 2017).
9. Essential features of a comprehensive communication programme for compliance, including training, according to Rogers (2022).
10. The elements that ensure compliance training is retained (Katz, 2014).

Research procedure

The quantitative survey was made available on the Microsoft Forms platform. The survey link was distributed to compliance contacts and skills development facilitators at the banks in South Africa, following the authorisation procedure for every bank. Informed consent was acquired from the employees responsible for skills development and the compliance contacts, some electronically and some verbally. After receiving approval, the survey was dispatched to compliance officers by the compliance contact, the employee responsible for skills development or another appointed person in the bank. Participation was voluntary. The questionnaire indicated that by completing it, participants gave implicit consent for the use of the research data as outlined in the questionnaire. Without consent, participants were unable to proceed with the survey.

The link to the survey was additionally posted to LinkedIn, enabling compliance officers within the banking compliance sector of South Africa to participate by invitation, after providing electronic consent. Additionally, the survey link was posted on the CISA’s website, and members were encouraged to participate. By exception, the survey included learning and development employees who designed compliance training, as they had detailed knowledge of best practices on training evaluation measures implemented within their banks. Out of the participant population, 341 surveys were completed, accounting for 27.7%.

Statistical analysis

IBM SPSS (version 28 of the IBM Statistical Package for Social Sciences) was used to perform the data analysis (Botha, 2024). Descriptive analysis was conducted on the combined responses to all questions across the scales for the 10 different effectiveness models of compliance training, aiming to evaluate the perceived usage of compliance training measures within compliance departments of banks in South Africa (Botha, 2024).

Ethical considerations

Ethical considerations were adhered to as follows: Ethical clearance was obtained from the North-West University’s Economic and Management Sciences Research Ethics Committee (EMS-REC) with reference number NWU-01902-22-A4. Additionally, written consent was obtained from the BANKSETA to access the email contacts of employees responsible for skills development through existing industry communities. Informed consent was acquired either electronically or verbally from chief compliance officers, employees responsible for skills development or compliance contacts, where they were used as entry points into the banks. Participation in the survey was voluntary and anonymous, and informed consent was received electronically from the respondents prior to the commencement of the survey.

Results

Validity and reliability of the research instrument measuring compliance training effectiveness

Variables curated from the literature study were validated using confirmatory factor analysis (Taherdoost et al., 2022) to form the constructs for assessing the effectiveness of compliance training. Factor loadings exceeding 0.298 are acceptable in samples greater than 300. Research by Hassim et al. (2020) has also considered variables with factor loadings below 0.3 to be suitable. Thus, this study deemed the confirmatory factor loadings to be acceptable. To further evaluate convergent validity, the study ensured that the average variance extracted for the model exceeded the 0.5 threshold (Baharum et al., 2023). Items on the borderline were still considered acceptable. The instrument measuring compliance training effectiveness was therefore regarded as valid (Botha, 2024).

In social and organisational sciences, Cronbach’s alpha is widely used to evaluate the reliability and internal consistency of research instruments (Gong, 2018). Shrestha (2021) states that a Cronbach’s alpha value above 0.7 is deemed suitable. Every factor of the compliance training effectiveness measurement instrument demonstrated a Cronbach’s alpha above 0.7, except for those related to training retention, which had a borderline value of 0.66 but were still deemed acceptable (Botha, 2024). Notably, the section in the questionnaire focusing on the retention of compliance training comprised only two questions. The survey questionnaire measuring compliance training effectiveness was regarded as reliable and demonstrated internal consistency in the data.

Descriptive statistics

A descriptive analysis was conducted on the total responses to all questions across the scales for the 10 different compliance training effectiveness models, aiming to assess the perceived effectiveness of compliance training within banks in South Africa. The results are discussed in the following text.

Summary of results

The descriptive statistics revealed the following:

More than half (53.48%) of the respondents agreed or strongly agreed that they implemented the training effectiveness statements (evaluation measures) proposed by the classical author Kirkpatrick (1996) within their workplace. Just 31.21% of respondents indicated agreement or strong agreement with applying the return on investment statements proposed by Marcheselli (2019) and Phillips (1996) within their banks. The results further revealed that 48.86% were undecided, while 19.94% disagreed or strongly disagreed. Most respondents (63.73%) agreed or strongly agreed that they applied the supportive learning environment statements in the workplace (Garvin et al., 2019). Similarly, 72.11% of respondents agreed or strongly agreed that they applied concrete learning processes (Garvin et al., 2019). Additionally, 74.25% of respondents agreed or strongly agreed that they applied measuring leadership that supports (reinforces) the process of learning within their banks (Garvin et al., 2019).

Most (69.52%) of the respondents agreed or strongly agreed that they applied the statements proposed by Young (2017) for measuring the attainment of business value after compliance training. Furthermore, 79.21% of respondents agreed or strongly agreed that they measured the impact of training and awareness programmes (Health Care Compliance Association, 2017). Most respondents (72.69%) agreed or strongly agreed that they applied the key factors for effective compliance training, as proposed by Dixon and Overton (2017). Similarly, 73.26% of respondents strongly agreed or agreed that they applied the essential features of a comprehensive communication programme for compliance, including training (Rogers, 2022). Finally, 58.50% of respondents strongly agreed or agreed that they applied the statements to ensure compliance training is retained (Katz, 2014).

Discussion

Outline of the results

The findings indicated that most respondents believed they applied the various compliance training effectiveness factors, except for calculating return on investment. It appears that most compliance functions within banks in South Africa are undecided or have not implemented the return on investment measure as part of compliance training effectiveness. Several factors may contribute to this phenomenon. The mandatory nature of compliance training can make the effort to calculate return on investment seem unnecessary. Furthermore, the intricacy of computing return on investment, coupled with a potential deficiency of knowledge or competence in performing such calculations, may be evident (Botha, 2024). This is highlighted by the fact that 48.86% of respondents were uncertain about whether they were applying return on investment within their banks.

Contribution

After validating the research instrument using confirmatory factor analysis, the factors contributing to compliance training effectiveness were integrated into a framework. This framework follows the analysis, design, development, implementation and evaluation (ADDIE) instructional design process (Afia & Laili, 2023) and includes a training administration stage. It also considers the components of the compliance training ecosystem. The Compliance Training Effectiveness Framework is shown in Figure 1.

FIGURE 1: Compliance training effectiveness framework for South African banks.

The factors from the corporate and compliance training effectiveness models in the literature review were organised into key clusters of best practices. These clusters shaped the Compliance Training Effectiveness Framework of banks in South Africa, as shown in Figure 1:

• Analysis and planning stages of compliance training
• Design and development stages
• Implementation stage
• Evaluation stage
• Administration stage

The outer section of the framework encompasses the compliance training ecosystem, which requires a supportive learning environment. This includes leaders who support the learning, communication programme and internal organisational processes. A supportive learning environment is crucial for effective learning. It makes learners feel safe to make mistakes and ask questions, values diverse perspectives, welcomes innovative thoughts and provides opportunities to reflect on successes and areas for improvement.

Leadership should reinforce what has been taught and learned. Effective learning processes should include formal methods for experimentation through innovative working methods. Information should be systematically produced, gathered, analysed and disseminated within the organisation. Workplace analysis should identify assumptions that may impact key decisions. Education and training should be valued, and information should be swiftly and accurately conveyed to essential stakeholders for decision-making purposes.

The inner section of the framework for compliance training effectiveness encompasses the phases of training analysis and planning, design and development, implementation, administration and evaluation. These phases follow a chronological and iterative process.

The analysis phase, in addition to preparing for compliance training, consists of the following:

• Conducting comprehensive risk assessments for high-risk employees to determine their specific training needs and identify control vulnerabilities that should be addressed. This involves integrating a consolidated risk assessment process (risk-based approach) within the compliance training framework.
• Performing a thorough target audience analysis to identify who needs training, how the training will be conducted and the content of the training programme.
• Establishing a compliance training policy.
• Implementing a compliance orientation programme for new employees.
• Keeping an updated rule book or compliance training plan that outlines which job titles require training on specific topics and the frequency of the training.

The design and development phase should comprise:

• Techniques that ensure training effectiveness and retention.
• A focus on the adverse societal stigma and distressing outcomes of non-compliance as part of training effectiveness.
• Role-specific, individually relevant training that leverages transformative learning interactions through workplace, digital and social learning, utilising multimedia-intensive methods. Integration of compliance training with broader strategic learning, reinforced and delivered when needed.
• Emphasising skills, knowledge, behaviours and decisions that result in lasting behavioural change.

The implementation phase includes:

• Confirming that third parties, senior leaders and board members receive training.
• Integrating training with other learning opportunities and incorporating it into a holistic learning plan within work processes.
• Using job families to identify similar roles and tailor content for different roles.
• Providing training on new regulations, laws, procedures and policies.
• Offering continuing education units to ensure employees are fit and proper for the roles they fulfil, where required.

The administration of compliance training involves several administrative duties, such as updating compliance content according to its shelf-life, maintaining records as required and providing reports to relevant forums. This includes periodically updating training materials and retaining training records as stipulated by regulations.

The training evaluation phase, based on Kirkpatrick’s (1996) proposal, includes:

• Acquiring learner reactions after a training event.
• Calculating whether training took place through various assessments.
• Evaluating changes in learner behaviour after training, such as through communication and inquiries with line leaders.
• Measuring the impact on business results, such as a reduction in compliance breaches after training.

The training evaluation phase also includes return on investment (Marcheselli, 2019; Phillips, 1996), where the financial benefits of compliance training, such as reduced training time or decreased costs from reduced breaches, are examined. Diagnostic measurements (Dixon & Overton, 2017) include training completion rates, assessment scores and training study time. Additionally, broader evaluation measures are monitored (Dixon & Overton, 2017; Health Care Compliance Association, 2017). For instance, compliance and policy shortcomings are assessed, and additional training is offered as needed.

The key contribution of this study is as follows: Compliance officers and managers responsible for compliance training in South African banks can effectively utilise the Compliance Training Effectiveness Framework. This framework enables the selection of various quantitative and qualitative effectiveness factors to measure, assess and demonstrate the value added by compliance training programmes. This is particularly pertinent given the substantial compliance costs and the inherent challenges in substantiating the value of the financial and temporal investments in compliance training. The framework helps communicate this value to senior stakeholders, governance forums and regulatory bodies.

Limitations and recommendations

The study participants were compliance officers within banks in South Africa who agreed to take part in the survey. Some banks were unwilling to distribute the survey link internally; consequently, the researcher used LinkedIn to reach compliance officers directly. Confirmation of compliance officers’ employment at banks in South Africa was sometimes necessary through direct messages. An electronic survey link was published on the CISA’s online platform. A message from CISA to its members helped encourage participation. As the survey link was shared on LinkedIn and CISA’s website, anyone could participate, potentially affecting the results. To address this, the questionnaire indicated that the study focused on the banking sector in South Africa and its compliance departments.

The research indicated that some banks rely on their in-house training departments or external vendors to create and assess all training programmes, including those for compliance. In contrast, other banks incorporate relevant learning and development employees directly into the compliance function. Compliance officers in banks in South Africa that have separate learning functions or use external providers may have struggled with questions about compliance training effectiveness measures applied within their banks. To account for this variation, the study also included learning employees or practitioners responsible for compliance training but who were not part of the compliance function.

Future research recommended

This study can be extended to additional regulatory functions, including legal or risk management departments. This would enable an assessment between the current research results and the recommended study to evaluate how well the effectiveness of compliance training aligns with the effectiveness of risk or general non-financial risk training. The current study could also be repeated as a longitudinal study over time to explore whether the perceived application of compliance training effectiveness measures within banks in South Africa increases over time.

Conclusions

A measurement instrument was developed from existing literature and validated through an empirical study to assess the extent to which compliance training effectiveness is perceived to be implemented within the training process in compliance functions within banks in South Africa. This tool may be used as is or adapted to suit specific banks’ needs to assess the perceived effectiveness of compliance training in banks in South Africa, as demonstrated in this study. The majority of compliance training effectiveness measures are perceived to be utilised within the banking industry of South Africa. However, it is evident that the concept of return on investment is either not widely known or not used. Upskilling compliance officers on the application of the effectiveness measures, including return on investment, could be highly beneficial. Moreover, the value added by compliance training could be effectively communicated to various governance boards and other stakeholders in an industry where costs are perceived to be high.

The design of the effectiveness framework for compliance training adapted a broad learning evaluation method, sourced from various origins, into a framework specifically tailored for compliance. This framework can be utilised by compliance heads and learning and development practitioners of banks in South Africa to demonstrate the value-add of mandatory and other compliance training, often perceived as a reluctant purchase. Given that 42% of all online training within companies involves compliance training (Dixon & Overton, 2017), this framework can be especially valuable to the compliance industry and, indeed, the broader learning and development community.

The significance of the Compliance Training Effectiveness Framework lies in its ability to demonstrate both quantitative and qualitative value added by training, awareness and communication programmes, which are often considered costly. Employee salaries and wages represent a significant expense and completing compliance training requires considerable time. Therefore, the ability to calculate both the quantitative and qualitative value added by compliance training is of significant importance.

Acknowledgements

This article is based on the author, M.D.B.’s thesis entitled ‘Proposing a compliance training success framework for South African banks to enhance an entrepreneurial business environment’, towards the degree of Doctor of Business Administration at the North-West University, with Professor Stephanus van der Merwe as supervisor, received April 2024. It is available here: https://repository.nwu.ac.za/bitstream/handle/10394/42546/Botha_MD.pdf?sequence=1.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

M.D.B. completed the data collection, analysis and interpretation, conceptualised the manuscript and was the lead writer. S.P.v.d.M. was the supervisor.

The authors reported that they received funding from the BANKSETA, which may be affected by the research reported in the enclosed publication. The author has disclosed those interests fully and has implemented an approved plan for managing any potential conflicts arising from their involvement. The terms of these funding arrangements have been reviewed and approved by the affiliated university in accordance with its policy on objectivity in research.

The data that support the findings of this study are not openly available because of confidentiality and are available from the corresponding author, M.D.B., upon reasonable request.

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.

References