In-role performance refers to actions that are official or known requirements for employees (Carpini et al., 2017; Motowidlo & Van Scotter, 1994). These behaviours could be viewed as the technical core (Borman & Motowidlo, 1997) that employees must demonstrate to be perceived as proficient and able to contribute to the achievement of organisational goals (Carpini et al., 2017). Unlike the performance models proposed by Schepers (2008), Koopmans et al. (2012) and Myburgh (2013), quality and quantity of work were divided into two separate, narrow performance dimensions, which align more with Viswesvaran et al.’s (2005) conceptualisation of performance. Technical performance was included under in-role performance, based on Borman and Motowidlo’s (1997) definition of task performance, namely the technical core tasks that employees perform, and was conceptualised in accordance with Campbell et al.’s (1993) definition. Finally, whereas following rules and organisational procedures was categorised by Borman and Motowidlo (1997) under contextual performance, it was included under in-role performance as an official (minimum and less voluntary) requirement of acceptable behaviour, as it is expected that employees will comply with organisational rules and procedures (Carpini et al., 2017; Katz, 1964). Definitions of the narrow dimensions comprising in-role performance are provided here:

The following hypothesis was formulated based on the conceptualisation of in-role performance as a hierarchical model:

Extra-role performance refers to acts orientated towards the future or change (Carpini et al., 2017), aimed at benefitting coworkers and the team (Organ, 1997), which are discretionary or not part of the employee’s existing work responsibilities (Borman & Motowidlo, 1997; Van Dyne, Cummings, & McLean, 1995). These behaviours shape the team in a way that creates a work setting that contributes to the achievement of organisational goals (Borman & Motowidlo, 1997). In the present study, altruism and initiative were untangled to form two separate, narrow dimensions, instead of the unidimensional construct measured by the IWPQ (Koopmans et al., 2012) and GPQ (Myburgh, 2013). Two narrow dimensions outlined by George and Brief (1992) and Podsakoff, MacKenzie, Paine and Bachrach (2000) as extra-role behaviours and applied as narrow, standalone dimensions in the GPQ (Myburgh, 2013) were further included to extend current conceptualisations of extra-role performance in South Africa, namely self-development and innovation. Self-development and innovation share conceptual overlap with adaptive performance (Pulakos, Arad, Donovan, & Plamondon, 2000), but were retained because of their discretionary and future-orientated nature (Carpini et al., 2017; George & Brief, 1992). The narrow dimensions of extra-role performance are defined here:

The conceptualisation of extra-role performance as a hierarchical model gave rise to the following hypothesis:

Adaptive performance relates to employees’ demonstration of the ability to cope with and effectively respond to crises or uncertainty (Carpini et al., 2017; Pulakos et al., 2000). It is also reflected in employees’ flexibility when dealing with novelty or working with coworkers who have different views (Pulakos et al., 2000). A broad multidimensional conceptualisation of adaptive performance (Pulakos et al., 2000), instead of a narrow dimension (Koopmans et al., 2012; Myburgh, 2013), was used as a primary orientation, but was adjusted to fit the performance model of this study. Firstly, dealing with uncertainty was reformulated as dealing with complexity to enlarge the scope of the construct and increase its relevance to important psychological predictors of performance, for example, cognitive ability (Sackett, Zhang, Berry, & Lievens, 2021). Secondly, interpersonal and cultural adaptability were collapsed into a single dimension for brevity of measurement. Thirdly, two narrow dimensions of adaptive performance identified by Pulakos et al. (2000), namely solving problems creatively and learning to extend existing knowledge on the job, were considered better suited to extra-role performance and were therefore not categorised under adaptive performance in this study. Finally, physical adaptability was conceived to be less generalisable across office and non-office settings and was thus excluded. Definitions of the narrow dimensions of adaptive performance are as follows:

The following hypothesis was formulated to test the tenability of the hierarchical model of adaptive performance:

Leadership performance refers to the effectiveness with which an employee can influence co-workers to achieve collective goals (Campbell & Wiernik, 2015; Hogan & Sherman, 2020; Yukl, 2012). Leadership does not necessarily have to be tied to a position of authority (Campbell & Wiernik, 2015; Myburgh, 2013) and could be portrayed by anyone who supports, directs and connects coworkers and changes coworkers’ views or approaches to doing things (Hedge et al., 2004; Yukl, 2012). Yukl’s (2012) taxonomy of leadership effectiveness was used as the primary orientation in the conceptualisation of leadership performance. External leadership was changed to network-orientated leadership, to reflect recent developments in complexity leadership (Uhl-Bien & Arena, 2017). The narrow dimensions of leadership performance are defined as follows:

The following hypothesis was formulated based on the conceptualisation of leadership performance as a hierarchical model:

Counterproductive performance reflects intentional or unintentional acts (Spector & Fox, 2005) by an employee who negatively affects the effectiveness with which an organisation achieves its goals and causes harm to its stakeholders (Campbell & Wiernik, 2015; Marcus et al., 2016). The present researchers deliberately chose forms of counterproductive performance that are of a lesser severity (Bennett & Robinson, 2000). Less severe behaviours (e.g. incivility) were chosen as narrow dimensions that are more frequently observable, less situation-dependent and easier to report on and utilise in performance feedback. Examples of more severe behaviours excluded are wilful damage to property of the employer, wilful endangering of the safety of others, physical assault and sexual harassment (Gruys & Sackett, 2003; Venter et al., 2014). These acts are viewed as forms of gross misconduct in South Africa – and warrant summary dismissal – which might be outside the purview of performance feedback and more in the realm of disciplinary action, which is subject to confidentiality (Venter et al., 2014). However, it must be acknowledged that more severe behaviours of counterproductive performance, such as physical assault, have serious consequences for organisations and there should be mechanisms in place to deal with such acts. Severe behaviours are less frequently endorsed in performance questionnaires and appear to increase the statistical multidimensionality of the construct. The deliberate exclusion of severe forms of counterproductive performance in this study might have increased the unidimensionality of the construct (Marcus et al., 2016).

In contrast to the unidimensional conceptualisation of counterproductive performance in the IWPQ (Koopmans et al., 2012) and CPQ (Myburgh, 2013), intrapersonal-focused and interpersonal-focused dimensions of counterproductive performance were differentiated in this study. Intrapersonal-focused counterproductive performance reflect avoidant behaviours such as withholding effort, aimed at escaping work situations. In contrast, interpersonal-focused counterproductive performance reflect approach behaviours such as interpersonal rudeness, which focuses on a dysfunctional engagement in work situations (Spector et al., 2006). Stagnation and stubborn resistance (Tepper, Schriesham et al. 1998) are seldom categorised under counterproductive performance (Marcus et al., 2016) but were included as narrow dimensions to extend the intrapersonal and interpersonal focus of counterproductive performance. Definitions of the narrow dimensions of counterproductive performance are as follows:

The following hypothesis was formulated to test the tenability of the hierarchical model of counterproductive performance:

A cross-sectional, quantitative research design was utilised in this study. The selection of a cross-sectional design ensured a composite view of the multifaceted nature of managers’ ratings of their subordinates’ performance at one point in time, as well as an efficient quantitative exploration of commonalities between a large set of variables across different organisational contexts (Spector, 2019).

The objective of the study was to determine the plausibility of five hierarchical dimensions (factors) of performance in the South African context. Confirmatory factor analysis (CFA) was performed, using version 0.6–8 of the lavaan package (Rosseel, 2012; Rosseel & Jorgensen, 2021) in R (R Core Team, 2016), to first inspect the interfactor correlations between all the narrow performance factors, whereafter the hierarchical factor structure of the broad performance factors was investigated. The guidelines recommended by Credé and Harms (2015) were followed in exploring the plausibility of alternative models (including hierarchical factor models), based on the five performance factors. The bifactor models were, in turn, subjected to analysis of bifactor statistical indices to determine whether unidimensional models better represented the structure of the five factors (Reise, Bonifay, & Haviland, 2013; Rodriguez et al., 2016).

Mardia’s multivariate skewness and kurtosis coefficients were 146297.60 (p < 0.001) and 105.56 (p < 0.001), which indicated that the data had a non-normal multivariate distribution. Given the medium (n = 448) sample size (Rhemtulla, Brosseau-Liard, & Savalei, 2012), the employment of rating scales with five numerical categories (Rhemtulla et al., 2012) and violation of multivariate normality (Satorra & Bentler, 1994; Yuan & Bentler, 1998), CFA results with robust maximum likelihood (MLM) estimation were deemed appropriate (Bandalos, 2014). Model–data fit of the CFA models was evaluated using the comparative fit index (CFI), the Tucker–Lewis Index (TLI), standardised root mean-square residual (SRMR) and root mean square error of approximation (RMSEA) (Brown, 2015; Hu & Bentler, 1999). The fit was considered suitable if the RMSEA and SRMR were ≤ 0.08 (Brown, 2015; Browne & Cudeck, 1992) and the CFI and TLI were ≥ 0.90 (Brown, 2015; Hu & Bentler, 1999). Even if comparative fit indices display a marginally good fit to the data (CFI and TLI in the range of 0.90 to 0.95), models might still be considered to display acceptable fit if other indices (SRMR and RMSEA) in tandem are in the acceptable range (Brown, 2015).

Table 1 provides the mean item score and standard deviation for each scale of the IWPR, along with the alpha and omega reliability estimates and standardised interfactor correlations of the 20 narrow performance factors. The interfactor correlations were obtained by conducting an oblique lower-order confirmatory factor model. The fit statistics for the oblique lower-order confirmatory factor model of the entire IWPR (χ² [df] = 4769.72 [2890]; CFI = 0.94; TLI = 0.93; SRMR = 0.05; RMSEA = 0.04 [0.04; 0.05]) were satisfactory (Brown, 2015; Hu & Bentler, 1999).

The interfactor correlations below the diagonal in Table 1 suggest that the narrow factors under in-role, extra-role, adaptive and leadership performance, respectively, covaried as expected. Narrow performance factors also covaried, with narrow factors outside of the broader related performance dimensions, suggesting that alternative theoretical configurations could be explored in future. The size of the interfactor correlations between all the narrow factors suggested that a general performance factor may exist across these factors in the IWPR (Viswesvaran et al., 2005). Evidence forwarded by Schepers (2008), Myburgh (2013) and Van der Vaart (2021) also suggested performance factors tend to covary.

The upper limit of 87% of the interfactor correlations in Table 1 were below the cut off (UL < 0.80, as proposed by Rönkkö & Cho, 2020) and therefore the majority of the narrow dimensions of performance displayed sufficient discriminant validity. Rönkkö and Cho (2020) considered interfactor correlations of 0.80 ≤ UL < 0.90 as marginally problematic and 0.90 ≤ UL < 0.10 as moderately problematic. According to this guideline, 13% of the upper limit correlations in Table 1 have lower discriminant validity. However, 6% of the upper limit correlations with marginal to moderate problematic discriminant validity are between narrow performance dimensions of the same broad dimension. The remaining 7% indicates marginally to moderately problematic discriminant validity between narrow performance dimensions that fall under different broader performance dimensions, which could be areas for future theoretical exploration and empirical replication.

Hierarchical CFA was performed to analyse the data gathered on employees using the IWPR. The purpose of hierarchical factor analysis was to provide a more parsimonious account, based on a pre-defined theory, of a latent variable that consists of various underlying narrow factors that have something in common (Brown, 2015). Credé and Harms (2015) argued that five sequential models should be tested before claiming a hierarchical structure amongst latent variables, namely orthogonal first-order, single-factor, higher-order, oblique lower-order and bifactor models. Figure 1 provides an example, using in-role performance, of how each of the factor models was specified. Not all the items of in-role performance are displayed in Figure 1 (the scale of each factor consists of 16 items).

As portrayed in Figure 1, both higher-order and bifactor models are hierarchical factor models. In-role performance narrow factors (quality of work, quantity of work, rule adherence and technical performance) mediated the relationship between the manifest variables and the in-role performance broad factor in the higher-order model (Beaujean, 2014). The broad performance factor therefore did not explain unique variance in the manifest variables beyond the narrow factors (Beaujean, 2014; Mcabee, Oswald, & Connelly, 2014). Bifactor models, in contrast, accounted for the unique variance explained in the manifest variables by the orthogonal broad performance factor, beyond the variance explained by the orthogonal narrow factors (Beaujean, 2014; Mcabee et al., 2014). The model–data fit of the different factor models outlined in Figure 1 is specified in Table 2.

The CFI, TLI, SRMR and RMSEA reported in Table 2 convey that a first-order and single-factor model fit the data poorly, whereas acceptable fit for an oblique lower-order factor, higher-order factor and bifactor model provided evidence of the existence of a hierarchical structure in the data. Overall, the bifactor models seemed to provide the best fit to the data, which supported H₁ to H₅.

Bonifay, Lane and Reise (2017) indicated that the superiority of bifactor models’ fit indices, relative to that of other confirmatory factor models, could be a symptom of overfitting. Rodriguez et al. (2016) recommended that bifactor statistical indices be calculated to determine the practical meaningfulness of group factors, such as the explained common variance (ECV), coefficient omega hierarchical (ω_h), construct replicability (H), factor determinacy (FD), percentage of uncontaminated correlations (PUC) and relative percentage bias (ARPB). Group factors were considered plausible when ω_h, H and FD² were > 0.50, 0.70 and 0.70, respectively (Dueber, 2017; Reise et al., 2013). The ECV for the general factor > 0.70 and PUC > 0.80 were indicative of unidimensionality (Reise et al., 2013). The ARPB of 10% to 15% indicated little difference in the factor loadings between a single-factor model and the general factor in a bifactor model (Rodriguez et al., 2016). Bifactor statistical indices were calculated using version 0.2.0 of the Bifactor Indices Calculator package (Dueber, 2020) in R (R Core Team, 2016). The bifactor statistical indices are reported in Table 3.

The bifactor statistical indices in Table 3 provide evidence of:

The IWPR is an attempt to address the bandwidth–fidelity problem in the measurement of performance by enabling researchers to inspect the validity of predictors and measure performance using a variety of narrow dimensions that still enable the in-depth measurement of the broader performance dimensions (Hunt, 1996). The broader dimensions may be of benefit when predictions need to be made about broader criteria in predictive studies. For example, in-role performance may be a more appropriate criterion when determining the predictive validity of the personality trait conscientiousness (Judge et al., 2013). The narrow performance dimensions quality and quantity of work may, however, be more appropriate when detailed predictions are required. For example, the personality aspects of industriousness (effort exerted in performing work) and orderliness (meticulousness with which tasks are performed) may explain unique variance in the quantity and quality of work produced by employees (Van Lill & Taylor, 2021). Evidence for the hierarchical structure of the IWPR could therefore enable practitioners to run predictive analytics at their preferred level of criterion, without compromising either accuracy or specificity.

The overall oblique lower-order model for the entire IWPR provides preliminary evidence of a fair degree of discriminant validity (87% of interfactor correlations) between the 20 narrow performance dimensions. The 20 narrow dimensions can be interpreted simultaneously with the five broader dimensions of in-role, extra-role, adaptive, leadership and counterproductive performance, based on the fit of the bifactor models. However, caution should be taken when interpreting each dimension as a hierarchical factor, as the bifactor indices suggested that each of the broader factors could be more parsimoniously interpreted as a unidimensional factor.

Contrasting evidence on the hierarchical structure of individual work performance merits two positions regarding the interpretation and use of scales based on the IWPR. Firstly, it may be more prudent to interpret the broader performance dimensions as total scores, especially when high-stakes decisions are made, for example, to identify, reward and promote star performers in organisations. By contrast, the fair degree of discriminant validity indicated that the narrow dimensions could capture differences between individuals, which could be useful in low-stakes situations, for example, tailoring performance feedback, setting performance goals and establishing new work habits to achieve performance goals. Carpini et al. (2017) argued that even in the presence of unidimensionality, narrow interpretations might still add to context when interpreting scores based on general factors, which, at times, could appear as more vague formulations. Cross-scale interpretations are further encouraged to enable a holistic understanding of employees’ performance.

The practical benefits of a carefully constructed measure of generic individual work performance for industrial psychologists and human resource professionals, such as the IWPR, include: