Validity

Validation Centric Development Approach

Saville Consulting Wave ® Professional Styles questionnaires are based on a Validation Centric development method where the validity of the item is a fundamental criterion (among other criteria) for item selection in a questionnaire. This approach is designed to maximise the validity of the questionnaire in predicting external job performance criteria. The questionnaire was developed to predict 36 work competency criteria.

Empirical Validity

The average validity of one Professional Styles scale in relationship to its work performance criterion is 0.39 and the composite validity of more than one Professional Styles scale across the criteria is 0.46.* This clearly indicates that the Professional Styles scales provide strong validity, ranging across a wide array of externally assessed work variables.

The composite validity* of each of the two Professional Styles forms in relation to overall job proficiency is 0.34 and 0.42 (n=325).

The composite validity* of each of the two Professional Styles forms in establishing external ratings of potential for promotion is 0.54 and 0.64 (n=324).

Practical Benefit

Validities of 0.30 can halve the selection errors under the appropriate circumstances, which can greatly improve the ability to spot and nurture talent.

*Note on Methodology

The composite figures cited are based on cross validation in a hold out sample and correction for attenuation based on unreliability in the criterion. No other corrections were applied (e.g. restriction of range, predictor unreliability).

Validity figures based on multiple regression equations cannot be used as an unbiased estimate of the validity of a questionnaire when they are not cross validated. As a matter of best practice we have cross validated the regression equations on an independent hold out sample to ensure that our validities are realistic.

Studies which do not use hold out samples and base their composite validities on the initial sample without cross validating results can give misleadingly high figures. Sometimes it is argued that corrections for shrinkage can be used to give a lower bound estimate of the validity, however, it is important to be wary as these adjustments incorrectly applied can seriously underestimate the downward correction which is appropriate. For example, if we were to use such an approach our validities would give an average validity around 0.60!

Professional Styles Single Dimension and Composite Validities Summary Table

Technical Information:

Group Differences

Reliability