Companies recognize the importance of the employer/employee relationship
and are anxious to gather information that will help improve it.
Sufficiently anxious that they become susceptible to the those willing
to sell nonsense in the guise of information.
There are at present two dominant forms of this nonsense:
1. Linking employee survey results with measures of corporate profitability.
2. Comparing survey results with those of other companies to identify
best employers.
1. Linking employee survey results with
measures of corporate performance such as profitability, shareholder
value etc.
One of today’s hot areas in employee research is the linking
of employee research results with measures of corporate performance.
In many ways, doing so is the ‘holy grail’ of human
resource practitioners and for good reasons. There is no doubt that
being able to demonstrate an empirical relationship between employee
attitude and satisfaction would be an exciting development allowing
human resource departments to show (finally) precisely how they
add value. This is undoubtedly why surveys making such promises
sell so well.
Unfortunately, what is being sold is a load of statistical nonsense
and organizations are well advised to stay well clear of any organization
promising to link employee survey results to corporate performance
measures. Once again, this practice amounts to little more than
‘lying with statistics’ in pursuit consulting profits.
Why is this so? The search for spurious correlation
To understand why this is lying with statistics, it is important
to understand what is being done. Some measure of corporate performance
(profit, shareholder value, stock price, etc.) is used as a dependant
variable – it is the effect we are seeking. Against this,
a massive set of explanatory variables are used to predict the outcome
of the dependant variable. These explanatory variables are the various
questions that make up the survey the particular consulting firm
uses.
Regardless of the statistical procedure used, what is being looked
for is correlation between the predictor or explanatory variables
and the dependant corporate performance variable. Statistically
significant correlations between the predictor and dependant variable
are identified and a model of performance is constructed.
Any graduate of a first year statistics (or science) course will
see the problem immediately. When looking for a correlation between
any measure of corporate performance and a hundred or more possible
predictors you will find all kinds of statistically significant
relationships -- merely by chance. There are also undoubtedly statistically
significant correlations between some of these variables and the
phases of the moon, the movement of the stock market or the number
of car accidents in Venezuela. The importance of these relationships,
however, beyond filling the pockets of consulting companies, is
questionable to say the least.
Consulting Voodoo
All this makes for consulting voodoo – conduct an employee
survey and the consultant consults his or her magic black box (a
computer filled with statistical routines the consultant in question
obviously fails to understand) and comes back with the magic potion,
in this case a model of the relationships between employee survey
results and performance.
Fortunately there is a simple scientific test companies can use
to see if they are being sold a ‘bag of goods’. It goes
back a few hundred years but has proved its worth time and time
again. The test is prediction.
If a model of the relationship between corporate performance and
employee attitudes is valid, it should be able to predict next year’s
corporate profits from this year’s employee attitudes. It
would be interesting to see how well these models predicted the
dot-com crash and the decline in corporate profits in the high tech
sector on the basis of employee attitudes.
The Simple Truth
The simple truth is that such relationships between corporate profits
and employee attitudes cannot be established empirically –
at least not within the constraints of an employee survey. Organizations
either have to accept or reject the notion that having positive
relationships with employees is good thing. If accepted, conducting
employee surveys to establish quantitatively, the current state
of affairs and how this state of affairs may have changed over time
simply makes sense, especially when done within a program of improvement,
where problem areas that surface are addressed.
Simple is better.
2. Comparing Employee Survey Results With Results From Other
Organizations.
Many organizations like to compare their survey results with the
results obtained from other companies or organizations to see where
they stand relative to everyone else or some selected group or sub-group
of comparative organizations. For reasons not often appreciated
by those without some statistical training, while such comparisons
can be made, they are essentially meaningless.
Comparing your organization’s results with the results of
a large number of other organizations is conceptually equivalent
of comparing your organizations results with a set of randomly drawn
numbers. Indeed, making the comparison to random numbers is not
only as valid, but considerably less expensive than comparing to
a database of previous results.
Comparing your organizations ‘scores’ with the results
obtained from other surveys is technically possible, but would require
a number of conditions to be met:
The data base of comparative organizations would have to be generated
within a relatively narrow time frame – that is the data gathering
of your employee information and the data gathering with similar
companies would have to be conducted simultaneously.
The comparative companies would have to be in similar industries
and face generally similar circumstances including location. Confounding
factors such as proportion of union employees, degree of centralization/decentralization,
average wage rates and similar variables would have to be controlled.
The companies selected for comparison would have to be selected
for specific reasons. Together, they would have to represent a specific
profile or profiles whereby comparisons would yield specific increases
in information or knowledge.
The data gathering instruments would have to similar with generally
the same question order and emphasis. The scales used, the general
administration of the survey and method of distribution would all
have to be highly similar.
Lastly, the effects of non-response would have to be eliminated
to ensure results were comparable and differentiating sources of
bias eliminated. This means there could be no self-selection –
all employees selected to participate in the survey (either a sampling
or a census) would be required to complete the survey. Past studies
have indicated that the validity of responses is destroyed by such
enforcement.
There are additional considerations but as it is, to our knowledge,
no such set of conditions exists in the field of employee research.
Making comparisons, and drawing conclusions from them, is best
charitably described as ‘lying with statistics’
The information gathered from your employees is best interpreted
by making internal comparisons (examining the differences among
operating units, locations, job functions and the like) and examining
differences over time as a means of identifying trends and emerging
problem areas. Organizations are advised to drop the meaningless
comparisons against some vague group of composing a ‘benchmark’
and get back to carefully examining their own data – identifying
and addressing opportunities for improvement.
|
