Given a set of variables, a Factor Analysis
tries to form coherent subsets that are as independent from
other subsets as possible, but where each variable in the
same subset is as highly correlated as possible. Thus, variables
that are correlated with one another and which are largely
independent of other subsets of variables are combined into
factors. Factors can then be thought of as the more fundamental
features that are measured by the variables.
For example, if we administered a large battery
of tests to high school students which included: addition,
multiplication, analogies, reading comprehension, symbol matching,
symbol rotation and so on, we might be able to extract 3 factors
which might be labeled as: Mathematical, Verbal and Spatial
processing abilities. Thus, tests of addition and multiplication
are highly correlated, but are largely uncorrelated to tests
of analogies or reading comprehension.
There are 2 features of Factor Analysis to
be aware of. First of all, the extraction of factors depends
on the measured variables fed to the analysis. Thus, if the
measured variables do not represent the entire spectrum of
variation, then the extracted factors will not cover the entire
range of possible motivations. In other words, you cannot
carve a pie so that you end up with more of it than you began
with.
Secondly, a Factor Analysis tells you
how well particular variables fit into specific factors, but
the factors are not then labeled for you. In other words,
one has to come up with the appropriate labels for each factor
after understanding which variables load heavily into each
factor. Oftentimes, it is hard to come up with a label that
encapsulates all the underlying variables of a factor.
