This paper describes an investigation into the reliability of an
automatic assessment method of the learner-build concept map by comparing it
with two well-known manual methods. We have previously proposed the
Kit-Build (KB) concept map framework where a learner builds a concept map
by using only a provided set of components, known as the set “kit”. In this
framework, instant and automatic assessment of a learner-build concept map has
been realized. We call this assessment method the “Kit-Build method” (KB
method). The framework and assessment method have already been practically
used in classrooms in various schools. As an investigation of the reliability of
this method, we have conducted an experiment to compare the assessment
results of the method with the assessment results of two other manual assess-
ment methods. In this experiment, 22 university students attended as subjects
and four as raters. It was found that the scores of the KB method had a very
strong correlation with the scores of the other manual methods. The results
suggest that automatic assessment of the Kit-Build concept map can attain
almost the same level of reliability as well-known manual assessment methods.