Criteria for Assessment of Oral Communication  

Historical Background

Assessment received increasing attention beginning in the 1970s and continuing into the 1990s and the new millenium. Initially appearing in the standards developed by state departments of education, by 1980 over half of the states had adopted statewide student‑testing programs. In Educational Standards in the 50 States: 1990, the Educational Testing Service reported that by 1990 statewide student‑testing programs existed in 47 states. By 1996, standards programs were established in every state in the country and assessment of student learning was required to qualify for national, state, and local funding.

As standards and assessment programs expanded, the number of different subjects and skills being tested increased, with additional attention devoted to assessment processes and testing methods. Organizations, such as the National Assessment of Educational Progress, intensified and expanded the scope of their assessment procedures and publicized the results of their findings nationally and annually.

By the end of 1989, the public recognized the significance of the national educational assessment movement. In the Phi Delta Kappan‑Gallup poll reported in the September 1989 issue of Phi Delta Kappan, 77 percent of survey respondents favored “requiring the public schools in this community to use standard­ized national testing programs to measure academic achievement of students” and 70 percent favored “requiring the public schools in this community to conform to national achievement standards and goals.”

Also toward the end of the 1980s, col­leges and universities began to realize that formal assessment issues were to affect them. In its 1989‑1990 Criteria for Accreditation, the Southern Association of Colleges and Schools—which provides institutional certification for over 800 colleges and universities in the South—held that “complete requirements for an associate or baccalaureate degree must include competence in reading, writing, oral communications and funda­mental mathematical skills.” They also held that the general education core of colleges and universities “must provide components designed to ensure competence in reading, writing, oral communica­tion and fundamental mathematical skills.” Similarly, the other regional accreditation associations began mandating campus-wide assessment programs in all academic units concerned with student learning, as an integral component of the accreditation process in higher education.

In 1990, a series of reports confirmed that systematic and comprehensive assessment should be a national educational objective. The National Governors’ Association stated that, “National education goals will be meaningless unless progress toward meeting them is measured accurately and adequately, and reported to the American people.”  The National Education Goals: A Report to the Nation’s Governors reinforced that line of reasoning and the Governors’ Association elaborated its commitment to assessment in Educating America: State Strategies for Achieving the National Education Goals: Report of the Task Force on Education. Additionally in 1990, in their report From Gatekeeper to Gateway: Transforming Testing in America, the National Commission on Testing and Public Policy recommended eight standards for assessment, arguing for more humane and multicul­tural assessment systems. 

By the end of the 1990’s, assessment of student learning became a permanent fixture at all grade levels, K through 16-18. It is now institutionalized in the accreditation and accountability processes that take place with regularity at academic institutions through the country.  Additionally, oral communication is now more extensively included in the curriculum, K-18, and therefore it has a presence in assessment programs as well.

NCA’s Role in Assessment 

The evaluation and assessment of public ad­dress has been of central concern to the discipline of communication since its inception and to the National Communication Association when it was organized in 1914. In 1970, NCA formalized its commitment to assessment when it created the Committee on Assessment and Testing (now known by the acronym CAT) for “NCA members interested in gathering, analyzing and disseminating informa­tion about the testing of speech communication skills.”

 As the assessment movement evolved, NCA has convened conferences and produced publications exploring methods for assessing oral communication.  These publications began to appear in the 1970s, proliferated during the 1990s, and now include in print the following: Large Scale Assessment in Oral Communication: Assessing College Student Competency in Speech Communication: 1994 NCA Summer Conference Proceedings; K-12 and Higher Education; Program Assessment in Speech Communication; The Conversational Skills Rating Scale: An Instructional Assessment of Interpersonal Competence; The Competent Speaker Speech Evaluation Form; and Assessing Motivation to Communicate.

Standards and Conferences

In 1979, in Standards for Effective Oral Commu­nication Programs, NCA adopted its first set of “standards for assessment.” Those standards called for “school‑wide assessment of speaking and listening needs of students,” “qualified personnel” to “utilize appropriate evaluation tools,” and a “variety of data” and “instruments” which “encourage” “students’ desire to communicate.”

In 1986, in Criteria for Evaluating Instruments and Procedures for Assessing Speaking and Listening, NCA adopted an additional 15 “content” and “technical considerations” dealing “primarily with the substance of speaking and listening instru­ments” and “matters such as reliability, validity and information on administration.” These criteria included among other concerns, the importance of focusing on “demonstrated” speaking skills rather than “reading and writing ability.”

In 1987, at an NCA Wingspread Conference, “conference participants recommended that any chosen instrument conform to NCA guidelines for assessment instruments,” and they specifically suggested that “strategies for assessing speaking skills” should be directly linked to the content of oral communication performances and student speaking competencies. Additionally, the Wingspread Conference participants considered strategies for assessing listening and for training assessors (see: Communication Is Life: Essential College Sophomore Speaking and Listening Competen­cies, Annandale, Va.: National Communication Association, 1990].

In 1988, an NCA Flagstaff Conference gener­ated a series of resolutions calling for a “national conference” and “task force on assessment” because “previous experience in developing standardized assessment has met with problems of validity, reliability, feasibility, ethics, and cultural bias.”

In July 1990, NCA and its Commit­tee on Assessment and Testing convened a national working conference on oral communication and its assessment. The Conference generated resolutions, which reaffirmed existing NCA assessment policies and provided criteria for resolving new issues in assessment.  A revision of those assessment criteria is contained in this publication.  The 1990 assessment conference also resulted in NCA’s publication of assessment instruments, for public speaking and for interpersonal communication.

In July 1994, NCA convened another assessment conference, focusing on oral competence assessment in higher education. The proceedings of that conference examine the philosophy, methods, and progress of assessment around the country and at a variety of academic institutions.

 Presently, a set of recommendations for engaging in oral communication assessment is available on the NCA home page at www.natcom.org. The recommendations include suggestions for developing successful assessment programs by institutions and academic departments, as well as recommended methods and techniques.

General Criteria for Assessing Oral Communication

1.      Assessment of oral communication should view competence in oral communication as a gestalt of several interacting dimensions. At a minimum, all assessments of oral communication should include an assessment of knowledge (understanding communication process, comprehension of the elements, rules, and dynamics of a communication event, awareness of what is appropriate in a communication situation), an assessment of skills (the possession of a repertoire of skills and the actual performance of skills), and an evaluation of the individual’s attitude toward communication (e.g., value placed on oral communication, appre­hension, reticence, willingness to communicate, readiness to communicate).

2.      Because oral communication is an interactive and social process, assessment should consider the judgment of a trained assessor as well as the impressions of others involved in the communica­tion act (audience, interviewer, other group members, conversant), and may include the self-report of the individual being assessed.

3.      Assessment of oral communication should clearly distinguish speaking and listening from reading and writing. While some parts of the assessment process may include reading and writing, a major portion of the assessment of oral communication should require speaking and listening. Directions from the assessor and re­sponses by the individual being assessed should be in the oral/aural mode.

4.      Assessment of oral communication should be sensitive to the effects of relevant physical and psychological disabilities on the assessment of competence. (e.g., with appropriate aids in signal reception, a hearing impaired person can be a competent empathic listener.)

5.      Assessment of oral communication should be based in part on atomistic/analytic data collected and on a holistic impression.

Criteria for the Content of Assessment

1.      Assessment of oral communication for all students should include assessment of both verbal and nonverbal aspects of communication and should consider competence in more than one communication setting. As a minimum assessment should occur in the one‑to‑many setting (e.g. public speaking, practical small group discussion) and in the one‑to‑one setting (e.g., interviews, interper­sonal relations).

2.      Assessment of speech majors and other oral communication specialists could include in addition assessment in specialized fields appropriate to the course of study followed or the specialty of the person being assessed.

Criteria for Assessment Instruments

1.      The method of assessment should be consistent with the dimension of oral communication being assessed. While knowledge and attitude may be assessed in part through paper and pencil instru­ments, speaking and listening skills must be assessed through actual performance in social settings (speaking before an audience, undergoing an interview, participating in a group discussion, etc.) appropriate to the skill(s) being assessed.

2.      Instruments for assessing oral communication should describe degrees of competence. Either/or descriptions such as “competent” or “incompetent” should be avoided, as should attempts to diagnose reasons why individuals demonstrate or fail to demonstrate particular degrees of competence.

3.      Instruments for assessing each dimension of oral communication competence should clearly identify the range of responses, which constitute various degrees of competence. Examples of such responses should be provided as anchors.

4.      Assessment instruments should have an acceptable level of reliability, e.g. test/retest reliability, split‑half reliability, alternative forms reliability, inter‑rater reliability, and internal consistency.

5.      Assessment instruments should have appropri­ate validity: content validity, predictive validity, and concurrent validity.

6.      Assessment instruments must meet acceptable standards for freedom from cultural, sexual, ethical, racial, age, and developmental bias.

7.      Assessment instruments should be suitable for the developmental level of the individual being assessed.

8.      Assessment instruments should be standardized and detailed enough so that individual responses will not be affected by an administrator’s skill in administering the procedures.

Criteria for Assessment Procedures and Administration

1.      Assessment procedures should protect the rights of those being assessed in the following ways: administration of assessment instruments and assessment and the uses of assessment results should be kept confidential and be released only to an appropriate institutional office, to the individual assessed, or if a minor, to his or her parent or legal guardian.

2.      Use of competence assessment as a basis for procedural decisions concerning an individual should, when feasible, be based on multiple sources of information, including especially a) direct evidence of actual communication performance in school and/or other contexts, b) results of formal competence assessment, and c) measures of individual attitudes toward communication (e.g., value placed on oral communication, apprehension, reticence, willingness to communicate, and readiness to communicate).

3.      Individuals administering assessment proce­dures for oral communication should have received sufficient training by speech communication professionals to make their assessment reliable. Scoring of some standardized assessment instru­ments in speaking and listening may require specialized training in oral communication on the part of the assessor.

Criteria for Assessment Frequency

Periodic assessment of oral communication competency should occur annually during the educational careers of students. An effective systematic assessment program minimally should occur at educational levels K, 4, 8, 12, 14, and 16.

Criteria for the Use of Assessment Results

The results of student oral communication competency assessment should be used in an ethical, non‑discriminatory manner for such purposes as:

1.      Diagnosing student strengths and weaknesses;

2.      Planning instructional strategies to address student strengths and weaknesses;

3.      Certification of student readiness for entry into and exit from programs and institutions;

4.      Evaluating and describing overall student achievement;

5.      Screening students for programs designed for special populations;

6.      Counseling students for academic and career options; and

7.      Evaluating the effectiveness of instructional programs.

Foundations for the criteria contained in this document were originally developed and adopted as resolutions at the NCA Conference on Assessment in Denver, Colorado, in July, 1990. Several of the criteria were authored by a subcommittee of NCA’s Committee on Assessment and Testing, composed of Jim Crocker-Lakness, Sandra Manheimer, and Tom Scott.  James W. Chesebro, then NCA Director of Educational Services authored the introductory sections to this publication in 1993, when the document was first published by NCA. The present iteration was revised in 1998by Sherry Morreale, NCA Associate Director and Philip Backlund, chair of NCA’s Assessment Commission.

For further information on the assessment of communication, contact the National Communication Association (formerly Speech Communication Association), 5105 Backlick Road, Annandale, VA 22003; 703-750-0533.  Or visit the assessment section of NCA’s homepage on the Internet: http://www.natcom.org.  If you have a particular question on assessing communication, direct your inquiry to experts in the field and other interested colleagues who are subscribers to an assessment listserv set up NCA.  Send an e-mail message to assessment@natcom.org.