Technology Literacy Assessment

Recently McREL assisted Wyoming’s Department of Education in determining how their districts and other states
assess the technology literacy of 8th graders as required by No Child Left Behind. The requirement to assess technology literacy does not specify how or by what criteria. Therefore, states are all defining it in different ways and using various assessment instruments. Some states have put a lot of effort into the assessment while others have given it little attention. Below is a summary of what we found. We know it is not a complete picture. What have we left off? Does your state have a different method of technology literacy assessment? Are there errors below? We welcome your comments.

State/Organization: Colorado

Link to Criteria: Levels were not found. Grade band profiles are found here.

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: TLAP

Strengths: Recently pilot tested and revised. Free to Colorado districts.

Weaknesses: Grant funded for Colorado only.

State/Organization: North Dakota

Link to Criteria: Unknown – you have to purchase the assessments to get the rubrics.

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: Atomic Learning – Tech Skills Student Assessment

Strengths: Focuses on how to use technology and how to apply it and allows easy identification of areas of greatest
instructional need. Includes customizable curriculum projects to target technology gaps.

Weaknesses: Unknown

State/Organization: Montana-based but used nationwide

Link to Criteria: Unknown

Standards Basis: 2002 ISTE NETS-S

Assessment Instrument: TAGLIT

Strengths: Includes assessments for administrators and teacher as well as students.

Weaknesses: Needs updating to the newest NETS-S.

State/Organization: South Dakota, Arizona, South Carolina, Georgia, Wisconsin, and other states

Link to Criteria: Technology skill set for 5th grade and 8th grade

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: Learning.com

Strengths: NETS-Aligned Resource. Blend of interactive, performance-based questions and multiple choice, knowledge-based questions to measure and report technology literacy and skills for elementary and middle
school students.

Weaknesses: Does not seem to support portfolio assessments.

State/Organization: New York (south central)

Link to Criteria: Unknown

Standards Basis: Unknown

Assessment Instrument: Tech Literacy

Strengths: This is a good example of what a Regional Education Service Center can accomplish.

Weaknesses: Small in scope with little background information.

State/Organization: Florida

Link to Criteria: Criteria found here.

Standards Basis: 2007 ISTE NETS-S modified for Florida.

Assessment Instrument: Student Tools for Technology Literacy

Strengths: After extensive feedback, indicators were modified. In April 2008, the complete tool was field
tested with over 1300 8th graders in several representative districts resulting in minor revisions prior to its availability statewide.

Weaknesses: Unknown – Florida only

State/Organization: Washington

Link to Criteria: Tiers of 8th Grade Technology Literacy Indicators

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: Washington Assessments for Education Technology

Strengths: Project based and integrated across content areas.

Weaknesses: Only social studies and the arts at this time.

State/Organization: North Carolina

Link to Criteria: Little found. Example report with some criteria found here.

Standards Basis: 2004 Computer/ Technology Skills North Carolina Standard Course of Study

Assessment Instrument: Test of Computer Skills

Strengths: Strong development process.

Weaknesses: Does not seem to incorporate project learning.

State/Organization: New Jersey

Link to Criteria: NJTAP-IN Rubric

Standards Basis: New Jersey Educational Technology Standards 8.1.

Assessment Instrument: No specific instrument has been identified, but the state has issued an RFI for one.

Strengths: Integrated with state planning and support structures found here.

Weaknesses: No specific instrument has been identified.

Other assessment sources used by schools:

State/Organization: InfoSource Learning

Link to Criteria: Unknown

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: Simple Assessments

Strengths: Used in over 1,200 districts nationwide. Free and easy to use.

Weaknesses: Seems oversimplified.

State/Organization: Intel

Link to Criteria: Each project has a rubric on the specific project guide page. An example can be found here.

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: Technology Literacy

Strengths: NETS-Aligned Resource.

Weaknesses: Unknown

State/Organization: State Educational Technology

Directors Association (SETDA)

Link to Criteria: Framework for Assessment of Technology Literacy

Standards Basis: 2007 ISTE NETS-S

Assessment Instrument: No specific recommendation. Analysis can be found here.

Strengths: A larger group of stakeholders from multiple states designed the toolkit.

Weaknesses: Based on older NETS-S standards.

State/Organization: National Assessment of Educational Progress (NAEP)

Link to Criteria: 2014 NAEP Technology and Engineering Literacy Framework – Pre-Publication Edition

Standards Basis: NAEP Standards developed by four cooperating organizations.

Assessment Instrument: NAEP Technology and Engineering Literacy Assessment (TELA)

Strengths: Large effort by national experts with collaboration from the International Society for Technology
in Education (ISTE)
International Technology and Engineering Educators Association (ITEEA), Partnership for 21st Century Skills (P21), and the State Educational Technology Directors Association (SETDA).

Weaknesses: Incorporates Engineering from STEM. (Some would consider this a strength)

RESEARCH NOTES:

In a report entitled Tech Tally: Approaches to Assessing Technological Literacy (Gamire & Pearson, 2006) it was determined that “doing” is central to students gaining technological literacy, traditional assessments will not work; technological literacy must be assessed in ways that are more authentic. According to the State Educational Technology Directors Association (SETDA), a knowledge-based assessment is insufficient on its own. If such an assessment is used, it should be used as a base in combination with a performance-based, portfolio-based or project-based assessment. The report developed six principles for guiding the development of assessments of technological literacy:

  1. Assessments should be
    designed with a clear purpose in mind.
  2. Assessment developers
    should take into account research findings related to how children and adults
    learn, including how they learn about technology.
  3. The content of an
    assessment should be based on rigorously developed learning standards.
  4. Assessments should
    provide information about all three dimensions of technological literacy—
    knowledge, capabilities, and critical thinking and decision making.
  5. Assessments should not
    reflect gender, culture, or socioeconomic bias.
  6. Assessments should be
    accessible to people with mental or physical disabilities.

Leave a Reply