Recently McREL assisted Wyoming’s Department of Education in determining how their districts and other states
assess the technology literacy of 8th graders as required by No Child Left Behind. The requirement to assess technology literacy does not specify how or by what criteria. Therefore, states are all defining it in different ways and using various assessment instruments. Some states have put a lot of effort into the assessment while others have given it little attention. Below is a summary of what we found. We know it is not a complete picture. What have we left off? Does your state have a different method of technology literacy assessment? Are there errors below? We welcome your comments.
State/Organization: Colorado
Link to Criteria: Levels were not found. Grade band profiles are found here.
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: TLAP
Strengths: Recently pilot tested and revised. Free to Colorado districts.
Weaknesses: Grant funded for Colorado only.
State/Organization: North Dakota
Link to Criteria: Unknown – you have to purchase the assessments to get the rubrics.
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: Atomic Learning – Tech Skills Student Assessment
Strengths: Focuses on how to use technology and how to apply it and allows easy identification of areas of greatest
instructional need. Includes customizable curriculum projects to target technology gaps.
Weaknesses: Unknown
State/Organization: Montana-based but used nationwide
Link to Criteria: Unknown
Standards Basis: 2002 ISTE NETS-S
Assessment Instrument: TAGLIT
Strengths: Includes assessments for administrators and teacher as well as students.
Weaknesses: Needs updating to the newest NETS-S.
State/Organization: South Dakota, Arizona, South Carolina, Georgia, Wisconsin, and other states
Link to Criteria: Technology skill set for 5th grade and 8th grade
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: Learning.com
Strengths: NETS-Aligned Resource. Blend of interactive, performance-based questions and multiple choice, knowledge-based questions to measure and report technology literacy and skills for elementary and middle
school students.
Weaknesses: Does not seem to support portfolio assessments.
State/Organization: New York (south central)
Link to Criteria: Unknown
Standards Basis: Unknown
Assessment Instrument: Tech Literacy
Strengths: This is a good example of what a Regional Education Service Center can accomplish.
Weaknesses: Small in scope with little background information.
State/Organization: Florida
Link to Criteria: Criteria found here.
Standards Basis: 2007 ISTE NETS-S modified for Florida.
Assessment Instrument: Student Tools for Technology Literacy
Strengths: After extensive feedback, indicators were modified. In April 2008, the complete tool was field
tested with over 1300 8th graders in several representative districts resulting in minor revisions prior to its availability statewide.
Weaknesses: Unknown – Florida only
State/Organization: Washington
Link to Criteria: Tiers of 8th Grade Technology Literacy Indicators
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: Washington Assessments for Education Technology
Strengths: Project based and integrated across content areas.
Weaknesses: Only social studies and the arts at this time.
State/Organization: North Carolina
Link to Criteria: Little found. Example report with some criteria found here.
Standards Basis: 2004 Computer/ Technology Skills North Carolina Standard Course of Study
Assessment Instrument: Test of Computer Skills
Strengths: Strong development process.
Weaknesses: Does not seem to incorporate project learning.
State/Organization: New Jersey
Link to Criteria: NJTAP-IN Rubric
Standards Basis: New Jersey Educational Technology Standards 8.1.
Assessment Instrument: No specific instrument has been identified, but the state has issued an RFI for one.
Strengths: Integrated with state planning and support structures found here.
Weaknesses: No specific instrument has been identified.
Other assessment sources used by schools:
State/Organization: InfoSource Learning
Link to Criteria: Unknown
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: Simple Assessments
Strengths: Used in over 1,200 districts nationwide. Free and easy to use.
Weaknesses: Seems oversimplified.
State/Organization: Intel
Link to Criteria: Each project has a rubric on the specific project guide page. An example can be found here.
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: Technology Literacy
Strengths: NETS-Aligned Resource.
Weaknesses: Unknown
State/Organization: State Educational Technology
Directors Association (SETDA)
Link to Criteria: Framework for Assessment of Technology Literacy
Standards Basis: 2007 ISTE NETS-S
Assessment Instrument: No specific recommendation. Analysis can be found here.
Strengths: A larger group of stakeholders from multiple states designed the toolkit.
Weaknesses: Based on older NETS-S standards.
State/Organization: National Assessment of Educational Progress (NAEP)
Link to Criteria: 2014 NAEP Technology and Engineering Literacy Framework – Pre-Publication Edition
Standards Basis: NAEP Standards developed by four cooperating organizations.
Assessment Instrument: NAEP Technology and Engineering Literacy Assessment (TELA)
Strengths: Large effort by national experts with collaboration from the International Society for Technology
in Education (ISTE) International Technology and Engineering Educators Association (ITEEA), Partnership for 21st Century Skills (P21), and the State Educational Technology Directors Association (SETDA).
Weaknesses: Incorporates Engineering from STEM. (Some would consider this a strength)
RESEARCH NOTES:
In a report entitled Tech Tally: Approaches to Assessing Technological Literacy (Gamire & Pearson, 2006) it was determined that “doing” is central to students gaining technological literacy, traditional assessments will not work; technological literacy must be assessed in ways that are more authentic. According to the State Educational Technology Directors Association (SETDA), a knowledge-based assessment is insufficient on its own. If such an assessment is used, it should be used as a base in combination with a performance-based, portfolio-based or project-based assessment. The report developed six principles for guiding the development of assessments of technological literacy:
- Assessments should be
designed with a clear purpose in mind. - Assessment developers
should take into account research findings related to how children and adults
learn, including how they learn about technology. - The content of an
assessment should be based on rigorously developed learning standards. - Assessments should
provide information about all three dimensions of technological literacy—
knowledge, capabilities, and critical thinking and decision making. - Assessments should not
reflect gender, culture, or socioeconomic bias. - Assessments should be
accessible to people with mental or physical disabilities.
