Education Development Center, Inc.
Center for Children and Technology

Domain Projects as Assessment Vehicles in a Computer-Rich Environment

CTE Technical Report Issue No. 5
August 1990

Prepared by:
Joseph Walters and Howard Gardner
Project Zero, Harvard Graduate School of Education


This paper reviews recent research that has been conducted at Project Zero at the Harvard Graduate School of Education on the topic of assessment. In the paper we argue that an assessment built around an interesting and challenging project that explores a particular domain (such as music, writing, drawing, mathematics, science, or programming) offers an important alternative to traditional standardized tests. As students work through such projects, they reveal most vividly the skills and aptitudes that we want to assess. We refer to these assessment instruments as domain projects.

The theory of multiple intelligences provides much of the conceptual underpinnings of our view of assessment, and the paper begins with a brief synopsis of this theory. Next, we review examples of domain projects under development in two different research initiatives at Project Zero--Arts PROPEL and Catalyst. Drawing from these examples, we list the salient characteristics of this project-based assessment and outline the design of one such project that uses computer technology.

The paper concludes by indicating directions for future research and development.

Assessment at Project Zero

For the past twenty years, Project Zero has taken as its research agenda the systematic study of children and their symbol-using skills by exploring such diverse areas as drawing, metaphoric language, musical ability, storytelling, written language, mathematics, and computer programming. The diversity of this research is responsible in part for the development of the view of human cognition that we call the theory of "multiple intelligences" (Gardner, 1983).

With respect to assessment, we draw two conclusions from this particular view of human cognition. First, an evaluation of achievement or aptitude must be specified within a domain of human activity. Second, the assessment must be drawn based on a true performance in that domain. In this section, we will build the argument by reviewing the central ideas of multiple intelligences and then outlining the implications of the theory for assessment.

Multiple Intelligences

The theory of multiple intelligences claims that human beings have evolved at least seven different forms of knowing or processing information. These different forms, called "intelligences," include the skills for manipulating language, logic and mathematics, musical ability, spatial information, bodily-kinesthetic information, knowledge of other persons (interpersonal), and knowledge of oneself (intrapersonal). All normal human beings possess some capacity in each of these intellectual spheres, but the interaction of genetic and environmental factors produce marked differences in the profiles of the various intelligences in individuals.

This theory was designed to serve two goals: (1) to synthesize a large set of findings about human cognition, including neurobiological evidence, cross-cultural analysis, and developmental milestones; and (2) to provide an alternative to the widespread belief in a single faculty- -intelligence--that can be adequately assessed by paper-and-pencil "intelligence tests."

In the theory of multiple intelligences, the specific skills of human cognitionthe intelligencesare mobilized for solving problems within particular domains of activity. Each intelligence is an evolved biopsychological potential that is manifested as a particular cognitive skill. In contrast, the domain of that potential is defined by the culture as an arena in which the various cognitive skills are mobilized. In the theory of multiple intelligences, analysis of problem solving requires consideration of both the cognitive functioning of the intelligences (and their combination) and the context of a domain specified within a particular culture.

The theory of multiple intelligences stipulates that the various intelligences are independent. For instance, when an individual displays a high degree of competence with one intelligence, this ability does not imply similar competencies in other areas. Similarly, disability in one intelligence does not imply disabilities in the others.

Although the intelligences are independently structured in this way, they do not function independently. Any reasonably complex adult task requires the simultaneous functioning of several of the intelligences. For example, the task of writing a research paper makes primary use of the linguistic intelligence, but it also taps the logical-mathematical intelligence at the same time. Playing the violin for an audiencefirst and foremost a musical task also makes demands on bodily-kinesthetic and interpersonal faculties. How an individual combines these separate intelligences is part of that individual's personal endowment, just as are the intelligences themselves.

These two outcomes of the theory of multiple intelligences that the intelligences are independent and that they operate in concert has important implications for the problem of assessment. First, an assessment must pose problems in which individuals work with the actual materials of the domain being examined. Second, a complete assessment must pose a number of problems that yield to a variety of solutions in order to reveal an accurate picture of the talents and skills of a given individual.

To illustrate the first point, consider the task of assessing an individual's musical competence. If the assessment consists of a number of questions in a multiple-choice format, the assessment is less a measure of the musical intelligence and more a measure of linguistic facility or test-taking skill. As an alternative, the assessment might ask the student to compose an ending to a simple melody. To tackle this problem, the student must manipulate musical notation, select musical sounds, and use musical terminology. In this task, the student makes demands directly on the musical intelligence the problem is not filtered through a linguistic or logical-mathematical assessment instrument, as it is in the multiple-choice example. This is what we mean when we say that to evaluate an underlying intelligence, an assessment must pose problems that require the individual to manipulate the actual materials of the domain of that intelligence. We must examine true performance and not just a verbalization of a problem solution.

Second, the fact that cognition is composed of several independent intelligences operating in concert implies that an assessment must pose a variety of problems that yield to different types of solutions if it is to establish a complete picture of the profile of intelligences for an individual. For instance, by asking a student to solve our music composition task, we do not learn much about that student's ability to write short stories. Similarly, we cannot infer that individuals who do not write well are equally disinclined to perform well on tasks that require interpersonal skills, spatial abilities, or musical prowess. Any simple task, even when it requires genuine performance with the materials of a domain, does not reveal the complete profile of an individual's talents. Therefore, a complete assessment of an individual must make demands on all the intelligences, not just a select few.

Finally, the theory of multiple intelligences underscores the importance of the "personal" skills in daily life. Working cooperatively in a group is recognized as an important feature of many adult situations, from manufacturing to family dynamics. Also, understanding one's self, the intrapersonal skill, is also highlighted in the theory. For instance, a strength in the musical realm may be underutilized by an individual who does not fully understand that strength, whereas an individual with a strong intrapersonal intelligence may be better equipped to combine modest strengths or to compensate for weaknesses efficiently. A complete assessment should take both of the personal skills into account, and the assessments we describe below consistently feature both small-group collaboration and reflection on learning.

Assessment through Projects

To summarize, assessments of ability and learning must engage students in performances in which they handle the actual materials of a given domain and mobilize a number of the different intelligences. We believe that problems that meet both criteria can often be formatted as projects.

At Project Zero, we have been devising a number of exemplar projects in a variety of domains. These projects are at once open-ended, structured, and oriented towards products. In order to pose rich, complex, and engaging problems, the projects are open-ended they offer opportunities to find alternative solutions and unexpected outcomes using different strategies and different combinations of intelligences. At the same time, the projects are structuredthe results can be analyzed in terms of what students are learning. Finally, as students work through projects, they are called on to create original products songs, essays, drawings and this gives them a stake in the outcome of the project.

The products that students create in working on a project are evaluated by the teacher and the student together. We follow the work of Collins and Frederiksen (1989) in designing these evaluations. First, the evaluations consider only the performances that reveal directly the skills and competencies that we are interested in; again, we ask students to exhibit skills, and not simply to describe those skills. Second, the scope of each project is designed such that every skill that we are interested in is exhibited; we do not design evaluations that test only a sample of the desired skills. Finally, all of these criteria are articulated at the outset of the project, making them transparent to the students. In the final section of the paper, we will return to these principles of problem design and evaluation.

Currently, two research initiatives at Project Zero are developing specific examples of assessment projects. Arts PROPEL is designing instruments in the arts; and Project Catalyst has created a series of computer-based projects. We will outline these projects in the next two sections. Following this, we will describe a project that we are developing for the Center for Technology in Education that embodies these several design principles.

Domain Projects in Arts

Arts PROPEL is a five-year research effort sponsored by the Rockefeller Foundation. Its primary goal is to create and test new techniques for fostering and assessing artistic development in secondary school students. The project brings together researchers from Project Zero and the Educational Testing Service and teachers of music, visual arts, and imaginative writing in the Pittsburgh public schools.

Arts PROPEL has designed two assessment techniques for arts classrooms: portfolios and domain projects. A portfolio is a collection of materials that the student assembles during the process of creating a finished piece and includes sketches, early drafts, and notes as well as the final piece itself. These selections may be supplemented with a written journal in which the student reflects on the process of creating the piece.

It is the second technique, the domain project, that is directly relevant to the present discussion. Each domain project in Arts PROPEL is composed of a clearly defined set of classroom tasks that focus students on a central issue in the art form. We will illustrate Arts PROPEL domain projects with examples drawn from the domains of music and imaginative writing, but projects are also developed in the visual arts. These descriptions begin with a summary of the project along with a description of the products that the students create. This is followed by a discussion of the various assessment techniques which includes reflection exercises and teacher judgments.

Sample Projects
Musical Performance: Ensemble Critique

In musical performance, rehearsal is the main avenue for developing instrumental skill. In a typical rehearsal session, the teacher leads a number of student instrumentalists, guiding their development by pointing out errors, highlighting certain areas for additional private practice, adjusting the ensemble sound, and suggesting stylistic interpretation.

In the Ensemble Critique project, students begin to make judgments about their performances of the sort that were previously made by the teacher. At the beginning of the project, the teacher leads a discussion of critique and gives the students special scoring sheets that they will use to critique a performance. The class then examines the score of the piece they will perform, considering the style, the key, and other important features.

Next, the students perform the piece as an ensemble and tape record their performance. They evaluate their performance from memory; then they listen to the tape and compare what they hear with what they remember. Finally, they fill out their score sheets as a formal critique of the performance, focusing on both the performance of their part as well as the entire ensemble.

This performance and critique process is repeated two more times during the semester. In later performances, students can compare their performance with earlier recordings. As the student musicians become increasingly proficient at the process of critique, they begin to take over other tasks of the rehearsal, including conducting the ensemble in rehearsal, determining which portions of the piece require additional attention, and even making stylistic changes in the performance.

Assessment of the Ensemble Critique project consists of the reflection activities that students carry out at each of the three check points. As they fill out the critique score sheets, they assess the performances of individuals as well as the ensemble as a whole. The teacher also evaluates these performances and students can compare their self-assessments with those of the teacher. With practice, students become more precise in their ability to evaluate the performance, and this precise evaluation in turn makes them more proficient in their ability to rehearse themselves.

This project is not designed as an assessment of performance. Instead, it is an assessment of the students' ability to critique a performance and to adjust their rehearsal accordingly. In this project the students begin to share responsibility for decisions that are usually made by the director alone.

Imaginative Writing: Writing Dialog

All studentsread plays in English class but very few write them. In the typical class, for example, the plays of Shakespeare are treated as literature that can be studied and analyzed, not as scripts to be performed and interpreted. The PROPEL project called Writing Dialog provides students with a set of structured activities that develop their skill at creating original dramatic dialogue. Brief passages from contemporary drama are examined and judged using the same criteria that are used to critique the students' work.

The Writing Dialog project is conducted over seven class sessions. Throughout these sessions, students are learning to articulate the criteria by which dialogues can be critiqued. In each session, written work is read out loud and discussed.

First, the teacher introduces the project by circulating several clipboards around the class. Each clipboard has written at the top a single line of dialogue and each student adds an additional line. When the sheet is filled, the class reads the resulting dialogues out loud and discuss the results. Next, working in pairs, students imagine a setting and write a simple scene with two characters that takes place in that setting. The students in each pair write alternate lines in the dialogue. Again, the class reads the scenes out loud and discusses the results. This process of writing and rewriting continues for several class sessions.

All writing done for the project is saved in a folder, and in this class session the students select a piece from that folder to work from, adding approximately ten lines of dialogue to the scene. At the conclusion of the project, students review their collection of dialogues. They record their observations of the changes they have seen in their own writing over the course of the project.

In this project, students begin writing dialogues with no instruction. They can then use these first efforts to begin to explore those features that make for a "good" dialoguesense of character, scene, motivation, and so on. As their dialogue writing continues, they elaborate these criteria and practice using them to evaluate their own dialogues as well as dialogues from published plays. By looking back over their earlier work, students can see a general improvement through the project; they can also see that they have a deeper understanding of what makes a dialogue work. In this way, the project helps the students to use their own productions to construct the criteria they need later to reflect on their own work.

Assessment in this project focuses on the collection of written material that the students create in the seven working sessions. At the end of the project, students evaluate their entire collection, looking for evidence of their developing skill at writing dialogue.
This self-examination is then compared with the teacher's review of the same material.

These sample projects, Ensemble Critique and Writing Dialog, illustrate the breadth of projects that have been created in Arts PROPEL. Currently, these domain projects, as well as a number of others, are being extensively tested in the Pittsburgh Public Schools. They will be available for broader dissemination in 1990.


To summarize, PROPEL projects are sets of activities that are presented over the course of the school year. Each project poses problems that students solve by creating original work. In producing original work, the students use the work of others or reproductions of master works for points of contrast. Throughout this process students critique their own work.

Teachers can use the PROPEL projects to assess students by evaluating the final products, by documenting the change in skills displayed over the course of the project or between two projects, and by measuring their students' growing abilities to recognize positive and negative elements in their own work. The PROPEL project assessments are not designed to measure static competencies nor to determine precisely what has been "learned" in a specific curriculum unit. Instead, they demonstrate learning over a period of time.

These projects are designed to incorporate the features of assessment drawn from the theory of multiple intelligences. The PROPEL domain projects require students to deploy a variety of intelligences as they engage specific problems. Students produce a product in each project, such as a musical piece or a dramatic dialogue. They also work in groups throughout the project, featuring the interpersonal skills of collaboration and group discussion. Finally, the PROPEL tasks tap intrapersonal skill by asking students to reflect on their unique approach to the problems posed in that project. In the PROPEL model, assessment is the continuing process of taking stock of one's current position and comparing that state to the desired state. As students become more sensitive to their current state and how it compares with the desired state, they begin to take on more responsibility for their own learning.

Second, the PROPEL projects provide students with multiple opportunities for problem solving. In
this program assessment does not rest on the results of a single PROPEL project but is drawn from students' performances over a series of projects given throughout the year.

Catalyst Projects

Catalyst is a five-year investigation funded by the John and Mary R. Markle Foundation to explore how children and adults learn with microcomputers. We focused this investigation on the experiences of individuals as they approached some area of endeavor for the first time, examining novices' performance in music and visual art, and more recently in mathematics, writing, and computer programming. Since the difficulties that novices experience in various areas are well documented, we wondered if the computer could alleviate some of those problems. We felt that this research question would shed some interesting light on the general issue of computer-aided learning.
This initial research effort demonstrated that even when assisted by powerful, straightforward, and inexpensive software tools, novices did not make a sustained entry into a domain when they worked on their own. Although these novices did not have difficulty with the software itself, their lack of experience with the domain prevented them from using the software effectively. At the same time, however, this early research determined that novices could use such software most productively when they were presented with an inviting task and considerable help.

Here is an example. We asked novice adults to use a music composition computer program to solve a number of simple harmony tasks. We (and the subjects themselves) were surprised to find that even with little musical training these beginners could solve the problems quite proficiently. However, when we then posed a more open-ended problem, like composing an original melody, we discovered that our novices were much less successful. Even with the computer as an aid, they could not tackle the more unstructured problem effectively on their own.

We took this research finding as a starting point for the design of computer-enhanced projects that would enable novices to work effectively in a new domain.Three prototype Catalyst projects have been developed in the past year to test this idea. What follows is a description of these projects along with the principles we used in designing them. Our current research initiative, also funded by the Markle Foundation, is to evaluate the efficacy of these projects.

Design of the Catalyst Projects

Each Catalyst project consists of three essential components:

1. A powerful software tool. Each project is built around a powerful computer program. We select software that is inexpensive, readily available, and powerful enough for professionals in the domain. Examples include music composition tools, computer program editors, and word processing software.

The software tool reduces the barrier posed by the prerequisite skills that make up the craft of the domain, which an expert would hone through years of practice. In music, for example, the computer produces the performance of the composition and can even flag several kinds of syntactic errors.

2. An on-disk library. Each project contains detailed information in the form of computer files. In the projects described below, this library can contain songs, programming procedures, or historical information.

The library of examples in the project gives the novice ready access to a repertoire of experiences, examples, or illustrations relevant to the problem at hand. The expert has internalized this repertoire through experience.

3. Strategies and instructions. The project also provides strategies in the form of step-by-step instructions along with expert advice and sample solutions. The user follows these instructions to complete the project.

The strategies and instructions of the project guide the novice through the solution of the problems inherent in each step of the task. The ability to find effective solutions to common problems quickly is the hallmark of expert problem solving.

Three Illustrations of Catalyst Projects

Three projects that meet these criteria have been created and are currently being tested at Project Zero: SongSmith (Walters, Meyaard, & Scripp, 1989); Just Enough Pascal (Walters & Morrison, 1988); and Immigrant 1850 (Project Zero, 1990).

Catalyst Project in Music: SongSmith

The Project
: To write a short poem (a limerick)
and set it to music by composing an original melody, duet part, bass line, and chords. Working alone, an individual with moderate musical experience can expect to spend 10 to 15 hours completing the project.

The Computer Tool: Deluxe Music Construction Set (Electronic Arts), an inexpensive but powerful music editing program that can perform the composition in up to four voices using a variety of synthesized timbres, and can print the composition as sheet music.

The Library: Examples of good solutions illustrate each step in the process of writing a song. Illustrations from experienced composers link the task to music history. Using nontechnical language, the project also makes connections with concepts in music theory.

The Instructions: The project guides the user through a step-by-step process of writing a song. It begins with the task of creating the lyric and setting it to an original melody. The project concludes with the creation of harmony with a bass line.

The Song Smith project poses the central problems of music compositiontext setting, voicing, contour, and harmony. It offers a chance for the beginner to explore various solutions to these problems and structures the experience in such a way as to guarantee a product (an original song) as an outcome.

Catalyst Project in Programming: Just Enough Pascal

The Project:
To assemble the Pascal program "GridWalker"a game in which small creatures intelligently find their way through mazesfrom a set of Pascal procedures supplied with the project. An individual with no Pascal programming experience can complete the project in about 25 hours.

The Computer Tool: THINK Pascal (Symantec), an integrated system for writing, editing, and debugging compiled Pascal programs. The debugging features of this development system pinpoint many of the the novice programmer's errors. Special windows give the programmer valuable information about the current state of the program and display the values assigned to variables at any given point.

The Library: The kit includes all the necessary Pascal pieces to assemble the complete program GridWalker. These program pieces are also fully explained in the accompanying manual.

The Instructions: The Just Enough Pascal kit provides concise instructions on how to assemble the pieces of Pascal code in the editor. The kit also includes instructions for "tinkering" with the program, making those changes to the program that reveal important underlying concepts such as loops, variables, conditionals, and events. It also exemplifies modular program design and issues of user interface.

Just Enough Pascal gives the novice an insider's look at programming through the process of building an elaborate program one step at a time.

Catalyst Project in Social Studies:
Immigrant 1850

The Project:
To explore the experiences of the Irish immigrants to Boston in 1850 by making the same decisions these immigrants made when they arrived in the United States. After a brief introduction to the computer software, an individual can complete the project in 5 hours.

The Computer Tool: Either Microsoft Works (Microsoft) or AppleWorks (Apple). These integrated programs include modules for word processing, database, and spreadsheet calculations.

The Library: Immigrant 1850 provides a complete list of the actual passengers of four ships that arrived in Boston from Ireland in 1850. It also provides information about available jobs (location, wages, necessary experience), housing (location and rent), and transportation costs of the time. A spreadsheet document displays a market basket of food and clothing prices in 1850.

The Instructions: The project poses the problem of adopting one family from the passenger list, and making several important decisions for that familyfinding them a place to live, jobs, transportation, food. The users record these decisions in the family diary (a word-processing document) and they summarize the family budget on a spreadsheet.

The Immigrant project takes on many aspects of a simulation in a game-like format. In working through the project, the user makes decisions and reviews the results. The full impact of each decision can be assessed by playing the game a second time and making particular decisions differently. Immigrant also demonstrates how structured information accessed through a powerful tool can be used to gather insights about an historical era.

The Immigrant project provides the individual with "raw" or unstructured information about a period in history, and the individual must organize that information to make decisions and to generate insights about the period and the social group that is being studied.

These three Catalyst projects were designed to give novices firsthand experiences with the materials and problems in three domains. At the end of each project the novice has created a finished piecea song, a computer program, or a diary. This product provides a "stake" for the student, the motivation and context for following the instructions of the task. For example, in SongSmith the discussions of musical notation, terms, and concepts are presented in the context of an individual's own emerging song. In Just Enough Pascal, the student creates a computer program, and in Immigrant 1850 a personal diary of experiences.

These three examples are different from the PROPEL projects in some important ways: They are not designed to facilitate assessment; they are not designed to fit seamlessly into the school curriculum; and they make explicit use of computer technology. Nevertheless, the Catalyst projects offer important features for our assessment projects by illustrating how projectssustained activity on a single problem to produce an outcomecan tap into specific intelligences and reveal how individuals approach the same task differently.

Designing Projects That Combine Technology and Assessment

As a partner in the Center for Technology in Education at Bank Street College, Project Zero researchers are developing assessment instruments that make the best possible use of technology. In our approach, technology functions in two distinct ways. First, technology is used directly in solving the problem at hand. For example, the music composition software actually performs the composition as it is being written, allowing the composer to judge by ear the quality of the work. Writers can use outlining software to study and manipulate the structure of an essay and use word processing programs to simplify the task of rewriting and editing a manuscript.

Technology operates in a second way in the assessment of student progress. The computer "opens up" the process of problem solving, making it more visible to both the problem solver and to the evaluator. For instance, students can easily revise and save drafts as they complete them for later review. When students work in small groups at the computer, their decisions, made visible on the computer screen, can be easily observed by both the members of the group and the teacher. In these ways, using the computer to compose allows the student and the evaluator to study the process of composition as well as the final product.

To develop assessment instruments that use technology as a facilitating instrument and that build on the design principles drawn from the theory of multiple intelligences, we are creating domain projects that embody features taken from the projects in both PROPEL and Catalyst:
Domain projects designed to these specifications allow students to work directly with the materials of the domain. These projects pose problems that can be solved in a variety of different ways. They provide an opportunity for students to develop a stake in the outcome of that problem solving. Finally, projects designed in this way incorporate technology as a tool that facilitates problem solving in the domain. The use of technology opens up the process of completing the project to assessment.

A Prototype Project

The domain project Immigrant 1850 provides a starting point for developing one such domain project. Like the other Catalyst projects, Immigrant 1850 uses a powerful computer tool (AppleWorks) to analyze information. The design of the assessment component is drawn from the domain projects constructed for Arts PROPEL. Because we are presenting this project as a prototype, we would like to describe it in a little more detail.

In the Immigrant 1850 project, students work in small groups. First, they examine the passenger list of a ship that arrived in Boston in 1850. This list includes the name, age, occupation, and country of origin of the passengers, all of whom list Ireland as their country of origin. Students sort the list to divide the names into groups that might make up individual families. For example, they find that the list contains several adults with the same last name, but by using the age and occupation information, they can make decisions about which individuals might plausibly be considered a family. Each working group "adopts" one family and begins to make a number of decisions from the point of view of this adopted family. Students begin a first-person-account diary for the family, which describes their voyage, their arrival in Boston, and the family's plans for the future.

Next, the students peruse other database documents. One document lists jobs that were available in Boston during that period, including the location of each job, the wage it pays, and the skills required. A second database records typical housing, its location, cost per week, and a brief description of the property. A third database covers the costs of commuting from one location in the city to another. Using these three sources of information, the students make the decisions that their adopted family had to make. The students locate their selected jobs and housing on a map of the Boston area and they calculate how much it would cost and how long it would take to commute from home to work. Using a spreadsheet document, students "purchase" food and dry goods supplies for their adopted family. As they select items from the list, the spreadsheet calculates the total cost.

With all of this information in hand--family members and their ages, jobs and salaries, the location and rent of housing, the cost of food and clothing, the distance and charges for transportation--students begin to calculate a yearly budget for their family. They enter all of the information they have gathered into the budget spreadsheet, which then computes the results of these expenditures over the course of one year. Students examine the result, and if the family is losing money, they return to the other documents to reconsider their decisions.

For example, the students may decide that the children in the family should be sent to work in factories instead of going to school. Or they might send older children to work as servants, because these jobs pay room and board in addition to a small wage. The parents in the family can decide to work six days a week instead of five. The family may select less expensive housing. In this way, the students adjust their decisions for the family so that income and expenses balance.

Throughout this process of getting their family settled, students enter the decisions they make into their family diary. They also compare the decisions they make for their family with the decisions of other working groups. For instance, the class can plot where individual families decided to live on a single map of Boston. This summary displays the distribution of this particular group of Irish immigrants across the city, which can be compared with published demographic maps of Boston of the period. Student also compare the experiences and decisions of different families. For instance, some families will have many children to support, others may have a single parent (many immigrants died at sea), and some families will consist only of young adults. The decisions and experiences of these different families will themselves be very different.

This core activity--using the various tools in AppleWorks to make decisions for an adopted family--is supplemented with several other activities. In one, students review newspapers written for the Irish immigrants during this period and use these as models for writing their own newspapers. In another activity, students produce a skit depicting some aspect of the Irish experience. In a third activity, students compare the cost of feeding a family of four in 1850 with the cost of making the same purchases in 1990.

Student Products

Immigrant 1850 supports the production of a range of products in a variety of formats. To illustrate, we will consider three written products that students create. First, they keep track of their decisions by writing a diary from the point of view of one member of their adopted family. To complete this diary, students supplement the factual information they are given in the various databases with historically realistic but fictional details. In writing this diary, students exercise their "historical imagination."

Students write a second essay as an historian or sociologist of the period. The questions that guide this piece may require additional research. For example, as historians, students would be asked to explain why so many Irish immigrants came to Boston at this particular time, and how their arrival changed the city. As sociologists, students could be asked to explain why the immigrants lived where they did, and to speculate on what impact adding a trolley system to the city might have, or how prejudice toward the immigrants was manifested.

In a third writing assignment, students consider questions about the unit itself. For example, they reflect on how they made decisions, how effectively their group functioned, and what they would do differently if they were to work through the project a second time. They consider what information is missing from the simulation and how the project would change if that information was added. Or, they discuss how an immigrant's experience today compares with that of the Irish immigrants 140 years ago.

Techniques for Assessment

Three different techniques are brought to bear in assessing this student work: (1) evaluation of the three written pieces; (2) assessment of group work using a checklist; and (3) review of students' self-assessment activities.

Written Work

As teachers read the written work, they offer several specific points for improvement. Students redraft the assignment taking these points into account, and the new draft is graded. Checklists of writing performance used in Arts Propel tasks will be adapted to this portion of the assessment. As in Arts Propel writing assignments, students create a portfolio of drafts that lead to the completion of the finished piece, which they use when reflecting on changes in their writing.

Checklist on Group Activities

In a second assessment technique, the teacher evaluates how the various teams of students function as a group to accomplish the assigned tasks. To facilitate this, the teacher uses a detailed checklist to record pertinent incidents and interactions. The aspects of group work that are listed on this checklist include interpersonal dynamics, group efficiency and creativity in problem solving, flexibility of roles, and responsiveness to suggestions.

This checklist is important for several reasons. First, it attempts to capture the interpersonal dynamics of the project, which may otherwise remain undetected because students rarely comment spontaneously on such features of their work in their own writing. Second, the checklist gives the teacher the opportunity to step back from the class and observe classroom interactions in a structured and purposeful manner. The checklist also provides the teacher a chance to observe the work of students who may not write well or speak up in class frequently.

Finally, the checklists offer yet another dimension to the grading processhere the teacher can take stock of the classroom atmosphere, how people are working as groups rather than individuals, and the differences in working style among these groups. Since group work is a large part of the project, it is important that assessment procedures take it into account. The results of the checklist will be shared with students as groups and individually when appropriate.

Student Reflection

Students' reflections on the project take two forms. They evaluate their own work (both process and product), and they comment on the project itselfits function, limitations, and assumptions. In both instances, reflection is linked directly to what students produce over the course of the project.

To structure the process of self-assessment, students create a portfolio of their drafts of the essays they write as they work through the project, along with their teacher's comments and suggestions. At the end of the project, they review all of the materials collected in the portfolio. They compare what they accomplished on this project with those of previous projects; or they contrast their observations and reflections with those of other students.

At the conclusion of the project, students critique the materials of the project itself, including the relationship of specific activities of the project to general questions in the subject area. Students can consider the nature of the simulation, its limitations and implications, and its relationship to history.

The goal of this reflective component is to get students to play a significant part in their own education by casting a critical eye toward their accomplishments and toward the project itself. To do that, they must think about what they are doing, why they are doing it, and how good the results are when compared with their capabilities and expectations. As these reflections become more precise, they can be used to guide students' work in later stages of the project.

Analysis of the Assessment Techniques

As a tool for assessment, the Immigrant 1850 project is designed with the theory of multiple intelligences in mind. In the project, students work with the materials of social studiesthe historical records and documents of the period being studied. Furthermore, students can approach the project from a number of different perspectives and engage the variety of intelligences in different combinations. They can mobilize the logical-mathematical intelligence to analyze quantitative data or to select from among competing sources of evidence. They can use the linguistic intelligence to fashion poignant written documents, the interpersonal intelligence to collaborate with their peers, and the spatial intelligence in locating neighborhoods on a map.

Although the theory of multiple intelligences informs the design of this assessment project, it does not speak to the issue of the actual assessment itself. How do we distinguish between an "adequate" solution of the Immigrant problem from an "inadequate" one? To consider this question, we will follow Frederiksen and Collins' analysis of assessment by considering three important features of assessment: directness, scope, and transparency.

First, as an assessment instrument, the domain project blends learning and evaluation and it targets directly the skills and concepts that the teacher wants the students to learn. For example, in a social studies class, students are evaluated on their ability to gather and collate information, to analyze and interpret the results, to get help when necessary, and to write a description of what they have uncovered. These are precisely the tasks they must perform to complete the project.

Because the skills of interest are evaluated directly, the domain project approach to assessment stands in sharp contrast to the traditional test situation. With direct assessment, both teacher and student focus on the skills themselves; in contrast, with tests the teacher must infer the presence of desired skills from students' performance on tests that measure those skills only indirectly. When a student answers a multiple-choice question correctly, the teacher must infer from that response that the student has some understanding of the concepts that provide a basis for the correct answer. Furthermore, as Frederiksen and Collins point out, indirect tests of skills often lead students to concentrate on the skills that are directly related to taking the test rather than on the skills that can only be inferred from the tests.

Second, the domain project assessment is more complete in scope than the traditional test in that it attempts to capture most of what is required to complete the desired activity. Interpersonal skills are manifested in the group interactions, mathematical and spatial abilities appear in the tasks of manipulating data and working with maps, and performance styles can be observed throughout the project.

Of course, different students will display very different profiles of performance along the dimensions represented by these different skills. The challenge to the assessment procedure is to document the breadth of these skills in useful detail. For example, the checklist of group activities directs the teacher to many skills that might easily be overlooked in the written components of the project. Similar instruments are yet to be devised for observing and recording other skills that are represented in the project.

In contrast, a traditional test must select a representative sample of skills for evaluation. From this sample, the teacher then infers the degree to which the untested skills have been mastered. Furthermore, certain important skillssuch as planning, negotiating, collaborating, revising, reflecting, or perseveringare rarely represented at all on multiple-choice tests.

Finally, the terms of assessment must be defined explicitly for both the teacher and the students. This makes the assessment transparent. In the Immigrant domain project, for example, all criteria of the assessment are shared with the students when the project begins. As the students complete portions of the project, they use these criteria to gauge their own work. They can even use the group process report card to evaluate the performance of their group.

Practice with Assessment

Because the domain project is direct in method and broad in scope, it allows students to "study for the test" or to practice with the features of assessment that permeate the project from beginning to end. Furthermore, if several domain projects are given throughout the year, an individual's accomplishments in one project can serve as signposts that direct learning on later projects. We hope that as they complete a series of these projects, students will gain confidence and
experience with the demands of assessment. And because these assessments tap the very skills that the teacher is interested in, this practice with assessment becomes part of the educational process itself. Ultimately, the goal of domain projects is to teach students to assess their own work efficiently and to use this self-assessment to improve their performance on similar tasks in the future.

Although adults might find reflection in a domain project to be straightforward, high school students probably will not. Therefore, the teacher must model the process by posing typical questions and then offering answers. As the students attempt to assess their own work, the teacher can review these first attempts and make suggestions where appropriate. Again, students will develop sophisticated skills of self-assessment only through repeated practice with feedback.

Concluding Remarks

Our goal in developing domain projects for assessment is to create a realistic model of an alternative to the traditional test-based approach to measuring competency or achievement. We will be examining this model in the classroom next year. We will attempt to determine if the assessment is practical and reliable and if it yields information about students efficiently with a reasonable amount of teacher effort. We will also consider the subjective nature of teacher judgments. We must determine which assessments capture insights and aspects of student performance that may have been overlooked otherwise and to what degree teachers agree with one another in these judgments.

The domain projects we are developing are not designed as large-scale replacements for what currently transpires in the classroom. Instead, they serve as models of how one might develop strategies for assessment that meet the criteria we have identified and can coordinate with existing curricula.

We hope that these domain projects will move the discussion of assessment away from tests, measurement, and large-group comparisons toward a consideration of careful documentation of student learning, self-assessment, and growth.


Collins, A. & Frederiksen, J. (1989). A systems approach to educational testing. Educational Researcher, 18, 27-32.

Project Zero. (1990). Immigrant 1850. Cambridge, MA: Harvard Graduate School of Education.

Garnder,H.(1983). Frames of Mind. New York: Basic Books.

Walters,J.(1988). Just Enough Pascal.Bedford, MA: Symatec.

Walters,J., Meyaard, J., & Scripp, L. (1989). SongSmith: Songwriting on the Macintosh. Cambridge, MA: Walters Associates

[ Home | About CCT | Projects | Newsletters | Reports | Staff | Links | EDC Home ]

Last Update: 11/18/96
Comments on the CCT Web site: Webspinner.

©1996 Education Development Center, Inc. All Rights Reserved.