Attachment 1

Questions and answers from the Science training.

Most of the questions are organized by criteria category. Under each category, the criterion referenced in the question(s) is stated first, followed by the question(s) and the answer(s). Questions not specific to the criteria are included at the end of this attachment.

Category 1: Science Content/Alignment with Standards

Category 1, Criterion 1

Content That Is Scientifically Accurate

Question 1: How many citations are expected for Category 1, Criterion 1, if the judgment is “yes”? (So, if it is “accurate,” how much proof is asked for?)

For each criteria category on the Report of Findings, you will be required to list several exemplary citations of where you as a panel found that the program met the criteria in that category. You may not have a citation for each criterion in that category, but you should have a representative number of citations that support your recommendation.

Category 1, Criterion 2

Comprehensive teaching of all California Science Standards at the intended grade level(s) as discussed and prioritized in the California Science Framework, chapters 3 and 4. The only standards that may be referenced are the California Science Standards. There should be no reference to national standards or benchmarks or to any standards other than the California Science Standards.

Question 2: Category 1, Criterion 2, all California science standards, does this include Investigation and Experimentation Standards?

Yes, the instructional materials must cover all the Investigation and Experimentation Standards.

Question 3: If the standards are covered in the teacher’s edition but not in the student’s edition, is that considered covering the standard?

If the standards are covered in the teacher’s edition but, according to the way the program is to be implemented, that material is not accessible to the students, then the standards are not being covered.

Question 4: How is it assured that textbooks are accurate on pages not referenced in the standards map?

It is assured because the Instructional Materials Advisory Panel (MAP) and Content Review Panel (CRP) members will be reviewing the whole program for content accuracy.

Question 5: Grade 5, Standard 1c, do publishers need to include the element name and chemical symbol: aluminum (Al)?

Grade 5, Standard 1c, states: “Students know metals have properties in common, such as high electrical and thermal conductivity. Some metals, such as aluminum (Al), iron (FE), nickel (Ni), copper (Cu), silver (Ag), and gold (Au), are pure elements; others, such as steel and brass, are composed of a combination of elemental metals.” When pure elements or a combination of elements are discussed in the program, the criterion requires that the publisher include examples such as those listed in the standard but would not have to include the elemental symbol.

Category 1, Criterion 3

Multiple exposures to the California Science Standards (introductory, reinforcing, and summative), leading to student mastery of each standard through sustained effort.

Question 6: “Multiple exposure,” is this for each standard or in general (Category 1, Criterion 3)?

The criterion states: “Multiple exposures to the California Science Standards (introductory, reinforcing, and summative), leading to student mastery of each standard through sustained effort.” Science programs must provide multiple opportunities for students to master each content standard at the appropriate grade level.

Category 1, Criterion 4

A checklist of California Science Standards in the teacher edition, with page number citations or other references that demonstrate multiple points of student exposure, and a reasonable and judicious allotment of instructional time for learning the content of each standard. Extraneous lessons or topics that are not directly focused on the standards are minimal, certainly composing no more than 10 percent of the science instructional time.

Category 1, Criterion 7

Evidence in the teacher edition that each hands-on activity directly covers one or more of the standards in the California Science Standards (in the grade-appropriate physical, life, or earth sciences strands); demonstrates scientific concepts, principles, and theories outlined in the California Science Framework; and produces scientifically meaningful data in practice. All hands-on activities must be safe and age appropriate.

Question 7: Can a hands-on activity ever be considered extraneous? Category 1, Criterion 7, places a requirement on every hands-on activity, which appears to exclude them from the 10 percent extraneous material allowed by Category 1, Criterion 4. So if we find a hands-on activity which does not “directly [cover] one or more of the standards,” must the proposal be rejected, or is the term “every” in Criterion 7 to be understood implicitly to have the 10 percent flexibility given in Criterion 4?

Category 1, Criterion 7, requires: “Evidence in the teacher edition that each hands-on activity directly covers one or more of the standards in the California Science Standards…,” so hands-on activities are excluded from the Category 1, Criterion 4, that allows for 10 percent extraneous lessons or topics.

Question 8: Clarify “no more than 10 percent extraneous lessons or topics,” keeping in mind that teachers of grades four and five require scaffolding lessons for their students who have never been taught science in the lower grades.

Extraneous means lessons or topics that students do not have to master in order to master the California Science Standards. Scaffold activities that are essential to understanding the science standards are, in fact, leading to mastery of the content standards and are not considered extraneous.

Question 9: Is material that appears out of grade level considered extraneous (e.g., grade-level three material that appears in the other grade-level text)? At what point is an extension to a topic (which now covers content at another grade level) considered extraneous?

If, for example, a grade three program includes topics or lessons that cover some standards in grade four, it would be considered building toward mastery of the content standards, so would not be considered extraneous.

Category 1, Criterion 5

A table of evidence in the teacher edition, demonstrating that the California Science Standards can be comprehensively taught from the submitted materials with hands-on activities composing at least 20 to 25 percent of the science instructional program. Hands-on activities must be cohesive, be connected, and build on each other to lead students to a comprehensive understanding of the California Science Standards.

Question 10: How do you define a “hands-on” activity specifically for the 20-25 percent criteria?

Criteria Category 5, Criterion 9, states: “Suggestions for how to adapt each hands-on activity provided to other methods of teaching, including teacher modeling, teacher demonstration, direct instruction, or reading, as specified in the California Science Framework.” Based on this criterion, teacher modeling, teacher demonstration, direct instruction, and reading are not hands-on activities. Generally, a hands-on activity can be defined as any educational experience where students are actively engaged in manipulating materials to gain knowledge or understanding of the science content standards.

Question 11: In the criteria for evaluating instructional materials in science, kindergarten (K) through grade 8, Category 1, Criterion 5, states hands-on activities at least 20-25 percent; yet the Science Framework, Chapter 1, page 11, states a maximum of 20-25 percent. Which do we go by?

The criteria governing this adoption are in the addendum included with your Science Framework (framework) identified as: “This reprint of Chapter 9 replaces the chapter printed in this framework (2004 edition).” The criteria are also included in the Invitation to Submit document (beginning on p. 49) and as a handout behind the “Evaluation Criteria” tab included in your training binder.

Question 12: “…20 to 25 percent of the science instructional program…,” is that 20 to 25 percent by time spent or grade outcome e.g., lab grade?

It is 20-25 percent of the instructional program.

Category 1, Criterion 6

Investigations and experiments that are integral to and supportive of the grade-appropriate physical, life, and earth sciences standards so that investigative and experimental skills are learned in the context of those content standards. The instructional materials must include clear procedures and explanations, in the teacher and student materials, of the science content embedded in hands-on activities.

Question 13: Does “clear procedures” imply a cookbook approach to experiments? Is there any room for some inquiry/exploration approach?

No, “clear procedures” does not imply a cookbook approach to experiments, and yes, there is opportunity for the inquiry/exploration approach. The framework provides guidance in Chapter 1, where it states: “Instructional materials need to provide teachers with a variety of options for implementation that are based on the science standards.” (Science Framework, Chapter 1, p. 6)

Category 1, Criterion 9

Extensive, grade-level-appropriate reading and writing of expository text and practice in the use of mathematics, aligned with the Reading/Language Arts Framework for California Public Schools and the Mathematics Framework for California Public Schools, respectively.

Question 14: Writing of expository text – should it be a direct lesson to students or stated in the teacher edition as a suggested follow-up?

If there are suggested activities for writing expository text in the teacher’s edition but, according to the way the program is to be implemented, that material is not accessible to the students, then this criterion is not met.

Category 1, Criterion 10

Examples, when directly supportive of the California Science Standards, of the historical development of science and its impact on technology and society. The contributions of minority persons, particularly those individuals who are recognized as prominent in their respective fields, should be included and discussed when it is historically accurate to do so.

Question 15: Examples, how much is enough?

There is no specific number of examples required. The absence of examples of the historical development of science and its impact on technology and society that directly support the standards would suggest that this criterion is not met.

Category 2: Program Organization

Category 2, Criterion 3

Clearly stated student outcomes and goals that are measurable and are based on standards.

Question 16: If the criterion states “…based on standards,” do the standards have to be explicitly stated?

No, if the criterion states “based on the standards,” the standards do not have to be explicitly stated.

Category 2, Criterion 5

A program organization that provides the option of preparing for or pre-teaching the science content embedded in any hands-on activities.

Question 17: Does preparing for or pre-teaching the science content refer to materials for the teacher to set up the activity? (“Discovery” approach option for pre-teaching)

This is not referring to the materials to set-up the activity. It requires that publishers provide the option of preparing for or pre-teaching the science content embedded in any hands-on activities. If publishers use the “discovery” approach, they must provide a way of preparing for or pre-teaching the science content embedded in the activity.

Category 2, Criterion 6

A program organization that supports various lengths of instructional time and helps make efficient use of small blocks of time (that may be available during the instructional day) in kindergarten through grade three.

Question 18: Does support various lengths of instructional time require a listing of approximate time to complete? Does this pertain only to Kindergarten through grade three?

No listing is explicitly required under this criterion, though it is a requirement in Category 5, Criterion 2, which states: “A checklist of program lessons in the teacher edition, with cross-references to the standards covered, and details regarding the instructional time necessary for all instruction and hands-on activities.” Category 2, Criterion 6, applies only to Kindergarten through grade three.

Category 2, Criterion 8

Support materials that are an integral part of the instructional program. These may include video and audio materials, software, and student workbooks.

Question 19: Program Organization #8, support materials that are an integral part of the program… Define “integral.” Clarify the difference between outcome and goal as perceived by the publisher.

Integral means support materials that are clearly connected to the instructional program.

In Publishers Bulletin 1, question 11, publishers were given the following guidance regarding the distinction between outcomes and goals.

Publishers Bulletin 1, Question 11: What constitutes “student outcomes and goals”? Are these specific questions that students should be able to answer if they have mastered material? How are outcomes different from goals?

Both the criteria and the framework provide the answer to this question. Category 2, Criterion 3, calls for materials that have “Clearly stated student outcomes and goals that are measurable and are based on standards.”

The Science Framework states: “Effective science programs include continual assessment of students’ knowledge and understanding, with appropriate adjustments being made during the academic year. Effective assessment (on a continuing basis through the academic year) is a key ingredient of standards-based instruction.” (Science Framework, Chapter 1, p. 11)

The Science Framework further states: “The science standards are set forth in terms of what students know. Therefore, mastery of an individual standard is achieved when students have actually learned the fact, skill, concept, principle, or theory specified. Mastery does not occur simply because students have received a particular explanation or participated in a particular activity.” (Science Framework, Chapter 3, p. 24)

Later the Science Framework discusses what the assessments should measure:

“The measures need to:

Reveal the student’s knowledge and skill in science and the ability to apply that knowledge and skill as a foundation for future learning.
Document the student’s progress (or lack of progress) toward mastering the content standards.
Provide information useful to planning and modifying future instruction in ways that will help all students master (or exceed) the content standards.
Help identify and reinforce effective instructional practices.”
(Science Framework, Chapter 6, p. 284)

Category 3: Assessment

Category 3, Criterion 2

Multiple measures of the individual student’s progress at regular intervals and at strategic points of instruction, such as lesson, chapter, and unit tests or laboratory reports

Question 20: With regard to Assessment Criterion 2, does “progress” in the phrase “Multiple measures of the individual student’s progress…” mean progress toward mastery of the standards? If not, where do we evaluate whether the publisher assesses the essence of the standards and not the trivial details.

“Progress” means that the program must provide multiple measures of students’ progress toward mastery of the science content standards.

Category 3, Criterion 3

Suggestions on how to use assessment data to guide decisions about instructional practices and to help teachers determine the effectiveness of their instruction.

Question 21: How should publishers show me the methods of using the information gathered in assessments? Do they need to specifically tell me what to do with my data or do I rely on my teaching experience and are they going to tell me in the teacher’s edition how to determine the effectiveness of my teaching instruction?

Publishers should provide suggestions on how to use assessment data to help teachers determine the effectiveness of their instruction.

Question 22: Where do you record within the criteria if the assessment tool and instructional materials are not in alignment?

There isn’t a specific criterion that requires that the assessments be aligned to the instructional material. You should use your Note-Taking Guide to determine if the program meets the Category 3, Assessment, criteria.

Category 4: Universal Access

Category 4, Criterion 1

Suggestions, based on current and confirmed research, for strategies to adapt the curriculum and the instruction to meet students’ identified special needs.

Question 23: What would current or confirmed research for special needs students mean, look like, and where would it be located? Who evaluates this research requirement? What criteria are we to use for “confirmed research” – federal definition or some other criteria?

Whenever the criteria state current and confirmed research, they are referencing the current and confirmed research that the framework is based on. So use the framework as your guide for the instructional strategies to adapt the curriculum and the instruction to meet students’ identified special needs. You are not expected to consult resources other than the framework or the content standards to evaluate the instructional materials.

Question 24: Does “special needs” in Category 4, Criterion 1, mean individualized education program (IEP) or 504 only, or does it include gifted and talented education (GATE) or a broader group?

“Special needs” means the following:

Students who are below grade level in science learning
Advanced learners
Students whose reading, writing, listening, and speaking skills are below grade level
Students with disabilities – those students with a 504 accommodation plan or IEP.

Category 4, Criterion 4

Suggestions to help teachers pre-teach and reinforce science vocabulary and concepts with English learners.

Question 25: If the publishers label a “suggestion” as appropriate for “English learners,” but IMAP members don’t feel it is English learner appropriate, must we accept it as a suggestion for English learners?

Your judgment is whether or not the suggestions help teachers to pre-teach and reinforce science vocabulary and concepts with English learners.

Category 5: Instructional Planning and Support

Category 5, Criterion 6

Clear, grade-appropriate explanations of science concepts, principles, and theories that are presented in a form that teachers can easily adapt for classroom use.

Question 26: Category 5, Criterion 6, Are these explanations to be included in the teacher edition, student edition, or both? (It doesn’t say where to include wording for the teacher to follow when planning instruction.)

The criterion does not state where the explanations must be placed; therefore, the publisher can put them anywhere in the teacher or student editions.

General Criteria Questions

Question 27: When the criteria refer to suggestions/strategies – how much is enough?

Use your professional judgment as to whether or not there are sufficient suggestions or strategies to meet the criteria.

Framework Questions

Question 28: Should vocabulary support the framework and standards, or can any terminology be selected? Is vocabulary not related to the framework part of the 10 percent extraneous material?

In general, the framework is considered a model curriculum and a guide for developing instructional materials, so publishers can adopt a different terminology for their instructional materials. The 10 percent extraneous refers to lessons or topics that are not directly focused on the standards, not vocabulary.

Question 29: In view of the sentence, …teaching of all California Science Standards…as discussed and prioritized in the California Science Framework…, just how should we incorporate the framework into our evaluation?

The framework is the state’s model curriculum, so it shows one way of weaving the California Science Standards into a coherent educational program; but it’s not the only way they can be so woven, and publishers can make different choices. You should use the framework as a guideline for the kind of instructional materials that might be expected in supporting the standards.

Edits and Corrections

Question 30: Will the publishers have a chance to correct specific pieces of materials according to reviewer’s recommendations? Inaccurate illustrations – is this enough to call content inaccurate, or can this be addressed in the report and changes made in illustrations?

In Publishers Bulletin 5, question 11, publishers were given the following guidance regarding the correction of errors:

Publishers Bulletin 5 Question 11: Is there a way for the correction of errors? For example, if a content error is found, can the publisher correct it?

The Edits and Corrections policy on page 31 of the Invitation To Submit (ITS) states: “Only the corrections and edits defined below will be permissible during the review of instructional materials for adoption.

Edits and Corrections are defined as:

Inexact language and imprecise definitions
Mistaken notations
Mislabeling of pictures, objects, animals, plants, and so on
Misspellings or grammatical errors
Computational errors and examples

Changes as defined below will not be allowed for instructional materials submitted for this adoption.

Revising the program to meet the criteria and standards
Rewriting of a chapter or section
Adding new content
Moving materials from one grade level to another
Incorrect data, including definitions and factual errors that require content experts to review the materials prior to approval”