Appendix_I.htm

APPENDIX I
METHODOLOGY: EVALUATING HIGH SCHOOLS

In attempting to evaluate the effects of alternative high schools on village children, we used many different sources of information, each of which has its own limitations.¹ By putting all these sources of information together, we hoped to arrive at the most accurate possible view.

Drop-Out and Transfer Statistics

The problem with most evaluations of high school programs for village students is that the major measure of program success used is the school's drop-out rate. Dropout is one useful indicator of student satisfaction. Appendix III presents dropout statistics of different types of high school programs for the years 1971-72 and 1972-73.

Using dropout figures as the sole measure of the school's success, however, is misleading from an evaluation standpoint and can actually harm the students in the school.

When a study depends solely on dropout to measure the success of a school (and therefore the performance of the staff), the staff feels pressured to reduce dropout to produce the lowest possible dropout figures. As a result, students who are suffering severe emotional problems from being separated from their families, for example, are not sent home but merely transferred to a different program. Thus, they do not become "drop-outs." The use of dropout figures alone to evaluate schools ignores what happens to children while they are in school; such a method only reveals that their bodies are still there.

After examining the severity of the social and emotional problems that village adolescents often developed in these programs, we began to regard dropping out, in many cases, as the most desirable outcome, at least if the student managed to get out before severe emotional damage was done.

Using dropout figures to evaluate the success of a high school program can also be misleading because dropout does not always mean a student is dissatisfied with a school. Village adolescents, like adolescents everywhere, tend to follow the examples of their friends. Thus, if one student from a village decides to drop out, frequently many of his friends will follow him, even if they are not at all discontented with the school. Indeed, several of the dropouts we interviewed spoke of being "forced" by their friends to leave a program they liked.

Dropout figures are also misleading, because they exclude transfers. Many students who are dissatisfied often transfer rather than drop out. Yet, transfer statistics are almost totally ignored.

A very distorted picture of students' satisfaction with a school program can emerge unless transfer and dropout statistics are combined into a measure of "total withdrawal" from the school program. For example, in 1971-72, the dropout rate of village students in boarding home programs in white majority cities was 11 percent. The dropout rate of village students in boarding home programs in Native majority towns was 15 percent. Does this mean that village students were more satisfied in the big cities? Not at all. In the city programs, 11 percent transferred; in the rural Native towns, 4 percent transferred. Village students unhappy in big cities were being transferred to rural Native towns because the staff felt they might find it easier to adjust. But students unhappy in rural Native towns usually were sent home because there was nowhere else to go. Total student withdrawal from urban white and rural Native boarding home programs was almost the same. These statistics agree with our other findings. Both types of programs create serious problems; the problems are just different.

Effects of Three Representative High Schools on Village Students

The major method used in evaluating the effects of schools on village children was to follow the progress of the 1971-72 village students who entered three representative high school programs. We attempted to examine the effects of the different schools on students' social and emotional problems and identity formation during their freshman and sophomore years.

High Schools Selected

We studied the following three high school programs because they appeared to us to offer the best available examples of particular high school alternatives with enough entering freshmen for a sample.

1. Bethel Boarding Home Program 1971-72/Bethel Dormitory and Regional High School 1972-73: We selected Bethel because in 1971-72 it was a boarding home program in a rural Native town close to students' homes. The Bethel Boarding Home Program represents a high school environment where students suffer a less abrupt cultural transition and less separation from their families than many other high school alternatives. Bethel, however, like other rapidly expanding regional towns, has a high level of social problems.

In 1972-73, a new dormitory and regional high school facility opened in Bethel. Most of the 1971-72 village students we were following moved to the dormitory. This contrast between the boarding home program and the dormitory with the same students in the same town was useful in examining the effects of each type of living situation.

2. Beltz Boarding School 1971-72/Nome-Beltz Regional High School 1972-73: We selected Beltz because in 1971-72 it was a boarding school enrolling only village students. Beltz is located a few miles outside of Nome, and students are fairly close to their homes.

In 1972-73, a school addition was built and the Beltz Boarding School was consolidated with the Nome High School. Thus, Beltz was no longer a small boarding school but rather a large comprehensive high school enrolling white and Eskimo town students as well as the village students living in the dormitory. Again, observing what happened to the same group of village students when the school environment changed was useful in examining the effects of different types of school situations.

3. Anchorage Boarding Home Program: We selected Anchorage because it was a boarding home program in a large, white majority city. Village students usually live with white families and attend large high schools enrolling primarily white students. This program represents a highly western environment where there is an abrupt cultural transition and breaking of family ties. However, the school offers the largest variety of courses and other educational resources. Experience in the city and in the boarding home family can also provide important educational opportunities.

Sample of Village Students

In each high school, we included in the study all freshman Eskimo students who had not previously been away to school. We selected only Eskimo students because they are the largest group (usually about 65 percent) of students who leave home to attend high school. We feared that including Indian and Aleut students as well would introduce additional problems of cultural differences between students. As the study progressed, it became clear that the schools were having basically similar effects on the students from each cultural group. Nonetheless, caution should be used in applying the results of the study to Aleuts and Indians.

The students sample consisted of:

The entire freshman class of village Eskimo students in 1971-72 at Bethel High School (23 students: 12 male and 11 female).
The entire freshman class of village Eskimo students in 1971-72 at Beltz High School (42 students: 17 male and 25 female).
A randomly selected sample (due to the large size of the entering class) of freshmen village Eskimo students in 1971-72 in the Anchorage Boarding Home Program (40 students: 21 male and 19 female).

Measures of School-Related Social and Emotional Problems

In education as in medicine, a basic principle must be, "First, do no harm." Thus, the first criterion for evaluating a school is to determine if the school experience itself actually causes social and emotional problems in village students.

Adolescence is generally a time of psychological stress, and not all social and emotional problems suffered by village adolescents result from their school experience. For this reason, we first evaluated the severity of a student's social and emotional problems, and then we evaluated the extent to which the problems appeared to result from the student's experience of going away to high school to a particular educational environment.²

A psychiatrist, who is highly experienced in cross-cultural mental health problems, rated the student's symptom patterns as (1) severe, (2) moderately severe, (3) mild, or (4) no disturbance.3 After rating the severity of the problem, the psychiatrist then determined the extent to which it was school-environment related. He judged the problem (1) "highly school-related" if it probably would not have occurred had the student not been in the school environment, (2) "somewhat school-related" if the school environment aggravated a problem the student would likely have had anyway, and (3) "not school-related" if the problem had little or nothing to do with the school environment.

The following types of information were used to evaluate the severity of students' social and emotional problems and their relationship to the school environment:

Background Information. Background information on social and emotional problems of the student before he entered the school program was taken from the student's application packet. In each student's application was an evaluation from the village teacher concerning the student's family strength, special social and emotional problems, and whether or not the student was a social referral. While some teachers provided excellent information, others did not. In addition, each student was asked to write an autobiography, and this provided clues to prior problems; for example, obsessive concern with death. Other biographical information was used to indicate possible problems before entering the program such as the loss of a parent or other type of unusual family situation, a history of family migration from village to village, being overage, or bizarre reasons for choice of school program.
Symptom Rating Scale. A common response to a stressful school environment is the development of such physical complaints as loss of appetite, insomnia, stomach aches, and headaches. Among boarding school students, psychiatrists generally recognize that such somatic symptoms serve as an outlet for the anxiety these children feel at being away from home.⁴

For this reason, the Health Opinion Survey interview, dealing with these types of health problems, was given to students when they entered high school and again at the end of their freshman year (See attachment to this Appendix). The Health Opinion Survey has been found to be a useful mental health indicator in epidemiological studies, and some evidence has shown that it is valid in studying mental health of Eskimos.⁵ Students' responses to the Health Opinion Survey generally agreed with those of the school nurse who was also asked about students' health problems.

Social and Emotional Problems Observed by School Staff. A major source of information about students' social and emotional problems came from interviews with school staff. In each program, the teacher with whom the student had had most contact (usually the language arts teacher) was asked both to anecdotally describe the students' problems and to rate common problems (depression, anxiety, homesickness, fearfulness, lack of classroom participation).

In a boarding school, the dormitory administrator and counselor were asked for anecdotal reports and also about such problems as drinking, suicide threats or attempts, trouble with the law, and drug use. In a boarding home program, the program coordinator and the boarding home parents were asked for similar information. When appropriate, probation officers and psychiatrists were interviewed. The types of information used in making ratings of social and emotional problems may be found in the summary rating form attached to this Appendix.

In many school programs, the staff did not know the students, and we were unable to obtain needed information. Thus, this study probably underestimates the severity of students' problems. However, if the background information did not signal prior social and emotional problems, this study probably overestimates the extent to which the problems are related to school experience.

Measures of Academic Progress

Since a major purpose of schools is academic growth, this was our second criterion in evaluating effects of schools on students. We examined:

Achievement gains in reading and language use as measured by the junior high school level California Achievement (reading and language sub-tests).
Types of courses taken and course grades.

Since dropout and absenteeism were so high, the number of students available for testing by the end of the sophomore year was small. This limitation should be kept in mind in evaluating academic progress.

Measures of Attitude Change

We also collected information through questionnaires on changes in attitudes (such as the belief that people are prejudiced against Natives) and in educational, occupational, and residential plans. All questionnaire and interview measures were evaluated by Native consultants and pre-tested with village students. We do not have a great deal of confidence in these measures because, in later interviews with students, we found that many were not reading the questions or were not interpreting them as intended. For this reason, we do not make much reference to the questionnaire results.

Measure of Student Withdrawal

In addition to collecting statistics on dropout and transfer, each student who left the program was sent a letter asking why he left the program and how well he liked what he was doing now. Of the 54 students who withdrew from the programs, 54 percent responded.

Measures of Later Success

To evaluate later effects of high schools on students, we followed up all village graduates from the three programs over the last 3 years (1970-1972). Aleut and Athabascan as well as Eskimo graduates were included.

Each graduate was mailed a questionnaire requesting his evaluation of his school experience and information about further education, employment, and participation in community and political activities. In addition, information was obtained from friends, school staff, and social service and vocational training personnel. An independent check of college records to determine college success of all graduates was also done. In addition, program graduates at the University of Alaska at Fairbanks and their counselors were interviewed regarding the students' adaptation to college.

Other Sources of Information

We surveyed dropout and social and emotional problems of village students in the new village ninth and tenth grade programs through a questionnaire to village teachers. Village teachers, of course, may be unaware of some of the social and emotional problems of their students. However, we also relied to a large extent on information supplied by teachers in the other schools. Thus, evaluations in both situations have similar bias. Moreover, since a number of students in village programs had left other high schools, this group probably had a higher proportion of students with originally bad problems.

In addition, we visited many other secondary school programs and learned much about internal operations of these programs through serving as consultants for 3 years to the Division of Regional Schools and Boarding Home Program.

Conclusion

Determining "effects" of school is a difficult endeavor, and there are many risks of incorrect conclusions. While we have confidence in the findings of the report, this confidence is due more to the harmony of the data taken as a whole and to the harmony of the data with educational theory and research than to any single data source. No attempt is made to assert the validity of any single source of data used in this study. Rather it is all the information taken together that suggests the conclusions.

STUDENT MENTAL HEALTH EVALUATION—BEHAVIOR RATINGS

STUDENT NAME ________________________ PROGRAM _________________

ANECDOTAL INFORMATION

Teacher (Confidence rating________)

Boarding Home Coordinator/Dorm counselor (Confidence rating________)

Boarding Home Parent (BHP students only) (Confidence rating________)

Student

Other

Student Mental Health Evaluation—Behavior Ratings
Program Retention
Dropped out of program?	1. YES	2. NO
Months in program before dropped:	1 2 3 4 5 6 7 8 9
Transfer to other program?	1. YES	2. NO
Months in program before transfer:	1 2 3 4 5 6 7 8 9
Expelled from program?	1. YES	2. NO
Months in program before expulsion:	1 2 3 4 5 6 7 8 9
Completed in program?	1. YES	2. NO
Returning next year?	1. YES	2. NO

Socio-Legal Events
Drug use:
	1. frequently	2. occasionally
	3. sometimes	4. never
Suicide threat/attempt:
	1. repeated attempt	2. attempt
	3. repeated threat	4. threat
Trouble with the law:
	1. YES	2. NO
Drinking:
	1. heavily and frequently	2. heavily and occasionally
	3. moderately and frequently	4. moderately and occasionally
Promiscuity:
	1. frequently	2. occasionally
	3. sometimes	4. never
Pregnancy:
	1. YES	2. NO

Home Behavior
Boarding home coordinator or dorm counselor confidence rating:
	1. very well	2. fairly well
	3. a little	4. slightly
	5. not at all
Staying out late:
	1. frequently	2. occasionally
	3. seldom	4. never
Not coming home at night:
	1. frequently	2. occasionally
	3. seldom	4. never
Fighting:
	1. frequently	2. occasionally
	3. seldom	4. never
Refusing to do chores:
	1. frequently	2. occasionally
	3. seldom	4. never
Self-confidence:
	1. high	2. fairly high
	3. somewhat low	4. very low
Present mood (or mood at termination):
	1. very cheerful	2. fairly cheerful
	3. matter of fact	4. depressed
General adjustment to program:
	1. very good	2. good
	3. fair	4. not very good
Number of homes lived in (BHP students only):
	1 2 3 4 5 6 7 8 9

School-Related Behavior
Teacher confidence rating:
	1. very well	2. fairly well
	3. a little	4. slightly
	5. not at all
Depressed—low spirits:
	1. frequently	2. occasionally
	3. seldom	4. never
Homesick:
	1. frequently	2. occasionally
	3. seldom	4. never
Lethargic—withdrawn:
	1. frequently	2. occasionally
	3. seldom	4. never
Fearful—apprehensive:
	1. frequently	2. occasionally
	3. seldom	4. never
Nervous—agitated:
	1. frequently	2. occasionally
	3. seldom	4. never
Aggressive—hostile:
	1. frequently	2. occasionally
	3. seldom	4. never
Staying away from school/class when not ill:
	1. frequently	2. occasionally
	3. seldom	4. never
Participation in class discussions:
	1. frequently	2. occasionally
	3. seldom	4. never
Work seems too hard:
	1. most of the time	2. sometimes
	3. not very much	4. not at all
Works hard in class:
	1. most of the time	2. sometimes
	3. not very much	4. not at all
Self-confidence:
	1. high	2. fairly high
	3. somewhat low	4. very low
Mood at present (or mood at termination):
	1. very cheerful	2. fairly cheerful
	3. matter of fact	4. depressed
General adjustment to the program:
	1. very good	2. good
	3. fair	4. not very good

FOLLOW-UP HEALTH OPINION SURVEY

STUDENT NAME ________________________ CODE:__________

PROGRAM ________________________________________________

HEALTH OPINION SURVEY

1. During this past school year, how has your health been on the whole?
	1. good	2. medium	3. poor
2. During this past school year, have you had any particular physical problems or health problems? How about accidents or injuries?
	1. no	2. yes

2a. If yes:	1	2	3
What illness or problem?
When did it start?
Does it still bother you?
Was it all the time or on and off?	1. all the time 2. on and off	1. all the time 2. on and off	1. all the time 2. on and off
Were you home from school because of it?	1. yes 2. no	1. yes 2. no	1. yes 2. no
If yes, how long?
Did you see the doctor or go to the hospital?	1. yes 2. no	1. yes 2. no	1. yes 2. no
How sick were you?	1. very 2. medium 3. little	1. very 2. medium 3. little	1. very 2. medium 3. little
Other notes:

3. During this past school year, have you at any time had to go easy on your school work or stay out of school for more than one week because of poor health?
	1. no	2. yes
4. During this past school year, did your hands ever tremble enough to bother you?
	1. often	2. sometimes	3. never
5. During this past school year, were you ever troubled by your hands or feet sweating so that they felt damp and clammy?
	1. often	2. sometimes	3. never
6. During this past school year, did you ever feel that you were going to have a nervous breakdown?
	1. often	2. sometimes	3. never
7. During this past school year have you ever been bothered by your heart beating hard?
	1. often	2. sometimes	3. never
8. During this past school year have you ever been bothered by dizziness and fainting?
	1. often	2. sometimes	3. never
9. During this past school year have you had a problem with feeling tired even when you wake up?
	1. often	2. sometimes	3. never
10. During this past school year have you had any trouble in getting to sleep and staying asleep?
	1. often	2. sometimes	3. never
11. During this past school year how often have you been bothered by having an upset stomach?
	1. nearly all the time 3. not very much	2. pretty often 4. never
12. During this past school year have you ever been bothered by nightmares (dreams which frighten or upset you)?
	1. many times	2. a few times	3. never
13. During this past school year have you been bothered by your arms or legs going to sleep?
	1. often	2. sometimes	3. never
14. During this past school year have you ever been bothered by "cold sweats?"
	1. often	2. a few times	3. never
15. During this past school year have you felt that you are bothered by all kinds of ailments or feeling sick in different parts of your body?
	1. often	2. sometimes	3. never
16. Have you smoked during this past school year?
	1. a lot (more than 1 pack per day)
	2. some (less than 1 pack per day)
	3. not at all
17. During this past school year have you ever been troubled by sick headaches?
	1. often	2. sometimes	3. never
18. During this past school year have you had loss of appetite?
	1. often	2. sometimes	3. never
19. During this past school year have you ever had a bad taste in your mouth?
	1. often	2. sometimes	3. never
20. During the past school year has your food ever seemed tasteless and hard to swallow?
	1. often	2. sometimes	3. never
21. During this past school year have you felt it was necessary to take vitamin pills for your health?
	1. often	2. sometimes	3. never
22. During this past school year have you taken medicines not given by your doctor?
	1. often	2. sometimes	3. never
23. During this past school year have you felt that you were more apt to catch contagious diseases than most people?
	1. yes	2. undecided
24. How would you say your health was this past year?
	1. excellent 4. poor	2. good 5. very poor	3. fair
25. Have you been happy during the past school year?
	1. most of the time	2. sometimes	3. very few times
26. During the past school year have you ever used medicines for your nerves or pills to help you get to sleep?
	1. often	2. sometimes	3. never
26a. If yes, do you know the name(s) of the pills?
	(Valium Librium Thorzine Stelazine Tofranil)
	______________________________________________
27. During the past school year have you ever used drugs?
	1. often	2. sometimes	3. never
27a. If yes, which ones?
	(Marijuana Acid Speed Heroin _______________)
28. Have there been times during the past school year when you felt low or hopeless?
	1. often	2. sometimes	3. never
29. During the past school year have you ever had a creeping feeling in your skin?
	1. often	2. sometimes	3. never
30. During the past school year have you ever been troubled by a feeling that your hair is standing on end and you have goose pimples?
	1. often	2. sometimes	3. never
31. During the past school year have you ever felt frightened without knowing why?
	1. often	2. sometimes	3. never
32. During this past school year have you ever had feelings of such great restlessness that you cannot sit long in a chair?
	1. often	2. sometimes	3. never
33. During this past school year have you had any fainting spells or loss of consciousness?
	1. often	2. sometimes	3. never

Endnotes

¹ Ideally, in a researcher's utopia, we would have randomly assigned a statistically large enough group of village students to a particular type of school and carefully measured their progress not only while they were in high school but after graduation, to assess long-term effects of a high school education. Of course, this was not possible. We approximated such a research design by combining a study of entering freshmen over 2 years with a follow-up study of graduates from the school.

² The development of social and emotional problems is, of course, not the result of any one single factor. When such problems are related to the school experience, they result from the combination of the stress of the school environment with the student's personal characteristics.

³ We would like to thank Dr. John Wreggit of Langdon Psychiatric Clinic for assisting in this work through a reliability study of our ratings.

⁴ Robert Leon, "Mental Health Considerations in the Indian Boarding School Program: In J. C. Cobb (Ed.) "Emotional Problems of Indian Students in Boarding Schools and Related Public Schools." Workshop Proceedings at the Albuquerque Indian School, Albuquerque, New Mexico (April 11-13, 1960). ERIC Files ED 047848.

⁵ See J. M. Murphy and C. C. Hughes. The use of psychophysiological symptoms as indicators of disorder among Eskimos. In J. M. Murphy and A. H. Leighton (Eds.) Approaches to Cross-Cultural Psychiatry, New York, Cornell University Press, (1965) 108-160. We are in the process of further examining the validity of the Health Opinion Survey by comparing responses of Eskimo adults hospitalized for psychiatric problems with those functioning well in their communities. While this research is not complete, preliminary results suggest the validity of this measure for Eskimo adults. Of course, a measure that is valid for adults may not be valid for adolescents and this is an important qualification to the use of the Health Opinion Survey. The measure seems fairly reliable for Eskimo adolescents. On re-administration of the questions to 26 Eskimo adolescents after a month's time, 83 percent of the students gave the same response both times.

Contents Page
[Alaskool Home]