Acceptable Research Methods
Students who participate in the Arizona JSHS may conduct the following types of research.
- Naturalistic research is observational research where an investigator observes the behavior of something.
It might, for example, be of interest to know whether the common periwinkle, Littorina obtusata is a negatively or positively effected by gravity at low tide, or perhaps is not
at all effected by gravity under exposed conditions.
- A thorough literature search may identify some mollusks that do migrate toward the low water line once they are stranded by receding tidal water and appear unaffected.
- It seems evident there should be few restraints on naturalistic observations.
- Accordingly, the researcher intervenes in the natural setting as little as possible.
- The natural behavior of the marine gastropod under real world conditions is of primary interest to the investigator.
- For those reasons the researcher imposes few if any controls on the subject or subjects intervening seldom if ever with the subjects being observed.
- When observations are made, however, they are made with consistency.
- Data is always gathered at the same time during the tidal cycle, perhaps at the time of extreme low water within each tidal cycle.
- Observations might be made from mid-tide on the ebb tide to mid-tide or on the flood tide.
- As you can see, the natural behavior of the periwinkle is important but not without trying to understand other factors, which might either support or nullify an eventual decision to
state that this organism is either negatively, positively or not effected at all by gravity.
- Observations for this type of research are recorded consistently each time they are made.
- To accomplish the task student researchers use a field or research notebook in which to record their data.
- Naturalistic observations may be conducted in both the physical and biological sciences.
- Suppose it is feared the construction of a new paper mill by International Paper in Old Town, Maine will effect local flora and fauna.
- Researchers could take stack gas samples for analysis in a laboratory.
- Beginning student researchers might gather rainwater samples from close and distant locations to the paper mill.
- The samples could then be analyzed for pH, and formaldehyde content.
- At the same time a record of the appearance of vegetation and animals at the two locations could be made.
- Other methods are of course, the same as those discussed previously.
- Field notebooks are used, and no attempts are made to alter existing conditions.
- What does this type of observational research provide investigators?
- Low constraint research of this type is often used to provide baseline information when a new line of research is beginning.
- We should always remember Charles Darwin's trip on the Beagle and his subsequent writings titled, On the Origin of the Species when we think of naturalistic observations.
- His success in providing a basis for much of the origins of biologically and ecologically focused science was his industry in observing and collecting facts.
Case Study Research
- Case study research, like naturalistic observations, is an observational type of research.
- Like its predecessor, in this type of research, few constraints are used.
- Close comparison, however, shows investigators to intervene somewhat more in this type research than in the former.
- While naturalistic observations are carried out usually in a natural setting as mentioned previously, case study research is carried out with one subject at a time in an interactive
or face-to-face situation.
- In case study research, investigators make no attempt to control independent variables, but look at fewer characteristics than they would when making naturalistic
- Many notable researchers have used case study research.
- Freud became world renowned for his clinical observations. His methods included interviews with subjects during which he allowed them to think freely. His subjects described their
dreams, fears, fantasies and more.
- Another example of case-study research is found in the work of Jean Piaget, the Swiss genetic epistemologist [who started his career as a marine paleontologist].
- Piaget studied one child at a time in natural or informal settings to gain an understanding of children's normal cognitive development.
- Piaget's case-study work had profound effects on contemporary science education.
- Today, grand debates rage about the best ways to teach science.
- It is, however, generally accepted that students construct their understandings of reality by gaining experience through direct or first hand experiences with concrete reality, a
direct spin-off of Piagetís work.
- When should beginning research students use case study research? The following list applies case-study research, and to naturalistic observations as well.
investigating a new area for which little information is available in the literature.
- When students desire to gain familiarity with typical characteristics of settings or subjects before conducting higher-constraint [more highly controlled] research for similar
settings or subjects.
- When research questions specifically focus on the natural flow of behavior and, or on behavior in natural settings.
- When a study focuses on a single individual, group or set of events, and the research questions are specifically about those people, settings or events.
- When examining demonstrations and illustrations like a demonstration of a new procedure.
- When researchers need to discover contingencies that can be used as a basis for higher constraint research.
- When, after completing higher-constant laboratory research, investigators want to know if laboratory findings hold true for the behavior in the natural environment.
- When it is necessary to describe events never before observed [like the astronomical collisions with Saturn].
- When it is necessary to identify contingent relationships among variables.
- When it is necessary to develop a basis for hypotheses to be used in higher constraint research.
- When researchers are attempting to negate general propositions. There are myriad examples of case study research. One high school student was interested in gaining an understanding
of what students in her school knew about the [flawed idea] relationship between the use of hard drugs and the smoking of marijuana. A ninth grade boy wanted to know something about the
effect of increasing incursion of Bull Moose into Anchorage, Alaska during rutting season, upon the fears of elementary students.
- Correlation research is more highly controlled [or constrained] than case study research and the relationship between two variables may be quantified.
- This, however, does not mean the variables are controlled.
- To the contrary, in research it is may not be possible, desirable or convenient to reduce the fluctuations of a variable.
- It may, however, be possible to record the natural occurring changes in a variable and subsequently compare the changes to the parameter being studied.
- The objective in this type of research is to identify some relationship, and describe it, usually mathematically using a range of numbers between -1 to +1.
- An interesting example of correctional research is found in a study reported by a high school student named Gilman.
Gilman noticed that there was a significant difference between wood production in a pine forest fronting on a rural road and a like pine forest fronting on a major highway.
She reasoned that a high rate of exhaust gas pollution caused a decrease in wood production. Further, she predicted that traffic counts would be good predictors of wood production. She
examined traffic count data for an 18-year period, correlating it with tree ring areas in square millimeters. She found tree ring area highly, negatively correlated to traffic count.
That is the higher the traffic count, the lower the tree ring area.
- We note that neither tree ring growth nor traffic count was controlled in this study. A cause-effect relationship seems to have been established. Traffic count [the amount of exhaust
gas pollution] controlled wood production or stated another way; wood production depended negatively upon traffic count. When Gilman collected an evaluated her data she used two
approaches. The first was tabular as shown in Table 1.
The tabular form was simply an easy way of arranging data in her field notebook.
- Once the data were collected, a scatter plot was constructed, and a best fit curve drawn so that the apparent relationship could be examined.
- In differential research, two or more groups are separated based upon some variables that existed before the research started.
- One student researcher knew that within limits, the ages of surf clams, Spisula solidissima could be determined by counting the rings on the shells. Prior to beginning a study of the
effect of hurricane Zelda on the Spisula population at Old Orchard Beach, Maine, the student divided Spisula into four age groups, from one to 15 years.
- The preexisting variable is age and it is also a quantitative variable.
- Another student, interested in studying the survival rates of the marine amphipod, Ampithoe lacertosa isolated in tidal pools subjected to constant sunlight, divided the amphipods in
two categories before beginning her research; male, and female.
- In the latter example the variable was sex, and it is qualitative.
- In differential research, the classification variable is the independent variable in the study to be conducted. In the first example, age is the independent variable and in the
second study, sex is the independent variable.
- The behavior measured in different groups is the dependent variable.
- In the first example, the dependent variable cannot yet be identified. In the second example, the survival rate is dependent upon sex. Another feature of research conducted in this
category is that the investigator manipulates neither variable. Rather, the variables are only measured.
- Researchers use differential research designs most frequently when manipulation of the independent variable is impractical, impossible or inappropriate. With two examples provided
previously, it would be impossible to manipulate the independent variables: age and sex.
- True experimental research boasts five characteristics.
- Predictions - It includes one or more hypotheses about predicted causal effects of independent variables upon dependent variables. One student predicted wood production, in trees
adjacent to herbicide-treated fields, would be less than wood production in trees adjacent to non-herbicide-treated fields. Another student predicted that population densities of lichens
in the genera Leprarid and Sobaria would be inversely related to the sulfur dioxide [SO2] content of the surrounding air in a particular locale.
- Variables - The research includes at least two levels or categories of the independent variable. As previously noted independent variables reflect some conditions existing prior to
the commencement of research. Two examples were used earlier: sex and categories of ages. In the case of the sex, there were two categories of the independent variable, male and female.
With age there were several levels of the independent variables, ages 1, 2, 3 . . . years.
- Subjects - Subjects are assigned to groups or conditions in an unbiased manner. Assignments for example may be made using a table of random numbers.
- Hypothesis Testing - The experimental design includes specific procedures for testing the hypothesis. A student, studying the effects of acid rain upon the upper centimeter of ocean
and fresh water following April rain storms when the prevailing winds blow paper company stack gases over Sprout lake close to Alberni Inlet on Vancouver Island, hypothesizes the rain
water would consistently increase the pH a greater amount in the lake water than in the Inlet. To test the hypothesis, a highly controlled system is developed to take water samples from
the lake each day at a specific time and following each period of rain as well. As part of the experiment, control samples were also drawn daily from a cove on the Strait of Georgia and
a pond close to the Strait. While not described here, procedures for taking water samples, the depth at which the samples were drawn, and the times at which they were collected, as well
as associated weather analysis were meticulously described. All procedures were highly controlled.
- Controls - The research includes controls for major threats to internal validity. Internal validity concerns itself with whether the independent variable was responsible for the
observed variance in the dependent variable or whether some random intervening or extraneous variable caused the variation. The more tightly and carefully the controls are placed upon
the research project, the less likely the results are to be confounded by extraneous factors and the more confidence users of the research will have that the research results can be used
beyond the sample used in the study. Without using a control, like sampling in two different areas, it is easy to see how some cause-effect relationship between two variables might be
suspect. If weather conditions in a particular area apparently cause an increase in the acidity of surface water, without a control not subject to the weather conditions and close
proximity of stack gases laden with acidic and acid forming compounds, it could be argued that the change in the acidity was the result of spring overturn mixing surface waters with
nutrient rich waters from lower depths. Obviously using a control area or group from which to sample does not rule out all possibilities that the observed phenomenon is not caused by
some as yet unidentified variable. The better the research design, the more tightly will be the controls and the more stringent will be the criteria for establishment of a control
The following list of possible intervening variables was compiled.
- Maturation. The animal being observed becomes bigger, stronger, more experienced and more sophisticated. Feeding habits of infant mice may be due to maturation and not the increased
ability of food variety.
- History. History is something that happened in the past that could have a confounding influence upon the present study. A research team decided to examine the effect of caloric
intake level upon weight and height gain in 12-year-old males strictly lectured upon composition of adequate and proper diet. The study was to be conducted over a six-month period, the
males weight and height data being collected weekly by the school nurse. After three months, some students reverted to previous behavior, eating excess quantities of starchy high calorie
foods. The previous habits of some students confounded the data. Had the data-gathering period been shorter, the subjects might not have reverted to previous habits.
Testing. For many years B.F. Skinner, his students and others demonstrated that when tests were repeated, subjects became test wise, subsequently performing better as the number of
tests they took increased. Beginning research students interested in studying animal behavior should be aware of this factor. Monkeys may only be interested in eating bananas if all they
ever are provided as food are bananas.
Instrumentation. Several things can happen to instruments in the data gathering process. Researchers sometimes forget to calibrate instruments. Instruments also age, becoming worn in
different places. Sometimes instruments are employed differently from subject to subject and from setting to setting.
Regression Toward the Mean. Regression toward the mean is related to sample selection from the population. A student, interested in studying an aspect of gerbil behavior felt more
intelligent gerbils would respond more rapidly, demonstrating the particular behavior. A test was subsequently located the score of which would identify raw gerbil intelligence. The test
publisher provided mean test score figures. It was decided to select the sample randomly from all gerbils scoring one standard deviation or higher above the published mean test scores.
After the sample of gerbils to participate in the research had been selected, the student decided to administer the test again. As a result of the second test administration, only 50% of
the gerbils scored one standard deviation or higher above the mean published test score. The scores of the remaining gerbils had regressed toward the mean.
- History is an especially important factor when beginning researchers use human subjects in their studies.
Sample Selection. When it is necessary to select a sample of anything from a larger population, that selection should be accomplished using random selection techniques. This
statement applies whether an investigator is examining bridge bolts, trees, or mice from larger populations to serve as subjects.
Attrition. Ainley, Leheshe and Shaden conducted a long-term study of the breading biology of Addelie penguins. One interest they had was establishing the age at which the birds
become sexually mature. Had their sampling procedures been inappropriate, or had several year groups of immature birds succumbed to catastrophic environmental events or some other
milady, the study might not have been a success. Too few individuals of known age might have remained to support continuation of the research, a problem exacerbated because marine birds
while sexually immature appear like sexually mature adults.
- What happened in the first testing? It could be that on the first test the gerbils were lucky!
A similar problem arose when a Junior Science & Humanities Symposium researcher decided to examine the sleep habits of high school women. Many young women volunteered to be
subjects in the study, the majority of whom were serious. The young researcher forgot that many high school seniors are not in school the second semester of the senior year because they
have met the graduation requirements and so chose to work so they could have extra money to pay college bills. This simple error destroyed her study. Only a few underclass students
remained as subjects.
Experimenter Effects. A high school research team observing the behavior of mice noticed a preference for one food over others. They hypothesized that juvenile mice fed with a
specialized protein enhancer would show a more rapid weight gain over time than mice fed a regular diet. Experimental group mice then were provided an endless supply of food through a
specially built automatic feeder. The control group was fed its normal diet and given the same quantities of food they normally received and consumed. The experimental group mice gained
weight faster than the control group mice did. This was the result expected by the experimenters. We note here that the results were biased by experimenter expectancies.
Experimental group feeding behavior was influenced by the availability of an endless food supply. Mice behavior was influenced in support of the hypothesis. The researchers
originally reasoned that since mice appeared to eat only until their bodies regulated, they had consumed enough food. They further reasoned that use of the automatic feeder would not
have an effect on the results. Finally, the students had only one automatic feeder available to them thus both control and experimental groups did not have the same access to food
Student researchers often unintentionally tend to make decisions and choices that favor the hypothesis being tested. Their actions, however, are unintentional, and data are not
deliberately or knowingly falsified.
Their mentors must constantly reintroduce beginning student researchers to the idea of experimenter bias.
Student investigators come from success oriented school environments. They tend to view research projects that generate data not in support of hypothesized outcomes, as failures.
Throughout their school careers they have been taught to make predictions, carry out some experimental procedures, and the expected results will surface.
But, what if the expected results do not occur? Was the experiment a failure? From a research perspective, the answer is no. Acquisition of specific results can only be predicted.
All student researchers have in the end are the data they gathered. If experimenter bias is not present, then the data is simply data.
It is not correct or incorrect data. The data either completely supports, supports to some degree, or does not support the previously formulated hypothesis.
Rejection of an hypothesis is a finding as significant as acceptance of the hypothesis. Mentors are well advised to hold regular discussions with their students regarding research
project success. Well-designed, highly controlled, experimental research free of biases is always a success.
Whether a project hypothesis is accepted or rejected is not part of the success equation.
Researchers the world around work diligently in the search of drugs to combat the AIDS virus.
Each time a new drug shows promise after adequate evaluation using acceptable screening procedures a study is designed to test the effects upon human subjects.
Two possible outcomes of such research exist:
- the drug will not be successful controlling the virus, and;
- the drug will be successful controlling the virus. In the first case, the researcher was successful since the work led to the elimination of another chemical from a long list thought
to have potential. There are hundreds of chemicals with potential. Eliminating those that will not control is a Herculean task at best requiring the efforts of thousands of researchers.
That the second researcher was successful will be heralded throughout the world, and no doubt the researcher will become a Nobel Laureate, in no way decreases the importance of the first
Back To Top
- After students have identified the types of research they will conduct, they begin to identify variables. A note of caution should be observed here. In order to be classified as research, studies do not necessarily need to have dependent and independent variables. While Ainley, LeReshe and Slader  were interested in breeding behavior of Addelie penguins, they did not have variables other than those they established as a result of their
investigations. Interestingly enough, however, they did establish some apparent correlations as a result of their attempts to identify baseline-breeding behavior. Theirs was a study that falls in the category of naturalistic observations. There is another caution for students. There is a tendency among people who hold advanced degrees in
fields other than science to guide, even force-beginning students to do experimental research. The perception is that any study that does not manifest a highly controlled situation in
which there are treatment groups, control groups, and before-after comparisons [pre-post], is not a true study or true research. The converse is true. There are many types of research
all equally valid depending upon the research objectives.
Back To Top
The scientific method is used in all research projects.
- The method generally includes the following:
- Identifying a Problem. Apparent problems are identified through observations of the real world, as a result of searching the literature or through some combination of events.
- Data Collection. Gathering all pertinent problem related data.
- Hypothesis and Research Question Formulation. Formulating an hypothesis or research question/s [may be more than
- Observations, Experiments, and Data Gathering. Performing experiments to test the hypothesis, or gathering data to help answer the research question/s.
- Interpretation of Findings. Interpreting the results of the experiments [usually accomplished through data analysis and comparison of new data with what is already known about a
- Drawing Conclusions Based Upon Evidence. Drawing one or more conclusions about the hypothesis or research question/s based upon evidence.
Back To Top
Library & Internet Research
- Students often conduct library and INTERNET research projects falsely thinking that such endeavors are a correct application of the scientific method. Generally, this is not the case. While literature searching is part of the science research process, it is not, in and of itself, acceptable as a Junior Science & Humanities Symposium project. Proper application of
the scientific method by student researchers is weighed heavily when research papers are evaluated. The evaluation determines which projects [therefore who] will be presented formally at
the Symposium. Projects, which do not involve the steps of the research process, will not be presented formally at the Symposium.
How To Write A Research Paper was in part provided by Pacific Junior Science
& Humanities Symposium.
Back To Top