Foundations.We all start somewhere. It's important to think through the reliability process when starting a research project. A few key questions are listed below to help you get started. Click on each question or icon to learn more.
|
|
What's your proposal?The first step, which will influence all aspects of reliability you undergo, is to develop your research proposal. This process typically includes reading scientific literature, having discussions, developing hypotheses and predictions, brainstorming dependent variables that address your question of interest, and determining your approach to data collection.
|
|
What are you measuring?The next step is to consider what type of information you're collecting. Ethologists often collect information in a variety of ways, producing different types of data, like time spent engaged in a behavior or features of the animal being studied. Your approach to reliability will likely depend on the type of data you aim to collect.
|
|
Who is involved in data collection?The level of detail used in reliability training often depends on 2 key factors: the number and experience level of the observers involved. We often scaffold our training, or slowly increase complexity, in response to these.
|
|
How do you train those involved?We often use multi-fold training exercises, based on our answers to the above questions. This often includes an initial phase of familiarizing trainees with definitions, examples, and sometimes short practice tests. This is then followed with a more formal training stage, where trainees either collect data synchronously with the expert, or score video and photo sets asynchronously to compare against an expert's score of the same material.
|
|
How do you know you're ready to start data collection?Through training, you will generate data sets that you can run statistics on to determine if individuals and techniques are repeatable and reliable enough to start collecting data. Evaluating consistency is crucial prior to starting data collection, but revisiting this process throughout the experiment can produce more robust and reliable outcomes.
|
This process laid out here is often not so straightforward. Problem solving is often required at each stage, and the outcomes of that can lead to structural changes throughout, including changes to definitions, types of data collected, and the training (and re-training) process.
