Foundations.We all start somewhere. It's important to think through the reliability process when starting a research project. Key questions are listed below to help you get started. Click on each question or icon to learn more.
|
|
What's your proposal?The first step is to develop your research proposal. This process typically includes reading scientific literature, having discussions, developing hypotheses and predictions, brainstorming dependent variables that address your question of interest, and determining your approach to data collection.
|
|
What are you measuring?The next step is to consider what type of information you are collecting. Ethologists often collect information in a variety of ways, producing different types of data, like time spent engaged in a behavior or features of the animal being studied. Your approach to reliability will depend on the type of data you aim to collect.
|
|
Who makes up your data collection team?The level of detail used in reliability training often depends on 2 key factors: the number and experience level of the observers involved in our teams. We often scaffold our training, or slowly increase complexity, in response to these characteristics.
|
|
How do you train those involved?We often use multiple training exercises, based on our answers to the above questions. We often begin by familiarizing trainees with definitions, examples, and sometimes short practice tests. This is then followed with a more formal training stage, where trainees either collect data synchronously with the expert, or score video and photo sets asynchronously to compare against an expert's assessment of the same material.
|
|
How do you evaluate your training with metrics?Through training, you will generate data sets that you can run statistics on to determine if individuals and techniques are repeatable and reliable enough to start collecting data. Evaluating consistency is crucial before starting data collection, but revisiting this process throughout the experiment can produce more robust and reliable outcomes.
|
|
What next? This process laid out here is often not so straightforward. Problem solving is often required at each stage. Move on to “Diving Deeper” to learn more about how reliability training is an iterative process.
|
