Consistency.In animal behavioral research, we quantify what the animals do. We also measure other parameters about the animals and their environment. Consistency matters.
|
When do we consider consistency?
Consistency is an iterative process that can be continuously evaluated throughout the experiment timeline, but is initially assessed before the start of data collection. We look for consistency within and across individuals.
YourselfTest intra-observer reliability: compare your results over 2+ time points or, if an expert standard exists, compare to that. Advancement can occur once one of these is confirmed. |
Trusted colleagueTest inter-observer reliability: compare your results to those of colleague. Advance once confirmed, otherwise revise data collection plan or definitions and return to the previous step. |
Entire teamTest inter-observer reliability: compare your results to each team member. Individuals can advance once confirmed. If challenges arise, revise plan or definitions and return to the first step. |
Who is the expert?In general, it is best to compare the most experienced observer to others. Expert status would ideally be confirmed and documented with intra-observer reliability, or inter-observer reliability between 2 experienced observers.
|
Who is the trainee?Observer consistency is influenced by who is being trained. Key considerations can include the background level or skill of the trainee, whether you intend to train them as a generalist (can reliably perform all types of data collection) or specialist (exclusively performs 1 specific type of data collection), and the balance of your learning objectives for your trainee vs. your experimental goals.
|
What outcomes can be evaluated for consistency?
There are 3 main categories that we evaluate reliability in. These are mutually exclusive, and increase in complexity For each, we start by establishing training criteria and developing tests that set observers up for success. Specifics about how to test and evaluate success for each type can be found by clicking below.
Tasks and techniques |
Special circumstances
If the behavior or parameter of interest is rare, evaluating consistency can look different.
