Conceitos Iniciais

Desenho Experimental e Análise Avançada de Dados Ecológicos

João O. Santos

joao.santos@ispa.pt

ISPA

2025-09-23

Back to Basics

• Philosophy of Science

• Research Methods

• Data analysis

• Three sides of the same prism

Similarities and Differences

Philosophy of Science

what is this thing called science cover

Chalmers (2013)

Karl Popper

• Hypothesis must be falsifiable

• Go find the black swan!

• See Platt (1964)

Thomas Kuhn

• Paradigms and incommensurability

• Normal science, crisis, and scientific revolutions

Imre Lakatos

• Theories have a hard core and a belt of auxiliary hypothesis

• Progressive and degenerative research programs

• We should be more leniant towards incipient research programs

New Experimentalism

• Science progresses by collecting data from theory-independent severe tests

• Severe tests are more informative

• See Mayo (1996; 2018)

Basic Concepts

• Variability

• Measurement

• Causality

Variables

• As the name implies…they vary (not constant)
  • There is variability
• Measurement scales:
  • Qualitative: nominal, ordinal
  • Quantitative: interval or ratio; discrete or continuous
  • Measuring accurately and reliably is hard
• Distributions:
  • Normal, binomial, exponential, logarithmic, quadratic, cubic, etc…
  • Variables can vary according to known distributions
    • The same is true for errors (see next episodes)

Distributions

• Central tendency:
  • Mean, median, mode, etc…
  • If variables follow a distribution, where’s the center?
• Dispersion:
  • Standard deviation, IQR, range, etc…
  • If variables follow a distribution how wide is it?
  • or how much they deviate from the center (or other known locations)?
• Shape:
  • Skewness and kurtosis
  • If variables follow a distribution what’s its shape?
• An area in a probability distribution is a probability!

Causality: It’s Complicated…

• Dependent and independent variables (outcomes and predictors)

• Moderators and interactions

• Mediators and causal paths

Our Toolbox

• Measurement

• Sampling

• Replications (and pseudo-replications)

• Control, control, control

• Manipulate

You Know the Tools

• Learn them well

• Know their power and limitations

• Mix and match

• Master them

• Learn from the data and improve the design

Ideas, Theories, Models, and Hypothesis

• Abstract ideas

• General theories

• Theoretical and statistical models

• Hypothesis

Testing Theories

• Test if their assumptions hold

• Test if their predictions are accurate

• Find unknown boundaries

• Find moderators

• Reverse the causality

• See (McGuire 1997)

Statistics

• The more you learn to model your hypothesis
  • The more theories you’ll be able to test
  • The more severe the tests you’ll design
• Less is more:
  • More or less…

The Model Comparison Approach

• Models
  • Idealized descriptions of reality
  • Capable of making predictions
  • They err
• We test hypothesis by comparing models (Judd et al. 2017)

Model Properties

• Models differ in how much they err in their predictions

• Models differ in their complexity

• We will compare errors and complexity

One Formula to Rule them All

• Reality = Model + Error

• Data = Model + Error

• Linear models:

  • \(DV = \hat{DV} + Error\)
  • \(\hat{DV} = \beta_0 + \beta_1 \times IV_1 + \beta_x \times IV_x\)
  • \(Error = DV - \hat{DV}\)

Model Comparisons

• We need a way to compare models

• We want accurate models

• We prefer simple models

• We need a formula that weighs their errors and their complexity

F Statistic

\(F = \frac{MSR}{MSE}\)

\(MSR = \frac{SSE(m0) - SSE(m1)}{df_{factor}}\)

\(MSE = \frac{SSE(m1)}{df_{error}}\)

• MSR = Mean reduction in error
• MSE = Mean error that is left
• df_error = degrees of freedom in estimating the error
• df_factor = differnce in the number of paramenters between m0 and m1

t Statistic

\(t = \sqrt{F}; F = t^2\)

\(t = \frac{Estimate}{Error}\)

\(t = \frac{M}{SE} = \frac{M}{\frac{s'}{\sqrt{N}}}\)