연구 수행의 전반적인 과정과 구조를 정리한 가이드입니다.

Main idea: Without data to analyze, you don't have a research project.

What is Experimental Research?

Experimental research is a structured process for testing ideas and gathering evidence.

In a nutshell, conducting experiments involves:

Planning: Design a method to answer your question
Collecting: Gather observations or measurements
Analyzing: Look for meaningful patterns
Interpreting: Explain what the results mean

Core Stages of Conducting Experiments

Before designing an experiment, return to your Research Question. Ask yourself: “What exactly am I trying to discover?” A weak question creates weak data.

1. Define Your Variables

Variables are the factors you measure or control.

Independent Variable: The factor you change.

e.g. Type of chatbot feedback

Dependent Variable: The outcome you measure.

e.g. Student quiz scores

Control Variables: Factors kept consistent.

Same lesson material
Same testing conditions
Same time limit

Key point: Clear variables make experiments easier to understand and repeat.

2. Choose a Research Method

Different questions require different methods.

Common research methods include:

Method	Best For
Experiments	Testing cause and effect
Surveys	Opinions and behaviors
Interviews	Detailed experiences
Observations	Natural behavior
Simulations	Modeling systems
Case studies	In-depth analysis

3. Design the Experiment

A good experimental design minimizes bias and confusion.

Consider:

Who will participate?
What materials will be used?
What steps will participants follow?
How long will the experiment last?
How will results be measured?

Good experiments are:

Clear
Repeatable
Fair
Ethical

Sampling & Participants Your sample is the group you study. Questions to consider: How many participants do you need? Who should be included? Is the sample representative? Example: Studying only computer science students may not represent all university students.
Ethical Considerations Research should avoid harming participants. Common ethical principles: Informed consent Privacy protection Transparency Voluntary participation Right to withdraw Some research may require institutional approval (IRB review). Key point: Ethical research protects both participants and researchers.
Collect Data Data is the evidence collected during research. Two major categories: Data Type Description Quantitative Numerical data Qualitative Descriptive or experiential data

Examples of Quantitative Data Test scores Response times Survey ratings Number of errors Examples of Qualitative Data Interview responses Observations Open-ended survey answers Participant reflections

Maintain Data Quality Poor data leads to unreliable conclusions. Important principles: Consistency Collect data the same way each time. Accuracy Record measurements carefully. Transparency Document your methods clearly. Bias Reduction Avoid influencing participants or results unintentionally.
Organize Your Data Before analysis, clean and organize the dataset. Common tasks: Remove duplicate entries Correct formatting issues Label variables clearly Handle missing values Separate categories consistently Good organization saves significant time later.
Analyze the Data Data analysis helps identify patterns and relationships. Quantitative Analysis Uses numbers and statistics. Examples: Averages Percentages Correlations Statistical tests Graphs and charts Questions to ask: Are there measurable differences? Are patterns statistically meaningful? Are results consistent?

Qualitative Analysis Focuses on meaning and interpretation. Examples: Identifying themes Grouping responses Coding interview transcripts Comparing experiences Questions to ask: What ideas appear repeatedly? What explanations do participants provide? Are there unexpected perspectives?

Visualize the Results Visuals help communicate findings clearly. Common visualizations: Visualization Best For Bar charts Comparing categories Line graphs Trends over time Pie charts Proportions Scatterplots Relationships Tables Precise values

Key point: A graph should make patterns easier to understand, not harder.

Interpret the Results Analysis alone is not enough. Interpretation answers: “What do these results actually mean?” Consider: Do the results answer the research question? Are there alternative explanations? Were there unexpected outcomes? What limitations affect interpretation? Example: Improved scores may result from increased motivation rather than chatbot feedback itself.
Understand Limitations No experiment is perfect. Common limitations: Small sample size Limited time Measurement errors Participant bias Uncontrolled variables Good researchers acknowledge limitations openly.
Reproducibility & Reliability Strong experiments can be repeated. Reliability Would repeated testing produce similar results? Reproducibility Can another researcher follow your method and obtain similar findings? Key point: Research becomes stronger when others can verify it.

Key Concepts Correlation vs Causation Correlation Two variables change together. Causation One variable directly influences another. Important: Correlation does not automatically prove causation.

Validity Validity asks: “Are you actually measuring what you intended to measure?” Example: A survey about “learning satisfaction” may not accurately measure actual learning performance.

Pilot Testing A small-scale test before the full experiment. Useful for: Finding unclear instructions Identifying technical problems Improving procedures

Common Beginner Mistakes

Starting experiments without a clear question
Collecting unnecessary data
Changing methods midway
Using samples that are too small
Ignoring ethical concerns
Confusing correlation with causation
Jumping to conclusions too quickly
Overinterpreting weak data
Forgetting to document procedures

Think of Experiments As: “A structured way to test ideas using evidence.”
Not: guessing, collecting random information, proving what you already believe.

Checklist

🧅 The Research Onion Checklist

Work through these layers to ensure your design is logically sound:

Philosophy: Are you looking for objective facts (Positivism) or human experiences (Interpretivism)?
Approach: Are you testing an existing theory (Deductive) or building a new one from data (Inductive)?
Strategy: Is this an Experiment, a Case Study, a Survey, or Action Research?
Choice: Will you use one method (Mono-method) or a mix of stats and interviews (Mixed-methods)?
Time Horizon: Is this a “snapshot” of right now (Cross-sectional) or a study over years (Longitudinal)?

🛠️ The “Methodology Integrity” Checklist

Before you start collecting data, ask these four questions to “stress-test” your design:

Construct Validity: Does your “test” actually measure what you claim it measures?
Reliability: If someone else followed your exact steps, would they get the same result?
Internal Validity: Have you accounted for “noise” or outside factors that could skew results?
External Validity: Can your findings be applied to the real world or other groups?

📋 The Practical “Must-Haves”

When writing the actual Method section, check for these specifics:

The “Why”: Did you justify why you chose a survey over an interview?
The Sampling: How did you pick your participants? (e.g., Random vs. Convenience)
The Procedure: Is it written like a recipe? (Could a stranger replicate it?)
Data Analysis: What software (SPSS, NVivo) or statistical tests will you use?

🚀 Pro Tip: Look for a “Symmetry of Design”.

If your research question is about feelings, your method should likely be qualitative.
If it’s about numbers, stay quantitative.

To give you a more specific checklist:

Are you doing Qualitative (interviews/text) or Quantitative (stats/surveys) research?
What is your target population?

To ensure your research design is airtight, use these separate checklists for Quantitative (numbers and logic) and Qualitative (meanings and experiences).

📊 Quantitative Checklist (The “Power” List)

Focus on precision, objectivity, and replicability.

Operationalization: Have you clearly defined how you will measure abstract concepts (e.g., measuring “stress” via heart rate or a 1–10 scale)?
Power Analysis: Is your sample size large enough to actually detect a statistical effect?
Variable Control: Have you identified “confounding variables” that might mess up your results?
Instrument Validation: If using a survey, has that specific survey been tested and “validated” by other researchers before?
Analysis Plan: Do you know which statistical test (t-test, ANOVA, Regression) you will use before you collect data?
Eliminating Bias: Is the study “blind” (participants don’t know the goal) or “double-blind” (researcher doesn’t know either)?

📝 Qualitative Checklist (The “Rigorous” List)

Focus on depth, context, and trustworthiness.

Saturation Plan: How will you know when to stop? (Usually when new interviews stop providing new information).
Interview Protocol: Do you have a list of open-ended “probing” questions to keep the conversation flowing?
Triangulation: Are you using multiple data sources (e.g., interviews + documents + observation) to verify the story?
Member Checking: Will you show your findings to the participants to see if they agree with your interpretation?
Reflexivity: Have you documented your own biases? (How might who you are influence how people answer you?)
Audit Trail: Are you keeping a “researcher diary” to track why you made certain decisions during the study?

💡 Top Tips for Both

For Quantitative:

Pilot your survey: Send it to 5 people first to see if any questions are confusing.
Think “Hard” about “Soft” data: If you’re turning opinions into numbers (Likert scales), ensure the “neutral” option is actually useful.

For Qualitative:

Record everything: Never rely on memory. Use two recorders if possible.
Transcribe early: Don’t wait until the end of 20 interviews to start typing; themes often emerge after the first three.

📌 Visual Anchor: Quantitative is a Microscope (zooming in on specific, measurable parts). Qualitative is a Wide-Angle Lens (capturing the whole scene and the “why”).

Conducting Experiments & Collecting Data