Experimental Methodology #CH30SP #swayamprabha

Experimental Methodology in Human Research

Introduction to Experimental Design

• The speaker introduces the concept of experimental methodology, emphasizing its role in designing experiments for data collection and annotation to understand emotions and develop emotionally intelligent machines.

Ethical Considerations in Human Research

• Importance of protecting human research subjects' rights and welfare is highlighted, necessitating the involvement of an Institutional Review Board (IRB).

• Researchers are advised to consult their organization's head or department regarding the existence of an IRB before proceeding with experiments involving humans.

Preparing for IRB Approval

• The necessity of obtaining ethical approval from the IRB is stressed, ensuring that all data collected is valid and ethically sourced.

• Key documents required for IRB submission include a draft or abstract detailing research methods, participant recruitment strategies, and experiment design.

Informed Consent Process

• An informed consent form must be prepared to ensure participants agree to participate under specified conditions; this form should be detailed (1-2 pages).

• Guidelines from organizations like APA can assist in creating informed consent forms that must accompany the IRB application packet.

Confidentiality and Risk Assessment

• A dedicated paragraph on confidentiality measures for participant anonymity and data privacy must be included in the proposal.

• Researchers need to identify potential risks associated with their experiments, such as psychological effects from stimuli used during testing.

Participant Recruitment Strategies

• Clear strategies for recruiting participants should be outlined, including advertisements or social media posts; copies of these materials must also be submitted to the IRB.

Data Collection Instruments and IRB Approval Process

Overview of Data Collection Instruments

• The data collection instruments include various tools such as surveys and questionnaires designed to assess the ground truth.

• Hardware devices like audio-visual equipment, webcams, and physiological sensors are essential for collecting diverse types of data, including brain signals.

• Understanding the research method and analysis techniques (quantitative or qualitative) is crucial for developing a robust research proposal.

Importance of Research Protocols

• Detailed protocols for interviews, questionnaires, and focus group discussions must be established to ensure effective data collection.

• These documents serve not only for IRB submission but also aid in refining the research methodology.

Ethical Considerations in Data Collection

• It is unethical to begin data collection without obtaining IRB approval; researchers must wait for this approval before proceeding.

• Engaging with the IRB committee can help expedite the approval process if necessary.

Key Factors Reviewed by the IRB Committee

• The IRB evaluates whether risks to subjects are minimized or reasonable compared to anticipated benefits from the study.

• Selection of subjects must be equitable, ensuring no biases during participant recruitment.

Informed Consent and Vulnerable Populations

• Informed consent is mandatory for all participants; special considerations apply when involving minors or vulnerable populations.

• Plans must be in place to monitor data collection while ensuring privacy, confidentiality, and anonymity of subjects.

Preparing for Experiments Post-IRB Approval

• Additional safeguards may be required when conducting research with vulnerable groups such as children or individuals with behavioral disorders.

Understanding Study Design in Research

Development of the Study Concept

• The initial phase involves understanding the setup of the experiment, which is crucial for developing a solid study concept.

Types of Study Designs

• There are three primary types of study designs: within-subject design, between-subject design, and mixed model design. Familiarity with these concepts is beneficial for researchers.

Within-Subject Design

• In this design, the same participants are used to study different conditions. For example, comparing emotional responses in VR versus computer settings using the same group of users.

Between-Subject Design

• This approach involves different groups of participants for each condition. One group experiences VR while another uses a PC to analyze effects across distinct user sets.

Mixed Model Design

• A combination of both within and between-subject designs allows researchers to leverage advantages from both methodologies.

Considerations for Choosing a Study Design

• The choice of study design depends on various factors including participant availability and resource constraints. Each design has its pros and cons that should be explored further.

Group Structure in Studies

• Typically, studies may involve at least two groups: a control group (no emotion elicitation) and a treatment group (emotion elicitation). The number of independent variables influences this structure.

Determining Sample Size

• Sample size refers to the number of participants in each group. It's essential to determine an adequate sample size based on statistical power considerations.

Ideal Sample Size Guidelines

• A common rule suggests having at least 30 participants per variable studied as an ideal sample size for robust results.

Methods of Evaluation

• Evaluating emotional expressions can be approached through various methods categorized into five main types.

Self-assessment Method

Understanding Emotional Assessment in Research

Methods of Assessing Emotions

• Self-assessment is crucial; participants should articulate their emotions and the intensity they felt during the experiment.

• Conducting informal interviews post-experiment can provide insights into general emotional responses without direct questioning.

• Psychophysiological methods involve analyzing brain signals (e.g., EEG) to understand emotional expressions during specific conditions.

• Observational techniques allow researchers to analyze recorded videos of participants, often with expert annotations to gauge emotional responses.

• Task performance evaluation focuses on behavioral data, assessing how external stimuli (like music) affect participant performance on tasks such as solving arithmetic problems.

Evaluating Ground Truth in Experiments

• The ground truth can be established by comparing user performance under different conditions, such as positive versus negative music influences.

• A mixed-method approach may be beneficial; researchers can choose one or more of the five assessment methods based on their study's needs.

Study Location Considerations

• Experiment settings can vary from controlled lab environments to naturalistic settings, impacting data collection and participant behavior.

Understanding Emotion Recognition in Different Devices

The Impact of Device Type on Emotional Response

• Researchers have explored how emotions differ when users engage with content on mobile phones compared to computer screens, indicating that device type can influence emotional responses.

Addressing Hardware and Software Failures

• In studies involving multiple sensors, it's crucial to develop protocols for identifying potential failures during data collection and annotation processes.

• For instance, if one sensor fails while collecting physiological signals (e.g., heart rate), researchers should have contingency plans to proceed with the remaining functional sensors.

Participant Recruitment Criteria

• Establishing inclusion and exclusion criteria is essential for participant recruitment. Inclusion criteria might specify age ranges (e.g., 18-24 years old), while exclusion criteria could involve recent health issues like COVID-19.

• An example of exclusion criteria includes excluding participants who recently recovered from COVID-19 due to its potential impact on study outcomes.

Preparing for Study Execution

• After preparing Institutional Review Board (IRB) documents and obtaining approval, researchers can begin recruiting participants for their studies.

• Successful execution of the study will yield valuable data necessary for analysis.

Tools in Affective Computing

Categories of Tools Used by Researchers

• Various tools are utilized in affective computing research, aiding both beginners and experts by improving efficiency and performance.

Types of Tools Available

1. Data Collection Tools: Software designed to facilitate a wide range of experiments.

2. Data Annotation Tools: Assist in labeling collected data accurately.

3. Signal Analysis Tools: Analyze physiological signals gathered during experiments.

4. Affect Classification Tools: Primarily based on machine learning methods to classify emotional states.

5. Expression Tools: Help express or visualize the identified emotions.

Overview of Data Collection Software

Introduction to PsychoPy Tool

Overview of PsychoPy

• The speaker emphasizes the importance of creating customized scripts developed in-house for experiments.

• PsychoPy is introduced as a popular, free, and cross-platform tool used primarily by neuroscientists, psychologists, and linguists for conducting various experiments.

• It features a flexible and intuitive Builder interface that allows users to drag and drop elements to design experiments easily.

Experiment Design Features

• Users can select different types of stimuli (images, videos, music) and responses (e.g., keyboard input), with options to integrate additional sensors like EEG devices.

• The Builder interface supports integration with various input/output devices for enhanced experimental capabilities.

Programming Capabilities

• PsychoPy is based on Python, allowing users familiar with programming to utilize Python libraries for more complex experiment designs.

• For those less comfortable with coding, the Builder interface provides an accessible way to create trials without programming knowledge.

Data Collection and Annotation

Importance of Data Annotation

• After data collection through controlled experiments, data annotation is crucial as it establishes the ground truth necessary for machine understanding of emotions.

Tools for Data Annotation

• Various tools are available depending on the modality used; for instance, self-assessment tools like SAM (Self-Assessed Mannequins) are commonly employed in emotional assessments.

Modality-Specific Annotations

• Different modalities such as audio or video have specific tools designed for emotion annotation during user interactions.

SAM Mannequins: A Popular Tool

Functionality of SAM Mannequins

• SAM mannequins measure three dimensions: pleasure, arousal, and dominance based on the PAD model of emotion representation discussed in earlier lectures.

User Interaction Process

Self-Assessment and Data Collection Tools in Affective Computing

Self-Assessment Mode with Sam Mannequins

• The Sam mannequins are utilized for self-assessment, allowing users to rate their levels of pleasure, arousal, and dominance on a scale of nine.

• This method is specifically designed for self-assessment data collection, indicating its limitations in broader applications.

Signal Processing and Analysis Tools

• The third category of tools focuses on signal processing and analysis, which encompasses various modalities such as audio, physiological signals, images, and videos.

• These tools help analyze collected data by pre-processing signals to remove noise and extract relevant features from different types of modalities.

Popular Tools for Audio Analysis

• Notable tools like PR Open Ear, Open Smile, and WB Surfer are commonly used for audio signal analysis.

• These tools facilitate both the pre-processing of signals and feature extraction necessary for further analysis.

MediaPipe: A Framework for Image and Video Analysis

• MediaPipe by Google has gained popularity as a framework for analyzing image and video modalities; it powers functionalities behind Google Assistant interactions.

• It supports machine learning algorithms applicable across platforms including desktop servers and Android applications.

Features Offered by MediaPipe

• MediaPipe provides ready-to-use APIs that allow users to perform tasks like face detection without needing extensive coding knowledge.

• After detecting faces, users can analyze facial expressions to correlate them with emotions based on previously collected ground truth data.

Data Mining Tools in Emotion Analysis

• Following data collection and annotation processes using previous tools, researchers can employ machine learning techniques to understand emotions present in the dataset.

Overview of Machine Learning Tools

Introduction to VEA Tool

• The discussion begins with an overview of various machine learning tools, particularly focusing on the VEA tool, which is popular among practitioners in the field.

• VEA is an open-source software that provides a range of algorithms for different machine learning tasks, enhancing accessibility for users.

Features and Capabilities of VEA

• The tool supports multiple functionalities including data preprocessing, classification, clustering, association identification, feature extraction, and visualization.

• It features a user-friendly graphical user interface (GUI), allowing users to easily navigate through data files and select appropriate filters or techniques for analysis.

Algorithm Selection Process

• Users can choose specific algorithms based on their needs; for instance, selecting between decision trees or support vector machines for classification tasks.

• A foundational understanding of machine learning concepts is recommended to effectively utilize the tool; beginners are encouraged to pursue relevant courses or tutorials.

Importance of Emotion Analysis

• After analyzing data using tools like VEA, understanding emotions within the data becomes crucial. This emotional insight can guide how machines express these emotions in applications.

Tools for Emotional Expression

• The next category discussed involves tools designed for emotional expression in machines. These tools vary significantly in complexity and required expertise.

ICT Virtual Human Toolkit

• One notable tool is the ICT Virtual Human Toolkit, which aids developers in creating virtual human conversational agents with varying levels of sophistication.

Robotics Operating System (ROS)

• In contrast to simpler tools like ICT's toolkit, ROS requires more advanced knowledge but offers greater capabilities for robotics applications.

Exploring Available Resources

• The ICT Virtual Human Toolkit consists of modules and libraries tailored to assist researchers in developing interactive agents capable of natural language processing and non-verbal behavior analysis.

Customization Options

• Users can customize available agents by selecting from various options provided within the toolkit’s interface.

Conclusion on Tool Categories

• The session concludes with a summary highlighting different categories of tools available for data collection, annotation, analysis, classification, and emotional expression.