Trustworthiness of AI models: Improving NLP with Causality

Name: Trustworthiness of AI models: Improving NLP with Causality
Uploaded: 2021-11-13T18:41:06.000Z
Duration: 2 h 6 min 32 s

Introduction and Welcome

The speaker, Hayah, is introduced and welcomed to the episode. She expresses her excitement to talk to the audience and shares her interest in the subject.

Hayah's Introduction

Hayah is excited to be a speaker on the episode.

She saw a blog post by Hayah that caught her attention.

The host is happy to have Hayah as a speaker.

Explanation of Navigation Bar and Q&A

The host explains how the navigation bar works and encourages viewers to ask questions during the session.

Navigation Bar and Q&A

There is a navigation bar for easy access.

Viewers can use the Q&A feature to ask questions at any time.

No question is considered silly, all questions are welcome.

Location of Speaker

The host asks about Hayah's location.

Location of Speaker

Hayah is based in Toronto.

This is the first time they have had someone from Toronto as a guest.

Professional Background of Speaker

The host asks about Hayah's professional background.

Professional Background

Hayah works as a data scientist.

Currently consulting with the World Bank on various projects.

Enjoys the challenging nature of the work outside of corporate settings.

Educational Background in Data Science

The host asks about Hayah's educational background in data science.

Educational Background

Hayah is self-taught in data science.

Initially pursued an undergraduate degree in biology with plans to become a veterinarian.

Discovered coding and machine learning along the way, which sparked her interest.

Passion for Data Science and Machine Learning

The host discusses Hayah's passion for data science and machine learning.

Passion for Data Science

Hayah fell in love with machine learning after discovering it.

Excited about the constant developments in the field.

Considers data science and machine learning as a long-term career path.

Poll to Understand Audience Background

The host conducts a poll to understand the audience's level of knowledge and profession.

Poll Results

The poll aims to gather information about the audience's background.

Viewers are encouraged to share their profession in the chat.

Results show a mix of data scientists and project managers, with some participants new to NLP.

Interest in Causality and NLP

The host asks Hayah about her interest in causality and NLP.

Interest in Causality and NLP

Hayah has always been curious about asking "why" throughout her life.

Recently discovered the mathematical aspect of causality, particularly econometrics.

Fascinated by deriving causes and exploring causal inference using NLP techniques.

Blog Posts on Causality

The host mentions that Hayah has written blog posts on causality.

Blog Posts on Causality

Hayah has written two blog posts on causality so far.

Another blog post is planned but may be challenging to complete.

Conclusion of Poll Results

The host concludes the poll results before moving forward with the discussion.

Conclusion of Poll Results

Most participants have answered the poll questions.

A variety of professions are represented in the audience.

The level of knowledge in NLP is split between novice and experienced participants.

Invitation for Questions

The host invites viewers to ask questions and assures them that all questions are welcome.

Invitation for Questions

Viewers are encouraged to ask any questions they may have.

The host and Hayah are happy to answer them.

No need to worry about asking questions, as it is appreciated.

Conclusion of Section

The host concludes the section, expressing gratitude for the viewers' participation and excitement for the upcoming discussion.

Conclusion of Section

The poll results have been shared.

Appreciation is expressed for the viewers' participation.

The host looks forward to discussing various topics with Hayah.

Bias in AI

The speaker starts with a quote about bias in AI, emphasizing how algorithms can amplify existing prejudices. They highlight the importance of designing AI systems carefully to avoid encoding biases and discuss personal concerns about flawed tools used as data scientists.

Algorithms act like convex mirrors that bluntly reflect human biases and do not allow for polite fictions.

Poorly designed AI systems can encode and potentially amplify societal prejudices.

Concerns are raised about flawed tools used by data scientists that are difficult to interrogate and change.

Transparency and awareness of what is happening in AI discussions are crucial.

Importance of Causality in NLP Models

The speaker explains their interest in applying causality to NLP models and why this topic is important. They express concern about the flaws in current tools used as data scientists and invite others to share similar concerns. The discussion goal is highlighted as transparent AI.

Personal interest sparked by the connection between causality and NLP models.

Flaws in current tools used as data scientists are concerning due to difficulty in interrogating and changing them.

Transparent AI is essential, requiring awareness of stakeholders about what is happening.

Invitation for a casual conversation on improving NLP models using causality.

Introduction to Talk Structure

The speaker outlines the structure of their talk, which consists of five sections. They provide an introduction followed by an overview of causality, then focus on improving specific aspects of NLP models such as robustness, fairness, bias, and explainability. Case studies will be used for examples, with relevant literature citations shared at the end.

Talk divided into five sections: introduction, causality overview, improving specific aspects of NLP models.

Case studies and literature references will be used for examples and further reading.

Encouragement to ask questions and create a casual conversation atmosphere.

Importance of NLP and AI Language Models

The speaker explains the significance of NLP and AI language models in our daily lives. They mention how we rely on products built with NLP, such as search engines like Google and Bing, which use AI language models like BERT and GPT. The progress made in natural language processing over the years is highlighted.

Daily interaction with products built with NLP, such as search engines.

Google Search powered by BERT, Microsoft Bing uses OpenAI's GPT.

Progress in natural language processing has been revolutionary for businesses and researchers.

Bias, Robustness, and Explainability Issues

The speaker discusses the prevalent issues faced by NLP models, including bias, robustness, and explainability. They mention how powerful NLP models can harbor biases towards historically marginalized groups. Additionally, they highlight vulnerabilities related to generalization during deployment and concerns about black box nature in fields like healthcare.

Powerful NLP models can harbor biases towards marginalized groups.

Vulnerabilities related to generalization during deployment affect model performance.

Concerns about black box nature raise trust issues in fields like healthcare.

Trusting Unexplainable Tools

The speaker emphasizes the increasing reliance on machines for everyday decision-making. They mention scenarios where tools like Google Maps provide route recommendations or when an NLP model ranks job candidates based on resumes. Trust becomes a concern when tools are unexplainable or opaque.

Increasing reliance on machines for decision-making.

Trust concerns arise when tools are unexplainable or opaque.

Examples include route recommendations and job candidate ranking.

Impact of Biased NLP Models

The speaker presents a hypothetical scenario where a biased NLP model could lead to unfair outcomes, such as female applicants being discriminated against in job opportunities. They recommend reading "Algorithms of Oppression" by Sophia Noble for an in-depth understanding of bias reinforcement in search engines.

Hypothetical scenario: biased NLP model leading to discrimination against female applicants.

Recommendation to read "Algorithms of Oppression" by Sophia Noble for insights on bias reinforcement in search engines.

Timestamps have been associated with the corresponding sections based on the provided transcript.

New Section

In this section, the speaker introduces the concept of causality and its importance in understanding cause and effect relationships. They recommend a book called "The Book of Why" by Judea Pearl as an introductory resource on causality.

Understanding Causality

Causality refers to cause and effect relationships.

Traditional machine learning focuses on prediction based on correlations, but causal inference requires determining cause by eliminating other possibilities.

Randomized trials are commonly used in medicine to establish causal relationships.

The speaker will provide an overview of causality and its application to NLP models.

New Section

This section provides a general introduction to four key concepts related to causality: counterfactuals, potential outcomes, spurious correlations, and causal graphs. These concepts will be connected with an NLP case study.

Key Concepts in Causality

Counterfactuals are hypothetical scenarios that explore what would have happened if different choices were made.

Potential outcomes framework describes a statistical model of cause and effect with counterfactual conditionals.

Spurious correlations refer to misleading associations between variables that do not imply causation.

Causal graphs are graphical representations that depict causal relationships between variables.

New Section

This section explores the concept of counterfactuals further by using a hypothetical scenario involving choosing between art and music in school. It explains how interventions play a crucial role in causal inference.

Understanding Counterfactuals and Interventions

Counterfactuals involve imagining what would have happened if different choices were made.

Interventions are necessary for defining counterfactuals and establishing causal relationships.

An intervention involves changing the treatment or choice being studied while keeping other variables constant.

Causal effects are defined as the magnitude of change in an outcome variable due to a unit-level interventional change in treatment.

The average treatment effect (ATE) describes the difference in outcomes between the real world and the counterfactual world.

New Section

This section discusses the fundamental problem of causal inference, which is the inability to directly observe counterfactual outcomes. It introduces the potential outcomes framework attributed to Imbens and Rubin, who developed methodology for causality.

The Fundamental Problem of Causal Inference

Counterfactual outcomes are impossible to directly observe, leading to a missing data problem.

The potential outcomes framework provides a statistical model for cause and effect with counterfactual conditionals.

Imbens and Rubin's work on developing methodology for causality earned them a Nobel Prize in Economics.

The book "Mostly Harmless Econometrics" by Angrist is recommended as an introduction to causality.

New Section

This section further explains the potential outcomes framework, which involves describing cause and effect using counterfactual conditionals.

Understanding Potential Outcomes

The potential outcomes framework describes cause and effect using two potential outcomes: one when treatment is applied and another when it is not applied.

Counterfactual conditionals are statements that would be true under different circumstances.

The framework helps analyze causal relationships by comparing factual observations with their corresponding counterfactuals.

Ignorability and Conditional Ignorability

This section discusses the concept of ignorability in causal inference and the need for conditional ignorability when random assignment is not feasible or ethical.

Ignorability

Ignorability requires that a treatment assignment be independent of the realized counterfactual outcomes.

Random assignment is usually used to achieve ignorability.

In observational data, random assignment may not be feasible or ethical.

In such cases, conditional ignorability is necessary.

Conditional Ignorability

Conditional ignorability assumes that there are no unobserved confounders.

Confounders cause spurious correlations and can lead to biased results.

Ignorability means that we can ignore how an individual ended up in one group versus another when calculating potential outcomes.

Omitted variable bias should be avoided.

Consistency Assumption

This section explains the consistency assumption in causal inference and its implications for observed outcomes at different treatment statuses.

Consistency Assumption

Consistency requires that the observed outcome at a given treatment status for a given unit is the same as would be observed if that unit was assigned to the treatment.

Also known as the stable unit treatment value assumption (SUTVA).

Two assertions of consistency:

No interference: The outcome for a unit is affected only by its own treatment status, not by the treatment status of other units.

Only one version of treatment: There is only one choice to be made, not both or none.

Positivity Assumption

This section discusses the positivity assumption in causal inference and its importance in ensuring a range of treated and untreated units are observed.

Positivity Assumption

Positivity assumes that the probability of receiving treatment is bounded between zero and one.

It requires the ability to observe units with and without treatment.

Positivity also implies that treatment status cannot be perfectly predicted given the observed covariates.

The presence of exogenous factors should prevent perfect prediction.

Spurious Correlations and Confounding

This section explains spurious correlations and confounding in causal inference, highlighting their differences and implications.

Spurious Correlations

Not all spurious correlations are confounders, but all confounders lead to spurious correlations.

Spurious correlation example: Regressing poor health outcomes on chewing gum may show a positive relationship, but it is a spurious correlation as chewing gum does not cause poor health.

Confounding

Confounders have an effect on both the treatment and the outcome.

Example: In smoking and poor health outcomes, diet or exercise can act as confounders affecting both variables.

Confounding occurs when a variable has an effect on both the treatment and outcome, causing a spurious correlation.

Causal Graphs (DAGs)

This section introduces causal graphs or directed acyclic graphs (DAGs) as tools for understanding causality in causal inference.

Causal Graphs (DAGs)

Causal graphs are graphs with direction indicated by arrows and no feedback loops or cycles.

DAGs help determine causality by visualizing relationships between variables.

DAGs were popularized by computer scientist Judea Pearl in his book "The Book of Why."

Israeli Army Checkpoints and Unemployment in Palestine

This section discusses the potential impact of Israeli army checkpoints on unemployment rates in Palestine and the resulting cycle or feedback loop of violence.

Impact of Checkpoints on Unemployment

Israeli army deploying more checkpoints in Palestine could lead to greater unemployment for Palestinians.

Higher unemployment rates in Palestinian towns could potentially lead to more violence.

The causal effect between Israeli checkpoints and Palestinian employment becomes difficult to distinguish due to the presence of a feedback loop.

Using Instrumental Variables for Causal Inference

This section explains how instrumental variables can be used to address the challenge of distinguishing causal effects between Israeli checkpoints and Palestinian violence on employment.

Abraham's Solution with Instrumental Variables

Abraham proposes using a clever instrumental variable approach to determine causality.

By introducing an instrumental variable, it becomes possible to differentiate whether it is the Israeli checkpoints or Palestinian violence that causally affects Palestinian employment.

A paper discussing this approach is linked at the end for further reference.

Directed Acyclic Graphs (DAGs) in Causality

This section introduces directed acyclic graphs (DAGs) as a graphical representation of causal relationships.

Understanding DAGs

A DAG consists of vertices (nodes) and edges, where each edge is directed from one vertex to another without forming a closed loop.

DAGs represent causal relationships without feedback loops.

A causal DAG example is presented, where treatment (T) represents the quality of a text or email, while Z represents confounding qualities such as topic or writing style.

The text itself is denoted as W, and other covariates are represented by X.

Arrows in the DAG indicate causal effects and correlations between variables.

Causal Effects in DAGs

This section explains how causal effects are represented in DAGs and the distinction between true causal effects and spurious correlations.

Representation of Causal Effects

The arrow from treatment (T) to text (W) represents the true causal effect of the treatment on the outcome.

The arrow from confounding variable Z to text (W) represents a spurious correlation, as Z is a confounder.

Changing the value of treatment (T) can alter the distribution of the outcome (Y).

Clarifying Causality and Spurious Correlations

This section clarifies the concept of causality and distinguishes it from spurious correlations.

Understanding Causality

Causality refers to a genuine causal relationship between variables, not just a correlation.

An example is given where poor health outcomes may seem correlated with chewing gum, but there is no actual causal effect.

Social science and other sciences, such as medicine and epidemiology, use causality to understand what causes certain phenomena.

Applying Causality to NLP - Case Study

This section discusses applying causality to improve natural language processing (NLP), specifically in a case study related to medical diagnosis prediction.

Case Study: Medical Diagnosis Prediction

The case study involves building a classifier to predict medical conditions based on written clinical narratives.

Multiple hospitals provide labeled training data for this task.

Variables include hospital (Z), physician-assigned diagnosis label (Y), written clinical narrative (W), and classifier prediction (Y-hat).

The challenge is to intervene on the hospital variable while holding the diagnosis label fixed, to obtain counterfactual narratives and predictions.

Spurious Correlations in Medical Diagnosis Prediction

This section highlights the issue of spurious correlations between writing style and medical diagnosis predictions in the case study.

Spurious Correlations and Out-of-Distribution Performance

Different hospitals may have different tendencies for diagnosing specific conditions.

Writing styles used at specific hospitals can influence clinical narratives.

Building a correlational predictive model without considering these factors can lead to poor out-of-distribution performance.

The challenge lies in addressing the spurious correlation between writing style and medical diagnosis predictions.

The summary has been provided in English as per your request.

New Section

This section discusses the issue of spurious correlation between hospital z and label y in training data, which leads to a lack of robustness in predictors.

Correlation Between Hospital Z and Label Y

The training data shows a spurious correlation between hospital z and label y.

Predictors learn the parts of x that are informative of hospital z, but this relationship does not hold during deployment.

Lack of robustness in predictors can lead to poor generalization.

New Section

This section introduces the dimensions of improvement for NLP models: robustness, fairness, and explainability. It focuses on discussing robustness.

Dimensions of Improvement for NLP Models

The three dimensions for improving NLP models are robustness, fairness, and explainability.

Robustness refers to a model's ability to generalize well to inputs different from the training data.

Testing the model on data from hospitals not included in the training data is an example of assessing its robustness.

Lack of robustness can result in poor performance when applied to novel data.

New Section

This section explains what it means for a model to be robust and provides examples related to medical diagnosis classifiers.

Understanding Robustness

A model is considered robust if it can generalize well to inputs different from the training data.

In the context of medical diagnosis classifiers, testing the model on data from hospitals not included in the training data is crucial for assessing its robustness.

Lack of racial representation in training data can lead to poor performance when diagnosing anxiety in patients from different racial backgrounds.

Overfitting to training data results in poor generalization.

New Section

This section explores how causality can address the issue of spurious correlation and improve model generalization.

Causality and Model Generalization

AI language models often make errors in out-of-distribution settings due to their reliance on spurious shortcuts.

Sensitivity and invariance tests are used to assess robustness.

Incorporating domain knowledge of causal structure into learning objectives can improve model performance.

Counterfactual data augmentation is one approach that utilizes causally motivated methods for improving models.

New Section

This section explains sensitivity and invariance tests as evaluations of counterfactuals to ensure predictions are not based on wrong reasons.

Sensitivity and Invariance Tests

Invariance tests assess whether a predictor behaves differently when given counterfactual inputs.

Counterfactual inputs involve changing a causally irrelevant factor, such as the label of a cause, to test the predictor's behavior.

Models with invariant predictions across counterfactuals tend to perform better on test distributions with different relationships between variables.

Sensitivity tests change the label while holding other causal influences constant to evaluate minimal changes necessary for switching the true label.

New Section

This section discusses how economists use sensitivity and invariance tests in regression setups, highlighting their relevance for improving models.

Sensitivity and Invariance Tests in Economics

Economists use regression setups with potential confounders as controls to test for sensitivity and invariance.

Adding controls ensures that regression coefficients remain stable, indicating robustness.

Stable coefficients provide evidence of invariant results despite changes in control variables.

While economists aim to prove causality, the focus here is on utilizing causally motivated approaches to enhance model performance.

New Section

This section introduces counterfactual data augmentation as a causally motivated approach for improving models.

Counterfactual Data Augmentation

Counterfactual data augmentation is a method that utilizes counterfactuals to improve model performance.

By generating samples with counterfactual inputs, models can learn from different causal scenarios.

Causally motivated approaches, such as counterfactual data augmentation, leverage sensitivity and invariance concepts to enhance models' robustness and generalization.

Understanding Counterfactuals in NLP

In this section, the speaker explains the concept of counterfactuals and how they are used in natural language processing (NLP).

What is a Counterfactual?

A counterfactual refers to imagining what would have happened if a different choice had been made.

In the context of NLP, a counterfactual observation considers what would have happened if a certain treatment or action was not applied.

Counterfactual Data Augmentation

Counterfactual data augmentation involves constructing counterfactual instances and incorporating them into the training data.

This method can be used for both invariance and sensitivity.

For invariance, a learning objective term can be added to penalize disagreements in predictions for counterfactual pairs.

Sensitivity testing involves augmenting the training data with label counterfactuals to test for sensitivity to noise and improve generalization.

Generating Counterfactual Examples

There are several ways to generate counterfactual examples:

Manual post-editing: Accurate but expensive.

Heuristic replacement of keywords: Works in some scenarios but lacks coverage and generalizability.

Automated text generation: Ideally combines accuracy, coverage, and speed. However, it may introduce spurious correlations.

Spurious Correlations and Data Augmentation

All three options for generating counterfactual examples carry the risk of introducing spurious correlations.

Keyword substitution approach may introduce new spurious correlations if lexicons are incomplete.

Manual editing and automated text generation can also introduce new spurious correlations during the generation process.

Fairness and Bias in AI Models

This section discusses fairness and bias in AI models, specifically focusing on protected attributes and the concept of fairness through unawareness.

Protected Attributes and Fairness

Causality provides a language for specifying desired fairness conditions across demographic attributes like race and gender.

Anti-discrimination law requires AI models that do not perpetuate existing inequalities, particularly regarding protected attributes.

Fairness through unawareness suggests ignoring protected attributes, but this approach is ineffective due to redundant encodings.

Counterfactual Fairness

A fairness metric can be motivated by causal interpretations of the data generating process (DGP).

Counterfactual fairness is defined as a predictor being fair if its prediction remains the same for a counterfactual version of an individual created by changing a protected attribute.

Historical data may contain biases based on these protected attributes, making counterfactual fairness necessary.

The summary has been provided in English as requested.

Counterfactual Reasoning and Intervention

This section discusses the concept of counterfactual invariance in prediction and the legitimacy of treating certain protected attributes as variables that can be subject to counterfactual reasoning or intervention. It also explores the use of observable proxies, such as names, to test for invariants.

Counterfactual Invariance and Protected Attributes

In a counterfactual world, changing a protected attribute like race should have a counterfactual invariance in prediction. However, there are questions about the legitimacy of treating certain protected attributes as variables that can be intervened upon.

While race and sexuality cannot be changed, there is an ethical question about whether gender and religion should be treated as something that can be intervened upon.

Observable proxies, such as names, have been suggested as a way to test for invariants.

Counterfactual Data Augmentation

Counterfactual data augmentation has been used to reduce bias in pre-training models like BERT.

The extent to which biases in pre-trained models propagate to downstream applications remains unclear.

The role of counterfactual data augmentation is to reduce the model's reliance on spurious correlations for predictions.

Explainability and Interpretability of NLP Models

This section focuses on the explainability and interpretability challenges posed by NLP models like BERT. It explores the need for diagnosing errors and building trust by providing both explainability of the method and interpretability of the results.

Challenges with NLP Models like BERT

NLP models like BERT act as inscrutable black boxes, making them difficult to explain and interpret.

There is a need for diagnosing errors so that decision-makers can trust the results.

Both the explainability of the method and interpretability of the results are necessary.

Approaches to Explainability

One common approach is to exploit network artifacts, such as attention weights, to generate explanations. Attention weights are computed along a path to generating a prediction.

Another approach is to estimate simpler and more interpretable models by using perturbations of test examples or their hidden representations.

However, both attention-based explanations and perturbation-based methods have limitations that can make them misleading.

Sensitivity and Invariance for Model Interpretation

This section discusses sensitivity and invariance as approaches for interpreting models. It highlights the importance of generating counterfactuals with minimal changes needed for different model predictions.

Sensitivity Approach

The sensitivity approach aims to generate counterfactuals with minimal changes required to obtain a different model prediction.

It improves explainability by highlighting the changes needed to change a model's prediction.

Plausible counterfactuals and diversity of counterfactuals are necessary for identifying invariances.

Invariant Approaches

One invariant approach involves generating counterfactual examples using data augmentation and comparing predictions for each sample with predictions from generated counterfactual examples.

Another invariant approach uses contrast sets to evaluate a model's decision boundary. Test instances are manually perturbed in small but meaningful ways to create these contrast sets.

These approaches aim to improve the interpretability of models by incorporating sensitivity and invariance tests into the model development process.

Manipulating Text Representations for Interpretability

This section explores an alternate approach where text representations are manipulated instead of changing the text itself. It introduces the concept of a counterfactual language representation model that aids in interpretability.

Counterfactual Language Representation Model

Instead of changing the text, the representation of the text is manipulated.

A counterfactual language representation model can be created by pre-training an additional instance of a language representation model like BERT with an adversarial auxiliary task that controls for confounding concepts.

This approach allows for interpretability by comparing counterfactual representations.

Conclusion

The transcript concludes by acknowledging the inspiration from a survey paper and emphasizes the importance of incorporating invariance and sensitivity tests into the model development process for greater interpretability and explainability.

New Section

In this section, the speaker thanks the audience for listening and invites them to connect on LinkedIn for any further questions.

Connecting with the Speaker

The speaker thanks the audience for listening and expresses hope that the talk was interesting.

The audience is encouraged to reach out on LinkedIn if they have any unanswered questions or want to connect.

New Section

In this section, a question from Brandon is addressed and the possibility of using counterfactual data augmentation is discussed.

Counterfactual Data Augmentation

Brandon asks about swapping words in counterfactual data augmentation.

The speaker explains that there are different approaches to counterfactual data augmentation, including a keyword-based approach.

Swapping words for their opposites can be one way to apply counterfactual data augmentation, especially when addressing gender bias.

However, keyword approaches may not work in all situations, and manual editing is currently considered the best option.

Brandon suggests using generic terms instead of opposite words, but the speaker explains that eliminating mentions of certain attributes may not eliminate bias entirely.

New Section

In this section, Brandon continues his discussion on counterfactual data augmentation and its limitations.

Limitations of Keyword Approach

Brandon asks if using generic terms instead of opposite words would achieve similar results in counterfactual data augmentation.

The speaker explains that even ignoring protected attributes like race may not eliminate bias completely because it can still be predicted through other features.

Keyword approaches have limitations and do not generalize well across languages or complex datasets with multiple attributes.

New Section

In this section, feedback from the audience is shared and plans for sharing code examples are discussed.

Feedback and Code Examples

The speaker acknowledges positive feedback from the audience regarding the presentation.

Plans for sharing code examples are mentioned, but currently, there are no specific code examples available.

The speaker directs the audience to an automated generation framework called Taylor for generating counterfactuals using perturbations.

New Section

In this section, the speaker expresses gratitude to the audience and discusses the availability of the recorded talk.

Gratitude and Availability of Recorded Talk

The speaker thanks everyone for attending and expresses enjoyment in discussing NLP.

The intensity of the talk is acknowledged, and it is mentioned that the recording will be uploaded on YouTube for those who want to watch it again.

Appreciation is expressed towards the organizers for their preparation and patience.

New Section

In this section, final remarks are made by both the speaker and a participant named Peter.

Final Remarks

Both participants express their enjoyment of the talk and mention that some parts may require rewatching or further reflection.

Thanks are given once again to all attendees, and well wishes are extended for a nice day or evening depending on their time zone.