Women in AI

Welcome to the Ninth Unit of the Lecture Series

Introduction and Overview

• The speaker welcomes attendees to the ninth unit of a lecture series, highlighting collaboration with Women in AI Austria and the City of Vienna.

• The discussion will delve deeper into topics previously introduced by Julia Eisner, including goals of the association and issues surrounding algorithmic bias.

• Key questions include how biases are reproduced in AI development and what awareness is being raised regarding these issues.

Speaker Introductions

• Dr. Natalie Segur Kabernak is introduced as Policy Lead at Women in AI Austria and Data Protection Officer at Magenta Telekom.

• Mira Reisinger, a data scientist specializing in natural language processing (NLP), will share insights on trustworthy AI.

• Eugenia Stambolev, a postdoctoral researcher focusing on AI ethics and critical studies, is also part of the panel.

Engagement Encouraged During Discussion

Format of the Session

• Attendees are encouraged to participate actively by submitting questions via QR code during the lecture for later discussion.

• The session consists of three presentations followed by a collective discussion.

Presentation by Natalie Segur Kabernak

Overview of Women in AI Austria

• Natalie introduces herself as a young member involved in policy work within Women in AI Austria while also serving as a Data Protection Officer.

• Women in AI Austria is described as an interdisciplinary network primarily composed of women focused on diversity and equality within AI development.

Goals and Activities

• The organization aims to address gender equality throughout all stages of AI life cycles, emphasizing continuous growth and learning among its diverse members.

• They frequently engage as speakers at various events, showcasing their expertise while promoting diversity-related discussions.

Impactful Contributions to Society

Advocacy and Collaboration

• The organization has published policy papers outlining their positions on key issues related to women’s representation in technology fields.

Introduction to the Event

Overview of the Event

• The speaker invites attendees to engage with various events organized by their team, emphasizing a welcoming atmosphere filled with curiosity and joy.

• Transitioning to the first speaker, there is an acknowledgment of a shift in mood as they prepare to discuss more serious topics.

Ethical Considerations in AI Research

Introduction to Ethical Topics

• The speaker introduces the ethical aspects of AI research, indicating that multiple themes will be covered during the session.

• They highlight that discussions on algorithmic bias must also consider societal and power structures that contribute to these biases.

Gender Bias in AI

• The discussion begins on why AI systems often fail to recognize women, pointing out complexities in programming inclusive technologies.

• Citing extensive research, the speaker confirms the existence of gender bias within AI systems, referencing "Invisible Women" by Criado Perez as a significant work on this topic.

Understanding Algorithmic Bias

Mechanisms Behind Bias

• The speaker explains how existing societal biases are embedded into algorithms, leading to further marginalization of women and other groups.

• They note that data sets used for training AI often lack representation of women, which contributes significantly to biased outcomes.

Team Composition and Application Design

• Emphasizing that teams developing these technologies are predominantly male-dominated leads to applications not being designed with women's needs in mind.

The Nature of Artificial Intelligence

Defining Artificial Intelligence

• The speaker discusses misconceptions surrounding artificial intelligence (AI), suggesting it is often mischaracterized as purely intelligent when it involves complex data processing systems.

Complexity and Transparency Issues

• They argue that many view AI as lacking true intelligence due to its reliance on human input for functionality and decision-making processes.

The Broader Context of AI Development

Infrastructure Requirements for AI

• Highlighting that effective AI requires substantial infrastructure including data centers and energy resources beyond just datasets.

Narratives Surrounding AI Technology

• The conversation shifts towards how media narratives can misrepresent the scientific basis behind current developments in artificial intelligence.

Power Structures Related to AI

Economic Implications

Understanding Bias in AI and Its Ethical Implications

The Nature of Bias in AI

• Bias often arises from power structures, leading to ethical issues within AI systems. It is crucial to recognize that there isn't always an individual responsible for these biases; rather, they are systemic.

• Complexity in large systems can lead to negligence and poor judgments due to a lack of oversight and understanding of data sets, which may be too small or homogeneous.

Historical Context and Gender Representation

• Often, the needs of women are overlooked in applications because historical patterns continue without questioning their relevance today. This perpetuates existing inequalities.

• AI serves as a mirror reflecting structural inequalities rather than solely exacerbating them. Recognizing this can help address underlying societal issues.

Regulatory Developments

• The new AI Act aims to enhance accountability by ensuring more people scrutinize specific applications and interfaces, addressing bias before it leads to discrimination.

• Understanding bias in computer science differs from its interpretation in sociology or psychology; it can sometimes simply reflect the information provided to algorithms.

Addressing Systematic Discrimination

• Strategies must be developed not only technically but also socially to make AI more inclusive. Identifying when bias becomes systematic discrimination is essential.

• Feminist approaches can provide insights into how historical examples inform current practices, highlighting areas where women's contributions have been ignored.

Intersectionality and Collaborative Research

• Discrimination affects various groups beyond just women; thus, intersectional approaches are necessary for comprehensive understanding and solutions.

• Collaboration across different research fields can foster new areas of study that benefit multiple marginalized groups simultaneously.

Structural Counterstrategies

• Public discourse on these issues is vital for progress. Collective efforts can democratize both society and AI technologies.

Understanding Structural Counterstrategies in Team Design

Importance of Diversity in Teams

• The design of teams is crucial for addressing biases and discrimination, starting from organizational levels.

• Diverse teams can identify errors related to bias more effectively, as those affected by discrimination are often aware of their disadvantages.

Progress and International Efforts

• There is ongoing international work aimed at promoting diversity and networking within teams, which is essential for progress.

Introduction to Trustworthy AI Practices

Background of the Speaker

• The speaker, Mir Reisinger, introduces herself as a linguist focused on language technologies and discrimination in language use.

• She works with a startup in Vienna that specializes in fair and trustworthy AI.

Challenges with Unfair AI Systems

Example of Bias in Facial Recognition

• A screenshot from a video illustrates how facial recognition systems fail to recognize individuals with darker skin tones due to inherent biases.

• The software misidentified a person wearing a white mask without facial features but recognized them when they had darker skin.

Community Initiatives Against Discrimination

• The Algorithmic Justice League was founded by Dr. Joy Buolamwini to address these biases; she emphasizes community involvement in making discrimination visible.

Creating Framework Conditions for Fair AI

Knowledge Sharing and Visibility

• Establishing communities for knowledge exchange significantly impacts the fairness of AI systems by raising awareness about discrimination issues.

Regulatory Challenges

• Upcoming regulations like the AI Act aim to enforce fairness but pose challenges for companies trying to implement these standards effectively.

Steps Towards Fairness in AI Development

Integrating Fairness into Development Processes

• Companies must prioritize fairness throughout all stages of system development—from ideation through deployment and retirement.

Defining Problems Clearly

• It’s essential to define the specific problem an AI aims to solve; not every issue requires complex solutions or even AI technology.

Identifying Stakeholders Affected by Bias

Understanding User Impact

• Developers should consider who will use the system and who may be affected by it, ensuring that potential biases are addressed early on.

Questions for Consideration

How Does Facial Recognition Fail for People with Dark Skin?

The Inequality of Facial Recognition Technology

• The speaker discusses the inadequacy of facial recognition systems for individuals with dark skin, highlighting that such technology fails to recognize them, indicating a quality issue in the system.

• When training models, developers must consider what constitutes correct predictions and how to evaluate errors effectively.

• Developers optimize models based on their intended functions, emphasizing the importance of understanding these technical aspects during development.

Collaborative Problem Definition

• It is crucial for teams to collaboratively define problem descriptions and solutions rather than relying solely on data scientists or machine learning engineers.

• Fairness evaluation should be integrated into the solution process, underscoring the significance of teamwork in addressing issues related to AI fairness.

Diversity in Development Teams

• Diverse teams contribute varied experiences that enhance AI development; including decision-makers is vital for effective problem-solving approaches.

• If team members had different ethnic backgrounds, they might have identified flaws in facial recognition technology earlier through testing.

The Importance of Model Evaluation and Testing

Selecting and Evaluating Models

• Developers need to carefully select models while considering potential system steps involved in their application.

• Model evaluation is critical; testing often gets overlooked but becomes increasingly important when discussing fair AI and algorithmic fairness.

Statistical Fairness Testing

• There are various fairness tests available that can mathematically assess a system's performance regarding fairness metrics.

• Fairness definitions vary significantly; it’s essential to determine who benefits from fairness assessments as not all metrics can cover every context.

Understanding Explainability and Transparency

Key Aspects of Model Interpretability

• Important considerations include explainability, transparency, and interpretability when selecting models. This involves understanding input-output relationships within complex systems like LLMs (Large Language Models).

Linguistic Perspectives on AI Systems

• The complexity of input-output testing poses challenges for interpreting results from LLM outputs. Linguists may find insights from these systems' language processing capabilities intriguing.

Regulatory Implications for High-Risk AI Systems

Compliance with EU Regulations

• Companies developing high-risk AI must provide evidence demonstrating transparency and documentation as part of compliance with regulations like the EU Act.

Data Considerations in AI Development

Data Collection and Machine Learning Concepts

Importance of Data Quality and Expertise

• Emphasizes the significance of not just data appearance but also understanding who creates datasets and their expertise.

• Highlights the necessity for datasets to meet quality standards, being complete, representative, and balanced across groups.

Supervised vs. Unsupervised Learning

• Introduces the distinction between supervised learning (where output is known) and unsupervised learning (where insights are derived from large datasets without predefined outputs).

• In supervised learning, predictions can be made based on labeled data, such as classifying individuals into categories like student or employee.

Feedback Loops in Predictive Models

• Discusses feedback loops where models make predictions that influence real-world actions (e.g., predictive policing), potentially leading to biased outcomes.

• Warns about the dangers of reinforcing biases through continuous training on skewed data resulting from previous model predictions.

Hybrid Intelligence Concept

• Introduces "Hybrid Intelligence," emphasizing human decision-making over AI assistance while maintaining control and responsibility for decisions made with AI support.

• Expresses optimism about the future of AI, highlighting its potential when humans collaborate effectively with technology.

Challenges in Implementing AI in Organizations

Practical Considerations for Companies Using AI

Digital Regulations and Innovation in the EU

Overview of EU Digital Regulations

• The speaker discusses the numerous digital regulations introduced by the EU, particularly in relation to the Green Deal and Digital Strategy.

• Key regulations include those focused on platforms, cybersecurity, and data regulation, emphasizing the importance of compliance for companies operating in Europe.

• Companies face challenges in understanding and implementing these complex regulations, requiring specialized legal expertise.

Impact on Businesses and Employees

• Continuous changes in regulatory environments necessitate that businesses evolve while maintaining innovation and profitability.

• There is significant pressure on European companies to remain competitive against global counterparts, especially regarding new technologies.

GDPR and Data Regulation Evolution

• The General Data Protection Regulation (GDPR) is highlighted as a core area of focus; it governs personal data rights but is evolving towards broader data regulation frameworks.

• New regulations like the Data Act are being introduced, indicating a shift from solely personal data protection to encompassing non-personal data as well.

Emotional Response to Regulatory Changes

• The speaker references Kübler-Ross's grief model to illustrate how employees react emotionally to significant regulatory changes like GDPR or AI regulations.

• Initial reactions often include shock and denial, which can hinder adaptation within organizations.

Cultural Adaptation for Implementation

• Successful implementation of new regulations requires cultural adjustments within companies; employees must be prepared for change.

• Organizations need proper tools, processes, checklists, policies, and support systems established early on to facilitate smooth transitions during regulatory shifts.

Value of Data Post-GDPR

• Since GDPR's introduction, companies have recognized the value of their data assets; understanding where valuable data resides has become crucial for monetization opportunities.

Ethics and Compliance in AI Systems

Importance of Ethics in AI Development and Use

• Emphasizes the significance of ethics not only during the development but also in the deployment of AI systems, particularly concerning personal data.

• Highlights the necessity for transparency regarding personal data processed by AI systems, including its origin and the ability to modify or delete it as required by GDPR.

Data Governance and Employee Training

• Stresses the need for employee training on data governance rules, especially when using high-risk AI systems to ensure compliance with regulations.

• Discusses challenges related to data accessibility within organizations, emphasizing that all employees should have appropriate access to necessary data.

Data Classification and Confidentiality

• Advocates for classifying data based on confidentiality levels to protect sensitive information from unauthorized access.

• Notes that increased compliance responsibilities arise from implementing AI technologies, which can create anxiety among employees about job security due to automation.

Ethical Guidelines and Supplier Relationships

• Urges companies to define their ethical standards clearly, considering customer needs and product sourcing while ensuring supplier compliance with these standards.

• Points out that maximum transparency is essential under GDPR; companies must adapt contracts with suppliers to reflect this requirement.

Transparency in Agile Work Environments

• States that many companies transitioning towards agile methodologies are already achieving greater transparency, which is crucial for effective collaboration.

• Warns that while automation through AI can enhance efficiency, it may also lead to employee fears about job displacement.

Addressing Inequality in Access to AI Tools

• Highlights potential disparities in access to AI tools within teams, suggesting that those without access may struggle compared to their peers who do have it.

• Encourages organizations to be aware of these inequalities and work towards equitable access across all team members.

Risks Associated with Implementing AI Technologies

• Advises against adopting off-the-shelf AI tools without careful consideration; organizations should monitor technological developments closely.

Discussion on AI and Data Management

Concerns about AI in Contract Signing

• The speaker emphasizes the importance of ensuring that contracts or documents signed under the "six eyes principle" are not merely signed by AI, which could lead to liability issues.

Positive Outlook on AI Tools

• The speaker expresses optimism about utilizing tools that can help identify personal data across unstructured systems, highlighting the challenges faced by data protection officers in locating and classifying such data.

Upskilling Employees for Digital Transformation

• There is a strong emphasis on the need for employee training and development to align with digital strategies, focusing on where employees want to grow and what skills are necessary.

Importance of Data Strategy and Ethics

• A comprehensive data strategy and digital ethics guidelines are deemed essential. Training for data personnel is crucial so they understand their roles within regulatory frameworks.

Emotional Reactions to Technological Developments

Reflection on Recent Technological Changes

• The speaker connects emotional responses to technological advancements, referencing a course at the University of Vienna that explored democracy amid digital revolutions, particularly after the introduction of ChatGPT.

Collective Emotional Response to AI Integration

• The discussion highlights a collective experience of confusion and adjustment as educational institutions adapt to new technologies like ChatGPT during exam preparations.

Current Emotional State Regarding AI in Society

• The speaker questions where society stands emotionally regarding AI discussions, suggesting there is excitement but also an ongoing realization of significant changes ahead.

Comparative Analysis of Past Technological Shifts

Speed of Change Compared to Historical Context

• There's speculation about how quickly society will adapt to changes brought by AI compared to past technological shifts, noting that current developments feel more rapid than those seen with earlier innovations like spreadsheets in the 1970s.

Impact of Mobile Technology as a Game Changer

• The introduction of smartphones (e.g., iPhone in 2007) is likened to current advancements in technology; both have drastically changed daily life and accessibility.

Concerns Over Monopolization in Tech Industry

Questioning Human Replacement by Technology

• The speaker argues against framing discussions around whether AI will replace humans; instead, it should focus on how technology evolves alongside human roles within cultural contexts.

Issues with Corporate Power Dynamics

Discussion on AI and Bureaucracy

The Changing Landscape of Technology

• The rapid evolution of technology is creating a shift in the landscape, which isn't necessarily negative but highlights a lack of tools to manage monopolistic companies that dominate the internet.

• As a media scholar, there is curiosity about what differentiates AI from previous technologies like photography, raising questions about its unique impact.

AI as a New Form of Bureaucracy

• AI represents a new bureaucratic structure; it will be utilized by people rather than being used directly. This raises concerns regarding democratic data management.

• The high resource demands of AI must be considered, especially during climate crises, prompting discussions on whether more data centers are necessary.

Calls for Pausing AI Development

• Experts have suggested pausing AI development amidst ongoing excitement and concern. This proposal aims to bring calm into the current chaotic phase surrounding technology.

Market Dynamics and Hype Around AI

• There is skepticism about the feasibility of halting progress in AI due to market pressures; many companies are profiting significantly from advancements in this field.

• The hype around capabilities like ChatGPT may be waning as more individuals challenge dystopian narratives suggesting machines will replace humans.

Human-Centric Approaches to Technology

• Emphasizing human agency in developing technology can alleviate fears; not everyone needs to constantly consider how AI impacts their lives.

• Digital humanism was discussed as an approach that prioritizes human needs over technological advancement.

Concerns About Equality and Data Usage

Ensuring Equal Opportunities Through AI

• Discussions highlighted that while AI could promote equal opportunities through unbiased selection processes, challenges arise with limited training data available in Europe.

Risks Associated with Data Scarcity

• A potential issue arises when attempting to ensure fairness in training models; if certain demographics are underrepresented, it could lead to biased outcomes.

Historical Biases Affecting Model Training

• Historical biases can skew model predictions; for instance, Amazon's hiring algorithm favored male candidates due to historical applicant data leading to unintentional discrimination against women.

Ongoing Challenges with Data Representation

Discussion on Data Quality and Algorithmic Bias

The Loop of Data Quality

• The speaker discusses a feedback loop in data systems where declining data quality leads to self-referential issues, likening it to genetic traits that can become problematic over time.

Market Dynamics and Company Longevity

• There is an ongoing discussion about the sustainability of companies like OpenAI, questioning how long they will remain relevant while still disrupting markets.

Transformation in Journalism and Media

• A significant transformation is occurring within journalism and media due to evolving data values, with synthetic data playing a crucial role in this change.

Human Involvement in Data Processing

• Despite advancements, human labor remains essential for sorting and labeling data, emphasizing the unseen effort behind these processes.

Challenges of Balanced Datasets

• The speaker highlights the risk of introducing personal biases when attempting to create balanced datasets that do not reflect real-world complexities.

Concerns Over Algorithmic Fairness

AMS Model Controversy

• Reference is made to the AMS model which has been criticized for being sexist and discriminatory by categorizing unemployed individuals into groups based on their perceived chances.

Collective Response from Organizations

• The speaker mentions a collaborative letter being drafted by civil society organizations expressing concern over the AMS model's implications.

Importance of Societal Reflection

• The need for societal reflection on algorithmic systems is emphasized as a way to ensure ethical standards are maintained amidst technological advancements.

Historical Context of Algorithm Issues

• Discussion includes past failures with algorithms using historical data that perpetuated existing biases, highlighting the importance of transparency in algorithm development.

Successful Protest Movements

Discussion on AI and Social Sciences

The Role of Public Engagement in AI Development

• The speaker reflects on the importance of public protests by journalists and NGOs that led to a reconsideration of certain AI developments, indicating that there was no malicious intent behind initial decisions.

Concerns Regarding AI in Human Resources

• A concern is raised about the introduction of an AMS algorithm related to ChatGPT, highlighting its potential risks in human resources and labor markets under the EU's AI Act.

Qualitative vs. Quantitative Methods in Bias Detection

• A question is posed regarding the use of qualitative approaches, such as narrative analysis from social sciences, to uncover indirect biases not easily identified through quantitative metrics.

Interdisciplinary Collaboration for Fair Design

• The speaker discusses their organization's commitment to combining technical and social science methods within research projects like "fair design," emphasizing the need for diverse teams with varied expertise.

Challenges in Bridging Technical and Social Perspectives

• It is noted that integrating different methodologies can be resource-intensive but essential for addressing biases effectively; collaboration between technologists and social scientists is crucial yet challenging due to differing focuses on problems versus solutions.

Reflections on Gender Issues in AI

Learning from Other Disciplines

• The speaker emphasizes learning from feminist approaches and other disciplines that have previously highlighted similar issues, noting a disconnect between problem identification by social scientists and solution-oriented thinking by technologists.

Addressing Societal Reflections Through AI

• There’s acknowledgment that current discussions around sexbots reflect broader societal issues, with AI serving as a magnifying glass revealing underlying truths about gender-based violence rather than creating new narratives.

Organizational Structure and International Collaboration

Overview of Women in AI Austria's Network

Global Exchange and AI Ethics

The Nature of Global Exchange

• The speaker emphasizes that the topic of exchange transcends national borders, highlighting its global nature. They acknowledge their privileged position as a researcher in this field.

• There is a concern about the dominant North American focus in discussions around companies and technologies, suggesting it may overshadow other international perspectives.

Discussion on AI Bias and Ethics

• The speaker references Joy Buolamwini's work on bias in AI, specifically mentioning the documentary "Coded Bias" available on Netflix. They encourage viewers to watch it for deeper insights into discussed topics.

• A historical context is provided regarding artificial intelligence (AI), noting that the term was first coined in 1956 during a conference in the USA. This sets the stage for discussing ethics within AI.

Historical Context of AI and Ethics

• The speaker questions why ethical discussions are more prevalent today compared to earlier developments like image recognition software, indicating a shift towards generative AI as a focal point.

• They discuss how different fields have historically influenced public discourse on technology, with robotics being a significant area of research since the 18th century.

Military Influence on Technology Development

• The origins of many technological advancements, including the internet and GPS, are linked to military funding and interests in North America. This connection raises questions about underlying motives behind technological development.

• The concept of "intelligence" is explored beyond simulating human thought; it also relates to security systems, which complicates ethical considerations surrounding AI.

Ethical Considerations in Robotics

• Ethical concerns become more pronounced when robots enter social contexts rather than industrial settings. This transition prompts deeper inquiries into moral implications.

• The discussion highlights that while robotics has roots dating back centuries, current conversations are driven by substantial financial investments in technology.

Addressing Inequality in Technology Development

• A question arises regarding representation within tech development—specifically how to ensure fairness for women and marginalized groups amidst capitalist influences.

How to Shape the Future of the Internet?

The Evolution of the Internet and Its Impact

• The internet has transformed from a niche, cozy space in the 90s to a commercially driven platform dominated by global players. This raises questions about how to navigate this evolution healthily.

Women in STEM Fields

• There is an ongoing discussion regarding women's representation in STEM fields, particularly in technology and AI. This conversation builds on previous efforts to increase female participation in these areas.

• Many women are entering scientific fields like data science through alternative educational paths, often after pursuing other careers. This indicates a growing interest among women in tech-related disciplines.

Visibility and Representation

• Increasing visibility for women in technology is crucial. Events showcasing female professionals can inspire others and demonstrate that women are actively engaged in these fields.

• Creating spaces for women within organizations can significantly enhance their presence and encourage more females to apply for roles traditionally held by men.

Supportive Frameworks and Regulations

• Organizations can implement specific job designs aimed at attracting female candidates. Establishing supportive frameworks is essential for fostering women's interest in tech careers.

• Regulatory measures have proven beneficial, but enhancing visibility remains a key focus area for promoting gender equality within tech industries.

Engagement Opportunities with the Organization

• For those interested in connecting with the organization or participating in upcoming events, information is available on their website, including policy papers and LinkedIn profiles.

• Numerous events and seminars are scheduled across various regions of Austria, providing opportunities for individuals to engage with experts live.