8. előadás: E2E tesztelés és Accessibility

Understanding End-to-End Testing and Accessibility

Introduction to End-to-End Testing

• The speaker introduces the topic of end-to-end testing, indicating that it will cover two main subjects: end-to-end testing itself and accessibility.

• End-to-end testing is described as a comprehensive approach where everything from one end of the application to the other is tested, particularly relevant for web and mobile applications.

• The necessity of end-to-end testing arises because unit tests alone cannot ensure that all components work together seamlessly in an application.

• A key motivation for conducting these tests is to validate complete workflows within applications, ensuring they function correctly as intended.

Importance of User Interface Testing

• User interface (UI) elements are challenging to test without the underlying application logic; thus, UI tests are crucial for verifying functionality.

• End-to-end tests run in environments similar to production settings, allowing real-world scenarios to be simulated effectively during testing.

• Integration tests check interactions between different components but do not encompass full system behavior like end-to-end tests do.

Automation Support for Testing

• Automation plays a significant role in supporting end-to-end tests across various platforms including web, mobile, and desktop applications.

• The focus on UI support is emphasized since API testing can often be conducted with existing techniques.

Accessibility Considerations

• The second major topic discussed is accessibility—making applications usable for individuals with disabilities such as visual or hearing impairments.

• Accessibility aims to ensure that digital solutions cater to a broad audience, including those with physical limitations or temporary conditions affecting their ability to use technology.

Standards and Tools for Accessibility

• There’s an emphasis on creating accessible applications so that users with varying abilities can interact effectively with software products.

• Many software solutions are expected to meet certain accessibility standards; this includes considerations for users who may have temporary impairments or age-related challenges.

Common Requirements for Both Topics

• Both end-to-end testing and accessibility require specific tools and methods applicable across different platforms (web, mobile, desktop).

• Key requirements include observability of the application during both testing processes and ensuring users can navigate through interfaces effectively.

Understanding Application Observability, Control, and Testing

Key Concepts of Observability

• The operational system must have the capability to inform users about their current context within an application, such as indicating where to input a bank account number.

• Observability is crucial; it allows applications to communicate their state and actions effectively to users or external systems.

• Applications should be controllable from the outside, enabling automated interactions like filling in fields based on clipboard data.

• Verification is essential for testing; applications need to provide feedback on whether operations were successful or not.

• A well-observable application supports its verifiability, creating a strong link between observability and verification.

End-to-End Testing Objectives

• End-to-end (E2E) tests aim to evaluate the entire application architecture from start to finish, ensuring all components are interconnected properly.

• E2E testing includes verifying infrastructure setups by checking if resources like databases and servers function correctly after deployment.

• Successful login processes and data retrieval from databases are critical indicators of proper backend functionality during E2E tests.

• If an application can list all movies from a database upon logging in, it confirms that connections between frontend and backend are functioning as intended.

• Additional tests may be necessary for specific functionalities like retrieving comments once basic connectivity is confirmed.

Workflow-Based End-to-End Testing

• Another interpretation of E2E testing focuses on workflow completion; it assesses whether users can achieve their goals using the application without hindrance.

• An application's value lies in its ability to fulfill user tasks completely; partial success (e.g., reaching 95% of a process but failing at the end step) renders it ineffective.

• Identifying issues through comprehensive workflow testing is vital since failures indicate significant problems within the application's core functionalities.

Importance of Manual Testing and Automation in Software Development

Manual Testing as a Foundation

• Developers should manually test their work at least once to ensure correctness before moving on. This process can be documented, creating a test case.

• Once a manual test is established, it can be automated, leading to the creation of an automated end-to-end test that can be added to the testing database.

Benefits of Automated Testing

• Automated tests help prevent regressions by ensuring that previously functioning features remain intact after new changes are made.

• A software release with 25% new features but also losing 25% of existing functionality is ineffective; maintaining working features is crucial.

Regression Testing and Its Significance

• Automated tests are particularly effective for catching regressions since they cover complex workflows and provide broad coverage of the codebase.

• If a regression occurs, developers must determine whether to adjust the test or fix the underlying issue caused by new changes.

Critical Path Analysis in Testing

• Identifying critical components within a system is essential; these areas require thorough testing to avoid major failures (e.g., login functionality).

• Tests for critical paths are often slower than unit tests but are necessary for ensuring overall system reliability.

Smoke Tests and Their Role

• Smoke tests serve as preliminary checks to confirm that basic functionalities work before more extensive testing occurs.

• These tests are typically run frequently (e.g., on every commit), while other regression tests may only need daily execution or post-release checks.

Technologies Used in End-to-End Testing

Overview of Automation Tools

• Various platforms support web automation tools like Selenium and Playwright, which have been discussed previously.

• For mobile testing, Appium is highlighted as a key tool for automating mobile application tests across different operating systems.

API Testing Considerations

Desktop Application Testing Techniques

Overview of Desktop Testing Tools

• The discussion begins with an introduction to desktop application testing tools, specifically mentioning the Winup driver, which operates similarly to web drivers used in web solutions.

• Older technologies such as UVP, WinForms, and VPF are highlighted; these are not commonly developed today but still have applications in existing software.

• Appium is noted for its mobile testing capabilities but can also be utilized for any UI-based application, including Windows and macOS apps.

Challenges in UI Testing

• The speaker outlines challenges faced when testing UIs, particularly focusing on the fragility of tests due to changes in application design or layout.

• A key issue is that if elements within the application are rearranged or redesigned, it can lead to test failures. Solutions need to ensure stability despite these changes.

Specific Problems Encountered

• The first major problem identified is unstable testing caused by frequent changes in UI components that lead to test failures.

• Another challenge discussed is related to speed; unit tests can run concurrently without issues, but many applications do not support parallel execution effectively.

Parallel Execution Limitations

• The difficulty of running multiple instances of an application simultaneously is emphasized. Most applications are designed for single-instance operation due to user settings or resource constraints.

• Desktop applications may allow some level of parallel execution compared to mobile apps where running multiple instances is often impossible due to OS limitations.

Speed and Complexity Issues

• The complexity of preparing tests for end-to-end scenarios contrasts with simpler unit tests. End-to-end tests require a comprehensive setup that cannot easily fit on a single screen.

End-to-End Testing Challenges and Expectations

Complexity of End-to-End Tests

• The end-to-end (E2E) testing process can involve complex workflows, potentially consisting of 354 steps that require meticulous coding.

• E2E tests differ significantly from unit tests in structure; they often lack the straightforward "arrange, act, assert" format typical of unit tests.

• These tests can become lengthy as they cover multiple interactions within an application, such as clicking buttons and entering text across different pages.

• Preparing for E2E testing can be complicated due to dependencies on background systems and ensuring all necessary endpoints are accessible during the test execution.

• A robust framework is essential to address these complexities effectively, focusing on what functionalities should be expected from a testing framework.

Expectations from Testing Frameworks

• Key expectations for an E2E testing framework include automation capabilities for browsers, whether UI-based or headless.

• For mobile or desktop applications, UI-less versions cannot be tested; thus, frameworks must adapt to specific application types like Electron apps that support headless operation.

• Built-in waiting mechanisms are crucial in E2E tests to handle asynchronous operations effectively without causing failures due to timing issues.

• Understanding how synchronous versus asynchronous code operates is vital; unit tests typically do not face this challenge as they operate under predictable conditions.

• In UI testing scenarios, testers must prepare for interactions with separate processes (e.g., browser instances), which complicates the synchronization of commands issued by the test scripts.

Handling Asynchronous Behavior in UI Tests

• UI tests often involve multiple threads where actions like button clicks or text entries do not return immediately but rely on the application's event handling cycle.

• Testers issue commands that trigger events in the application's graphical thread rather than receiving immediate feedback upon command execution.

• This necessitates a clear understanding of how user interactions translate into events processed by the application’s main thread.

Application Testing and Screenshot Functionality

Waiting Mechanisms in Applications

• The process of integrating text into applications involves waiting for the application to reflect command results, indicating a need for built-in waiting mechanisms.

• A typical solution is a "glorified wild loop" that waits until the desired output appears, suggesting a preference for more efficient built-in wait functionalities.

Network Monitoring Considerations

• The discussion highlights that network monitoring and interception are not addressed in this seminar, although they are crucial when testing desktop or any applications.

Importance of Screenshots in Testing

• Screenshots are essential across various platforms (mobile, desktop), allowing testers to capture application states visually during interactions.

• There are two primary uses for screenshots: documenting errors when tests fail and providing visual evidence of application states during testing processes.

Use Cases for Screenshots

• In case of failures, screenshots can serve as proof of what went wrong at specific points in the testing process, aiding in reporting issues effectively.

• Visual documentation is also necessary for app submissions to stores like Google Play or Microsoft Store, where images help convey how the app looks to potential users.

Analyzing Test Failures with Screenshots

• If discrepancies arise between expected outcomes (e.g., images shown during testing vs. actual app behavior), automated systems will flag these inconsistencies without human intervention.

Utilizing Screenshots within Tests

• Screenshots can be integrated directly into tests as a method to verify UI elements against expected pixel-perfect designs; however, changes in themes may cause tests to fail due to pixel mismatches.

Efficiency through Visual Comparison

• For extensive data sets displayed on screens (e.g., 842 fields), taking screenshots allows testers to perform quick visual comparisons rather than checking each field individually.

Limitations and Challenges

Application Development and UI Testing Insights

Importance of Software Architecture in Application Development

• A good software architecture can simplify application setup, reducing the need for extensive configurations (e.g., 35,000 settings).

• The goal of UI tests is to interact with graphical elements effectively, ensuring that necessary actions like button clicks and text inputs are performed correctly.

Interaction with UI Elements

• UI testing involves searching for elements on the interface and interacting with them to avoid fragility in tests.

• Stable identifiers are crucial for reliably referencing UI elements during interactions.

Naming Conventions for UI Elements

• Each UI element should have a unique name; generic names like "label 1" or "button 1" can lead to confusion when multiple instances exist.

• Proper naming conventions help maintain clarity in code, especially when dealing with similar buttons across different forms.

Unique Identifiers for Application Elements

• Unique identifiers should be structured to reflect their context within the application (e.g., loginform.button.OK vs. userlogin.transfer.button.OK).

• These identifiers must be unique across the entire application to prevent conflicts and ensure accurate test automation.

Standards for Web Element Identification

• In web development, HTML tags serve as the foundation where attributes can be added for identification purposes.

• Custom attributes such as data-test-id are commonly used in various technologies to facilitate automated testing.

Accessibility Identifiers in Mobile Applications

• iOS applications utilize accessibility identifiers extensively, which aid in automating tests by providing clear references to UI components.

Understanding Custom Attributes in Application Development

Importance of Custom HTML Attributes

• The discussion begins with the significance of custom attributes, particularly custom HTML attributes, which enhance element identification within applications.

• These identifiers are crucial for automation platforms to locate and interact with UI elements effectively.

Challenges in Desktop Application Testing

• The speaker highlights the complexities associated with desktop application testing, noting that despite being the oldest platform, it presents significant challenges.

• A variety of frameworks exist for desktop development, complicating the testing process due to differing standards and practices across platforms.

Automation IDs and Framework Limitations

• Each UI element can have an automation ID assigned, which is a string used by automated testing frameworks to identify elements.

• The diversity of UI frameworks on Linux adds complexity; developers must navigate various windowing systems that affect application behavior.

Accessibility Standards: ARIA

• The acronym ARIA stands for Accessible Rich Internet Applications, emphasizing the need for applications to be accessible beyond just web contexts.

• The goal is to create applications that are easily accessible and usable by all individuals, including those with disabilities.

Principles of Accessible Application Design

• Four key principles guide the design of accessible applications: perceivable, operable, understandable, and robust.

• Perceivable: Applications must present information clearly so users can understand it.

• Operable: Users should be able to navigate and control the application effectively.

Enhancing Understandability through Alternative Text

• Providing alternative text (alt text) for images is essential; this practice has evolved from web development into desktop applications as well.

• Alt text allows screen readers to convey information about buttons or images to visually impaired users.

Input Handling and Navigation Control

User Interface Navigation and Usability

Importance of Virtual Keyboards

• Virtual keyboards are essential for accessibility, allowing users with disabilities to navigate applications without a physical keyboard.

• They can be controlled via voice commands, enhancing usability for those who may have difficulty using their hands.

Enhancing Application Operability

• Applications should support navigation through various input methods (e.g., tabs, enter keys), improving user experience significantly.

• Users expect logical navigation when filling out forms; for instance, pressing the tab key should move them to the next relevant field seamlessly.

Tab Order Management

• Proper tab order is crucial in application design; it allows users to navigate efficiently between fields without unnecessary clicks.

• Developers must assign integer values (tab indices) to UI elements so that they follow a logical sequence during navigation.

Focus Management in Applications

• Managing focus is vital, especially when modal windows appear; developers need to ensure that focus shifts appropriately between different windows.

• When a modal window opens, focus should automatically shift there unless specified otherwise, maintaining workflow continuity.

Understandable User Interfaces

• UI elements must be intuitive and clear; users should easily understand their functions upon interaction.

• Consistency in UI design is critical; similar actions or buttons across different contexts should yield predictable results to avoid confusion.

Robustness of User Interfaces

• UIs need to be resilient against future changes; they should function correctly despite updates or modifications in underlying systems.

Accessibility Standards and Guidelines

Importance of Browser Updates and Accessibility

• The speaker discusses the need for browser updates to ensure compatibility with new screen reader applications, emphasizing that applications should not rely on outdated versions.

• Reference is made to WCG standards, which compile principles for accessibility in technology, originally stemming from browser descriptions.

Overview of WCG Standards

• The WCG standard outlines expectations for application support regarding accessibility, detailing how applications can be designed to meet these standards.

• The speaker highlights that the WCG standard defines levels of preparedness for applications concerning automatic processing and usability by individuals with disabilities.

Levels of Accessibility Compliance

• Three distinct levels of accessibility are identified:

• Level 0: No accessibility features; usable only by sighted users.

• Level A: Minimal automation; includes alt texts and basic navigation aids.

• Level AA: Fully functional with assistive technologies; meets legal requirements in many jurisdictions.

Legal Implications and Validation

• Level AA compliance often aligns with legal expectations in the EU, where digital solutions may receive additional points in proposals if they meet accessibility standards.

• Automated validators exist to check web applications against these standards, identifying areas lacking proper descriptions or IDs necessary for accessibility.

Design Guidelines for Accessibility

• The guidelines suggest ensuring strong color contrasts between text and backgrounds to enhance readability.

• Specific contrast ratios (e.g., a minimum of 4.5:1) are recommended to avoid poor visibility issues that could hinder user experience.

Common Design Pitfalls

• An example is provided illustrating poor design choices, such as using low contrast colors (e.g., orange text on a red background), which can lead to significant readability issues.

Discussion on Frustration and Improvement Suggestions

Addressing Frustration and Stress

• The speaker discusses the presence of frustration and stress among individuals, which leads to suggestions for improvements in various applications.

• Emphasizes the importance of testing web and mobile applications to identify areas needing enhancement or where issues have arisen.

Conclusion of the Session

• The speaker indicates that they have reached the end of their prepared material, leaving a little over 10 minutes for questions.