3 Тест-дизайн: Стратегии, техники и практические кейсы для тестировщиков

Introduction to Test Design

Overview of the Session

• The session begins with a confirmation that participants can see the screen.

• The focus will be on test design and techniques related to it.

Importance of Test Design

• Test design is an ongoing activity for testers, essential for effective testing practices.

• Awareness in test design is crucial; without it, practical application may not be correct.

Key Components of Test Design

Initial Stages of Testing

• One of the initial stages in software testing involves planning and designing test cases based on predefined quality criteria.

Goals of Test Design

• The primary goal is to write tests for applications while ensuring they are minimal yet sufficient.

• Writing numerous tests does not guarantee quality; fewer, well-designed tests can cover more user expectations effectively.

Analyzing Requirements

Understanding Requirements

• Analyzing requirements is critical; without understanding them, writing effective tests becomes challenging.

Risk Assessment

• Evaluating potential risks associated with product usage is vital for identifying what needs testing.

Practical Considerations in Testing

Realistic Testing Scenarios

• Often testers create complex scenarios that may not reflect real-world usage, leading to ineffective testing outcomes.

User-Centric Focus

Understanding Testing Techniques

Regression and Sanity Testing

• The discussion begins with the concept of regression testing, noting that it doesn't always apply uniformly to all scenarios.

• Introduction of "sanity testing," which involves thorough examination of functionality before its first production release.

• Emphasis on checking all sub-functionalities deeply to identify potential issues prior to deployment.

• Differentiation between basic and extended sanity tests, highlighting their roles in ongoing test design processes.

• Clarification that some tests are conducted only once before releasing functionality into production.

Test Design Considerations

• Importance of decomposing applications and functionalities for effective test design is discussed.

• Dekomposition (decomposition) is defined as breaking down functionality into smaller parts for better understanding and testing.

• Acknowledgment that complete technical specifications (TЗ) may not always be available, necessitating independent research for information.

• Mention of the need to anticipate user actions, such as confirming deletions, even when detailed descriptions are lacking.

• Insight into how often testers rely on their intuition or experience when specific details are missing.

Logical Relationships in Testing

• Discussion about the necessity of establishing logical relationships within test cases to understand cause-and-effect dynamics in software behavior.

• Highlighting the importance of identifying which tests will serve as sanity checks—those performed once for verification purposes.

• The speaker expresses a preference for a more structured approach to understanding causal relationships in testing scenarios.

• Reinforcement that every outcome has an underlying cause, emphasizing accountability in software failures or bugs.

• Recognition that even user errors should be treated as bugs if they lead to system failures without appropriate error messaging.

Addressing Software Failures

• Critique of testers who claim software fails without any action taken; emphasizes the need for understanding root causes behind failures.

• Stress on the importance of recognizing causality in order to effectively address and rectify issues within software systems.

• Encouragement towards maintaining awareness of logical connections throughout testing processes to ensure comprehensive coverage.

Techniques in Test Design

• The significance of knowledge regarding various test design techniques is highlighted alongside practical application experiences.

• Skepticism towards metaphysical concepts related to testing; belief rooted in tangible foundations and reasons behind outcomes is expressed.

• Assertion that there are always underlying principles leading to specific results within software development contexts.

• Mentioned examples illustrate how unforeseen issues can arise from overlooked factors during development cycles.

Understanding Equivalence Class Testing

The Concept of Equivalence Classes

• The speaker emphasizes that whether one has heard of a concept or not does not negate their understanding of it.

• Design is based on assumptions, which help reduce the number of checks and speed up testing processes. A person questioning these assumptions can lead to deeper insights.

Importance of Boundary Testing

• Boundary values are crucial in testing; they should be checked thoroughly within defined ranges.

• The likelihood of hitting boundary values varies depending on application specifics, highlighting the need for careful consideration during testing.

Real-world Application Examples

• In a retail scenario, if a customer spends 500 UAH or more, they receive a 3% discount. This example illustrates how boundaries affect user experience.

• If the system incorrectly defines "more than 500 UAH," customers may miss out on discounts due to programming errors.

Challenges with Exact Values

• Discusses the difficulty in defining exact weights (e.g., one kilogram), as real-life measurements often fluctuate around these values.

• When boundaries exist, additional techniques must be employed to ensure accurate testing outcomes.

Techniques for Effective Testing

• Introduces boundary value analysis as an extension of equivalence class testing, focusing on points where application behavior changes.

• Highlights potential issues when users expect greater discounts at higher spending thresholds and how this affects their purchasing decisions.

Conclusion: Simplifying Complex Concepts

• The technique simplifies error identification at boundaries while ensuring comprehensive coverage across equivalence classes.

Testing Methodologies and Exploratory Testing

The Nature of Testing Approaches

• The speaker discusses the concept of exploratory testing, emphasizing that it can be conducted without pre-designed test cases. This method allows testers to rely on their intuition and experience.

• The effectiveness of exploratory testing is highlighted, especially in startup projects where rapid feedback and iteration are crucial for success.

• The speaker notes that when testers operate without predefined test cases, they can generate tests organically based on their understanding and insights during the process.

Challenges in Team Dynamics

• A scenario is presented where a lead tester must onboard a new team member. The lead's challenge lies in conveying the rationale behind testing decisions without relying solely on documented test cases.

• It’s suggested that sharing personal experiences and insights about what needs to be tested can be more effective than simply providing written documentation.

Balancing Formality with Flexibility

• The discussion shifts to the limitations of strictly following exploratory testing methods, particularly when onboarding new team members who may require structured guidance.

• In active development environments, testers often work from task descriptions rather than formal test cases, adapting their approach based on immediate requirements.

Addressing Unforeseen Issues

• When faced with unexpected problems reported by users after deployment, the speaker advocates for an exploratory approach to quickly identify issues rather than creating extensive documentation first.

• If there is no clear problem description or formalization available, transitioning to exploratory testing becomes essential for efficient troubleshooting.

Importance of Documentation

• Despite advocating for flexibility in testing approaches, the need for some level of documentation is emphasized. Testers should maintain records of critical checks even if not all scenarios are formally documented.

• The speaker warns against neglecting documentation entirely as it complicates knowledge transfer within teams and hinders onboarding processes for new members.

Misconceptions About Testing Techniques

• There’s a caution against over-relying on specific techniques like "pairwise testing," which may seem appealing but have limited applicability across different scenarios.

Understanding Event States in Ticketing Systems

Key Parameters and Their Values

• The discussion begins with the introduction of three parameters, each capable of taking two values, which are crucial for understanding the system's behavior.

Internal vs. External Events

• It is emphasized that events can be categorized as either external or internal, impacting how the system responds to user actions.

State Transitions in Ticketing

• An example is provided where if a ticketing system remains idle (e.g., a cart filled but not checked out) for 10 minutes, it clears the cart and returns tickets to availability.

• The speaker reflects on personal experience working at a ticket counter, providing context for the technical discussion.

Transition States Explained

• There are three main transitions from the "Main" state: cancellation by user action (external), cancellation due to inactivity (internal), and payment completion (external).

• Two transitions are classified as external since they require user interaction, while one transition occurs internally when the system acts without direct user input.

Status Changes Upon Cancellation

• When a cancellation occurs (either by user action or timeout), the system updates ticket statuses accordingly—tickets revert from booked to available.

• In cases of payment completion, additional attributes like "Paytime" or "Paid" are added to tickets, indicating their new status post-payment.

Real-world Application and Challenges

• The speaker discusses practical scenarios involving ticket printing at physical counters versus online systems, highlighting differences in handling ticket states.

Issues with Ticket Scanning Systems

• A specific incident is recounted where delays in scanning led to confusion about whether tickets had been used or not due to local network issues affecting real-time data synchronization.

Consequences of Delayed Data Synchronization

Understanding User Stories and Decision Tables in Agile Development

Introduction to User Stories

• The discussion begins with the importance of understanding user stories, emphasizing that differences arise from the tickets used and the time they were utilized.

User Story Example

• An example is provided about a user story involving a user who wants to swipe tickets for a favorable arena display on their screen.

Agile Process Insights

• The speaker mentions that they will elaborate on user stories within the context of the Agile process, indicating its relevance to training sessions offered by their company.

Certification Discussion

• There is a note on certification, explaining that it incurs additional costs which affect course pricing. The speaker clarifies that certification differs from standard training offerings.

Training Experience

• A personal anecdote is shared about conducting Scrum training at a bank, highlighting practical experience in teaching Agile methodologies.

Decision Tables: A Tool for Testing

Preference for Diagrams Over Tables

• The speaker expresses a preference for diagrams over tables when discussing decision-making processes, stating they find tables less appealing but acknowledges their utility in organizing information.

Explanation of Decision Table Technique

• The decision table technique is introduced as a method for documenting various input combinations and corresponding system behaviors in tabular form.

Interface Example

• A simple interface example is presented where users can create, delete, or modify accounts. This sets up the context for how decision tables can be applied practically.

Building and Optimizing Decision Tables

Conditions and Outputs

• In decision tables, conditions are outlined clearly; if all fields are filled correctly, expected outputs are logically derived based on inputs provided.

Importance of Comprehensive Documentation

• Emphasizes documenting all possible scenarios to avoid overlooking logic during testing phases. Each scenario should be meticulously recorded to ensure thoroughness.

Handling Edge Cases

• Discusses edge cases where conflicting options (like modifying vs deleting an account simultaneously) cannot occur together. This highlights the need for clear rules in decision-making processes.

Refining Decision Tables Through Iteration

Optimization Process

• After initial creation of decision tables, there’s an emphasis on refining them through iterations to reduce complexity while maintaining clarity in documentation.

Practical Application

• Once optimized, these tables serve as effective tools for identifying errors during testing phases by systematically reviewing each rule against expected outcomes.

Preconditions for Using Decision Table Techniques

Understanding Requirements

• Questions are posed regarding prerequisites necessary to effectively utilize decision table techniques. It suggests that understanding requirements is crucial before applying this methodology.

Understanding External Testing

The Nature of External Testing

• The speaker discusses the challenge of reviewing a product, emphasizing the importance of understanding the correct product context.

• They highlight that even in simple applications with two buttons, predicting user interaction speed and environmental factors is complex.

• A reference to a philosophical quote about knowledge is made, indicating that external tests often reflect an approach rather than a definitive state.

Criteria for Effective External Testing

• The speaker poses a question regarding what constitutes effective external testing, acknowledging that it may not be fully exhaustive.

• They suggest that identifying objective possible scenarios is crucial since hypothetical scenarios can lead to endless possibilities.

Contrasting Testing Techniques

• The discussion contrasts traditional testing techniques aimed at resource efficiency with external testing, which focuses on thoroughness without guaranteeing complete coverage.

• An example involving PR-WICE is provided to illustrate how certain methods can catch 90% of errors but may not suffice for critical products like pacemakers.

Importance of Comprehensive Testing

• When discussing pacemaker testing, the speaker emphasizes the need for rigorous testing due to life-dependence on device functionality.

• They argue against cutting corners in testing processes when human lives are at stake, stressing the necessity for comprehensive checks despite time constraints.

Contextual Understanding in Testing

• The speaker refers back to SDQB (Software Development Quality Benchmark), highlighting its role in understanding test contexts and evaluating potential savings on checks.

Future Discussions and Practical Applications

Upcoming Topics in Testing Methodology

• Future sessions will cover bug reporting and evaluation methods, indicating an ongoing exploration of practical aspects within software testing.

Engaging with Participants' Experiences

• The speaker invites feedback from participants regarding their experiences with the course content and its applicability to their work environments.

Challenges in Current Testing Processes

• A participant raises concerns about working with poorly documented or outdated products, complicating the testing process significantly.

Collaboration Between Developers and Testers

Discussion on Testing and Development Processes

Importance of Documentation in Testing

• The speaker emphasizes the necessity of documentation during testing phases, highlighting that even without formal documents, having knowledgeable personnel (like a director) can guide the process.

Engaging Developers in Scrum Methodology

• The speaker suggests organizing sessions to engage developers in understanding the Scrum development process, indicating their readiness to share extensive knowledge due to their certifications in Scrum methodologies.

Openness to Questions for Deeper Understanding

• There is an acknowledgment that while the initial training may seem basic, deeper insights can be shared if participants ask questions. The speaker expresses a willingness to elaborate on various development methodologies like Kanban and others.

Motivation and Engagement Strategies

• The need for strategies to motivate developers is discussed. The speaker humorously suggests unconventional incentives (like beer), stressing the importance of engaging motivated individuals who are eager to improve processes.

Challenges with Unmotivated Participants

• Reflecting on past experiences, the speaker recounts challenges faced when conducting training sessions where participants were not genuinely interested or engaged, making it difficult to facilitate meaningful discussions.

Recognizing Participant Interest Levels