Session 3- Selenium with Python

Name: Session 3- Selenium with Python
Uploaded: 2022-03-29T14:30:11.000Z
Duration: 3 h 2 min 4 s

Introduction to XPath

Overview of Previous Session

Discussed various types of locators including ID, name, link text, class name, and tag name.

Explored CSS selectors and their combinations with tags and attributes.

Focus on XPath

Today's session centers on XPath, a crucial locator for identifying elements on complex web pages.

Emphasized the dynamic nature of element values and attributes that change at runtime, making XPath preferable in such scenarios.

Understanding XPath

Definition and Purpose

XPath stands for XML Path; it is a syntax or language used to locate elements on a webpage using XML path expressions.

It operates based on the Document Object Model (DOM), not directly on HTML. The DOM structure is essential for XPath functionality.

Importance of DOM

The DOM is an API interface created by browsers that represents the structure of a webpage's HTML code as nodes (e.g., body, head).

Browsers automatically generate the DOM when a page loads, allowing XPath to navigate through these nodes to find specific elements.

How XPath Works

Navigation Through Nodes

XPath identifies elements by navigating through each node in the DOM structure rather than relying solely on HTML code. Each part of the HTML corresponds to different nodes within the DOM tree.

Addressing Elements

An important takeaway is that every element has an address within the DOM; without this address, identification is impossible. Thus, we can think of XPath as providing an "address" for locating any element effectively.

Reliability of XPath

Unlike other locators which may fail under certain conditions, XPath consistently identifies elements regardless of their location or state within the webpage's structure. This reliability makes it indispensable in automated testing environments where dynamic content is common.

Understanding XPath: Types and Usage

Introduction to XPath

XPath is a powerful tool for identifying elements in web pages, often preferred over other locators due to its effectiveness.

There are two main types of XPath: Absolute (or Full) XPath and Relative (or Partial) XPath.

Types of XPath

Absolute XPath: This type provides the complete path from the root element to the target element. It is commonly used but can be lengthy.

Relative XPath: This type offers a shorter path that starts from a specific node rather than the root, making it more flexible for dynamic content.

Practical Application of XPath

To identify an element using XPath, one can either write it manually or capture it directly from the webpage's HTML code.

Inspecting an element allows users to copy its corresponding full or relative XPath directly from the browser's developer tools.

Capturing Full and Relative XPaths

When inspecting an element, options like "Copy XPath" provide relative paths while "Copy Full XPath" gives absolute paths.

The copied full path represents all nodes leading to the target element, while the relative path focuses on key identifiers.

Differences Between Absolute and Relative XPaths

Both types can locate the same element but differ significantly in structure; absolute paths include every parent node while relative paths do not.

Understanding these differences helps in choosing which type of path to use based on context—absolute for static structures and relative for dynamic ones.

Navigating HTML Structure with XPaths

Absolute XPaths navigate through each tag in the DOM structure until reaching the desired element, starting from <html> down through <body>.

Each tag represents nodes within this structure; understanding how these tags relate is crucial for effective navigation using XPaths.

Understanding XPath: Absolute vs. Relative

Overview of XPath Structure

The discussion begins with the explanation of closing tags in HTML, specifically focusing on <ul> tags and their corresponding closing tags.

It is noted that within a specific <ul>, there are multiple <li> tags, emphasizing the need to navigate to the third <ul> for further examination.

The speaker identifies that the target element is located within the first <li> tag of the third <ul>, illustrating how absolute XPath navigates from root nodes through each node until it finds the desired element.

Absolute vs. Relative XPath

Absolute XPath starts from the root node (HTML), traversing through every node to locate an element, while relative XPath allows direct access to an element based on its attributes or position.

An example is provided where an anchor tag with minimal attributes demonstrates how relative XPath can directly jump to elements without navigating through all parent nodes.

Navigating Elements Using IDs

The importance of IDs in elements is highlighted; relative XPath can quickly locate elements by starting at any node with a specified ID, such as header_iphone_nav.

It’s explained that even if certain parent nodes do not have IDs, identifying a key parent node can facilitate navigation downwards in the hierarchy.

Practical Examples of XPath Usage

A practical demonstration involves copying both absolute and relative XPaths for an image element, showcasing how each type functions differently when locating elements in a DOM structure.

The speaker illustrates how relative XPaths begin with a slash followed by attributes or tag names, emphasizing that they require at least one attribute specification for effective targeting.

Key Differences Between Absolute and Relative XPath

A clear distinction is made: absolute XPaths always start from the root HTML node and navigate through all child nodes until reaching the target element.

An example involving nested elements (a, b, c) illustrates how one would write an absolute path to find a deeply nested child like c1 by starting from its root ancestor.

Understanding XPath: Absolute vs. Relative Paths

Introduction to XPath Elements

The discussion begins with the identification of two nodes, c1 and c2, within a parent node c. The focus is on capturing the c1 element using an absolute XPath.

An explanation follows about relative XPath, which does not start from the root node but instead uses attributes like ID to navigate directly to elements.

Differences Between Absolute and Relative XPath

The syntax for writing a relative XPath involves specifying an attribute (e.g., ID), allowing direct access to the desired element without traversing from the root.

Key differences are highlighted:

Absolute XPath starts with a single slash (/) and navigates from the root node.

Relative XPath starts with double slashes (//) and can jump directly to elements based on specified attributes.

Syntax Characteristics

In absolute XPath, only tags or nodes are used without any attributes. This contrasts with relative XPath, which incorporates attributes for navigation.

A summary of three main differences:

Absolute paths begin at the root; relative paths do not.

Absolute paths use single slashes; relative paths use double slashes.

Absolute paths rely solely on tags; relative paths utilize attributes.

Writing XPaths Manually

The speaker transitions into how to manually write XPaths, emphasizing that while tools exist for automatic generation, understanding manual creation is crucial for interviews and dynamic elements.

Manual creation of XPaths is presented as a skill that may be tested in interviews despite automation being common in practice.

Best Practices in Writing XPaths

When writing an absolute XPath, it’s recommended to work from bottom to top through the HTML structure. This method helps identify where specific elements reside more effectively.

An example is provided involving locating a "register" element by inspecting its HTML structure step-by-step through its parent tags.

Conclusion

The session emphasizes understanding both absolute and relative XPaths' structures and their practical applications in web development. Mastery of these concepts aids in effective navigation of XML/HTML documents when automating tasks or testing web applications.

Understanding XPath: Manual and Relative Writing Techniques

Manual XPath Creation

The discussion begins with compressing a URL tag within a div tag, emphasizing the need to simplify the structure by stating "ul/ul" for clarity.

The speaker explains navigating through nested div tags, identifying the first and second parent div tags in the hierarchy.

A focus on counting div tags is highlighted, noting that there are multiple instances present under the body tag, leading to confusion if not organized properly.

The importance of understanding HTML structure is stressed; recognizing how many div elements exist helps in writing effective XPath queries.

The speaker mentions using Ctrl + F to verify if an absolute XPath written matches an element in HTML, showcasing practical validation techniques.

Challenges with Absolute XPath

Writing absolute XPaths can be cumbersome due to lengthy pages; missing a tag can lead to errors requiring re-evaluation from scratch.

The complexity of creating absolute XPaths is discussed as a significant challenge when dealing with extensive HTML structures.

Introduction to Relative XPath

Transitioning from absolute to relative XPath is introduced as a more efficient method for locating elements within HTML documents.

The syntax for writing relative XPaths starts with double slashes followed by specifying the element's tag name and attributes enclosed in brackets.

Practical Example of Relative XPath

An example illustrates how to write relative XPath for an input box by using its ID attribute, demonstrating practical application of previously discussed syntax.

Verification of the created relative XPath is shown using Ctrl + F again, confirming its accuracy by highlighting the corresponding HTML element.

Flexibility in Writing Relative XPath

If unsure about a tag name, using an asterisk (*) allows searching across all tags for specified attributes, enhancing flexibility in locating elements.

Understanding XPath and Tools for Automation

Differences Between Absolute and Relative XPath

The discussion begins with a focus on the differences between absolute and relative XPaths, emphasizing the importance of syntax when writing relative XPaths.

Writing XPaths can be time-consuming and requires significant effort, prompting the need for tools to automate this process.

Historical Tools for XPath Generation

Firebug and Firepath were previously popular extensions for Firefox that allowed users to automatically generate XPaths but are no longer available due to security concerns.

Current browsers have integrated features that allow users to capture XPaths directly by right-clicking on HTML elements.

Built-in Browser Options

Most modern browsers provide built-in options such as "Copy XPath" or "Copy Full XPath," which can be accessed through right-clicking an element in the browser's inspect tool.

Users can highlight HTML code, right-click, and copy either type of XPath directly from their browser.

Selector Hub: A Modern Solution

Selector Hub is a widely used extension compatible with various browsers (Chrome, Firefox, Edge), offering numerous features beyond just generating XPaths.

It provides suggestions for different types of locators (ID, name, CSS selectors), making it versatile for identifying web elements.

Features of Selector Hub

Selector Hub allows users to test whether their locators correctly identify elements on a webpage.

The official website offers extensive resources about its capabilities and user support; it has gained popularity among testers due to its efficiency in locator management.

Installation Process for Selector Hub

Instructions are provided on how to install Selector Hub as a Chrome extension. Users should search for "Selector Hub extension" in Chrome Web Store.

After locating the extension page, users will see an option to add it to Chrome; once installed, it integrates into the browser's inspect tool interface.

This structured overview captures key insights from the transcript while providing timestamps for easy reference.

Using Selector Hub for XPath Generation

Introduction to Selector Hub

The process begins with adding the Selector Hub extension, which appears as a small icon in the browser.

The interface includes HTML and CSS sections, indicating that web pages are built using both languages.

Navigating Developer Tools

Users can find elements by clicking an arrow in the Selector Hub interface, which highlights corresponding HTML elements.

Right-clicking on highlighted elements allows users to inspect them further.

Generating XPath

Upon selecting an image element, suggestions for various XPath formats appear automatically.

Relative XPath starts with //, and users can filter through multiple elements based on tag names and attributes.

Understanding XPath Suggestions

Different XPaths may identify the same element; users should choose one that uniquely identifies it (e.g., @class attribute).

The tool provides options for both XPath and CSS selectors, enhancing flexibility in locating elements.

Testing and Validating XPaths

Users can verify if their generated XPaths correctly point to desired elements within the page.

The extension offers features to copy relative XPaths directly from the context menu for ease of use.

Additional Features of Selector Hub

Besides generating XPaths, Selector Hub suggests different locator types and provides a user-friendly interface for testing them.

It is important to understand which type of XPath (absolute vs. relative) is preferred in various scenarios during testing.

Why Use Relative XPath Over Absolute XPath?

Understanding the Preference for Relative XPath

The discussion begins with the recommendation to use only relative XPath in automation scripts, highlighting that absolute XPath is often lengthy and cumbersome.

While many believe that relative XPath is preferred due to its simplicity, the speaker emphasizes a more critical reason: stability. Relative XPath is deemed more stable than absolute XPath.

Reasons for Choosing Relative XPath

The first reason for preferring relative XPath is its resilience against changes in the HTML structure. If a developer adds new elements within the hierarchy, it does not affect relative XPaths as significantly as it does absolute XPaths.

An example illustrates how introducing new elements can corrupt an absolute XPath because it relies on a fixed hierarchy, making it prone to breakage when changes occur.

The second reason discussed is related to positional changes of elements. If an element's location changes (e.g., moving from center to left or right), any absolute XPaths referencing that position will fail.

Stability of Relative vs. Absolute XPath

The speaker concludes that absolute XPaths are inherently unstable due to their reliance on specific locations within the document structure, which can change frequently.

In contrast, relative XPaths utilize attributes of elements rather than their positions in the hierarchy, making them less susceptible to breaking when modifications occur in the web page layout.

Additional Options with XPath

Beyond choosing between relative and absolute paths, various operators and functions can be used with both types of XPaths. These include methods like and, or, contains, starts-with, and text().

Practical Application of XPath

Transitioning into practical examples, the speaker prepares to demonstrate how to implement both types of XPaths using Python and Selenium in an application context.

A new directory and Python file are created for this demonstration, indicating a shift from theory to hands-on coding practice involving different versions of XPaths.

The session continues by setting up necessary imports from Selenium before launching a browser instance where various types of XPaths will be explored through automation code.

Understanding Absolute and Relative XPath

Introduction to XPath

The discussion begins with an introduction to absolute XPath, focusing on how to identify elements like the search box and search button using this method.

Using Absolute XPath

To find the absolute XPath for an element, one can inspect the element and use tools like Selector Hub. This allows copying of the full XPath directly.

The speaker emphasizes that while relative XPaths are often provided by default, capturing the full absolute XPath is essential for precise element identification.

After identifying the search box, a value (e.g., "t-shirts") is sent using sendKeys, demonstrating interaction with web elements through absolute XPath.

The process of clicking on the search button is similarly executed by obtaining its absolute XPath and performing a click action.

Transitioning to Relative XPath

Following successful execution with absolute XPaths, the speaker transitions to using relative XPaths for the same elements, highlighting their flexibility in locating elements without needing a complete path.

Practical Application of Relative XPath

The importance of syntax in relative XPaths is discussed; attributes should be enclosed in single quotes within double quotes to avoid errors during execution.

The speaker demonstrates how to obtain relative XPaths from Selector Hub and emphasizes their utility in simplifying element identification.

Advanced Operators in Relative XPath

Utilizing Logical Operators

Beyond basic usage, various operators such as 'or' and 'and' can enhance relative XPaths. These allow for more robust searches when dealing with dynamic web pages where attributes may change.

Understanding 'Or' Operator

The 'or' operator enables matching if either of two specified attributes exists. If one attribute fails but another succeeds, it still identifies the element effectively.

Understanding 'And' Operator

Conversely, the 'and' operator requires both specified attributes to match. This ensures stricter criteria for locating elements but increases reliability when both conditions are met.

This structured approach provides clarity on utilizing both absolute and relative XPaths effectively while introducing advanced logical operators that enhance web automation tasks.

Understanding XPath Operators in Automation

The Role of the OR Operator

The identification of an HTML element using XPath requires that both specified attributes match; if either attribute does not match, the element cannot be found. This is a fundamental aspect of the OR operator.

In practical application, when inspecting a search box element, an XPath is constructed using the input tag and specifying one attribute (id).

An example XPath includes both id (search_corridor) and name (search_query) attributes to ensure accurate identification of the element.

The OR operator allows for flexibility; as long as at least one attribute matches, the element can still be identified. If both attributes are incorrect, no elements will match.

Thus, when using the OR operator, it is crucial to understand that only one correct attribute value is needed for successful identification.

Implementing the AND Operator

Transitioning to another example with a button element, an XPath can be created using multiple attributes such as name and class.

When utilizing the AND operator, both specified attributes must match for successful identification; if either fails to match, no elements will be found.

A practical demonstration shows that removing or altering any part of an attribute leads to failure in identifying the desired element when using AND.

After constructing a locator with both operators (OR for one element and AND for another), executing these locators successfully interacts with web elements like entering values or clicking buttons.

Advanced Functions: Contains and Starts With

Moving forward, additional functions such as contains and starts-with are introduced as essential tools for dynamic element identification in automation scripts.

These functions are particularly useful when dealing with elements whose identifiers may change frequently but retain common prefixes or substrings.

A scenario illustrates how a button's ID changes from starter to reflect its state (start/stop), emphasizing why dynamic methods like contains or starts-with are necessary in automation contexts.

Understanding Dynamic XPath Selection for Button States

Dynamic ID Changes Based on Button State

The button's ID changes dynamically between "start" and "stop" states based on user actions. This dynamic behavior is crucial for identifying the element using XPath.

Writing Basic XPath Expressions

An example of a basic XPath expression is provided: //button[@id='start']. This will only match when the button is in the "start" state. If the button switches to "stop," this XPath fails.

Limitations of Simple XPath Expressions

A second example, //button[@id='stop'], illustrates that this expression will only work when the button is in the "stop" state, failing if it’s in "start." Thus, neither simple XPath can identify the element under both conditions.

Need for a Universal XPath Solution

The goal is to create a single XPath that can identify the button regardless of its state (either "start" or "stop"). This requires more advanced functions like contains or starts-with.

Using Contains Method for Dynamic IDs

The contains function allows matching elements with partially common attributes. For instance, both IDs ("start" and "stop") share 'st' as a common substring, enabling identification through: //button[contains(@id,'st')].

Alternative: Starts-With Function

Alternatively, one can use the starts-with function since both IDs begin with 'st'. The syntax would be: //button[starts-with(@id,'st')], which works effectively when there’s a consistent starting pattern across values.

Differences Between Contains and Starts-With

While starts-with requires matching from the beginning of strings, contains can find matches anywhere within an attribute value. This flexibility makes it useful when commonality exists at different string positions.

Combining Conditions with OR Operator

Another approach involves using an OR operator in an XPath expression to check either condition directly:

//button[@id='start' or @id='stop']

This method identifies elements by checking if either ID matches without relying on partial string matches.

Conclusion on Custom XPaths

Custom XPaths are essential when default generation does not suffice; they allow incorporating additional logic such as combining conditions or utilizing specific functions like contains and starts-with to enhance element identification strategies effectively.

XPath Functions in Selenium Automation

Practical Application of XPath Functions

The speaker discusses the practical use of XPath functions, specifically focusing on two elements and their respective functions.

For the search element, an XPath is created using the contains method to identify an input tag with a specific ID that partially matches "search".

The contains method checks if a portion of the ID attribute exists, allowing for flexible matching without needing the full value.

Differentiating Elements with XPath

The speaker explains how to use the starts-with function for identifying button elements by their name attribute.

An example is given where only part of the name ("submit") is used to match multiple elements; adjustments are made to ensure uniqueness.

By modifying the name string further (adding underscores), a unique match is achieved.

Executing and Understanding XPath Functions

The execution of both contains and starts-with functions demonstrates how they can be applied in practice, leading to successful interaction with web elements.

A brief overview indicates that these methods will be explored further in future examples, emphasizing their importance in understanding XPath.

Introduction to Additional XPath Functions

The text function is introduced as another useful method for identifying elements based on inner text content.

An example shows how to write an XPath using the text function to find an anchor tag containing "women", demonstrating its application effectively.

Summary of Key Concepts and Future Directions

A recap highlights various frequently used functions alongside XPath: absolute vs. relative paths, logical operators like 'and', and other advanced options.

The session concludes with encouragement for practice and exploration of tools like Selector Hub for verifying XPaths, setting up anticipation for continued learning in subsequent sessions.