6. előadás: CI/CD megoldások és live monitoring

Discussion on CD-R and Software Development

Origins of Software Development Concepts

• The discussion begins with the historical context of software development, tracing its roots back to the 1990s. The speaker emphasizes that the concept of software delivery is not new.

• It is noted that no software can be considered "finished." Instead, there are always updates or improvements needed, indicating a continuous cycle of development.

• The primary goal has shifted from merely completing software to ensuring it is in a usable state that provides value to users. This reflects a change in focus towards delivering functional products.

Importance of Delivery in Software Development

• The speaker highlights the importance of rapid delivery after development, stressing that new features should reach users quickly for maximum impact.

• There are two critical aspects: providing timely updates to users and managing multiple parallel developments without divergence leading to integration issues.

Challenges in Integration

• A significant challenge arises when different teams work on separate features; if these diverge too much over time, integrating them becomes difficult.

• Reflecting on past practices from the 1990s, the speaker describes how developers had to manage with less sophisticated tools than today’s version control systems like Git.

Historical Context and Integration Hell

• Developers faced challenges merging their work due to primitive tools available at the time, leading to fragmented developments.

• The term "integration hell" is introduced, describing situations where more time was spent integrating code than developing it. This often resulted in frustration among developers.

Solutions Proposed for Continuous Integration

• To address integration challenges, solutions were sought out by developers who wanted smoother processes. Continuous integration emerged as a key strategy.

• Continuous integration aims to streamline workflows so that developers can integrate their changes into a shared repository frequently and efficiently.

Philosophical Shift Towards Continuous Integration

• The philosophy behind continuous integration focuses on working in smaller increments and integrating those changes rapidly into the main codebase.

Continuous Integration and Development: Key Concepts and Evolution

The Importance of Tooling in Continuous Integration

• The integration process is crucial, but without proper tooling, it becomes ineffective. Development tasks should not exceed 8 hours to ensure timely integration.

• Developers must integrate their work by the end of the day; otherwise, they risk working overtime. This practice aims to maintain a continuous integration workflow.

Emergence of CI Pipelines

• In the early 2000s, CI pipelines began to emerge, focusing on automating the integration process and linking workflows with appropriate tools.

• Extreme programming (XP) was significant for agile development practices that supported continuous integration through pair programming.

Early Tools Supporting CI

• CruiseControl, introduced in 2001, was one of the first systems designed to support CI processes through effective tooling.

• Another early alternative was the HZone system, which also aimed at automating various tasks within CI pipelines.

Version Control Systems and Their Impact

• Version control systems were essential for managing codebases effectively. During this time, many projects used shared drives or SVN repositories before Git became prevalent.

• The introduction of Git allowed developers more flexibility in version control without compromising software integrity.

Automation and Developer Experience

• Automation enabled builds and tests to run automatically upon commits, reducing issues where new code could break existing functionality.

• Running builds on dedicated servers rather than local machines alleviated pressure from developers who often faced long build times.

Transition Towards Continuous Delivery

• By the mid-2000s (around 2005), there was a shift towards continuous delivery as teams recognized that stable code could be released more frequently.

• The emergence of platforms like Flickr highlighted this transition; however, challenges remained regarding delivering features directly to users.

The Evolution of Software Development and Testing

Rapid Release Cycles in Software Development

• Flicker.com exemplifies rapid feature releases, achieving a frequency of about ten deployments daily. Developers could see their work live almost immediately after completion.

• Traditional CI pipelines allowed developers to confirm code integration but did not guarantee functional correctness, leading to potential issues further down the line.

• In the early 2000s, automated testing was limited; while small tests existed, comprehensive testing frameworks for software applications were not yet developed.

Challenges in Automated Testing

• Early automation efforts involved basic mouse movement recording rather than sophisticated UI testing frameworks that are common today.

• Manual testing remained prevalent due to concerns over existing applications' reliability; many teams hesitated to fully trust automated processes.

• Slow release cycles were common as extensive manual testing was required before deploying new features or updates.

Deployment Processes and Marketing Considerations

• Frequent deployments (e.g., ten per day) can dilute marketing impact; significant changes require more substantial development time and strategic planning.

• Not all software projects necessitate daily deployments; some may opt for fewer releases based on project needs and marketing strategies.

Advancements in Testing Tools and Infrastructure

• By the late 2010s, advancements in tooling enabled comprehensive test automation across various platforms (web, mobile).

• Cloud platforms emerged around the same time as deployment automation tools, simplifying resource management and application deployment processes.

Transitioning to Modern CI/CD Practices

• The introduction of cloud services revolutionized deployment practices by automating many previously manual steps involved in releasing software.

• Jenkins became a pivotal tool for running automated tests and builds, streamlining continuous integration/continuous deployment (CI/CD).

Future Directions: AI in Operations

How AI and CI/CD Tools are Transforming Software Development

The Rise of Context-Based Software Development

• Discussion on the increasing popularity of context-based software development models that integrate AI, allowing for tailored automation based on specific user subscriptions and needs.

Historical Overview of CI/CD Tools

• Mention of the historical context surrounding CI/CD tools, noting their evolution since before 2020 and their significant impact on workflow efficiency.

• Emphasis on how modern infrastructure management has alleviated the burden from developers, enabling them to focus more on coding rather than server maintenance.

Jenkins: A Pioneer in Automation

• Introduction to Jenkins as a foundational tool in CI/CD, highlighting its open-source nature which was crucial for early adoption in organizations hesitant to share source code.

• Explanation of Jenkins' self-hosted model requiring extensive configuration but offering flexibility in integrating with existing services and workflows.

TeamCity: An Alternative Solution

• Overview of TeamCity as a commercial product with both free and paid versions, emphasizing its ease of configuration compared to Jenkins while being particularly strong within the .NET ecosystem.

Cloud-Based Solutions: GitHub vs. GitLab

• Comparison between cloud-based platforms like GitHub and GitLab, noting their similar functionalities but differing limitations regarding free usage for teams.

GitLab vs. GitHub: Key Differences and Insights

Overview of GitLab and GitHub

• When setting up a Java web server, both GitLab and GitHub provide relevant action items for configuration, with GitLab being open-source, making self-hosting easier compared to the proprietary nature of GitHub.

• On GitHub, users register their machines to run tasks; however, the source code remains under GitHub's control. Users pay for computing resources while managing their own infrastructure.

• Azure DevOps is likened to GitHub but developed internally by Microsoft. It diverged from its initial development alongside GitHub due to different target markets.

Target Markets and User Experience

• Azure DevOps targets larger enterprises with complex workflows, aiming to enhance user experience for those already using Microsoft infrastructure, while GitHub caters more towards small to medium-sized businesses.

• The acquisition of GitHub by Microsoft has led to a perception that both platforms serve distinct purposes rather than competing directly against each other.

Demonstration of CI/CD Systems

• A demonstration includes two systems: TeamCity and Jenkins. TeamCity shows project management capabilities similar to those in GitHub but offers more visibility into ongoing processes.

• Both systems integrate with version control (like git), allowing tracking of changes through visual representations akin to graphs seen in version control systems.

Agent Pools and Configuration

• In TeamCity, users can view pending jobs across various agent pools. This feature allows monitoring of workload distribution which is not as easily managed in either GitLab or GitHub.

• Setting up a TeamCity agent requires specific configurations on the host machine including IP address settings and environmental variables necessary for builds.

.NET Integration and Build Management

• The integration between TeamCity and .NET is highlighted as particularly well-developed, facilitating smoother setups compared to other programming environments like Java.

Team City vs. Jenkins: A Comparative Overview

Resource Management in Development Environments

• The speaker discusses the challenges of resource allocation in development environments, emphasizing that having multiple servers can significantly enhance testing capabilities for a small development team.

• Team City is highlighted for its strong customization features, particularly useful when systems become complex and require tailored configurations.

Jenkins System Overview

• An introduction to Jenkins is provided, showcasing its user interface and the various teams involved in different tasks within the system.

• The speaker notes the ability to monitor builds actively, indicating how users can track ongoing processes and their statuses.

Build Process Insights

• A standard workflow example is presented, illustrating how build processes can be configured with varying parameters to optimize performance based on specific needs.

• The flexibility of parameterized builds in Jenkins allows developers to specify which tests should run, enhancing control over the build process.

Error Tracking and Notification Systems

• Discussion shifts to an older system used for tracking error messages from applications, highlighting its role in proactive issue detection without customer reports.

• Sentry is mentioned as a modern tool that automates similar functions previously handled manually by developers.

Build History Analysis

• The speaker examines a frequently built project’s history, demonstrating how changes are tracked over time and their impact on build success rates.

• Unique features of Jenkins are contrasted with GitHub's offerings; Jenkins provides detailed insights into failures linked to specific commits or pipelines.

Debugging Failures

• The output console of runners executing builds is discussed as essential for diagnosing issues when failures occur during execution.

Jenkins vs. Team City: CI/CD Tools Overview

Introduction to CI/CD Tools

• The speaker introduces Jenkins and Team City, indicating that today's discussion will be more relaxed and improvisational.

• A summary table is mentioned, which outlines key aspects of different CI/CD systems based on various criteria such as ease of setup and pipeline configuration.

Jenkins Features

• Jenkins is highlighted for its extensive plugin ecosystem, allowing for significant customization and flexibility in configurations.

• The speaker notes that Jenkins supports a wide range of container environments, with Docker being a common choice for running pipelines.

Comparison with Other Systems

• The comparison indicates that while GitHub Actions and GitLab have some capabilities, they lack the extensive customization options available in Jenkins.

• Transitioning between CI/CD tools is described as manageable; switching from one system to another (e.g., from GitHub Actions to Jenkins or Team City) can be done without major disruptions.

Cost Considerations in Technology Choices

• The speaker discusses the cost implications of changing technologies, emphasizing that switching between CI/CD tools typically incurs less expense compared to larger software migrations (like moving from Java to .NET).

Monitoring Application Performance

• A new section on monitoring application performance is introduced, focusing on quality code deployment and user statistics collection.

• Two main areas of focus are identified: capturing crashes/errors and tracking feature usage within applications.

Importance of Data Collection

• Emphasis is placed on the need for anonymous usage statistics to understand application performance better.

How to Analyze Application Functionality and Performance

Understanding Button Functionality and User Interaction

• The discussion begins with the need for animation on buttons, specifically focusing on how user interaction with the red or green button can dictate further actions in application development.

• Emphasizes the importance of collecting functional statistics to understand user behavior, suggesting that even poorly designed features may be frequently used due to their necessity.

• Highlights the significance of monitoring network conditions where software is utilized, as performance can vary greatly depending on users' environments.

Data Collection Strategies

• Discusses methods for gathering data through local databases that periodically send information back for analysis, stressing the need for a tailored approach to data collection.

• Suggests developing custom instrumentation code to track specific functionalities within applications, ensuring developers have control over what data is collected and how it is processed.

Analyzing Crash Reports

• Introduces the concept of crash report aggregation, explaining that understanding relationships between different crashes can provide insights into underlying issues affecting multiple users.

• Stresses the importance of identifying whether multiple crashes stem from a single bug or if they are isolated incidents, which can significantly impact user experience.

Utilizing Technology for Analysis

• Explains how various technologies can be employed to analyze crash reports effectively. It mentions using platforms that provide stack traces and memory dumps based on programming languages used.

• Discusses leveraging databases like ElasticSearch for storing and analyzing large datasets efficiently while also being capable of processing linguistic elements in error messages.

Enhancing Developer Collaboration

Understanding Bug Reporting and Prioritization in Software Development

The Role of User Statistics in Bug Management

• Developers focus on user statistics rather than crash reports to prioritize bug fixes, as they can be more informative about the impact of issues.

• When developers receive a bug or crash report, they assess whether similar issues have been reported by other users to determine priority based on frequency and severity.

Strategies for Prioritizing Bugs

• Developers evaluate multiple bugs by considering their relevance and the number of affected users, allowing them to prioritize effectively.

• Ideally, developers should be able to fix bugs without needing detailed reports from users; proactive identification is key to improving software quality.

Challenges in Bug Reporting

• Users reporting issues indicates a significant problem; it reflects that the software has reached a threshold where user frustration prompts action.

• Effective software should allow developers to identify bugs through code analysis rather than relying solely on user-submitted data.

Quality Assurance Through Monitoring

• High-quality software development involves systems that automatically track errors without requiring user input, enhancing overall user experience.

• Developers may discover unreported bugs through monitoring tools before users even notice them, which is indicative of high-quality software maintenance.

Evolution of Monitoring Systems

• The shift towards cloud-based services has led to the emergence of standardized monitoring solutions that facilitate better error tracking and management.

• Tools like Azure's Upinights help streamline data collection and analysis processes for developers managing applications in cloud environments.

Instrumentation Solutions for Data Handling

• Modern solutions provide SDKs that simplify instrumentation tasks, allowing developers to focus on coding rather than data transfer complexities.

Understanding Instrumentation and Telemetry in Software Development

Overview of Distribution and Instrumentation

• The speaker discusses the importance of understanding a 100% distribution to interpret data from diagrams, emphasizing that this knowledge is unique to the presenter.

• It is noted that regardless of the cloud service provider used, telemetry data can be collected effectively across platforms.

• Managed services like Centri provide higher-level instrumentation compared to non-managed solutions, allowing for more detailed insights into application performance.

Differences Between Managed and Non-Managed Services

• In managed services, such as those provided by Centri, users do not need to modify code extensively; they simply specify which applications to monitor.

• Notifications are automatically generated for exceptions in the code without requiring manual intervention or extensive coding changes.

Detailed Analysis of Application Performance

• The speaker demonstrates how a browser interface can show detailed metrics over a specified period (e.g., last 90 days), including application launches and user error rates.

• Users can drill down into specific exceptions (like null reference exceptions), gaining insights into device types and operating system versions involved in errors.

Insights on Error Tracking and User Interaction

• The discussion highlights how structured data allows developers to analyze issues with precision, identifying problems without needing personal user data.

• Examples illustrate how developers can track issues related to specific devices (e.g., Samsung mobile running Android 12), enhancing their ability to troubleshoot effectively.

Challenges in Internal Telemetry Implementation

• The speaker points out that achieving such precise internal telemetry is often unrealistic for smaller software teams due to resource constraints.

Remote Monitoring and Ethical Considerations

Overview of Remote Monitoring Capabilities

• A leader in testing discusses the ability to track data through a unique ID associated with a phone, showcasing how much information can be accessed without additional tools.

• Various application types (web, mobile, desktop, cloud API) can easily integrate monitoring features; the speaker emphasizes that remote monitoring is becoming more ethical.

Ethical vs. Unethical Monitoring Practices

• The unethical side of web applications includes tracking user behavior without consent, such as mouse movements and eye tracking; many applications do this by default unless users opt out.

• Developers can analyze heatmaps to understand user interaction with UI elements; excessive mouse movement before clicking indicates poor design.

Incident Reporting and Exception Handling

• Tools for remote monitoring fall under usage telemetry rather than incident reporting; they help identify unhandled exceptions in software.

• An example illustrates how language settings affect error messages displayed to users, highlighting challenges in maintaining consistency across different languages.

Debugging and Communication Challenges

• The speaker describes difficulties in connecting similar exceptions across different platforms due to varying user interfaces and error reporting systems.

• When issues arise during testing phases at companies like Microsoft, communication becomes challenging as developers cannot directly interact with testers.

Solutions for Effective Application Testing

• Information can be sent into the system for tracking purposes without being classified as an error message; this helps monitor application performance effectively.

• Applications must pass rigorous verification processes before being published on platforms like Google Play or Apple Store; poor-quality software is not accepted.

Case Study: Microsoft Testing Experience

• A specific case where Microsoft rejected an application due to crashes led to further investigation revealing compatibility issues with Windows versions used by testers.

• The speaker explains how remote debugging was utilized to trace errors back to their source despite limited communication with Microsoft's team.

Conclusion on Remote Monitoring Tools

• Remote monitoring tools allow developers to investigate crashes effectively while ensuring compliance with privacy regulations since no personal data is collected during network traffic analysis.