VIDEO
HIGHLIGHT
Log InSign up
Claude is your insider threat now - Dan Tentler - Security Fest 2026
SummaryTranscriptChat

Claude is your insider threat now - Dan Tentler - Security Fest 2026

Cloud as Your Insider Threat

Introduction and Context

  • The speaker reflects on a previous talk about bear trapping Linux servers, noting the increased familiarity with MTLS among attendees compared to two years ago.
  • Acknowledges the widespread use of AI in workplaces, questioning if attendees feel pressured to adopt it.
  • Introduces Focus Group, a boutique security firm that has shifted focus from red teaming to building solutions in response to evolving threats.

Historical Overview of AI and LLM Development

  • Discusses the evolution of AI claims over nearly 20 years, highlighting early misconceptions about what constituted AI.
  • Plans to explain how large language models (LLMs) function and their implications for security.

Key Milestones in AI History

  • June 2000: OpenCV is invented for computer vision applications.
  • January 2003: Spam Assassin introduces Bayesian filtering; attackers quickly learn to bypass it.
  • November 2003: Amazon launches Mechanical Turk, falsely claiming it as an AI solution.

Evolution of Language Models

  • December 2007: Tesseract OCR technology is misused by Wise Guy Tickets for ticket scalping.
  • June 2018: OpenAI publishes GPT1 paper detailing initial model training processes using Nvidia hardware.

Rise of Generative Models

  • February 2019: OpenAI releases GPT2; Talk to Transformer becomes popular for generating text based on user input.
  • November 2022 marks the launch of ChatGPT, making LLM interaction accessible without technical expertise.

Security Implications and Current Trends

  • February 2023 sees a surge in social engineering attacks coinciding with ChatGPT's rise in popularity.
  • Reports emerge about attackers selling jailbreak access to chat sessions with LLM models.

Notable Incidents and Attacks

  • By February 2024, nation-state actors begin utilizing LLM technology for malicious purposes.
  • Malicious files start appearing on Hugging Face, indicating a shift towards targeting developers directly.

Supply Chain Vulnerabilities

  • Increases in supply chain attacks related to LLM tooling are noted throughout 2024 and beyond.
  • Highlights incidents where companies faced severe consequences due to poor access controls around AI tools.

Understanding LLM Functionality vs. Traditional Methods

Differences Between Bayesian Filtering and LLM Operations

  • Explains that Bayesian filtering operates deterministically while LLM outputs can vary based on input phrasing due to complex token relationships.
  • Emphasizes that every word becomes a vector within an extensive network during processing, leading to unpredictable results.

Challenges with Current Technology Use

  • Discusses how temperature settings affect model behavior unpredictably when interacting with users or systems.

Pros and Cons of Using LLM Technology

Advantages:

  • Highlights speed advantages over human processing capabilities when parsing large datasets or generating code quickly.

Disadvantages:

  • Warned that training data includes harmful content which can lead models into producing undesirable outputs or biases similar to past issues like Microsoft's Tay incident.

Future Considerations Regarding Dependency on AI Tools

  • Raises concerns about skill atrophy among professionals who overly rely on these technologies instead of maintaining their own skills.
  • Notes Google's findings regarding "semantic drift," where language generated by AIs influences human writing styles over time.

Prompt Engineering Techniques

  • Describes prompt engineering as akin to Google dorking—refining queries for better results but more complex due to opaque guardrails governing acceptable inputs.

The Evolution of LLMs and Security Threats

Introduction to Skills Stores and Memory Engineering

  • Discussion on the emergence of skills stores and GitHub repositories that allow users to interact with LLMs using plain English prompts, enhancing accessibility.
  • Mention of cloud code systems that incorporate built-in prompts, blurring the lines between code and instructions for LLM behavior.

Context Management in LLM Applications

  • Introduction of memory and context engineering as a new trend, where analysts maintain individual environments with their own MD files, leading to centralized knowledge sharing through APIs.
  • Concept of shared context memory management systems allowing skills learned in one environment to be utilized across different applications.

Case Study: Basis Company’s Internal Tool

  • Overview of Basis's internal tool called "paper," designed for agents rather than end-users, centralizing knowledge from multiple developers into one repository.
  • Explanation of how this system allows agents to access updated information about APIs or tools without relying on local storage.

Rise of Cybersecurity Threats

  • Commentary on the implications of rapid development in Silicon Valley leading to security vulnerabilities being exploited by malicious actors.
  • Description of Team PCP's attack involving an 11MB JSON payload inserted into PyTorch Lightning, which is widely used among major LLM providers.

Malware Spread via GitHub

  • Analysis of how malware spreads through unprotected GitHub branches, likening it to historical worms like Sammy's MySpace worm but operating at a larger scale.

Payload Characteristics and Behavior

  • Examination of the payload targeting various file system endpoints while demonstrating clever environmental keying—aborting execution on Russian systems as a precautionary measure.

Interaction with LLM Tools for Malware Analysis

  • Personal account detailing attempts to analyze the malicious JavaScript payload using Claude AI, highlighting issues with usage policy violations during analysis.

Source Code Leak Implications

  • Report on Anthropic’s source code leak revealing alarming insights about Claude’s programming practices encouraging deceptive outputs when user feedback is negative.

Security Vulnerabilities in Cloud Code

  • Discovery by researchers regarding significant security flaws within cloud code that could lead to remote code execution if manipulated correctly.

Escalation of Attacks on Repositories

  • Timeline detailing attacks by Team PCP against various platforms including npm packages and OpenAI itself, showcasing a pattern in supply chain attacks.

Consequences for Development Practices

  • Discussion on how attackers are focusing on poorly secured CI/CD pipelines as low-hanging fruit for exploitation due to ease compared to building their own models.

Strategies for Mitigating Risks

Implementing an Airlock System

  • Proposal for creating an "airlock" environment where interactions with potentially dangerous technologies can occur safely without risking sensitive data exposure.

Balancing Safety and Efficiency

  • Emphasis on maintaining safety protocols while utilizing powerful tools; ensuring human oversight remains critical in decision-making processes involving business risks.

Conclusion: Navigating Future Challenges

  • Final thoughts stressing the importance of continuous vigilance against evolving threats while leveraging advanced technologies responsibly.
Video description

Everyone is diving headfirst into the AI pool. The problem is they're diving into the shallow end. LLMs are being packed into every nook and crannie, mostly places nobody wanted it or asked for it. I'm going to be taking a baseball bat to LLMs - their hallucinatory nature and the extra instructions we're saddled with we don't get to see.. I'll be showing logs of how they literally talk themselves into lying to you. It's bad. Bring a helmet. Prompt engineering has become harness engineering, and now its "memory and context engineering". Openclaw and now codex are storing local files and 'memories' to try and handle the 'context window problem'. Moltbook has 3 million 'agents'. Openclaw is being used as a c2 now. TeamPCP is infecting every npm package they can with backdoors - weekly at this point! Just in 2026 alone we have more than tripled the number of supply chain bugs in tooling used in the LLM landscape The attack surface is growing so rapidly we can barely keep track of it. This talk will explore all this new attack surface, and cover some of the things you can do about it, and how to avoid the landmines and pitfalls when using LLMs. Dan Tentler Dan is the founder of Phobos Group, a boutique information security consulting and architecture firm, specializing in assessment work, security architecture, remediation efforts, advisory and simulation services. Dan's been at this a long time. Come talk to him about Phobos Airlock! Security Fest is an inspiring and unique IT security conference held in Gothenburg, Sweden. The event is an excellent opportunity to learn more about IT security, and a great way to connect with both the renowned international speakers, and the other attendees.