Trail of Bits

There were >60 AI-related talks at BSidesLV, Black Hat , & DEF CON this year. ✨ Let me save you 𝒉𝒐𝒖𝒓𝒔: I put the talks in one place and summarized them. + slides, recordings, etc. where possible. 📊 The Process The goal of this was to help quickly get an idea of everything AI-related people shared at Hacker Summer Camp. I grouped each talk by Category, and summarized its abstract in 1 sentence and then 2-3 sentences, so you can quickly get a lay of the land. Time permitting, I want to do more detailed summaries, but many slides and recordings are not out yet. 📊 Top Categories The most popular categories, in descending order, were: 1. Attacking AI 2. AI for Defense (Blue Team) 3. Public Policy 4. Securing AI 5. AppSec 📖 Talk Summaries 📖 There were far too many talks for me to describe here, but let me give you a flavor: 🛡 Securing AI Rich Harang shared lessons learned from NVIDIA’s AI red team from securing dozens of LLM-powered applications. Christina Liaghati, discussed MITRE ATLAS, a public knowledge base of AI adversary tactics. Cyrus Nikolaidis and Faizan Ahmad shared CyberSecEval Prompt Injection benchmarks and PromptGuard. 👿 Attacking AI Hillai Ben Sasson and Sagi Tzadik used malicious models to break security boundaries in AI-as-a-Service platforms. Shachar Menashe analyzed the attack surface of the 6 most popular OSS MLOps platforms. Harriet Farlow bypassed casino biometrics, Ryan Tracey et al bypassed person detection in a popular security camera. Suha S. Hussain and Vasilios Mavroudis discussed backdooring models. Tooling: Leon Derczynski presented garak, Ian Harris FuzzLLM. Ofir Yakobi and Shir Sadon presented AI Goat. Maya Pavlova, Aaron Grattafiori et al on red teaming Llama 3. ✍ AppSec Kirill Efimov & Eitan Worcel 🎗 on AI autofixes. Chris Wysopal on AI-generated code introducing more vulnerabilities. Jay Chen & Ravid Mazon on automatically finding AuthZ vulns in web apps. 😷 Adam Shostack: LLMs for threat modeling. 🎣 Phishing Arun Vishwanath, Fred Heiding, Simon L. on building a self-improving phishing bot. Josh Kamdjou showed real, in-the-wild phishing attacks with fabricated contents. Perry Carpenter created a powerful backend for real-time, interactive voice-enabled cyber scams. 🔬 Blue Team Sheng-Hao Ma et al described a neural-network-based symbolic execution LLM that can analyze malware, even if protected by commercial packers. Ann Johnson & Sherrod DeGrippo on how Microsoft is leveraging its global-scale threat intelligence + AI. Ezz T.: using AI to map SIEM data to MITRE ATT&CK. Adel K.: building an LLM-powered web honeypot. 🗡 Red Team Michael Kouremetis: a methodology for evaluating LLM's offensive cyber capabilities. Joel Noguera & Diego Jurado Pallarés on creating an AI agent to augment bug bounty and pentesting workflows. ...and so much more 👇 https://lnkd.in/d3rmuPRN #cybersecurity #ai

It's uncommon for companies to have both AI/ML expertise and cybersecurity expertise in the same team. That's a potential risk and the right time to ask for help. Trail of Bits has an experienced team that knows how to bridge this gap, including in supply chain risk. Even better, our research informs our consulting and vice versa. See how we can help ensure safety and security for your ML pipelines at https://lnkd.in/eXxAVrEn.

We took on the challenge of provisioning cloud infrastructure the Max Power way—by relying on AI tools like ChatGPT and Claude. The result? Functional code that works but comes with serious security flaws. These AI-generated scripts often include hard-coded credentials and weak password generation methods, exposing critical vulnerabilities in your cloud setup. https://buff.ly/3X54ZSN

Researchers uncover 20+ vulnerabilities in ML software supply chains, posing serious security risks to MLOps platforms. These flaws could lead to arbitrary code execution or even allow malicious datasets to infiltrate systems, affecting the integrity of AI-driven operations. Read: https://lnkd.in/gfQTdYPB

We are excited to be sponsoring SekaiCTF again this year! Best of luck to all the teams competing! https://ctf.sekai.team/

New position!! We are actively seeking a Technical Content Manager to join our marketing team at Trail of Bits! This is an awesome opportunity to work at the leading edge of fields like AI/ML, appsec, blockchain, cryptography, and open-source security by working closely with Trail of Bits' security engineers and the marketing team to effectively communicate our offerings. Some highlights! - You will be a part of one of the most well-respected security organizations in the world! - Create impactful technical content (newsletters, websites, blogs, podcasts, social media etc) that resonates with our industry! - Get to work with the amazing Holly Womack to craft content that authentically captures the Trail of Bits' voice! - Even though the position is US-only, we are a remote-first company that offers a $1,000 stipend for any items you need to feel more comfortable working remotely! Take a look at the position below and apply if you think you would be a good fit for the role. Feel free to send me a DM if you have any questions regarding the role! https://lnkd.in/ge9usFTR

“YOLO” is not a valid hash construction. Our recent blog explores the risks associated with custom hash constructions. While seemingly straightforward, these ad-hoc solutions are often just plain wrong. We examine common pitfalls in: 👉 Multi-hash functions 👉Homegrown MACs 👉Custom password-based key derivation functions We also provide guidance on standardized methods that offer robust security: For multi-hashing: TupleHash or BLAKE3's stateful hash objects For MACs: HMAC, KMAC, or BLAKE2/3's keyed hashing mode For password-based key derivation: Argon2 or scrypt These vetted solutions have undergone rigorous testing and analysis by the cryptographic community, offering far greater security assurances than any custom construction.

If you missed our Burp Suite webinar featuring James Kettle, you can watch the recording now on Youtube: https://buff.ly/4ds1dd7 We cover: advanced web research techniques using Burp Suite how to discover ideas and targets optimizing your setup & utilizing Burp tools in various scenarios + Q&A with Albinowax

That's a wrap on Hacker Summer Camp for me! A huge thank you to everyone who attended my presentations at DEFCON, BSidesLV, and HOPE! I really enjoyed discussing my AI/ML security research at Trail of Bits. I'm very grateful to the many people who made my project and presentation possible. If you missed my talks, you can catch up here: -- Slides: https://shorturl.at/Ha6uD -- HOPE recording: https://lnkd.in/eTjN7JwA -- BSidesLV recording: https://lnkd.in/e6uSna3K (at 28:50ish) It’s an exciting time to be working in this space. Feel free to reach out if you have questions about my talk or want to discuss AI/ML security!

📚 tl;dr sec 243 Talks and Tools from BSidesLV, BlackHat, and DEF CON ✨ Highlights 👨💻 Web Security 👨💻 - GraphQler: A dependency-aware GraphQL API fuzzing tool - Omar Tsai - Splitting the email atom: exploiting parsers to bypass access controls - Gareth Heyes - Listen to the whispers: web timing attacks that actually work - James Kettle 😈 Hacker Summer Camp 😈 - BlackHat Innovators & Investors Quick Hits - Darwin Salazar, MSc - 100+ Founders, Investors, and Operators: Francis Odum's Key Takeaways - Watch How a Hacker’s Infrared Laser Can Spy on Your Laptop’s Keystrokes - samy kamkar - Apple Prototypes and Corporate Secrets Are for Sale Online—If You Know Where to Look - Matthew Bryant ☁ Cloud Security ☁ - AWS Honey Token Manager for Creating and Monitoring Access Keys - Steven Smiley - Shorten your detection engineering feedback loops with Grimoire - Christophe Tafani-Dereeper - Bucket Monopoly: Breaching AWS Accounts Through Shadow Resources - Yakir Kadkoda, Ofek Itach, Michael Katchinskiy ⛓ Supply Chain ⛓ - ArtiPACKED: Hacking Giants Through a Race Condition in GitHub Actions Artifacts - Yaron Avital - Grand Theft Actions: Abusing Self-Hosted GitHub Runners at Scale - Adnan Khan, John Stawinski 🤖 AI + Security 🤖 - NVIDIA AI Red Team Course - Will Pearce, Rich Harang, Becca Lynch, Joseph Lucas, John Irwin - Trail of Bits’ thread overview of AIxCC - Dan Guido https://lnkd.in/guqEmarZ #cybersecurity #security #ai #ciso