PromptArmor (YC W24)

Imagine if an attacker could steal any Slack private channel message. We've disclosed a vulnerability in Slack AI that allows an attacker to exfiltrate your Slack private channel messages via indirect prompt injection, and phish users. Slack's August 14th change to include documents in input increases the surface of this attack. Read more about the attack and what settings changes you can make to mitigate this risk: https://lnkd.in/gfzf9sw5

Products using LLMs are increasingly embedded in business-critical systems, making them a focal point for security concerns. Real-world incidents have demonstrated how LLMs can be manipulated to steal data, spread misinformation, and perpetrate fraud if left unmonitored. Some of the key threats to applications that use LLMs are: 1️⃣ Malicious Instructions from Prompt Injections: Attackers can manipulate LLMs by injecting confusing prompts, leading to unauthorized actions or data leakage 2️⃣ Data Leakage from Prompt Extractions: Vulnerabilities in LLMs can result in unintentional disclosure of sensitive information, as seen in instances where employees disclosed confidential data through applications using LLMs 3️⃣ LLM-Enabled Phishing Opportunities: LLMs create new avenues for phishing attacks where attackers can trick users into clicking malicious links embedded in AI-generated content 4️⃣ Sensitive Information Disclosure: LLM applications may inadvertently reveal proprietary information, algorithms, or confidential data, posing risks of unauthorized access and privacy violations 5️⃣ Supply Chain Vulnerabilities: The supply chain of LLMs can be compromised, leading to biased outcomes, security breaches, or system failures due to tampering with pre-trained models or training data supplied by third parties 6️⃣ Model Theft: Companies using LLMs face the risk of their proprietary models being stolen or copied if adequate security measures are not in place In the recent Microsoft for Startups and Y Combinator event in SF, I met with Shankar Krishnan who is building PromptArmor (YC W24). After reading about the product I was very fascinated and wanted to share it with all of you. PromptArmor (YC W24) offers a proactive security solution tailored for monitoring and securing LLMs in production systems. Their cutting-edge threat detection engine auto-updates to identify new attack vectors by analyzing model input, output, and actions for anomalies. Suspicious activity is flagged in real-time, enabling rapid incident response. With PromptArmor, organizations gain complete visibility into emerging LLM threats across their product portfolio. The consolidated dashboard reveals the source and characteristics of attacks, empowering leadership to make data-driven decisions around model usage policies and security strategies. If you are an AI professional, I recommend you check out PromptArmor which will help you develop rigorous security for your applications using LLMs. #ai #llm #prompting #securitybreach #privacy #dataprivacy

The security risks of products using LLMs are vast. PromptArmor (YC W24) is the LLM application security platform for enterprise-level threats: data exfiltration, phishing, and system manipulation. Any product sending data to an LLM is at risk, and awareness is low. Most think traditional data security measures (models in a VPC, secure data storage) are enough, but LLMs are now the weakest link in the chain. Attacks in the wild have shown that ignoring these risks leads to disastrous consequences, like customer data being exfiltrated to attackers, or phishing links being shown to users within your own trusted application. Threats are evolving every day, and PromptArmor’s threat intelligence keeps its detection engine up to date so customers don’t have to. Their engine analyzes input, output, and actions and performs anomaly detection to determine likely threat vectors – making it a comprehensive end-to-end solution. Companies use PromptArmor to detect, monitor, and respond to LLM security threats across their product portfolio, with a centralized dashboard for leadership to get visibility into the threat vectors and where they are originating from. Congrats Shankar Krishnan and Vikram J. on the launch – and welcome to YC!

Hi everyone, we published a new blog post - check it out! https://lnkd.in/gRJp9_pN

Last week we hosted our 3rd 'AI in HR' at Indeed's SF office. It's a unique event for future-of-work focused founders to connect with each others, investors, HR professionals and Indeed. The pace of gen ai innovation in HR is picking up and it was great to see 10+ demos. Searchlight for showcasing TRACY Commit and its GPT-4 private network for software devs Recruitment.ai ai assessments and RPO receruiterGPT the chrome extension to make recruiters more productive Teamforce AI - engage front line workers to help retention PromptArmor (YC W24) - protection against prompt injection attacks Girri M Palaniyapan Muthhukumar Malaiiyyappan dialme.at - an sdk for voice agents Vidal Graupera showing interviewigniter, an ai toolset to help job seekers get jobs in tech RAII - a recruiting ai agent to help with candidate sourcing, screening, and scheduling CloudApper AI enabling enterprises to build and integrate ai & llms easily