Shobhit Varshney’s Post

Great keynote by Matt Wood at AWS Summit NYC. AWS showcased some meaningful incremental progress on Gen AI. Here is my personal take: 𝗧𝗵𝗲 𝗔𝗵𝗮! ✅ IBM Consulting earned the AWS Generative AI competency! Super proud of our joint work in scaling GenAI responsibly for our clients. https://ibm.co/4cSKxL5 ✅ Amazon Q in Sagemaker studio.. yes please. providing tailored guidance, code generation, and error troubleshooting, to build, train, and deploy models efficiently. https://lnkd.in/g_pf_ncP ✅ Guardrails for Amazon Bedrock adds hallucination detection and now supports 3rd party LLMs. Any time a vendor does more for responsible AI, it's a win for all of us https://lnkd.in/gRmjXsci ✅ Agents for Amazon Bedrock now offer Memory to retain user context and Code Interpreter to dynamically execute code snippets. AWS is incrementally adding features for agent orchestration to catch up. https://lnkd.in/gRp8QNsC ✅ Personalizing Amazon Q Developer to enterprise code base with adapters is a great add. https://lnkd.in/gQsXuZcq ✅ Incremental feature updates eg. more connectors with Knowledge bases for Bedrock ✅ Great to see deeper partnerships with Scale AI and Perplexity 𝗧𝗵𝗲 𝗨𝗵 𝗢𝗵! 🤨 From an enterprise perspective, I don't understand the obsession with no-code app development. It's great for college kids to whip up an app quickly, but in enterprises there are design standards, reusable approved components, software guidelines, security protocols etc before you can 'publish' an app. I am lukewarm about AWS App Studio. It's a decent set of starter features, but shouldn't market it as enterprise-grade. Most of our GenAI-enhanced software development has been human in the loop modular approach. https://lnkd.in/gpbThM4c 🤨 Amazon Q Apps is another ultra light app creation tool. There are very few workflows where a web app is the appropriate UX to expedite a process. I think a conversational interface of IBM watsonx orchestrate or Microsoft co-pilot is a more interactive way to augment a workflow. https://lnkd.in/gQiG3yrC 🤨 Finetuning Anthropic Claude 3 Haikui for data scientists is incremental, but adding enterprise data to Claude 3.5 Sonnet would have been a baller move. I would have loved to see more accelerators to build training data, ontologies and non-coders being able to contribute skills and knowledge. IBM's InstructLab is a better approach. For a more detailed discussion of AWS, please reach out to our team Mahmoud Elmashni, Dan Bailey, MBA, Matthew Candy, Becky Carroll, Manish Goyal, Javier Olaizola Casín, Glenn Finch, Balu Angaian, Shiva Bayati. Thanks for the partnership Seiji Shinozaki, Ryan Keough, Amit Chowdhury. #awssummit #aws #genai #ibmconsulting -------- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv --------

Would love 🙋♂️🎤 audience questions in advance… Looking forward to hosting a panel with Lavanya Ramani (Palo Alto), Barak Turovsky (Cisco) and Vivek Gangasani (AWS) at Arize AI’s Observe event this Thu in San Fran. It’s the industry’s leading event for AI/LLM Observability for developers, business leaders and researchers. We will be discussing practical techniques of how enterprises are continuously monitoring their AI/LLM models in production, and what lessons we have learned over the last couple years. Please don’t hesitate in dropping a few questions for the panel in the comments below. Stay connected, and I will recap our discussion after the event.

🔥 Hot take on Claude 3.5, BIRD-SQL and AI slop🔥 💬 "Absolutley stunning. Our team had been struggling with accurately reading an image of an anlaog dial guage with GPT 4o and Gemini 1.5 Pro. Claude 3.5 nailed it!" 💬 "Despite the impressive vision capabilities, the scale at which enterprises do computer vision in manufacturing, QA etc., I still can't recommend LLMs over a classic ML pipeline. The ROI and reliable accuracy is still not there" 💬 "IBM granite models crush the BIRD-SQL text-to-sql benchmarks #1" 💬 "GenAI gets a lot of bad rep for creating junk content, but GenAI has been incredible at detecting fake/ hate/ misinformation. #aiforgood" 💬 "Ultimate usecase of AI video generation is letting me insert myself virtually into a movie I am watching, say a scene with Beyonce or Avengers !" As always, loved joining the Mixture of Experts podcast with: 👨💻 Bryan Casey, brilliant host and digital marketing guru 👩💻 Marina Danilevsky, leads some of our incredible work on LLMs in IBM Research 👨💻 Michael Glass, who is an expert at using AI for text to SQL In this episode we talk about: 🧠 Anthropic Claude 3.5 Sonnet https://lnkd.in/gwk4bxn7 🔢 BIRD-SQL https://lnkd.in/gsNzGdFH benchmark for text-to-SQL 😰 AI Slop - creepy AI generated content Listen in and subscribe @ Youtube https://lnkd.in/gXZ7V9rf Spotify https://lnkd.in/gfT2RSDw Apple https://lnkd.in/gttTjXuB Podcast https://lnkd.in/gpa2Yiz3 For a deeper technical discussion with our brilliant team at IBM Research, please reach out to Dario Gil, Sriram Raghavan, Lisa Amini, Luis Lastras, Kush Varshney For AI for business, pls reach out to Mohamad Ali, Kelly Chambliss, Matthew Candy, Javier Olaizola Casín, Jesús Mantas, Armand Ruiz #mixtureofexperts, #ai, #genAI, #aiforgood, #ibm, #claude3.5 -- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv

Pumped about speaking at Arize AI Observe.. it's shaping up to be the most exciting LLM & AI Observability annual conference. Jason Lopatecki & Aparna Dhinakaran have built tremendous momentum in the market with Arize AI. It's been an absolute pleasure partnering with them to scale GenAI responsibly in large enterprises. Lots of learning opportunities with 3 different tracks: 🛠 AI Builders Guild 💡 AI Innovators 🔬 AI Research Frontiers Register @ https://lnkd.in/guPreQMK Incredible lineup of speakers from all the movers and shakers across the AI landscape. #arize #observe2024

ISG's Ventana Research just released their buyer guide on GenAI, AI, MLOps and Data platforms.. summary attached below. Full guides at https://lnkd.in/gyupgQtY. I realize these market reports are conversation starters, directional evaluations and need to be tailored to clients, and most analysts offer such services. ISG applies a different product & customer lens via RFIs and their own index for these guides. ↪ Proud to see IBM top right across all 4 ↪ Surprisingly they ranked Oracle #1 across all 4 - GenAI, AI, MLOps, Data. Personally I have a lot of respect for the future direction of Oracle's AI stack, and our team just had some massive wins! Making the right choices, open ecosystem, tailoring it to their target audience. OpenAl is expanding beyond Microsoft to Oracle Cloud Infrastructure (OCI) to provide additional capacity. https://lnkd.in/g2n7eAgK But my personal large scale client delivery experience would put Microsoft, AWS, Google, IBM, Databricks ahead of them. ↪ Puzzled by some of their choices eg. Teradata, Anthropic as GenAI platforms ?? InterSystems instead of Databricks & Snowflake in Data ?? As an AI strategy advisor to my clients, my team and I invest a lot of time shaping our clients' strategy & tech bets across Data & AI landscape. On the surface everyone offers everything : ) Need 3 ingredients to get it right: 1️⃣ Very very deep hands-on experience in deploying top vendors at scale, so we are looking beyond marketing and RFI responses when comparing capabilities 2️⃣ Deep understanding of what business problem client is trying to solve, so we can establish what capabilities are truly meaningful in their context 3️⃣ Make recommendations on how to close the capability gaps, ROI associated with that transformation etc. Be able to simplify the message from c-suite to developers What are your thoughts on these specific guides? or how relevant are analyst guides to your organization? #ai #genai -- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv

Super pumped to see such intense competition in Gartner's MQ for the Data Science & ML platforms. There has never been a better time to be a data scientist or AI engineer. The barriers to entry have been considerably lowered with AutoML, GenAI code copilots, one-click deployments, end2end evals, and simplified MLOps. Below I tried to show the progress vendors have made since the last Jan 2021 Gartner MQ, pre-GenAI days. 𝗚𝗼𝗼𝗴𝗹𝗲:  𝗔𝗵𝗮! - The only major cloud vendor that makes it's own frontier LLMs - Gemini. They invest heavily in Gemini to protect their other mega businesses. Great job balancing both GenAI and AI/ML. Our data scientists love the remarkable 500+ updates to Vertex AI in 2023. 𝗨𝗵 𝗢𝗵! - Data governance & lineafge is fragmented across the ML pipeline. 𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁:  𝗔𝗵𝗮! - Great range of frontier models from OpenAI and open-source small models (eg. their Phi3) alongside AI ML. Azure AI Studio + Azure ML, gives our team full flexibility of AutoML, evals etc. Investments in massive compute for OpenAI trickles down to developers. 𝗨𝗵-𝗢𝗵! - Notoriuous for going all in and then sunsetting products. Need to understand the total cost of all licenses etc, particularly co-pilots. 𝗔𝗪𝗦:  𝗔𝗵𝗮! - Ton of options across the AI lifecycle. Anthropic was a great bet! Our AI team loves full control, super scalable components, security, lots of integrations, modular services. AWS has invested a lot in AI infrastructure eg. SageMaker HyperPod for efficient LLM training.  𝗨𝗵-𝗢𝗵! - Bedrock is still missing some features like Azure's Prompt Flow. Options get overwhelming for beginners. Titan LLMs are closed. 𝗗𝗮𝘁𝗮𝗯𝗿𝗶𝗰𝗸𝘀:  𝗔𝗵𝗮! - A very unifed data-first approach to simplifying Data & ML workflows. Nailed data governance & lineage for ML training. MossaicML was a brilliant acquisition. 𝗨𝗵-𝗢𝗵! - Product upgrades are a bit messy. Need to optimize model deployment flexibility and tuning LLMs w/ enterprise data. 𝗗𝗮𝘁𝗮𝗶𝗸𝘂:  𝗔𝗵𝗮! - Great technical teams & business collaboration. Visual flow compliments pro-code. Great with large datasets and complex projects.  𝗨𝗵-𝗢𝗵! - The user community is not big enough. On-prem deployment requires a lot. Gets expensive quickly. 𝗜𝗕𝗠:  𝗔𝗵𝗮! - Ideal for enterprises with hybrid clouds (on-prem, private, public). Has focused heavily on openess (watsonx.ai) and responsible AI governance (watsonx.gov). Granite LLMs are ranked high by Forrester, are fully open-sourced and very transparent with the training data. InstructLab helps enterprises collaboratively train LLMs with priorietary client data.  𝗨𝗵-𝗢𝗵! - Need more pre-built integrations with services provided by client's cloud vendor. Need an integrated LLM app builder. What are your thoughts on these vendors? Any surprising omissions or inclusions? #AI #MachineLearning #DataScience #Gartner -- 🔔 Pls connect, comment, repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv

❤️ ⚽ Sevilla Football Club has fundamentally transformed recruiting in the sports industry in partnership with IBM watsonx. So incredibly proud of the brilliant work Elias Zamora Sillero & team have led. Vamos Sevilla FC ! Raise your hand if this sounds familiar to your enterprise.. 💡 Decades of proprietary intelligence locked in unstructured data, burried across silos 💡 It's not humanly possible to make sense of millions of data points and tease out the insights that define your competitive advantage 💡 You are looking for trusted AI that can understand the nuances of your specific domain (vs. prioritizing sounding like Scarlett Johansson) ...you need watsonx - a Data & AI platform that's built for business! We are augmenting decision making across every industry by following a consistent playbook: ☑ Step 1 - Start with a trusted base model that delivers performance & transparency, broad commercial rights & Indemnification ☑ Step 2 - Efficiently represent your enterprise data in the model ☑ Step 3 - Deploy, scale and create value with AI anywhere Learn more about 'Future of AI is Open' in Dario's talk https://lnkd.in/g-EUFaTz Deeper dive into AI for sports: ⚽ https://ibm.biz/BdmLAK for more info on IBM-Sevilla's GenAI partnership ⛳ https://www.ibm.com/sports for more examples of IBM's GenAI partnerships in sports (Masters, US Open, Wimbledon, ESPN etc.) 🎾 https://lnkd.in/gKrSpngq for latest on watsonx at Wimbeldon Super proud of the Sevilla FC partnership led by IBM client engineering team Arturo Guerrero Díaz-Pintado and Miguel Gimenez de Castro, Fernando Suárez León, Victoria Gómez. Congrats to the IBM team for consistently focusing on Trusted AI for Business Arvind Krishna, Rob Thomas, Dario Gil, Dinesh Nirmal, Kareem Yusuf Ph.D, Ritika Gunnar, Madison Gooch, Matt Kosinski, Armand Ruiz, Mohamad Ali, Kelly Chambliss, Matthew Candy, Salima Lin, Javier Olaizola Casín, Jesús Mantas, Glenn Finch & team -- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv

🥇 🥈 🥉 Battle of the Titans !! with Nvidia selling power ups ! Let this graph sink in.. OpenAI has been the undefeated Alpha Titan since 2022.. and within the last ~100 days: ▶ Claude briefly took the crown, then back to OpenAI ▶ Google edged ahead and OpenAI won it back again ▶ and now Claude 3.5 stole the crown again In those ~100 days Jensen Huang added 💰$1.1 Trillion💰to NVIDIA market cap. Disclaimers: ☑ Public benchmarks aren't represantative of real world performance ☑ The right way is to evaluate performance per $ vs. just raw performance ☑ This analysis takes an average of the popular benchmarks to create a blended metric, but they are equally important for enterprises ☑ Without transparency of training data, we can't be sure if the benchmark test data wasn't leveraged Talk to our IBM Consulting team to create your own benchmarks and test data that represent your specific usecases you have in production. It's critical to focus on the full orchestration - before and after the LLM to unlock value. Enterprises should take a very modular approach to building LLM-based apps, with a maniacal focus on evals and value tracking. -- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv

Anthropic just had a 🎤💥 mic-drop moment ! This is how you flex 💪🏽 - you take your middle sized model, and put it on steroids to outcompete your own best-in-class expensive model & the leading vision model. Oh, and deliver 2x faster responses, 80% cheaper. It's incredible how well it's doing on benchmarks and taken over the crown from OpenAI GPT 4o so quickly. Take the time to review these video that showcase the power of Claude 3.5 Sonnet. 🎦 Vision for documents, graphs: https://lnkd.in/equ6YmNR 🎦 As a writing partner: https://lnkd.in/eV8DpB8i 🎦 For agentic coding: https://lnkd.in/emu8HP2e 🎦 For sparking creativity: https://lnkd.in/eYzYyGez ▶ $3 for 1M input tokens, $15 for 1M output tokens ▶ 200K token context window ▶ Claude 3.5 Sonnet is ASL-2 on safety ▶ Available on Anthropic, Amazon Bedrock & Google Vertex AI ▶ Claude 3.5 Haiku & 3.5 Opus coming soon ▶ New feature Artifacts help better interactions with the model, and will soon support collaboration. ▶ Full blog here: https://lnkd.in/gwk4bxn7 But remember, in the big picture, these are marginal improvements on public benchmarks that may or may not translate into meaningful performance in real world business usecases. Our team at IBM Consulting builds custom evals and benchmarks leveraging client/domain specific data to thoroughly test each new model on what matters to your business. 🤙a friend to talk : ) Now I even more anxiously waiting for titans - Claude 3.5 Opus, Meta's LLAMA 3 400B, and eventually GPT 5. It's Game on ! Huge congrats to Dario Amodei, Daniela Amodei & team ! you guys crushed it. -- 🔔 If you like such content, I encourage you to connect, comment and repost ♻️ Subscribe to my free ‘AI with Whiskey’ newsletter https://lnkd.in/gK4zrSYv