Together AI

Today marks an inflection point for open-source AI with the launch of AI at Meta Llama 3.1 405B, the largest openly available foundation model, that rivals the best closed-source models in AI, rapidly accelerating the adoption of open-source AI with developers and enterprises. We are excited to partner with Meta to bring all the Llama 3.1 models (8B, 70B, 405B, and LlamaGuard) to Together Inference and Together Fine-tuning. Together Inference delivers horizontal scalability with industry-leading performance of up to 80 tokens per second for Llama 3.1 405B and up to 400 tokens per second for Llama 3.1 8B, which is 1.9x to 4.5x faster than vLLM while maintaining full accuracy with Meta’s reference implementation across all models. Together Turbo endpoints are available at $0.18 for 8B and $0.88 for 70B, 17x lower cost than GPT-4o. This empowers developers and enterprises to build Generative AI applications at production scale in their chosen environment – Together Cloud (serverless or dedicated endpoints) or on private clouds. As the launch partner for the Llama 3.1 models, we're thrilled for customers to leverage the best performance, accuracy, and cost for their Generative AI workloads on the Together Platform while allowing them to keep ownership of their models and their data secure. Function calling is supported natively by each of the models, and JSON mode is available for the 8B and 70B models (coming soon for the 405B model). Together Turbo endpoints empower businesses to prioritize performance, quality, and price without compromise. It provides the most accurate quantization available for Llama-3.1 models, closely matching full-precision FP16 models. These advancements make Together Inference the fastest engine for NVIDIA GPUs and the most cost-effective solution for building with Llama 3.1 at scale. https://lnkd.in/gFwBNQhJ

We're back! Lux Capital is running a Bio x ML Hackathon with EvolutionaryScale and Enveda Biosciences Bio from Oct 10-20! By bringing together the world’s top minds, we’re driving forward the next frontier of science with AI-driven tools and scientific imagination. Build on top of the latest foundation models like ESM3 (98B parameters, GPU access) and proprietary datasets to predict protein activity, model across therapeutic modalities, and develop next-gen biology applications. Model APIs, datasets & compute powered by Amazon Web Services (AWS), OpenAI, NVIDIA, DigitalOcean, EvolutionaryScale, Modal, LatchBio, Together AI, Enveda Biosciences, and RunPod. Open globally. Get started here: hackathon.bio

Introducing the Together Rerank API: a new serverless endpoint for enhancing enterprise search and RAG systems. As the exclusive launch partner for Salesforce AI Research's LlamaRank, we're bringing a powerful reranker model to developers everywhere. This partnership combines Salesforce's AI expertise and enterprise search knowledge with Together's production-ready infrastructure, featuring fast inference and developer-friendly APIs. Key features of the Together Rerank API with LlamaRank: ▶ 8K context length for documents ▶️ Handles semi-structured data (JSON, email, tables, code) ▶️ Improves search relevance and reduces LLM compute costs in RAG systems ▶️ Compatible with Cohere's Rerank API for easy migration Developers can start using the Together Rerank API with LlamaRank today. Read more in our blog post: https://lnkd.in/g2Me2x7E We can’t wait to see how developers will enhance their search and RAG applications with this new API.

Congratulations to our partners at AI21 Labs on the release of their Jamba 1.5 models – powerful long context models for the enterprise. This language model family introduces: ▶️ 256K context window - longest among open models  ▶️ Up to 2.5X performance boost on long contexts  ▶️ Benchmark scores surpassing larger proprietary models  ▶️ Developer features: structured JSON output, function calling, citation mode  ▶️ Multilingual capabilities across 9 languages We’re proud to announce the upcoming addition of Jamba 1.5 to our platform, furthering our commitment to providing developers with access to state-of-the-art, open source AI models. 👏 Incredible work from AI21 Labs Jamba 1.5 – Coming Soon on Together.ai

Announcing Together Analytics in beta! You can now view your usage (requests, latency, and TPM) over time. This was a highly requested feature from our customers to be able to see their usage in a more granular level. You can now see usage over time and see successful requests, errors, latency, and TPM for all your usage on Together in one place. We're also already working to make it even better! Available today in beta for all customers, directly in your Together dashboard.

We’re excited to work with Together AI to bring open models to Daily Bots! Developers can build voice-to-voice AI and real-time multimodal agents with all three sizes of Llama 3.1. #llama #generativeai #ai

We made it simpler to run voice ai and partnered with Anthropic Cartesia Deepgram Together AI for this launch. Our goal is to combine the best tools, best developer ergonomics, and best infrastructure for real-time AI into a single platform. Daily will host and launch the bots for you, these bots are connected to STT, LLM, and TTS. We have several options under each category, plus you can bring your own API Keys. In addition, we offer function calling to third-party, or to your own infra (using webhooks and post requests). Further, since this is built on RTVI, you as a developer have the ability to augment pr change the config settings and services on the fly, mid-conversation. On the SDK front daily bots comes with a client component and a thin server component. Clients are an RTVI client app and we have web, Android, and iOS SDKs and sample apps. The server side is basically an init, the first request from the client initiates /bot/start. The server is also helpful in the case of telephony, wherein there isn’t a rtvi client per-say and you need a good default config, service and prompt. In an upcoming update you maybe be able to bind dtmf tones to external function calls, build a dynamic IVR for example.

Developers now can build voice-to-voice AI with Llama models! We’re excited to partner with Daily to bring the power of Llama models to ultra low latency voice and video agents.

Our guide: Optimizing GPU Clusters for Generative AI Model Training was featured on blockchain.news. Click to read a summary of insights into our comprehensive acceptance testing processes for new GPU clusters: https://lnkd.in/ek8t4amF Or check out our detailed process (with links to github repos and code snippets) in the comment below ↓

There's a dangerous sentiment described in the SB 1047 around how to make AI safer with legislation. Instead of sectoral regulation directed at nefarious activities, it aims to restrict the development and study of AI systems in the open. Historically, what's been successful is targeting uses: eg: hacking is illegal, but developing programming languages is not. The view in SB 1047 is equivalent of saying if a programming system can be used to develop anything harmful, then the development of a system should be restricted. I am quoted in the story below. SB 1047 should be viewed very critically as it can have a massive negative impact on AI safety (by limiting its study to a few large companies) and make it very difficult to build leading open-source AI models. https://lnkd.in/gT9t3Cgx “Senate Bill 1047 is fundamentally flawed as it targets AI technologies rather than their applications, posing a significant threat to the competitiveness of US AI companies, particularly smaller ones and open-source projects,” Vipul Ved Prakash, CEO and founder of Together AI, told PYMNTS. “We believe this bill will stifle innovation and unfairly burden startups. Open-source AI, crucial for responsible, sustainable and safe AI advancements, would suffer greatly.”