The #AIHardwareSummit is just around the corner, and I'm excited to see the latest breakthroughs but before I dive into the details next week, I wanted to zoom out and consider the bigger picture...
There have been HUGE changes since the last Summit. Models are now trillions of parameters while token costs are approaching zero, both of which seemed far off twelve months ago.
The proliferation of democratic AI seems much closer to becoming reality but making AI ubiquitous won't be based on any one company. It’s going to take a global community of companies and each has to determine how to offer exceptional hardware performance, scalability, and attractive cost of ownership to create success. That’s challenging calculus to get right.
In a recent conversation with Bob Beachler he described what he thinks is a “sweet spot” on the spectrum of specificity and generalization. Untether is building general-purpose accelerators with excellent flexibility and scalability, owing to their at-memory compute architecture and emphasis on software development.
The VP of Product at Untether AI knows a thing or two about this topic - he’s insatiable about studying the market and listening to their customers, so he recognizes the hazards of indexing too far one way or the other.
He told me, “We’ve literally gotten hundreds of models from our customers saying, ‘we don’t want to run this model, we’ve changed the architecture a little bit.’ So, understanding that and making sure that you keep your architecture optimized for inference acceleration but flexible enough so you can adapt to what’s going to happen in the future is really important.”
Untether seems to have dialed in their recipe for success in this rapidly growing and competitive arena. In the most recent ML Perf analysis, they won a battery of tests including the highest single-card throughput in Datacenter and Edge categories for ResNet-50 and the fastest-ever recorded latency for ResNet-50, along with other accolades.
Though he’s not presenting at this year’s Summit, Bob’s insights will resonate with me as food for thought to sharpen my focus at the Summit, where I’ll ask other product leaders and founders where they stand in helping to usher in the “Era of AI Everywhere”.
My utmost appreciation goes to Bob for sharing his perspectives, which you should hear in the video below. Then share yours in the comments… do you think the future of AI is about more powerful generalization, increased specificity, or something in between?
#semiconductorindustry #artificialintelligence #aihardwaresummit
