I have a business renting high performance compute. I want to democratize compute to make it more easily available on short term basis to anyone who wants access. I talked to TT about a year ago; also one of their customers as well. I was really impressed with everyone I talked to and I'd love to work with them.
What I realized though is that as much as I'd like to buy and host it and make it available, I'm not sure the economics or interest are there yet. The focus today is so dedicated to Nvidia that "fringe" gear is still just that. People who really want to play with it, will just buy it themselves. They've probably been doing that with hardware for a long time.
So, it is a bit of a catch-22 for me right now. Hopefully interest grows in these sorts of offerings and demand will pick up. The world needs alternatives to just Nvidia dominance of all of AI hardware and software.
> Hopefully interest grows in these sorts of offerings and demand will pick up.
Well, looking at their (as far as I can see highest end) accelerator the n300s we get:
- 24GB of memory
- 576GB/s of memory bandwidth
- $1400
As a hobbyist this is still not compelling enough to get excited and port my software - same amount of memory as a 4090/3090, half the bandwidth as a 4090/3090, slightly cheaper (than a 4090), more expensive (than a 3090), much worse software support. Why would I buy it over NVidia? This might be more compelling to bigger fish customers who would buy thousands of these (so then the lower price makes a difference), but you really need small fish people to use it too if you want to achieve good, widespread software support.
However if they'd at least double the amount of memory at the same price, now we'd be talking...
Yeah, clearly the low-hanging fruit is simply to offer more memory.
In a recent thread, just about everybody was talking about how Intel should have released its "Battlemage" line of cards with 48GB+ and how they'd actually have a very compelling offering on their hands had they done so. https://news.ycombinator.com/item?id=42308590
Missed opportunities.
That said, I know some of the Tenstorrent guys and they're extremely smart and capable. They'll figure it out, and the company is probably going to 10x.
> Yeah, clearly the low-hanging fruit is simply to offer more memory.
Yep. And just to be clear, this isn't necessarily a strategy to directly make money, as the market for people running these things locally is probably not very big (as a lot of people in the Intel thread said). But it's a strategy to beat the CUDA monopoly and get everyone and their dog to support your hardware with their software. I know I would be porting my software to their hardware if it was actually compelling for small fish, and I know plenty of other people who would too.
AMD also somewhat falls into this trap. They're slightly cheaper than NVidia, and their hardware is roughly good enough, but because their software stack and their hardware support sucks (just look at their ROCm compatibility list, it's a complete joke with, like, 3 models of GPUs, vs NVidia where I can use any GPU from the last 8 years) no one bothers. Being good enough is not good enough, you need to offer some sort of a unique selling point for people to tolerate the extra software issues they'll have with your hardware.
If someone built a mediocre GPU (~4070 level) with 48GB or 96GB, then the community would build the software stack for you.
Granted, you would not own that software and it would be ported to other cards in the future, but if you are trying to topple the king (nvidia) it would be a powerful strat.
Can you sell or re-sell colo space to a handful of customers who might put TT or other “weird” hardware there? Doesn’t scale, but it hedges your own business. Requires the right customer though, like somebody who might buy Nervana / Xeon Phi but then buy NVidia from you when it blows up.
Kind of. It is easier for us to be the capex/opex (meaning buy and run equipment) and then rent it out. I have the backing to make large investments without too much drama. We can do that with any hardware as long as we have longer (1-2 year) contracts in place. We have the space/power available in a top tier data center (Switch).
What I realized though is that as much as I'd like to buy and host it and make it available, I'm not sure the economics or interest are there yet. The focus today is so dedicated to Nvidia that "fringe" gear is still just that. People who really want to play with it, will just buy it themselves. They've probably been doing that with hardware for a long time.
So, it is a bit of a catch-22 for me right now. Hopefully interest grows in these sorts of offerings and demand will pick up. The world needs alternatives to just Nvidia dominance of all of AI hardware and software.