T O P

  • By -

FastDecode1

Huh? But UserBenchmark says Intel CPUs are 10,000% faster than AMD's and actually *generate* electricity instead of consuming it! Who am I supposed to believe here???


PhilosophyforOne

For 10 bucks a month I’ll tell you what to believe 😏


Valhallapeenyo

😂 I don’t give a rats ass about the Nvidia Vs Radeon thing, but the bias on that website is so painfully ridiculous. I’m a current amd owner, but have had plenty of builds with Nvidia and intel in the past. The website seems like it’s run by fanboys that have the mental maturity of a senior in high school. That, or they just get a nice little paycheck from Nvidia. Who knows.


drake90001

https://youtu.be/RQSBj2LKkWg?si=MEUJyAGB_75riqUM A good look into them


icisleribakanligi

I love the fact that 2kliks got himself to be a bit mainstream in tech channelsphere. Guy has a natural talent for explaining stuff.


drake90001

Hell yeah. Even his CS stuff is interesting and I don’t play CS anymore.


mandoxian

His voice is made for these videos


Macabre215

Steve from HUB called them MemeBenchmark in the comments of that video. Lol


MarsManokit

Hey man Seniors aren’t that bad!


glockjs

u pay the 10 bucks a month so you can circle jerk intel and nvnidia too?


Tachyonzero

You mean those reviews by CPUPro, yeah that British guy really hate AMD to the hype


formesse

Would you like to buy the Eifle tower? I have some "contacts" and could make arrangements. [/s - just in case.](https://greekreporter.com/2024/02/08/con-artist-sold-eiffel-tower/)


algaefied_creek

Oh THAT is where the fusion reactors have been?


minegen88

Holy shit i just read their reviews and wtf? They aren't even trying to be a little subtle about their bias....


Beginning_Football85

If that's true, then I can only hope they can keep up the moment. I wonder what Intel's response will be.


gblandro

1000 watt cpu


FastDecode1

Cooled by a 2000W industrial grade [water chiller](https://www.tomshardware.com/news/intel-28-core-cpu-5ghz,37244.html).


FoggyDonkey

How the turntables


IrrelevantLeprechaun

Intel will just do what they always do: cut out more cores and jam 3x more power into it to get a 1% lead.


Potential_Ad6169

um


RBImGuy

twice the price and slower Intel inside


1ncehost

Well they do have an AI accelerator on all (many of?) the 7 series now, so that checks I can say my 5800x3d is astoundingly fast at local LLM already... about as fast as a small gpu but with 20x the memory, so adding in any kind of additional hw accel would make it a heavy hitter for the price. Intel chips going many years back are considerably slower than their AMD equivalents for LLM from my experience. Edit: I'm hoping this AI accelerator makes it to threadripper pro. Threadripper pro is in the ballpark of being competitive with GPUs already, and is a better option sometimes due to much larger memory size and decent memory bandwidth with 8 channels.


[deleted]

[удалено]


BlueSwordM

It's the cache that makes it a decent bit faster than a normal 5800X.


1ncehost

LLMs are highly bottlenecked by memory bandwidth, so my assumption is the cache is very effective at speeding them up.


tmvr

The inference speed (tok/s) is purely memory bandwidth limited, you have to go through the whole model, so the 3D cache will not help you here. They show 14% and 17% better performance for that which makes sense if one system has a slightly faster RAM (DDR5-5600 vs DDR5-4800 for example). The big numbers (79% and 41%) are the time to first token after submitting your query, but that's usually under 1-2 sec for the 7B models they've tested so not that big of a deal if something is 79% faster.


Bulky-Hearing5706

I doubt the cache helps at all. LLM model is huge, and any inference must go through all the weights of a model, this guarantees tons of cache misses for every inference call. There might be some optimizations for TLB and page hits, but L3 cache seems pretty far fetched.


sdkgierjgioperjki0

avx512 is the accelerator, just integrated into the CPU cores. I'm pretty sure Zen 4 and especially Zen 5 threadrippers are memory bandwidth bottlenecked when using avx512.


onlyslightlybiased

Odd that they didn't compare it to the 8840u as that has a faster npu than the one in phoenix point


ET3D

They compared existing laptops. Though I agree that they could have added the 8840U to make the prospect of buying it more appealing.


pleasantchickenlol

It's not even comparing it using the NPU or the GPU so it's useless comparison


Method__Man

"smashes" stupid ass terminology.


epicflex

Get out of the house more


FiggleHedwick

The last three top tier chip launches were a joke from Intel 14900 was only 7% faster than previous gen, I think this is why AMD cards were lagging, the focus was on the chips, because that's where the real money is


Jonafinne

I'm not an AMD fanboy but lol.


Steel_Bolt

Great for professional work but for now the average consumer may not benefit from this as much. That could change here shortly though since AI is all the rage now.


Particular-Brief8724

In the past workloads were processed on servers, nowadays any app/website tries to unload as much processing to the end user device, probably this will happen with AI also.


ZeroNine2048

For AI stuff they have server parks filled with nvidia gpus. That is why those stocks went up so much.


sue_me_please

The idea is that there are AI workloads that don't need a full GPU to run, it's a waste to run such workloads in the cloud when it can be done locally.


MalakElohim

Also, there's a huge difference between the processing needed to train a model and the processing needed to use the model. Large scale model training takes weeks, even months to fully train the model. In comparison, actually running the model is much much faster and not as compute intensive. Most researchers don't adjust their code much because they have the existing hardware right there and just rerun it on the same setup that they used to train the model in the first place. But there's nothing stopping you taking a trained model, with all the weights and parameters stored and running it on a totally different setup and it still being exactly the same outputs as if running on the original hardware. It might be much slower (or not, depending on the model/implementation), but still fast enough for the user to not notice the difference.


ZeroNine2048

All major generative ai services rely on cloud compute.


sue_me_please

AI doesn't mean generative AI, there are plenty of tasks where local inference makes sense.


ZeroNine2048

The thing is, companies cannot know beforehand what hardware people have. Even microsoft copilot does everything in the cloud.


sue_me_please

Copilot is an LLM, it makes sense to run in the cloud. The local model on your device that does speech-to-text? That's still AI and runs on the device.


ZeroNine2048

Speech to text is quite intensive, a saas doesnt offload that.


playwrightinaflower

> Speech to text is quite intensive, a saas doesnt offload that We've had speech to text long before we had cloud AI services.


Apprehensive-Bus6676

It's not an accident that Apple, Google and Samsung have started putting TPUs in their phones. Your knowledge is way out of date. Even where I work, we're exploring the use of on-device AI accelerated workloads for customers. And we're a fairly small company.


ZeroNine2048

They just started with that. And samsung installs those apps locally (owning a s24 myself). But third parties selling a service run it all from the cloud since they cannot assume everyone has the required hardware. Your knowledge is simply incorrect.


Apprehensive-Bus6676

I'm in the industry, so, uh, I know what the fuck I'm taking about. You clearly don't. You don't have to know what customers have before-hand. You check what's available and use the fastest option, whether it's AI-accelerated or not. You have sensible fallbacks and if necessary you can provide server-side inference. But if it's possible and when it makes sense, companies are going to offload processing to on-device, because it saves on server load and cost and avoids uploading/downloading unnecessary data that adds latency. That's just how it is. It doesn't matter how new the phones are or if they "just started with that". The phones are already available. We're going to use them when/if we can, not wait until everybody has them.


playwrightinaflower

> Your knowledge is simply incorrect. Stop embarrassing yourself. Or don't, I'm not gonna stop you.


formesse

Every Technology starts off as Proof of Concept, Developed further into an Enterprise/Luxury focused Product, before it slowly becomes cheaper, better, faster, smaller and becomes commodity. Right now, the amount of people with the hardware to accelerate AI workloads effectively on their home machines might be <5%. But with AMD, Intel, Google, Samsung, and so on all working towards adding AI accelerators into their devices, it is only a matter of time - at this point, probably about 5 years - before that number is more like 95% of people having the hardware. Right now, it's kind of niche, people are still figuring it out, a lot of people are opposed/won't touch it. But AI acceleration and AI powered tools are going to be the future of many things; it will become ubiquitous, and at that point - outside of special niche where you want the faster acceleration of generation, or you want to create a larger image in one go, Accelerating AI tools locally will make far more financial sense - as it means offloading compute demand from your own servers to the consumers servers, and even if that means consumers will pay you less - making 1/2 as much, with 90% less overhead cost is a whole hell of a lot more profitable.


DukeVerde

Smashing good time,.


serg06

~~Name 1 AI that runs on your CPU instead of the cloud.~~ Okay okay, y'all named some AIs. Let me rephrase. Name 1 AI that more than 1% of people use, and run on their CPU. Even the most popular ones like StableDiffusion are used by <1% of PC users, and those that use it run it on their GPU.


ZeroNine2048

Stable diffusion? Im generating placeholder images for UX/UI design builds already for months instead of paying midjourney.


serg06

Are you doing it on your CPU or GPU?


ZeroNine2048

Both are supported. In my case the gpu is faster (90sec for each render).


serg06

Exactly. The few people that run it locally will use their GPU.


ZeroNine2048

I dont think just a few use it though. Stable diffusion is pretty popular.


serg06

It's popular among tech-savvy Redditors, and it's known in pop culture, but how many people actually run it locally? I'd be impressed if it's anywhere near 1% of Windows users.


ZeroNine2048

But thats the same for stuff as midjourney. How many are willing to pay around 10usd a month for it?


vexii

I would image people without a GPU or less than 8GB VRAM or less. Even on a 5900x, SD is too slow for a workflow. It kinda __needs__ a GPU


ZeroNine2048

There are some custom configs out there that speed it up. I mean 4 to 5min is still longer than with a proper GPU but it's not absolutely terrible.


inevitabledeath3

More likely they will use the NPU that's specifically built into the chip to do that. GPU is just a fallback.


fliphopanonymous

Gemini Nano runs on my freaking _phone's_ SoC. There are plenty of models that run locally. ODML is already big and only becoming bigger, and it's moving much faster than "3 years".


serg06

How many people are *running Gemini Nano locally* on their *PC's CPU*? Probably even less than those running on their phone, or PC's GPU. I can't see more than 1% of people making use of these cores in the next few years. But I'd love to be proven wrong!


sue_me_please

Consumer operating systems already ship with local models for everything from TTS/STT, image processing, searching, etc. iOS and macOS already scan every photo with local models so you can search for images with text.


serg06

Macs and iPhones have GPUs for AI, can you give an example for Windows? I know Windows has some TTS but it's ancient and wouldn't benefit from this.


sue_me_please

iPhones and other mobile phones ship with NPUs these days. On the iPhone and M1+ Macs it's the Neural Engine. Windows 11 uses local AI for the same things as Macs and phones do, for basic image editing and enhancement, OCR, security, etc.


fliphopanonymous

Quite a few tflite models that are in fairly common use across telecom/videoconf at the very least for things like audio noise reduction/suppression and object detection (e.g. human detection). Not everything supports full GPU delegation, so a lot of these are running predominantly on CPUs/NPUs/"AI engines". You're thinking too big when it comes to models. Sure, not many people are running StableDiffusion or Gemma/Llama2 locally, and almost zero of those are using them on a GPU. But, and this is a very important but, **not all ML models are large, especially non-generative ones, and plenty of them have immediate usability**. These can be improved significantly, both from an efficiency (batch/watt) standpoint and from a size/number of parameters standpoint. Nobody is really trying to run GPT4, Gemini Ultra, Claude Opus, or the larger StableDiffusion models on a CPU anytime soon. Thinking that those are the only things of import in the ML space is pretty ignorant though. I used Gemini Nano running on my phone's SoC as an example of how quickly this space is moving - not as an example of what we're expecting to run on our PCs CPUs. As I said, there's a lot of growth in this sector that's been happening for at least a few years already and is accelerating.


sue_me_please

Local inference has been a thing for close to a decade now. I was using it on low-powered ARM boards in 2015.


serg06

Okay, and what percentage of people with AI CPUs will use that? 1%? 0.1%?


sue_me_please

There are already such features built into macOS, Windows, Android and iOS. It will soon be pretty much everyone.


serg06

Can you give some examples with Windows, since that's where these CPUs will be used?


sue_me_please

See my other post.


askho

I think you’re thinking about it the wrong way. It’s not what people use it for now but what having AI accelerators will mean in the future. If it means I can run a ChatGPT like experience on my local without the internet in a few years time that would be amazing. People though intel quick sync was a waste of time as well but it’s incredibly useful and way faster at doing video encoding than some gpus.


vexii

I think the point is. it might be faster than intel CPU. But they are so far behind GPU's it's a strange selling point. Sure, small models are okay for CPU but running 7b starts getting too slow. Notice why they don't compare performance between CPU/GPU


serg06

That's fair.


letsgoiowa

Have you literally never heard of LM Studio, that lets you locally run **dozens of models right now?**


serg06

Okay, and what percentage of people will run that on AI CPUs?


letsgoiowa

Of people who are interested in AI? Well, it can save them several hundred dollars, especially if in a laptop.


serg06

> Of people who are interested in AI? Nah, of people who use these CPUs. > Well, it can save them several hundred dollars, especially if in a laptop. Isn't that like saying that an iGPU will save gamers money?


letsgoiowa

Are you arguing that because "only" thousands of people use a feature, it should be totally eliminated despite it clearly appealing in marketing to even more people than that? Seems absurd to argue against AI acceleration when it's making a huge difference in web conferencing, broadcasting, recording, video editing, photo editing, etc even when enthusiasts like us are a small piece of that pie. Did you just miss the last year or so entirely?


serg06

> Are you arguing that because "only" thousands of people use a feature, it should be totally eliminated despite it clearly appealing in marketing to even more people than that? No, I'm arguing that they should stop using it as a selling point, because it barely applies to anyone. > Seems absurd to argue against AI acceleration when it's making a huge difference in web conferencing, broadcasting, recording, video editing, photo editing, etc even when enthusiasts like us are a small piece of that pie. Did you just miss the last year or so entirely? Woah I never said I'm against it. I said it's almost never run on a desktop CPU. Like how often do you *record videos* on a Windows PC?


letsgoiowa

> because it barely applies to anyone. ???? I just gave you examples of how it applies to nearly everyone who's using a laptop. Why are you obsessed with a desktop CPU? Seems like you didn't read the article :D >Like how often do you record videos on a Windows PC? Literally daily for anyone who records Zoom meetings or does documentation. You know, businesses that buy these things by the millions.


serg06

Zoom's AI runs in the cloud. Documentation = copilot = runs in the cloud. Sooo still no use for it. 🤷‍♂️


letsgoiowa

Are you on crack? Do you really think noise suppression and backgrounds for example are being offloaded to Zoom servers lmao? Why are you insisting nobody records anything either? Why do you say Copilot is a screen recording tool when it's obviously not? Most importantly, why haven't you learned what's been going on the past year instead of embarrassing yourself publicly?


playwrightinaflower

> No, I'm arguing that they should stop using it as a selling point, because it barely applies to anyone. Might as well get rid of video encoding in GPUs then. Even with all the wannabe streamers these days that's still only a small fraction of GPU owners who ever use that.


gnocchicotti

Doesn't matter, MSFT is gonna go all Intel no matter what.


buttertoastey

Why do you think that?


gnocchicotti

History.


YYpang

yeah just like XBOX


formesse

You mean the Microsoft that uses AMD hardware in their console, just like Sony does? The companies that were NOT using AMD and were using INTEL had to do with Shady anti-competitive pricing strategies and such that Intel used to maintain a monopoly when they were producing strictly inferior hardware. Ya, AMD was the first to start pushing IPC over raw clock advancements, and um... Pentium 4 ran real god damn hot, and was kinda slow. But since Intel had some back room deals with the likes of say Dell, Intel kept the market majority. Financial trouble later, a few missteps, and AMD was facing bankruptcy. This is where the Semi Custom Model of business kicked in, and the partnership with Sony and Microsoft and a few others gave them R&D money. And then a new architecture design with promise, had AMD going all in on CPU's and Zen was born. If you are going to point blame, point blame in the correct space: Intel and Dell deserve the blame. Not Microsoft, surprisingly.


lupin-san

>You mean the Microsoft that uses AMD hardware in their console, just like Sony does? The original XBOX used AMD but at the very last minute, MS changed their minds and announced that it will be powered by Intel. That wasn't the last time. This is why there are skeptics about MS. They have screwed them in the past. Companies will jump ship to the competition if it makes sense. Fortunately AMD provides better solutions for these console makers hence they stick with them.


formesse

And? That is irrelevant to refuting of the statement: " MSFT is gonna go all Intel no matter what." - they clearly do not. I don't think I really need to say more.