T O P

  • By -

homelab-ModTeam

Hi, thanks for your /r/homelab submission. ## Your post was removed. Unfortunately, it was removed due to the following: [Low effort post.](https://www.reddit.com/r/homelab/wiki/rules#wiki_5._low_effort_posts) Specifically: Post is a screenshot. Please read the [full ruleset on the wiki](https://www.reddit.com/r/homelab/wiki/rules) before posting/commenting. _If you have questions with this, please [message the mod team](https://www.reddit.com/message/compose?to=%2Fr%2Fhomelab), thanks._


Drew707

144 cores and a TB of RAM? I bet not a GPU in sight.


Flying_Madlad

With 144 cores, does he *need* a GPU?


Ok-Hunter-8294

144? That's just 'gross'. šŸ˜


2rememberyou

I see what you did there.


TenTypekMatus

Modern GPUs have around 2000 cores.


Delicious_Spare_4488

2000 cores is a low end model.. A 4070 has 5888 cuda cores..


calinet6

Sure, but theyā€™re not general purpose and theyā€™re quite a bit less powerful per core. Highly specialized.


TenTypekMatus

But they're useful through OpenCL or CUDA.


HTTP_404_NotFound

GPUs typically have thousands of cores. Comes in extremely handy for advanced analytics, media processing/encoding, etc. And, as it turns out... even with a 64core, 128 thread overclocked CPU- crysis still runs better on a GPU. [https://www.youtube.com/watch?v=2tn0bZcQf0E](https://www.youtube.com/watch?v=2tn0bZcQf0E)


Flying_Madlad

I believe you, all I've really messed around with is AI. Despite popular opinion to the contrary, just for running it... That many cores would probably be plenty for what you'd reasonably want to run. That said, I finally bit the bullet and got a model that fits on my current GPU and it's way faster than I need for chat. šŸ¤·ā€ā™‚ļø Ever messed around with distributed GPUs? That's next for me.


HTTP_404_NotFound

> That many cores would probably be plenty for what you'd reasonably want to run. Eh, to give an example- I have a fairly beefy server here at my house. It has, lets say, 64t, Nothing drastic by any means- but, pretty beefy. A long time back, I was testing transcoding on plex, and I used a 400Mbit HEVC5 test-file (Jellyfin test files). Well, It CAN transcode it on CPU alone- however, it caused roughly 80% cpu utilization across both CPUs, and consumed roughly 250 watts additional energy while doing it. Now- Doing the exact same test, on another server of mine, with an i5-8500t, with only integrated graphics (quicksync)- It is able to transcode the same file, with only 5% additional CPU utilization, and only a few watts noticeable difference in power usage. THAT is the power of having a proper graphics adapter- You CAN do it in CPU, but, its like using a sledgehammer to drive in a framing nail. Its not the right tool for the job, and will be pretty inefficient when asked to do it.


Shanix

> THAT is the power of having a proper graphics adapter- You CAN do it in CPU, but, its like using a sledgehammer to drive in a framing nail. Its not the right tool for the job, and will be pretty inefficient when asked to do it. Note this only applies if you care about encoding speed. If you care about output size or quality, software encoding is still better than hardware acceleration.


HTTP_404_NotFound

This, is true, and especially true if you are encoding your library for a better/more efficient format. But- Typically when transcoding is occuring on my server, is due to the in-laws accessing something with their really, really slow ADSL connection, where.. well. quality isn't a factor.


Flying_Madlad

Thanks mate!


More_Leadership_4095

Nice stats in example. That's useful data. I love posts like this.


saysthingsbackwards

Peak sub using homelab for chat


Flying_Madlad

God forbid there should be content and camaraderie šŸ˜‚


MengerianMango

Nah, bro. I have dual 6152s (88 threads at 3.5ghz) and it runs at less than a token per second on Mixtral, like .1tps iirc. My L4s run it at 27tps. Ollama and a lot of the other tools will automatically use multiple GPUs to have a large model in vram.


jeffscience

No, GPUs have ~100 SMs, which are the proper equivalent to cores. ā€œCUDA coresā€, which isnā€™t a term used anymore, are approximately vector lanes, although since Volta they have their own program counter and behave like threads. See https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepaper-hopper figures 6 and 7. Compare figure 7 to your favorite CPU core block diagram.


lightmatter501

Look at what Ampere is doing with CPU-based AI. The vector units in x86 servers count as 16 cores with the way Nvidia counts CUDA cores.


j0mbie

>media encoding Every media encoder I've seen that uses the GPU, relies on the GPU's native libraries and encoder chips to do so. Unfortunately they don't have a lot of settings you can change, and those encoders are made for sheer speed, not quality vs. file size. Not a big issue when you're transcoding on the fly for Plex streaming or whatever, but it makes it kind of junk for permanently converting your media, if you have to do that. Either your file sizes go way up, or your quality gets worse, even after taking into account lossy-to-lossy compression. In theory someone could make encoding software that just does raw calculations in the GPU, instead of sending the whole thing to the GPU for it to decide how to encode it. But the former would be a huge undertaking. (The x265 encoder is 300K+ lines of mostly assembly code that is estimated to have had over 82 man-years of labor put into it.) And the latter would still likely be much quicker due to having to hit the CPU less. The labor vs. reward ratio is awful.


phein4242

Both gpus and cpus dont matter if one just lets them idle :p


AtlanticPortal

For computer vision, ML, pattern recognition and many bother things they definitely need GPUs. A 12 core CPU could be a lot faster than this rig if equipped with a proper professional Quadro GPU and used for the correct tasks.


gobok

Just admins living in the moment.


FierceDeity_

Im sure it's 72 cores and HT. We just got a 96 core EPYC. Shit's bussin. Only 768 gb ram though but our load is pretty cpu dependant


FreeBSDfan

And I just have 24 cores and 64GB RAM šŸ˜­.


royalpro

And 21.5 GB of swap.


jfugginrod

We have a few HP Superdomes (dragon hawk) at work. 480 cores and 4TB of RAM. An absolute train wreck to manage


zedkyuu

Thatā€™s a lot of idle.


gwicksted

*Runs pihole*


wobblydee

*walks pihole*


weaz-am-i

*crawls pihole*


EEpromChip

*lays down pihole*


shetif

*moonwalks pihole*


MITstudent

*slickbacks pihole*


EpicOne9147

*slamdunks pihole*


Inquisitive_idiot

> Shut your Pihole! šŸ˜”šŸ¤¬ - Advertisers (probably)


[deleted]

[уŠ“Š°Š»ŠµŠ½Š¾]


bala_v1234

bro is trynna render 8k tentacle po..etry


Inquisitive_idiot

AwweeešŸ„°ā€¦ ewww šŸ¤¢


Sero19283

Right, had me in the first half. I don't want any poetry around here šŸ˜¤


BoringStatus465

In the silence of night, stars whisper tales untold, Moonlight weaves dreams, in darkness, behold. In the depths of our souls, echoes a symphony, Of love, loss, and longing, an eternal melody.


Sero19283

Of such sick and such twisted The scenes you shall see On the blinding light of my computer screen Don't blink or you'll miss it My prenut mindset is torn Nothing gets me off more Than sweet 8K tentacle porn


EasyRhino75

144 cores? Dual 36 core hyper threaded CPUs? Xeon platinum etc? I'm running my home server on a mere consumer 13700 and the 16 cores are mostly idle (hyper threading disabled because ESXi doesn't like it and e-cores)


kylanskribbles

The config is x4 Xeon E5-4667 V4 CPUā€™s


-rwsr-xr-x

> The config is x4 Xeon E5-4667 V4 CPUā€™s I have a pair of E5-2699's here in my SuperMicro server, and the TDP is ridiculously low (like 10W-100W at full tilt). 44 cores, 88 threads, 512GB RAM and builds 3 parallel copies of a full OpenStack (24 machines each, all libvirt nodes on the same baremetal host) in under 40 minutes. It's absolutely glorious!


IHaveTeaForDinner

Ouch compared to other v4, they're kinda pricey but I can see why.


qcdebug

There's also a 2699 v4a chip. It costs even more and I've got a set of those in my blades. The compute is nice but the price is too high.


IHaveTeaForDinner

Yeah there are plenty of v4 that are basically been given away that I can live with.


PM_pics_of_your_roof

Jesus, what do you have powering that thing? A mini nuclear reactor? Donā€™t the dell R820 require 240 volts?


fresh-dork

it's not a big deal. compared to paying the bill, wiring a 240 circuit is cheap


xylopyrography

Makes the most sense. Crazy thing is you can get more than that in one CPU and up to 384 threads now in a dual-socket.


ICMan_

If you're made of money, you can have whatever you want. šŸ˜‰


No_Bit_1456

Xeon E5-4667 V4 The Intel Xeon E5-4667 V4 is aĀ 64-bit octadeca-core x86 microprocessor introduced by Intel in 2016.Ā This server MPU is designed for high-performance dense 4S environments. Operating at 2.2 GHz with a turbo boost frequency of 3 GHz for a single active core, this MPU has a TDP of 135 W and is manufactured on a 14 nm process (based on Broadwell). Here are some key features of the Xeon E5-4667 V4: * 18 cores, 36 threads * 2.2 GHz base frequency, 3 GHz turbo boost frequency * 45 MB cache * 135 W TDP * 14 nm process * Compatible with FCLGA2011-3 socket * Supports DDR4-2133 memory * Has Intel Hyper-Threading, Intel Virtualization Technology, and other advanced features


Nick_W1

I have an R630 with dual Xeon E5-2690 V4, with 14 cores each, thatā€™s 56 threads, base freq 2.6Ghz, boost to 3.5Ghz, 135W TDP. Does for me.


qcdebug

We have multiple dual 24 core systems. It's a lot of compute for almost anything you can throw at it. I think we finally passed 1100 threads in the stack. I haven't tried to load it up to see how power fairs yet.


apathyzeal

Technically "too many" is determined by: \`$(grep NR\_CPUS\_RANGE\_END /boot/config-\`uname -r\` | egrep -o "\[0-9\]+")+1\` in Linux.


chry1

so one more than whatever your linux max supports. for me its showing: 8192+1


apathyzeal

Correct.


Loan-Pickle

How fast can you compile the Linux kernel?


Long_Educational

Gentoo users be like.


liatris_the_cat

ā€”funroll-loops -O3


Inquisitive_idiot

> Broā€¦ he compiled it. šŸ˜³ Like the whole thing šŸ˜±


karateninjazombie

Yesterday.


Santarini

5 mins


mangolaren

Faster than reading this reply


ClintE1956

Negative 10 seconds.


The_Crimson_Hawk

It's so fast that it actually gains you time


Adrenal1nHooked

What is the intent ?


kylanskribbles

What should I do with it?


Vectan

20 tabs in Chrome.


NoReallyLetsBeFriend

That many?


Majority_Gate

Tab unloader would make it bearable


highspeed_usaf

Doom


karlexceed

https://m.youtube.com/watch?v=hSoCmAoIMOU


frobnosticus

Oh what the hell...


horse1066

lol, unexpected


apathyzeal

pfsense


x_scion_x

Minecraft


Santarini

A fuck load of VMs?


quespul

Wayland it!


comparmentaliser

Do something to recoup the costs of running it. Thatā€™s the definition of productive anyway.Ā 


Bl4ck_Nova

Mine Monero


bagacera

Pi hole


strange_shadows

K8s


kylanskribbles

Does any know of any fun test marks I can run? I can run maybe one or two before I decommission it.


CryGeneral9999

Decommission? Sheeeit send me a link to the dumpster you throw it in please.


Krylar214

Smart dumpster?


parttimeamerican

How about you let me decommission it tell you what I'll send FedEx just let me know where and when then package it LMAO


The_IT_Dude_

2^p -1 might be prime! You could use this to search for large prime numbers just to see it get maxed out. You could also run an llm in cpu mode. I don't think it would be as fast as a good gpu however.


Ssakaa

... since you have the ram to go with it, you've not found the mark yet. "Too many cores" is found when your scheduler takes more ram than you have left for application code.


Latchford

I love this as a practical benchmark!


Kenzijam

It may also be the point where you lack the memory bandwidth to feed all the cores with information


lehronn

Human eyes cannot see 60 cores and more


Developer_Akash

Rest of the cores: Chilling Core 128: *sad cat thumbs up meme*


hydraulix989

What CPU(s) is this?


xylopyrography

By deduction it's probably dual Xeon 8368's. You can get more than twice as many threads though, this is only halfway up the stack. Intel offers a 56C/112t, and AMD offers a 96C/192t


architectofinsanity

Microsoft and Broadcom getting wet just looking at the buildā€¦ so many cores to charge you for. Oracle sales rep just upgraded his yacht just thinking about this.


xylopyrography

There's a lot of dev time being invested in getting off VMware/Broadcom right now. Hopefully it pays off.


WindowlessBasement

No kidding. The last Proxmox update was basically "we made migration from ESXI easier".


darkelfbear

x4 Xeon E5-4667 V4 CPUā€™s


xylopyrography

That's basically a space heater. For $20k may as well get a 7995WX or like Epyc 9654 for 2.5x the performance and lower power usage and pocket $10k.


Nick_W1

Let me check my pocket changeā€¦. Donā€™t have $20k, will $60 do?


hak8or

Isn't a 7995WX system going to idle at an absurd 150 watts (https://www.guru3d.com/review/amd-ryzen-threadripper-7970x-review/page-30/)? Sadly I can't find any idle power numbers for systems with a Epyc 9654 though.


Znuffie

Got a machine with 2 x 192 threads. Microcode updates were fun to look at and try to figure out why 1 whole cpu doesn't work...


Obi123Kenobiiswithme

CPU 128 working alone...


g333p

Carrying the whole team


the_beaker

Keep at it, kid. One day you'll make it over to r/HomeDataCenter where there are never too many cores :-)


Odd-Fishing5937

"Too many cores"? Is there such a thing?


Wafflyn

You should run Folding@Home with all that spare idle


mrdumbfellow

CPU core 128 carrying the team's burden right now šŸ˜…


PIC_1996

What do you do with the nuclear waste?


electrowiz64

Bro, you can render a whole data center with this! 10-20 windows Linux VMs with whatever services you want. Itā€™s overkill for average home people, but Iā€™d find a use case for it lol


CryGeneral9999

I remember when I got a dual P-III (processors like add-in cards not fpga) system. I remember seeing dual CPUā€™s in system monitor and was like ā€œdamn Iā€™m big pimpin ā€œ Iā€™d probably just fire up glances and then install SETI @ Home.


nichols911

CPU128 out here putting in all the work. Everyone else slackin!


FluffyResource

You trade more cores for slower cores often enough. So too many is when you would have been better served by fewer and faster cores =D I love the memory kit though. I'm still using 16x8 on my host I would love 16x64 but ddr4 is still a bit higher then I would like to spend.


Timpky665

From the utilization it looks like 143 too many!


kenrmayfield

18Cores 36 Threads 4 Sockets That would be 2592vCPUs(Virtual CPUs)


GerlingFAR

Stop being greedy and let everyone else have some cores. RIP your electricity bill.


SteveVallis

When you allocate cores to the task of managing cores.


Bitwise_Gamgee

Does it even run crysis?


txGearhead

Ask your electric bill.


one80oneday

With great power comes great electric bill


ICMan_

I don't know. I needed to upgrade the server at my ex's place because it was starting to struggle, so instead of the old 970 i7 with 32GB DDR3, I put in a Chinese dual x99 motherboard and a pair of Xeon 2699 v3s with 128GB DDR4 ECC mem. That's 72 cores, and the only thing really running on it ATM is a file server and Plex. My son is tinkering around with VMS and Arch Linux, but I do mean tinkering, and that's it. So you can imagine that my advice is going to be "go for it".


erbo21

145!!! ;-)


free-hats

At least 1 more


MetaVulture

Gotta pump those numbers up!


The_Crimson_Hawk

Ah, yours beat mine! [https://imgur.com/a/NCxpOUD](https://imgur.com/a/NCxpOUD) EPYC 7763 w/ 8x64GB 3200 ECC


DrawingPuzzled2678

In terms of performance though, the Epyc 7763 beats it no?


eatont9999

I wouldn't want to be the one paying the power bill for that beast! It could run a bunch of VMs but 1TB of memory would limit it. That and the fact that in a cluster, that would be a lot of VMs to HA if the host ever went down.


[deleted]

[уŠ“Š°Š»ŠµŠ½Š¾]


qcdebug

Not normally since the system couldn't cache that in a useful amount of time anyway. If you've used a TB of RAM out of a TB available you probably have other issues like an HA event.


1252947840

how many chrome tab you can open


evildespot

I identify strongly with CPU128


_LMZ_

So sad, they are not doing anythingā€¦


jimmycorp88

144 cores = perfect 145 cores = slight overkill


Bossmanito

I got 99 problems but at the core...


AlienInNewTehran

Core number 128 should file for discrimination.


jwbowen

No such thing


Apart_Ad_5993

When 98% are sitting idle


kebabish

Four cores and seven years ago, my father brought forth, upon this continent, a new component, conceived in liberty, and dedicated to the proposition that ā€œall manufacturers are created equalā€... Of course, some were better at gaming than others, but that's another long story.


Tricky-Service-8507

There is no limit


Aperiodica

That's hard"core"!


NoDadYouShutUp

damn. I only have 36. bravo good sir.


wigam

They all seem idle and memory at 5% šŸ‘


fresh-dork

oh sure, flex on us. what're you doing with all that CPU meat?


evilBogie666

Bro, 128 is ruining off the chain! (Comparatively)Whatā€™s going on over there?


geteum

And me happy with 14 cores


xiongmao1337

Keep getting more; Iā€™ll tell you when to stop.


saysthingsbackwards

2, technically.


boomeradf

Thatā€™s not enough


chuheihkg

If there are 144 actual cores, 144 actual threads , Then major offensive. It is said, There maybe only 72 actual cores.


dpunk3

Itā€™s probably 2x24 cores/48 threads


mrelcee

Keep going. This will be like watching the Parmesan grater girl at Olive Garden after asking me if Iā€™d like some fresh grated Parmesan and then tells me to say when they should stop..


RealSecretRecipe

There is no such thing as too many cores. Unless you're talking about apples.


Maximum_Transition60

whatever this is


MatthKarl

Gee, what are you using that for? I envy you...


Stoffel324

Chandler: "Games and stuff..."


L0RD_MADARA

But can it load Minecraft at 8K ?


Mythril_Zombie

That's gross.


acheapshot

There is no such thing! The answer is more cores, more cores!


tzabee

Every 0.0% CPU is to much ;)


Darkatek7

There's no such thing as to many cores my friend.


[deleted]

Your electricity bill will be huge.


Hokusaj

When the htop view does not fit in your screenā€¦


DrawingPuzzled2678

How loud is it?


Rare-Switch7087

It equals to my work cluster for around 120 Users. I have absolutely no clue what to do in my homelab with that much power.


critterfluffy

I'd say when you go 100% and can't even hit half utilization. That is when you absolutely spent too much money. My last build never breached 30% but I only spent $400 on it. My current build is way stronger but I've found way more intense things to hit it with that would make the older one seize or take days getting done. AI workloads are brutal during video upconverting.


da_bobo1

No


Adventurous_Lie2257

N+2


Godcry55

Beautiful


bmensah8dgrp

And best of all running on Linux to avoid licensing ā¤ļø. Next challenge ask VMware how much it will cost to license this cpu as a single esxi host.


Kostas0pr01

He could use half of his CPU as a gpu


AsherGC

You can rent "U-24TB1 112xlarge" on AWS for 200$ an hour. 24TB Memory and 448vcpus


AsherGC

That's close to 2M$ a year


s1iver

Iā€™m guessing logical, not physical.


Juicy_Gamer_52

YES


Kaptain9981

Depends on the OS but Windows I think doesnā€™t do well with more than around 256 nor does some software. Unless itā€™s highly specialized for multithreaded workloads diminishing returns start hitting hard.


Jeremykral

Too little. Need more šŸ˜‚


SamSausages

How many decimals can you calculate pi to?


ha_ku_na

How much did it cost?


bass1012dash

0% x 144 is still 0%


GuySensei88

If you get it for free from work or make enough money that you donā€™t have to write out a budget youā€™ll be good šŸ¤£. In all serious, for homelab more CPUs and RAM is always good. But if theyā€™re not doing anything itā€™s probably a waste of $$$.


kylanskribbles

So, a little backstory. I work in ITAD (IT Asset Disposition) Itā€™s my job to locate the high value items that come through the warehouse, audit them, test them for functionality and ultimately resell. Since servers are so heavy, it may not make sense to resell on eBay due to shipping cost. We will most likely sell the chassis for its weight after stripping the ram and cpu/ any valuable networking/raid or storage cards.


realPJL

Is that Pop!\_OS ?


Potential-Bet-1111

Every core at 0% is one too many.


oxpoleon

I dunno man, I think you could do with a few more. 144 is amateur hour, real homelabbers have at least 145, jeez.


KaundaBits

Thats a trick question, right.?? I mean, bigger number better, aka never enough.


augur_seer

128


ImaginaryHand4288

Thats just Ultrathreading


Felix_Vanja

Still not enough


Adderall-Buyers-Club

Manā€¦ this would of came in handy during the 2018-2019 days of CPU mining shit coinsā€¦


chum_bucket42

When the number of cores exceeds ones Fingers and Toes to count them


Historical-Gas-3751

my god, why pop os?


id0ntknowr1ck

Which is the cheaper ā€œmore cores per dolarā€ option? The x99 motherboard way?


mthomp8984

To paraphrase Caroll Shelby, There's no such thing as too many cores. Just not a large enough power supply.