10 years ago 32GB ram laptops sounded too much. 8 was enough. These days even I would get that much ram since it’s soldered. 64GB is higher end.

In a few years we should see such high end hardware commonplace. Working with a local LLM to get work done is the ideal way to go which has mostly hardware limitation as of now that gets solved in due time.

Yep, Siri AI; they’re doing it in public.

The iPhone 15 was almost entirely marketed based upon AI (I would say fraudulently so, advertising features they still haven't delivered), and a huge portion of the OS work was on local AI or AI integration.

And for that matter Apple has been dumping enormous sums into their own AI development. Their failure to have a lot to show for it doesn't void the fact that they tried really, really hard.

It's bizarre how often this "Apple sat on the sidelines and let the AI people fight...so smart!" narrative appears on HN. Apple hasn't gone down the path of spending hundreds of billions on nvidia GPU data centres, but they absolutely tried really hard to matter in AI.

I just want my butt ugly repairable beast machine to do the same trick. Why is my ram not unified? I have an iGPU in my server, but it can't access the 64 GB ram (I got last year for 150 euro) directly or something? It's on the CPU right? Why did only Apple go for this architecture? So many questions...

Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels).

But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't.

However, macOS on M-series macs is excellent for local models. (Maybe not as excellent as a box full of the best nVidia GPUs, but still excellent).

So if you're getting Apple hardware, like Linux, and want to run all of it locally, a fine setup for a machine to run local models, with agentic characteristics:

- macOS running one of the many local model runners. I used to use Ollama and Whisper, and now use llama.cpp instead of Ollama. Others use LM Studio, oMLX, etc. Provide HTTP endpoints to access the models.

- Linux in a VM for overall control and orchestration, with standard VM settings, and bridged networking so it appears as its own machine on your network. Also, in here provide a robust shared file server for shared state. Use this VM as your desktop and primary access to the machine, if you like Linux.

- Linux in a VM to launch ephemeral, volatile containers, with the containers using a memory-only tmpfs overlay on top of a read-only Linux filesystem in a VM disk image, with tools in this filesystem. Alternatively, a writable Linux filesystem in a VM disk image, with disk buffering set to use macOS host buffering and discard fsync requests. These settings optimise for container disk performance for data that's only ephemeral which will be deleted soon or on system shutdown. (You can combined both VMs, but need to use two VM disks to get equivalent behaviour, and be careful about VM disk configuration of the two disks.)

- Containers spawned within that second Linux VM can be spawned very quickly and run quickly, so are ideal for LLM agents that need a quick sandbox. These sandboxes generally run faster than a macOS sandbox, despite being on the same machine with VM overhead, because Linux is faster at some things. Teach the LLMs to store files and memories they want to keep in the shared file server.

It would take Apple one or two engineers to make Linux life much easier on macs. But Linux is outside their walled garden so it's ignored.

My old 3090 is typically significantly faster (almost 2x token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.

In most situations it's a better idea to just buy tokens. But there are definitely cases when that's not an option. And then a machine like the M5 Ultra can allow you to do things locally for a fairly limited budget. And in a simpler package to manage than a machine with multiple GPUs.

FWIW, Ollama, LM Studio and Lemonade (and oMLX) also wrap Apple's MLX framework.

It's the Hardware, Software and Services in combination. None would work without the other (to reach the scale apple is)

More specifically, they're a hardware dongle company first

Depends on where you live. Also keep in mind population decline so long term, im not so sure. Prices already dropping in less desirable places (everywhere outside of most blue areas) (Assuming you are talking about the US sorry if not)

problem solved /s

They’re very different things.

The more logical argument to me is that Apple uses its upgrade price points as more than just direct BOM and rather as a proxy for things that are amortized across all their sales like support/warranty/etc so higher SKUs subsidize the costs of the lower ones.

What makes this expensive & sell well is it's not very fungible at the moment. Where else are you going to get 512 GB of high speed memory with a well supported accelerator attached that you can throw in the corner of anyone's home and not really have them notice? There are plenty of lesser options, plenty of noiser/power hungry options, plenty of harder to support options, but not really something in direct competition at the moment. Even the next rounds of the integrated AMD/Nvidia solutions are only targeting 196 GB of much slower memory and compute.

For one thing, you can’t tell from a movie what the specs are. A $999 Mac Studio looks exactly the same as a $20,000 one.

For another, Apple updates the industrial design on their products so rarely, a 6-year-old Mac, iMac or MacBook also looks nearly indistinguishable from a brand-new one.

Even using multiple windows in parallel for as many as 5-10 hours per day, I find that I am not fully using my claude max (20x) and chatgpt pro (20x) accounts. I can for sure use up the claude max account, but chatgpt either gives me a free reset before I run out of tokens or I just fail to use the full quota. The quota for Sol seems like 10x that of Claude Opus at the same level, and forget Fable, you can use a 5 hour quota in 20 minutes.

But lets do the math:

Lets say a 20k workstation can run 1 inference at a time at the same speed you get with Sol hosted by openai (big assumption) and run an equally capable model (big assumption).

Each month this gives you about 100-170 inference hours on a Sol 20x Pro account, and 720 hours (if you utilize 24/7) on the workstation.

Assuming a 36 month amortization before the workstation has to be replaced due to no longer being able to run frontier models or is too inefficient due to electrical costs or what have you:

The monthly workstation cost is about $550 capex and $150 electricity -> $700/month

You would need about 6 Pro accounts to reach that capacity, which would cost you $1200 a month.

But this fails because:

- You most likely can't utilize the workstation 24/7. Your work hours will be concentrated into 6-10 hours per day.

- During work hours you are capable of utilizing more than 1 concurrent session. 6 Sol accounts would support as many as 20-30 during working hours, not all the time but if you could burst to that many (don't forget sub-agents and agent directed parallel agent workloads).

- In 1 year the cost of Sol level models is likely to cost a fraction of what it does now.

this leads to:

                       Workstation   1 Sol Pro   2 Sol Pro

  Monthly cost            $700          $200        $400

  Raw capacity (hrs)      720           120         240

  Usable capacity (hrs)   100-130       120         240

  Concurrent sessions     1             3-5         6-10

  $ per usable hour       ~$6.00        $1.67       $1.67

  Usable hours per $700   ~115          ~420        ~420

I edit 4K ProRes and H.265 footage, sometimes with multicam (up to 4 streams) and color adjustments, titles, etc. It's only after stacking 3-5 effects before things can stutter, really.

Or if you try doing something CPU-intense in the background _while_ running some heavy creative software. I just don't do that.

E.g how is the perf/$ vs Wildcat lake

https://commission.europa.eu/news-and-media/news/safer-and-m...

https://www.psychologytoday.com/ca/blog/the-digital-self/202...

ELIZA beat the Turing test and then everyone forgot about it. Humans are just really terrible at recognising robots.

https://en.wikipedia.org/wiki/ELIZA_effect

It turns out the limiting factor isn't how sophisticated algorithms are, it's how gullible humans are.

I've tried [1] and I almost 100% detect which is the AI. I really want to convince myself I have failed, does anyone know of a better site/resource for this?

I know it might be moving goalposts but I would consider AI to have passed in a well and truly undisputed manner when [2] is resolved.

But in a more practical sense, if AI can impersonate humans so well today then why are state of the art frontier models so obviously AI when they create PRs, commit messages, documentation, etc. Are the companies deliberately making them unnatural?

[1] https://turingtest.live/

[2] https://www.metaculus.com/questions/11861/date-when-ai-passe...

https://en.wikipedia.org/wiki/Turing_test

And the "popularized" version is faulty also since it uses an ideal, abstract human judge (like the "spheroidal economic agent").

But if you want to add declinations to the said popularized image of the Turing test, you may add Maxim Lott's IQ tests at trackingai.org . Between the end of 2024 and the beginning of 2025 LLMs reached an equivalent IQ of 100, for example.

I think there are elements showing lowering of performance and expectation.

Could you provide more details about the Epyc + DDR4 server?

Basically everyone that makes memory is building new fabs, meanwhile I'm not sure how much longer AI datacenter demand for ram will last. I think the decrease in AI ram demand and the new fabs will likely coincide leading to a collapse in pricing.

That is, of course, assuming the memory manufacturers don't pull their favorite trick and collude.

...

> I don’t think anyone truly knows when, but it will happen.

Do you know what cyclical means ... ?

These times are exciting and rough seas make good sailors. Find your path forward.

- Extreme poverty has dropped from 30% to under 10% globally. - Child mortality rates have dropped in half - Internet access has exploded from 10% to 70% - Solar energy costs have dropped 90% - Cancer death rates have declined by 30%

All of these massive improvements in less than 30 years.

While there certainly are issues to solve, and if you simply follow journalism you may think the world is worse off, but for many, their lives have been significantly improved.

That's one big plus.

You'll just have to be careful to match the enclosure to the ports on the system. The base-model M6 Mac mini still uses Thunderbolt 4, so a USB4v2 enclosure would be wasted.

Not being able to upgrade the SSD or the ram is a big issue, not so much on a portable laptop.

The prices are ridiculous though. I may just keep rolling with my Windows 10 setup.

This may depend on the size of your library. I tried installing Jellyfin on a Synology NAS, which runs Plex just fine, and it ran so poorly it was basically unusable. It “worked”, but it was painful.

"M6 also introduces a Dual 16-core Neural Engine, providing up to 2x the peak compute over previous generations to make on-device AI workflows run even faster" .

I ordered an ASUS Zephyrus G16 with 5090 NVIDIA card + 1.9kg (quite an overkill, and I know that I will have to limit power output), but hasn't arrived yet.

But what's fun is that I love QML+QuickShell with its hot reloading, Hyprland with its Lua support.

With AI nowdays it's just so easy to do deep UI changes that wasn't possible a year ago.

It’s a bit depressing because it means that if I ever feel forced to switch my daily driver, it won’t come without a dump truck load of friction, frustration, and lost productivity, which I’ve validated by using the various Linux desktops on secondary machines.

For example when using PyTorch I wanted to try to speed up my NN kernel by 2x by just using half precision and haven't noticed any speedup at all. Also I was missing the easy to use GNU tools that had to be mixed with Apple's tools.

I loved using Arc browser as well, and I'm missing it, but I guess I will do without it somehow (Chrome's vertical tabs are just not the same).

My main program missing from going back to Linux was ChatGPT Desktop, but now it's there.

I just checked out Hammerspoon, I'm happy for you that you wrote it, and looks great, but it has the same problem that I had: for security reasons Apple stopped allowing the window APIs to get all important information on other workspaces. You can only do it with Accessibility API. I was trying to fight with it but have up.

The most disruptive thing in Omarchy is not the keyboard, it's the tiling WM in my opinion.

The performance you get with M5/M6 in theory is crazy (especially having matmul cores with such an energy envelope and unified memory). By the time I can use it well in Linux there will be M7/M8 though.

With NVIDIA I can buy a Blackwell based core right now and use it (even if I will have much less GPU memory. Though it will be high bandwidth).

They don’t provide any source code or documentation (which they easily could) so if want to do literally anything with your “open” device you must first reverse engineer the entire MacOS driver stack like the Asahi project did. Great fun, but it’s a criminal waste of incredibly talented engineer hours.

On top of that, the reason Asahi doesn’t work on M4 and later is that Apple has intentionally modified the ARM core to prevent stock MacOS from running once the hardware is “unlocked,” which makes it way more difficult to reverse engineer.

Bastards.

That, or I can’t read.

There are emails unearthed in various lawsuits where you can read Bill Gates screaming at his subordinates: "why the hell can't our partners like Sony and Creative create a similar device? Give them all, give them early access to everything, work with them". In the end MS felt compelled to make their own.

The issue was the abysmal marketing and "me too" attitude Microsoft had (and still has).

But it had nothing to do with the iPod/iPhone release in his story.

(story starts ~1:15, but I highly recommend the entire talk)

Apple never came first ... but often just at the right moment and had marketing skills to make it a new trend.

[2017] Speaking at an event Tuesday, Scott Forstall, who lead Apple's iOS software division under Jobs, recounted the surprising history behind the company's first touchscreen prototype, which ultimately lead to the first iPhone.

It all began, Forstall said, because Jobs had an acquaintance at Microsoft who he really, really, didn't like.

"It began because Steve hated this guy at Microsoft," Forstall said during the event. "Any time Steve had any interaction with the guy, he'd come back pissed off."

The unnamed Microsoft exec, who was apparently the husband of a friend of Jobs' wife, apparently bragged so much it pushed Jobs over the edge.

"He just shoved it in Steve's face -- the way they were going to rule the world with their new tablets with their pens. He came in on Monday with a set of expletives and then was like 'let's show them how it's really done.' "

https://mashable.com/article/scott-forstall-steve-jobs-micro...

When plugged in... This caveat is so enormous it should almost be legislated. If your computer use is at all portable, a computer that scales down to 20 - 40% of GPU power when unplugged is an enormously significant factor. So far as I'm aware (could be wrong about arm devices?) there's no non-apple laptop that operates at 100% speed on the road.

Intel had to reduce the frequency of their processors when running AVX2 instructions and the AVX2 frequency of the processors were non-disclosable to anyone.

Also, benchmarking Intel processors and publishing these numbers were forbidden in some cases. I don't know whether this ban is still in effect.

x86 processors can't keep up with the ARM processors TDP and thermal profile wise. So they slow down a ton when running on battery. See Jeff Geerling's last video on Apple Neo vs. some Intel laptop. It's as "efficient", but slow as a newborn tortoise learning to walk when unplugged and trying to get the most endurance out of the battery.

My M1 Mac gets almost 2 days of low-intensity use after ~6 years of use, and it got warm once or twice because something ran away in the background for tens of minutes.

Hold up, in what metric/benchmark? Feel personally like we life in an age where, no matter the SOC vendor, something high performant and efficient is offered, so seeing a claim that any vendor, be it Intel, AMD, Qualcomm or Apple, is consistently outperforming another, I'd like to get more context on that.

The real question is whether all these Intel Windows laptops spin their fans at full speed when doing absolutely nothing. That’s something I can never go back to. I do some light gaming on my M2 Pro MBP and it gets hot when trying to push 120 fps. But my Lenovo Legion (that’s now collecting the dust) is so loud I could hear it through headphones.

Of course you can beat the entry level MacBook Neo by comparing it to larger, more powerful, more expensive laptops.

The M5 is an entire family with a range of performance. Intel/AMD have done a lot to improve performance but they’re not beating the high end M5 chips on performance or battery life yet.

THE (Col.) Billy Mitchell was actually a badass. His time in Alaska alone, running telegraph lines in the wilderness, made him a legend even before his forward thinking during WWI and beyond.

I enjoy it because of progress, not because of Apple.

5.25GHz Pentium 4s, intentionally lower binned Athlons, the era your CPU got obsoleted the moment you booted it for the first time.

I don't remember early 90s much. I was too young back then. I don't remember much stuff from that era. But late 90s, early 2000s.

Oh, boy.

P.S.: AMD64 was a great sucker punch though. One of the professors in our university rejected to believe and got mad when he learnt that Intel licensed AMD64 from AMD, heh.

Now we boot an embedded microcontroller (or CPU) which boots the main CPU which boots another OS semi-persistently to boot the main OS (if it's allowed).

Sometimes there are other processors needs to be up to allow processor to continue booting as well (these are mostly servers, but eh).

What OS could I have loaded onto my Macintosh SE/30 that wasn't from Apple?

Intel Panther lake beats the M5 in GPU and multi-core performance, and is only behind in single core (and not by much). Panther Lake is also on some Windows laptops that trounce MacBook battery life (e.g., Framework 13 Pro according to Just Josh Tech medium-brightness productivity benchmarking).

The Snapdragon X2 Elite beats the M5 multi-core performance (the HP Omnibook Ultra 14 is priced identically spec-for-spec with the MacBook Pro with M5). Still behind in single core and probably graphics, but squarely in the same realm.

Both of these chips are within small deviations of the M5. We could call Intel and Qualcomm 1 year behind at best.

My YouTube will load 0.00001s faster!

how about measuring returns in the sense that I no longer need to share my flagship idea with some random 3rd party just because it hosts the LLM I am using?

consumer electronics has literally never been an asset.

> Apple computers tend to have excellent resale value

do you think the new leasing category might change that? hmmmmmmmmmmmmmm

edit:

https://www.reddit.com/r/LocalLLaMA/comments/1vxzg6v/apple_i...

Nevermind that Apple still insists on providing base systems with only 512GB of non-upgradable SSD. The equivalent spec mini (4TB/64/10G) is almost $5k for half the VRAM. Not as good a CPU compared to the M5/6 but you also get 20 cores and full CUDA.

A PC with similar capabilities is going to sound like a jet taking off.

And, you have to include power. Where I am, the power for a rig with 2x 3090, running for four hours a day, would be around $70/month!

It's kinda like datacenter only exist to optimize compute cost through oversubscription of hardware/time share, cheaper business energy rates, and bulk discounts, compared to self hosting. ;)

It's not even in the top 5000, because it is objectively terrible. People would rather play the original Life is Strange one more time (despite a remaster being out), than play this game.

At some point you really have to start getting suspicious why it keeps coming up.

The number of reviews vs. number of actual players online doesn't make sense when compared to peers in its genre either. It has a similar number of reviews as Life is Strange Remastered, but it loses to the 4 years old re-release of an 11 years old story-based game by an order of magnitude. For a genre that that has extremely top-heavy player count charts this doesn't make sense.

Funnily enough Life is Strange Remastered had a bit of a revival around that game's release, probably because people decided they'd rather play LiS.

Entertainment media entertaining nobody is just objectively bad at its job and the blatant astroturfing and nepo-baby-pampering adds an unsavory dimension to it.

1: https://steamcharts.com/app/2582320

2: https://steamcharts.com/app/760640

Also, the UI has been in decline for a decade or so, with the regrettable decisions under Alan Dye. Thankfully, Zuck poached him and we're all grateful for that. I mean, what they've done to our icons is tragic, the glyphs in the menus was intern-level boneheaded, and reduction of contrast to "get out of the way" of our content has made everything less usable, especially for people with vision issues.

A bug that I filed about window-ordering when using the Cmd-` / Shift-Cmd-` shortcuts has been live since 2010 without getting fixed, though an industrious HN user posted a link to the background utility they made to fix it themselves[0].

0: https://github.com/tacomanator/sash

I've been running it for 3 years now. I am so macpilled that I had to install it on my new Framework 13 pro.

The Mac muscle memory was always my biggest hurdle with Linux. I always felt handicapped by it. But Toshy eliminates that.

Jeff Goldblum's 30 second "Presenting three easy steps to the internet" https://www.youtube.com/watch?v=rjY0xsoozs8

And now when you want to set up a fresh Mac it's a parade of dialogue boxes:

Do you want to turn on Siri? No. (the dialogue-choice is probably "maybe later" ugh)

Do you want to share app analytics with developers? Yes.

Do you want to add Touch ID? No, because I'm setting this up for someone else.

There are more!

I gave those examples -- each of which having a different answer or reason -- to show that the problem of onboarding is nontrivial, but Apple has spent no(?) effort toward fixing it. It's the same for new iPhones and iPads.

NB I don't know if you sign into your Apple account it pre-answers these questions. That'd be nice.

Great hardware. I don't miss OSX though.

I think it's very likely due to OS updates, like other people are commenting on. It probably doesn't happen to everyone, and maybe doing a clean reinstall helps (I tried all the standard stuff - rebuild caches etc).

I’m only using it as a travel machine now, trying to hold on from upgrading it until I can get an Air with OLED display - hopefully coming in <24 months…

Feels as snappy and impressive as the first day I got it. I never need to close apps, or reboot. I record multitrack audio on it without any perceptible latency. I write code on it. It's the best computer I've ever owned by a significant margin.

Best computer I've had in a long time.

And I don't feel like beating a very dead horse, so just google "macOS Tahoe design issues" and take your pick.

Sharing files with a windows hard drive. My workmate had a seagate that installed an NTFS driver on mac, which saved us a few times. Never had a problem on linux.

Python never seems to work right. Conda, and other stuff, just never worked smoothly with the bioinformatics code we were running. Cluster machines it was easy.

There is a wierd issue that you can't close a macbook and use it with an external monitor without plugging it in. It just sleeps.

Would you like to join iCloud. No really would you like to join iCloud. How about apple music? (To be fair MS is just as bad if not worse about onedrive...)

We had a pythonTK program to print labels from a CSV file. Every MacOS update broke it. We ended up just running it on a windows machine. back up was going command line on mac.

case insensitive file systems.

generally stability..

Its not bad, just seems the OS has gotten worse.

But, still:

- it slows down after a while

- design direction is questionable and controversial - that it got more inconsistent is factual - also, so much wasted space and everything got bigger and a little dumber

- so little control over system behavior, functionality, looks

On modern Linux, you get a more polished (wow, unbelievable thing to say - I first dabbled with Linux in 1997), robust, and infinitely more functional and customizable experience. After a while, I started feeling at home - like it's "mine" - and how distinctively MacOS feels like being a guest there, and that it was never "my environment".

I definitely realize how vague and unquantifiable most of this sounds - and how little value those things have for a lot of people - but I've come to enjoy my setup, I spend ~12h / day in it, and I discovered that how it "feels" matters to me.

I just use Linux PC as much as possible instead. And that 4$ M4 is just a fancy presentation computer I use when I travel.

Personally I would spend thousands on an actually great Linux laptop, but there are still none around.

My MBP probably got affected by OS updates like other people are saying - opening apps and responsiveness definitely got worse, and felt like slightly deteriorating over time (but it might have been MacOS updates along the way).

I’ve been moving all my machines to devuan with xfce (or freebsd with xfce).

I have zero issues despite not having to deal with wayland, systemd, snap, etc.

"Our Quarter Pounder may weigh only 1/8 oz, but we figured out how to pack double the flavor in each mouthful, so we're still selling it as a Quarter Pounder."