▲I don't know how many others here have a CoPilot+ PC but the NPU on it is basically useless. There isn't any meaningful feature I get by having that NPU. They are far too limited to ever do any meaningful local LLM inference, image processing or generation. It handles stuff like video chat background blurring, but users' PC's have been doing that for years now without an NPU.
reply▲I'd love to see a thorough breakdown of what these local NPUs can really do. I've had friends ask me about this (as the resident computer expert) and I really have no idea. Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with. Is there a single remotely killer application for local client NPUs?
reply▲The problem is essentially memory bandiwdth afiak. Simplifying a lot in my reply, but most NPUs (all?) do not have faster memory bandwidth than the GPU. They were originally designed when ML models were megabytes not gigabytes. They have a small amount of very fast SRAM (4MB I want to say?). LLM models _do not_ fit into 4MB of SRAM :).
And LLM inference is heavily memory bandwidth bound (reading input tokens isn't though - so it _could_ be useful for this in theory, but usually on device prompts are very short).
So if you are memory bandwidth bound anyway and the NPU doesn't provide any speedup on that front, it's going to be no faster. But has loads of other gotchas so no real "SDK" format for them.
Note the idea isn't bad per se, it has real efficiencies when you do start getting compute bound (eg doing multiple parallel batches of inference at once), this is basically what TPUs do (but with far higher memory bandwidth).
NPUs are still useful for LLM pre-processing and other compute-bound tasks. They will waste memory bandwidth during LLM generation phase (even in the best-case scenario where they aren't physically bottlenecked on bandwidth to begin with, compared to the iGPU) since they generally have to read padded/dequantized data from main memory as they compute directly on that, as opposed to being able to unpack it in local registers like iGPUs can.
> usually on device prompts are very short
Sure, but that might change with better NPU support, making time-to-first-token quicker with larger prompts.
Yes I said that in my comment. Yes they might be useful for that - but when you start getting to prompts that are long enough to have any significant compute time you are going to need far more RAM than these devices have.
Obviously in the future this might change. But as we stand now dedicated silicon for _just_ LLM prefill doesn't make a lot of sense imo.
You don't need much on-device RAM for compute-bound tasks, though. You just shuffle the data in and out, trading a bit of latency for an overall gain on power efficiency which will help whenever your computation is ultimately limited by power and/or thermals.
reply▲The idea that tokenization is what they're for is absurd - you're talking a tenth of a thousandth of a millionth of a percent of efficiency gain in real world usage, if that, and only if someone bothers to implement it in software that actually gets used.
NPUs are racing stripes, nothing more. No killer features or utility, they probably just had stock and a good deal they could market and tap into the AI wave with.
NPUs aren't meant for LLMs. There are a lot more neural net tech out there than LLMs.
reply▲aleph_minus_one21 hours ago
[-] >
NPUs aren't meant for LLMs. There are a lot more neural net tech out there than LLMs.
OK, but where can I find demo applications of these that will blow my mind (and make me want to buy a PC with an NPU)?
Apple demonstrates this far better. I use their Photos app to manage my family pictures. I can search my images by visible text, by facial recognition, or by description (vector search). It automatically composes "memories" which are little thematic video slideshows. The FaceTime camera automatically keeps my head in frame, and does software panning and zooming as necessary. Automatic caption generation.
This is normal, standard, expected behavior, not blow your midn stuff. Everyone is used to having it. But where do you think the computation is happening? There's a reason that a few years back Apple pushed to deprecate older systems that didn't have the NPU.
adgjlsfhk120 hours ago
[-] I've yet to see any convincing benchmarks showing that NPUs are more efficient than normal GPUs (that don't ignore the possibility of downclocking the GPU to make it run slower but more efficient)
reply▲NPUs are more energy efficient. There is no doubt that a systolic array uses less watts per computation than a tensor operation on a GPU, for these kinds of natural fit applications.
Are they more performant? Hell no. But if you're going to do the calculation, and if you don't care about latency or throughput (e.g. batched processing of vector encodings), why not use the NPU?
Especially on mobile/edge consumer devices -- laptops or phones.
Best NPU app so far is Trex for Mac.
reply▲I think they were talking about prefill, which is typically compute-bound.
reply▲throwaway29389223 hours ago
[-] I used to work at Intel until recently. Pat Gelsinger (the prior CEO) had made one of the top goals for 2024 the marketing of the "AI PC".
Every quarter he would have an all company meeting, and people would get to post questions on a site, and they would pick the top voted questions to answer.
I posted mine: "We're well into the year, and I still don't know what an AI PC is and why anyone would want it instead of a CPU+GPU combo. What is an AI PC and why should I want it?" I then pointed out that if a tech guy like me, along with all the other Intel employees I spoke to, cannot answer the basic questions, why would anyone out there want one?
It was one of the top voted questions and got asked. He answered factually, but it still wasn't clear why anyone would want one.
TitaRusell10 hours ago
[-] The only people who are actually paying good money for a PC nowadays are gamers- and they sure as hell aren't paying 3k so that they can use copilot.
reply▲nextaccountic3 hours ago
[-] Also professionals that need powerful computers ("workstations") in their jobs, like video editing
A lot of them are incorporating AI in their workflow, so making local AI better would be a plus. Unfortunately I don't see this happening unless GPUs come with more VRAM (and AI companies don't want that, and are willing to spend top dollar to hoard RAM)
So... what was the answer?
reply▲throwaway29389222 hours ago
[-] Pretty much the same as what you see in the comments here. For certain workloads, NPU is faster than CPU by quite a bit, and I think he gave some detailed examples
at the low level (what types of computations are faster, etc).
But nothing that translated to real world end user experience (other than things like live transcription). I recall I specifically asked "Will Stable Diffusion be much faster than a CPU?" in my question.
He did say that the vendors and Microsoft were trying to come up with "killer applications". In other words, "We'll build it, and others will figure out great ways to use it." On the one hand, this makes sense - end user applications are far from Intel's expertise, and it makes sense to delegate to others. But I got the sense Microsoft + OEMs were not good at this either.
> For certain workloads, NPU is faster than CPU by quite a bit
WTF is an NPU ? What kind of instructions does it support ? Can it add 3 and 5 ? Can it compute matrices ?
Probably a lot of jargon AI word salad that boiled down to “I’m leaving in Dec. 2024, you guys have fun.”
reply▲In theory NPUs are a cheap, efficient alternative to the GPU for getting good speeds out of larger neural nets. In practice they're rarely used because for simple tasks like blurring, speech to text, noise cancellation, etc you can get usually do it on the CPU just fine. For power users doing really hefty stuff they usually have a GPU anyway so that gets used because it's typically much faster. That's exactly what happens with my AMD AI Max 395+ board. I thought maybe the GPU and NPU could work in parallel but memory limitations mean that's often slower than just using the GPU alone. I think I read that their intended use case for the NPU is background tasks when the GPU is already loaded but that seems like a very niche use case.
reply▲If the NPU happens to use less power for any given amount of TOPS it's still a win since compute-heavy workloads are ultimately limited by power and thermals most often, especially on mobile hardware. That frees up headroom for the iGPU. You're right about memory limitations, but these are generally relevant for e.g. token generation not prefill.
reply▲> Everything I see advertised for (blurring, speech to text, etc...) are all things that I never felt like my non-NPU machine struggled with.
I don’t know how good these neural engines are, but transistors are dead-cheap nowadays. That makes adding specialized hardware a valuable option, even if it doesn’t speed up things but ‘only’ decreases latency or power usage.
I think a lot of it is just power savings on those features, since the dedicated silicon can be a lot more energy efficient even if it's not much more powerful.
reply▲"WHAT IS MY PURPOSE?"
"You multiply matrices of INT8s."
"OH... MY... GOD"
NPUs really just accelerate low-precision matmuls. A lot of them are based on systolic arrays, which are like a configurable pipeline through which data is "pumped" rather than a general purpose CPU or GPU with random memory access. So they're a bit like the "synergistic" processors in the Cell, in the respect that they accelerate some operations really quickly, provided you feed them the right way with the CPU and even then they don't have the oomph that a good GPU will get you.
cookiengineer17 hours ago
[-] My question is: Isn't this exactly what SIMD has done before? Well, or SSE2 instructions?
To me, an NPU and how it's described just looks like a pretty shitty and useless FPGA that any alternative FPGA from Xilinx could easily replace.
recursivecaveat16 hours ago
[-] You definitely would use SIMD if you were doing this sort of thing on the CPU directly. The NPU is just a large dedicated construct for linear algebra. You wouldn't really want to deploy FPGAs to user devices for this purpose because that would mean paying the reconfigurability tax in terms of both power-draw and throughput.
reply▲imtringued12 hours ago
[-] Yes but your CPUs have energy inefficient things like caches and out of order execution that do not help with fixed workloads like matrix multiplication. AMD gives you 32 AI Engines in the space of 3 regular Ryzen cores with full cache, where each AI Engine is more powerful than a Ryzen core for matrix multiplication.
reply▲So it's a higher power DSP style device. Small transformers for flows. Sounds good for audio and maybe tailored video flow processing.
reply▲Do compilers know how to take advantage of that, or do programs need code that specifically takes advantage of that?
reply▲It’s more like you need to program a dataflow rather than a program with instructions or vliw type processors. They still have operations but for example I don’t think ethos has any branch operations.
reply▲There are specialized computation kernels compiled for NPUs. A high-level program (that uses ONNX or CoreML, for example) can decide whether to run the computation using CPU code, a GPU kernel, or an NPU kernel or maybe use multiple devices in parallel for different parts of the task, but the low-level code is compiled separately for each kind of hardware. So it's somewhat abstracted and automated by wrapper libraries but still up to the program ultimately.
reply▲I have one as well and I simply don’t get it. I lucked into being able to do somewhat acceptable local LLM’ing by virtue of the Intel integrated “GPU” sharing VRAM and RAM, which I’m pretty sure wasn’t meant to be the awesome feature it turned out to be. Sure, it’s dead slow, but I can run mid size models and that’s pretty cool for an office-marketed HP convertible.
(it’s still amazing to me that I can download a 15GB blob of bytes and then that blob of bytes can be made to answer questions and write prose)
But the NPU, the thing actually marketed for doing local AI just sits there doing nothing.
SomeHacker4414 hours ago
[-] Also the Copilot button/key is useless. It cannot be remapped to anything in Ubuntu because it sends a sequence of multiple keycodes instead if a single keycode for down and then up. You cannot remap it to a useful modifier or anything! What a waste of keyboard real estate.
reply▲If you want a small adventure, you could see which HID device those keystrokes show up on, and they might be remappable courtesy of showing up on a HID device for that specific button. Failing that, they most likely come from either ACPI AML code or from the embedded controller (EC). If the former, it’s not
that hard to patch the AML code, and maybe Copilot could do it for you (you use standard open source tooling to disassemble the AML blob, which the kernel will happily give you, and then you make a patched version and load it). If the latter, you could see if anyone has made progress toward finding a less silly way to configure the EC.
(The EC is a little microcontroller programmed by the OEM that does things like handling weird button presses.)
There are also reports of people having decent results using keyd to remap the synthetic keystrokes from the copilot button.
(The sheer number of times Microsoft has created totally different specs for how OEMs should implement different weird buttons is absurd.)
If I had to steelman Dell, they probably made a bet a while ago that the software side would have something for the NPU, and if so they wanted to have a device to cash in on it. The turnaround time for new hardware was probably on the order of years (I could be wrong about this).
It turned out to be an incorrect gamble but maybe it wasn’t a crazy one to make at the time.
There is also a chicken and egg problem of software being dependent on hardware, and hardware only being useful if there is software to take advantage of its features.
That said I haven’t used Windows in 10 years so I don’t have a horse in this race.
aleph_minus_one21 hours ago
[-] > There is also a chicken and egg problem of software being dependent on hardware, and hardware only being useful if there is software to take advantage of its features.
In the 90s, as a developer you couldn't depend on that a user's computer had a 3D accelerator (or 3D graphics) card. So 3D video games used multiple renderers (software rendering, hardware-accelerated rendering (sometimes with different backends like Glide, OpenGL, Direct3D)).
Couldn't you simply write some "killer application" for local AI that everybody "wants", but which might be slow (even using a highly optimized CPU or GPU backend) if you don't have an NPU. Since it is a "killer application", very many people will still want to run it, even if the experience is slow.
Then as a hardware vendor, you can make the big "show-off" how much better the experience is with an NPU (AI PC) - and people will immediately want one.
Exactly the same story as for 3D accelerators and 3D graphics card where Quake and Quake II were such killer applications.
They are still including the NPU though, they just realised that consumers aren't making laptop purchases based on having "AI" or being branded with Copilot.
The NPU will just become a mundane internal component that isn't marketed.
pseudosavant5 hours ago
[-] I did some research on if the transistor budget for the NPU was spent on something else in the SoC/CPU, what could you get?
You could have 4-10 additional CPU cores, or 30-100MB more L3 cache. I would definitely rather have more cores or cache, than a slightly more efficient background blurring engine.
What we want as developers: To be able to implement functionality that utilizes a model for tasks like OCR, visual input and analysis, search or re-ranking etc, without having to implement an LLM API and pay for it. Instead we'd like to offer the functionality to users, possibly at no cost, and use their edge computing capacity to achieve it, by calling local protocols and models.
What we want as users: To have advanced functionality without having to pay for a model or API and having to auth it with every app we're using. We also want to keep data on our devices.
What trainers of small models want: A way for users to get their models on their devices, and potentially pay for advanced, specialized and highly performant on-device models, instead of APIs.
What seems to be delivered by NPUs at this point: filtering background noise from our microphone and blurring our camera using a watt or two less than before.
reply▲If it really is a watt or two less, that's a lot on a laptop.
reply▲If you do video calls for 7 hours a day and then ran out it means you could have maybe ~7.5 hours. Not nothing, but differences in things like screen backlight and other component efficiency still dominate battery interests over whether there is an NPU or not. If you don't spend your day on video calls it's more like a 0% increase (mic noise processing is much lower load).
Regardless if it does zilch or some minor good for you in the battery respect, the point was more NPUs don't deliver on the above reasons everyone was supposed to want AI for. Most likely, IMO, because they are far too weak to do so and making them powerful takes too much power+cost.
The idea is that NPUs are more power efficient for convolutional neural network operations. I don't know whether they actually are more power efficent, but it'd be wrong to dismiss them just because they don't unlock new capabilties or perform well for very large models. For smaller ML applications like blurring backgrounds, object detection, or OCR, they could be beneficial for battery life.
reply▲Yes, the idea before the whole shove LLMs into everything era was that small, dedicated models for different tasks would be integrated into both the OS and applications.
If you're using a recent phone with a camera, it's likely using ML models that may or may not be using AI accelerators/NPUs on the device itself. The small models are there, though.
Same thing with translation, subtitles, etc. All small local models doing specialized tasks well.
OCR on smartphones is a clear winner in this area. Stepping back, it's just mind blowing how easy it is to take a picture of text and then select it and copy and paste it into whatever. And I totally just take it for granted.
reply▲Not sure about all NPUs, but TPUs like Google's Coral accelerator are absolutely, massively more efficient per watt than a GPU, at least for things like image processing.
reply▲NPUs overall need better support from local AI frameworks. They're not "useless" for what they can do (low-precision bulk compute, which is potentially relevant for many of the newer models) and they could help address thermal limits due to their higher power efficiency compared to CPU/iGPU. but that all requires specialized support that hasn't been coming.
reply▲Yeah, that's because the original npus were a rush job, the amd AI Max is the only one that's worth anything in my opinion.
reply▲SomeHacker4414 hours ago
[-] I have a Strix Halo 395 128GB laptop running Ubuntu from HP. I have not been able to do anything with the NPU. I was hoping it could be used for OpenCL, but does not seem so.
What examples do you have of making the NPU in this processor useful please?
pseudosavant5 hours ago
[-] All the videos I've seen of AI workloads with an AMD Strix Halo with 128GB setup have used the GPU for the processing. It has a powerful iGPU and unified memory more like Apple's M chips.
reply▲The Apple M series chips are solid for inference.
reply▲Correct me if I'm wrong, but I thought everyone was still doing inference on the GPU for Apple silicon.
reply▲mrinterweb22 hours ago
[-] The Apple M series is SoC. The CPU, GPU, NPU, RAM are all part of the chip.
reply▲The RAM is not part of the SoC. It's a bunch of separate commodity RAM dies packaged alongside the SoC.
reply▲Dylan1680722 hours ago
[-] Is that because of the actual processing unit or because they doubled the width of the memory bus?
reply▲It's because it comes with a decent iGPU, not because of the NPU inside of that. The NPU portion is still the standard tiny 50 TOPS and could be filled with normal RAM bandwidth like on a much cheaper machine.
On the RAM bandwidth side it depends if you want to look at it as "glass is half full" or "glass is half empty". For "glass is half full" the GPU has access to a ton of RAM at ~2x-4x the bandwidth of normal system memory an iGPU would have and so you can load really big models. For "glass is half empty" that GPU memory bandwidth is still nearly 2x less than a even a 5060 dGPU (which doesn't have to share any of that bandwidth with the rest of the system), but you won't fit as large of a model on a dGPU and it won't be as power efficient.
Speaking of power efficiency - it is decently power efficient... but I wouldn't run AI on battery on mine unless I was plugged in anyways as it still eats through the battery pretty quick when doing so. Great general workstation laptop for the size and wattage though.
simulator5g21 hours ago
[-] If you do use video chat background blurring, the NPU is more efficient at it vs using your cpu or gpu. So the feature it supports is longer battery life, and less resource usage on your main chips, and better performance for the things that NPUs can do. E.g higher video quality on your blurred background.
reply▲Really, the best we can do with the NPU is a less battery intensive blurred background? R&D money well spent I guess...
reply▲The stacks for consumer NPUs are absolutely cursed, this does not surprise me.
They (Dell) promised a lot in their marketing, but we're like several years into the whole Copilot PC thing and you still can barely, if at all, use sane stacks with laptop NPUs.
NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard, and Apple's hardware can do it. NPUs are basically Microsoft trying to fudge their way to a llamacpp-on-Apple-Silicon experience. Obviously it failed, but they couldn't not try.
reply▲aleph_minus_one21 hours ago
[-] >
NPUs were pushed by Microsoft, who saw the writing on the wall: AI like chatgpt will dominate the user's experience, edge computing is a huge advantage in that regard
Then where is a demo application from Microsoft of a model that I can run locally where my user experience is so much better (faster?) if my computer has an NPU?
imtringued12 hours ago
[-] I think the reason why NPUs failed is that Microsoft's preferred standard ONNX and the runtime they developed is a dud. Exporting models to work on ONNX is a pain in the ass.
reply▲alfiedotwtf22 hours ago
[-] > AI like chatgpt will dominate the user's experience
I hope not. Sure they’re helpful, but I’d rather they sit idle behind the scenes, and then only get used when a specific need arises rather than something like a Holodeck audio interface
The NPU is essentially the Sony Cell "SPE" coprocessor writ large.
The Cell SPE was extremely fast but had a weird memory architecture and a small amount of local memory, just like the NPU, which makes it more difficult for application programmers to work with.
withinrafael18 hours ago
[-] The Copilot Runtime APIs to utilize the NPU are still experimental and mostly unavailable. I can't believe an entire generation of the Snapdragon X chip came and went without working APIs. Truly incredible.
reply▲simulator5g21 hours ago
[-] If you do use video chat background blurring, the NPU is more efficient at it vs using your cpu or gpu. So the feature it supports is longer battery life and less resource usage on your main chips.
reply▲I'm not too familiar with the NPU, but this sounds a lot like GPU acceleration where a lot of the time you still end up having everything run on the CPU since it just works everywhere all the time rather than having to have both a CPU and an NPU version.
reply▲I've got one anecdote: friend needed Live Captions for a translating job and had to get copilot+ PC just for that.
reply▲What software are they using for that, and how did they know ahead of time that the software would use their NPU?
reply▲Question - from the perspective of the actual silicon, are these NPUs just another form of SIMD? If so, that's laughable sleight of hand and the circuits will be relegated to some mothball footnote in the same manner as AVX512, etc.
To be fair, SIMD made a massive difference for early multimedia PCs for things like music playback, gaming, and composited UIs.
> circuits will be relegated to some mothball footnote in the same manner as AVX512
AVX512 is widely used...
NPUs are a separate accelerator block, not in-CPU SIMD. The latter exists for matrix compute, but only in the latest version of AVX which has yet to reach consumer CPUs.
reply▲> It's not that Dell doesn't care about AI or AI PCs anymore, it's just that over the past year or so it's come to realise that the consumer doesn't.
I wish every consumer product leader would figure this out.
People will want what LLMs can do they just don't want "AI". I think having it pervade products in a much more subtle way is the future though.
For example, if you close a youtube browser tab with a comment half written it will pop up an `alert("You will lose your comment if you close this window")`. It does this if the comment is a 2 page essay or "asdfasdf". Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort. The end result is I only have to deal with that annoying popup when I really am glad it is there.
That is a trivial example but you can imagine how a locally run LLM that was just part of the SDK/API developers could leverage would lead to better UI/UX. For now everyone is making the LLM the product, but once we start building products with an LLM as a background tool it will be great.
It is actually a really weird time, my whole career we wanted to obfuscate implementation and present a clean UI to end users, we want them peaking behind the curtain as little as possible. Now everything is like "This is built with AI! This uses AI!".
> Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort.
I read this post yesterday and this specific example kept coming back to me because something about it just didn't sit right. And I finally figured it out: Glancing at the alert box (or the browser-provided "do you want to navigate away from this page" modal) and considering the text that I had entered takes... less than 5 seconds.
Sure, 5 seconds here and there adds up over the course of a day, but I really feel like this example is grasping at straws.
It’s also trivially solvable with idk, a length check, or any number of other things which don’t need to 100b parameters to calculate.
reply▲This was a problem at my last job. Boss kept suggesting shoving AI into features, and I kept pointing out we could make the features better with less effort using simple heuristics in a few lines of code, and skip adding AI altogether.
So much of it nowadays is like the blockchain craze, trying to use it as a solution for every problem until it sticks.
andrekandre20 hours ago
[-] > Boss kept suggesting shoving AI into features, and I kept pointing out we could make the features better with less effort using simple heuristics in a few lines of code
depending on what it is, it would probably also cost less money (no paying for token usage), use less electricity and be more reliable (less probabilistic, more deterministic), and easier to maintain (just fix the bug in the code vs prompt/input spelunking) as well.
there are definitely useful applications for end user features, but a lot of this is ordered from on-high top-down and product managers need to appease them...
... And the people at the top are only asking for it because it sounds really good to investors and shareholders. "Powered by AI" sounds way fancier and harder to replace than "power by simple string searches and other heuristics"
reply▲The problem isn't so much the five seconds, it is the muscle memory. You become accustomed to blindly hitting "Yes" every time you've accidentally typed something into the text box, and then that time when you actually put a lot of effort into something... Boom. Its gone. I have been bitten before. Something like the parent described would be a
huge improvement.
Granted, it seems the even better UX is to save what the user inputs and let them recover if they lost something important. That would also help for other things, like crashes, which have also burned me in the past. But tradeoffs, as always.
Which is fine! That's me making the explicit choice that yes, I want to close this box and yes, I want to lose this data. I don't need an AI evaluating how important it thinks I am and second guessing my judgement call.
I tell the computer what to do, not the other way around.
You do, however, need to be able to tell the computer that you want to opt in (or out, I suppose) of being able to using AI to evaluate how important it thinks your work is. If you don’t have that option, it is, in fact, the computer telling you what to do. And why would you want the computer to tell you what to do?
reply▲> You become accustomed to blindly hitting "Yes" every time you've accidentally typed something into the text box, and then that time when you actually put a lot of effort into something... Boom. Its gone.
Wouldn't you just hit undo? Yeah, it's a bit obnoxious that Chrome for example uses cmd-shift-T to undo in this case instead of the application-wide undo stack, but I feel like the focus for improving software resilience to user error should continue to be on increasing the power of the undo stack (like it's been for more than 30 years so far), not trying to optimize what gets put in the undo stack in the first place.
poopooracoocoo23 hours ago
[-] Now y'all are just analysing the UX of YouTube and Chrome.
The problem is that by agreeing to close the tab, you're agreeing to discard the comment. There's currently no way to bring it back. There's no way to undo.
AI can't fix that. There is Microsoft's "snapshot" thing but it's really just a waste of storage space.
johnnyanmac22 hours ago
[-] I mean, it can. But so can a task runner that periodically saves writing to a clipboard history. The value is questionable, but throwing an LLM at it does feel overkill on terms of overhead.
reply▲> Wouldn't you just hit undo?Because:
1. Undo is usually treated as an application-level concern, meaning that once the application has exited there is no specific undo, as it is normally though of, function available. The 'desktop environment' integration necessary for this isn't commonly found.
2. Even if the application is still running, it only helps if the browser has implemented it. You mention Chrome has it, which is good, but Chrome is pretty lousy about just about everything else, so... Pick your poison, I guess.
3. This was already mentioned as the better user experience anyway, albeit left open-ended for designers, so it is not exactly clear what you are trying to add. Did you randomly stop reading in the middle?
reply▲>You become accustomed to blindly hitting "Yes" every time you've accidentally typed something into the text box, and then that time when you actually put a lot of effort into something... Boom. Its gone.
I'm not sure we need even local AI's reading everything we do for what amounts to a skill issue.
You're quite right that those with skills have no need for computers, but for the rest of us there is no need for them to not have a good user experience.
reply▲I have the exact opposite muscle memory.
reply▲I think this is covered in the Bainbridge automation paper
https://en.wikipedia.org/wiki/Ironies_of_Automation ... When the user doesn't have practiced context like you described, to be expected to suddenly have that practiced context to do the right thing in a surprise moment is untenable.
> if you close a youtube browser tab with a comment half written it will pop up an `alert("You will lose your comment if you close this window")`. It does this if the comment is a 2 page essay or "asdfasdf". Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort.
I don't think that's a great example, because you can evaluate the length of the content of a text box with a one-line "if" statement. You could even expand it to check for how long you've been writing, and cache the contents of the box with a couple more lines of code.
An LLM, by contrast, requires a significant amount of disk space and processing power for this task, and it would be unpredictable and difficult to debug, even if we could define a threshold for "important"!
I think it's an excellent example to be honest. Most of the time whenever someone proposes some use case for a large language model that's not just being a chat bot, it's either a bad idea, or a decent idea that you'd do much better with something much less fancy (like this, where you'd obviously prefer some length threshold) than with a large language model. It's wild how often I've heard people say "we should have an AI do X" when X is something that's very obviously either a terrible idea or best suited for traditional algorithms.
Sort of like how most of the time when people proposed a non-cryptocurrency use for "blockchain", they had either re-invented Git or re-invented the database. The similarity to how people treat "AI" is uncanny.
QuantumNomad_2 days ago
[-] > It's wild how often I've heard people say "we should have an AI do X" when X is something that's very obviously either a terrible idea or best suited for traditional algorithms.
Likewise when smartphones were new, everyone and their mother was certain that random niche thing that made no sense as an app would be a perfect app and that if they could just get someone to make the app they’d be rich. (And of course ideally, the idea haver of the misguided idea would get the lions share of the riches, and the programmer would get a slice of pizza and perhaps a percentage or two of ownership if the idea haver was extra generous.)
With Claude Code doing the implementing now, we'll have to see who gets which slice of pizza!
reply▲Difference is now, that person with an idea, doesn’t need a programmer or anyone to share the pizza with. They are free to gorge on all 18” of it.
reply▲johnnyanmac22 hours ago
[-] Well, until the other 10 people with that idea get a slice in. Likely speaking that 2 people get 7 slices of the 8 slice pizza, and the other 8 people fight over the last piece.
reply▲The difference between "AI" and "linear regression" is whether you are talking to a VC or an engineer.
reply▲> Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input.
That doesn't sound ideal at all. And in fact highlights what's wrong with AI product development nowadays.
AI as a tool is wildly popular. Almost everyone in the world uses ChatGPT or knows someone who does. Here's the thing about tools - you use them in a predictable way and they give you a predictable result. I ask a question, I get an answer. The thing doesn't randomly interject when I'm doing other things and I asked it nothing. I swing a hammer, it drives a nail. The hammer doesn't decide that the thing it's swinging at is vaguely thumb-shaped and self-destruct.
Too many product managers nowadays want AI to not just be a tool, they want it to be magic. But magic is distracting, and unpredictable, and frequently gets things wrong because it doesn't understand the human's intent. That's why people mostly find AI integrations confusing and aggravating, despite the popularity of AI-as-a-tool.
> The hammer doesn't decide that the thing it's swinging at is vaguely thumb-shaped and self-destruct.
Sawstop literally patented this and made millions and seems to have genuinely improved the world.
I personally am a big fan of tools that make it hard to mangle my body parts.
sawstop is not AI
reply▲Sure, where's the line?
If you want to tell me that llms are inherently non-deterministic, then sure, but from the point of view of a user, a saw stop activating because the wood is wet is really not expected either.
Yes, cutting wet wood on the sawstop sucks, but I put up with it. If clicking 'close' on the wrong tab amputated a finger, I'd also put up with it. However, I've closed plenty of tabs accidentally, and all my fingers are still attached.
reply▲Mm yeah, I see the point you're making.
(Though, of course, there certainly are people who dislike sawstop for that sort of reason, as well.)
also from the point of view from a user: in this example, while frustrating/possibly costly, a false positive is infinitely preferable to a false negative.
reply▲I mean, I wouldn't want sawstop to hallucinate my finger is a piece of wood.
reply▲But... A lot of stuff you rely on now was probably once distracting and unpredictable. There are a ton of subtle UX behaviors a modern computer is doing that you don't notice, but if they all disappeared and you had to use windows 95 for a week you would miss.
That is more what I am advocating for, subtle background UX improvements based on an LLMs ability to interpret a users intent. We had limited abilities to look at an applications state and try to determine a users intent, but it is easier to do that with an LLM. Yeah like you point out some users don't want you to try and predict their intent, but if you can do it accurately a high percentage of the time it is "magic".
> subtle UX behaviorsI'd wager it's more likely to be the opposite.
Older UIs were built on solid research. They had a ton of subtle UX behaviors that users didn't notice were there, but helped in minor ways. Modern UIs have a tendency to throw out previous learning and to be fashion-first. I've seen this talked about on HN a fair bit lately.
Using an old-fashioned interface, with 3D buttons to make interactive elements clear, and with instant feedback, can be a nicer experience than having to work with the lack of clarity, and relative laggyness, of some of today's interfaces.
reply▲> Older UIs were built on solid research. They had a ton of subtle UX behaviors that users didn't notice were there, but helped in minor ways. Modern UIs have a tendency to throw out previous learning and to be fashion-first.
Yes. For example, Chrome literally just broke middle-click paste in this box when I was responding. It sets the primary selection to copy, but fails to use it when pasting.
Middle click to open in new tab is also reliably flaky.
I really miss the UI consistency of the 90s and early 2000s.
Serious question: what are those things from windows 95/98 I might miss?
Rose tinted glasses perhaps, but I remember it as a very straightforward and consistent UI that provided great feedback, was snappy and did everything I needed. Up to and including little hints for power users like underlining shortcut letters for the & key.
johnnyanmac22 hours ago
[-] I miss my search bar actually being a dumb grep of my indexed files. It's still frustrating typing 3 characters, seeing the result pop up in the 2nd key stroke, but having it transform into something else by the time I process the result.
reply▲optimalquiet19 hours ago
[-] Inevitably windows search fails to highlight what I’m looking for almost all of the time, and often doesn’t even find it at all. If I have an application installed, it picks the installer in the downloads folder. If I don’t have an app installed, it searches Bing for it. Sometimes it even searches when I
do have the application installed!
Microsoft seems not to believe that users want to use search primarily as an application launcher, which is strange because Mac, Linux, and mobile have all converged on it.
The only one I can think of, literally the only one, is grouped icons.
And even that's only because browsers ended up in a weird "windows but tabs but actually tabs are windows" state.
So yeah, I'd miss the UX of dragging tabs into their own separate windows.
But even that is something that still feels janky in most apps ( windows terminal somehow makes this feel bad, even VS code took a long time to make it feel okay ), and I wouldn't really miss it that much if there were no tabs at all and every tab was forced into a separate window at all times with it's own task bar entry.
It's not like grouped icons wasn't technically infeasible on win95. And honestly, whatever they are more useful is quite debatable. And personally, I don't even have a task panel anymore.
The real stuff not on Win95 that everyone would miss is scalable interfaces/high DPI (not necessary as in HiDPI, just above 640x480). And this one does require A LOT of resources and is still wobbly.
I'm not sure what you mean by "Technically feasible", but it wasn't supported by explorer.
You could have multiple windows, and you could have MDI windows, but you couldn't have shared task bar icons that expand on hover to let you choose which one to go to.
If you mean that someone could write a replacement shell that did that, then maybe, but at that point it's no longer really windows 95.
I remember seeing one of those "kids use old technology" videos, where kids are confused by rotary phones and the like.
One of the episodes had them using Windows 98. As I recall, the reaction was more or less "this is pretty ok, actually". A few WTFs about dialup modems and such, but I don't recall complaints about the UI.
> But... A lot of stuff you rely on now was probably once distracting and unpredictable.
And nobody relied on them when they were distracting and unpredictable. People only rely on them now because they are not.
LLMs won't ever be predictable. They are designed not to be. A predictable AI is something different from a LLM.
> There are a ton of subtle UX behaviors a modern computer is doing that you don't notice, but if they all disappeared and you had to use windows 95 for a week you would miss.
Like what? All those popups screaming that my PC is unprotected because I turned off windows firewall?
I want magic that works. Sometimes I want a tool to interrupt me! I know my route to work so I'm not going to ask how I should get there today - but 1% of the time there is something wrong with my plan (accident, construction...) and I want the tool to say something. I know I need to turn right to get someplace, but sometimes as a human I'll say left instead: confusing me and the driver where they don't turn right, and AI that realizes who made the mistake would help.
The hard part is the AI needs to be correct when it doesn't something unexpected. I don't know if this is a solvable problem, but it is what I want.
Magic in real life never works 100% of the time. It is all an illusion were some observers understand the trick and others do not. Those that understand it have the potential to break the magic. Even the magician has the ability to fault the trick.
I want reproducibility not magic.
It is magic that I can touch a swith on the wall and lights come on. It is magic that I have a warm house despite the outside temperature is near freezing. we have plenty of other magic that works. I want more
reply▲If your light switch doesn't turn on the lights any more it's probably broken.
If your "AI" light switch doesn't turn on the lights, you have to rephrase the prompt.
Electricity, light, and heat aren't magic: they're science. Science is something well understood. Something that seems magical is something poorly understood. When I ask AI a question, I don't know whether it will tell me something truthful, mendacious in a verisimilitudinous way, or blatantly wrong, and I can only tell when it's blatantly wrong. That's magic, and I hate magic. I want more science in my life.
reply▲>For example, if you close a youtube browser tab with a comment half written it will pop up an `alert("You will lose your comment if you close this window")`. It does this if the comment is a 2 page essay or "asdfasdf". Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort. The end result is I only have to deal with that annoying popup when I really am glad it is there.
The funny thing is that this exact example could also be used by AI skeptics. It's forcing an LLM into a product with questionable utility, causing it to cost more to develop, be more resource intensive to run, and behave in a manner that isn't consistent or reliable. Meanwhile, if there was an incentive to tweak that alert based off likelihood of its usefulness, there could have always just been a check on the length of the text. Suggesting this should be done with an LLM as your specific example is evidence that LLMs are solutions looking for problems.
I've been totally AI-pilled because I don't see why that's of questionable utility. How is a regexp going to tell the difference between "asdffghjjk" and "So, she cheated on me". A mere byte count isn't going to do it either.
If the computer can tell the difference and be less annoying, it seems useful to me?
Who said anything about regexp? I was literally talking about something as simple as "if(text.length > 100)". Also the example provided was distinguishing "a 2 page essay or 'asdfasdf'" which clearly can be accomplished with length much easier than either an LLM or even regexp.
We should keep in mind that we're trying to optimize for user's time. "So, she cheated on me" takes less than a second to type. It would probably take the user longer to respond to whatever pop up warning you give than just retyping that text again. So what actual value do you think the LLM is contributing here that justifies the added complexity and overhead?
Plus that benefit needs to overcome the other undesired behavior that an LLM would introduce such as it will now present an unnecessary popup if people enter a little real data and intentionally navigate away from the page (and it should be noted, users will almost certainly be much more likely to intentionally navigate away than accidentally navigate away). LLMs also aren't deterministic. If 90% of the time you navigate away from the page with text entered, the LLM warns you, then 10% of the time it doesn't, those 10% times are going to be a lot more frustrating than if the length check just warned you every single time. And from a user satisfaction perspective, it seems like a mistake to swap frustration caused by user mistakes (accidentally navigating away) with frustration caused by your design decisions (inconsistent behavior). Even if all those numbers end up falling exactly the right way to slightly make the users less frustrated overall, you're still trading users who were previously frustrated at themselves for users being frustrated at you. That seems like a bad business decision.
Like I said, this all just seems like a solution in search of a problem.
Because in _what world_ do I want the computer making value judgements on what I do?
If I want to close the tab of unsubmitted comment text, I will. I most certainly don’t need a model going “uhmmm akshually, I think you might want that later!”
Because the computer behaving differently in different circumstances is annoying, especially when there's no clear cue to the user what the hidden knobs that control the circumstances are.
reply▲What about counting words based on user's current lang, and prompting off that?
Close enough for the issue to me and can't be more expensive than asking an LLM?
We went from the bullshit "internet of things" to "LLM of things", or as Sheldon from Big Bang Theory put it "everything is better with Bluetooth".
Literally "T-shirt with Bluetooth", that's what 99.98% of "AI" stickers today advertise.
>
Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical inputNo, ideally I would be able to predict and understand how my UI behaves, and train muscle memory.
If closing a tab would mean losing valuable data, the ideal UI would allow me to undo it, not try to guess if I cared.
Yeah. It's the Apple OS model (we know what's right for you, this is the right way) vs the many other customisable OSes where it conforms to you.
reply▲YouTube could use AI to not recommend videos I've already watched, which is apparently a really hard problem.
reply▲It just might be that lot of users watch same videos multiple times. They must have some data on this and see that recommending same videos gets more views than recommending new ones.
reply▲Is there a way to tell if people are seeking out the same video or or if they are watch it because it was suggested? Especially when 90% of the recommendations are repeats?
There isn't even an "I've watched this" or "don't suggest this video anymore" option. You can only say "I'm not interested" which I don't want to do because it will seems like it will downrank the entire channel.
Even if that is the case, I rarely watch the same video, so the recommendation engine should be able to pick that up.
I work for YouTube. You’re hired.
reply▲The problem is the people like me who DO rewatch youtube videos. There are a bunch of "Comfort food" videos I turn to sometimes. Like you would rewatch a movie you really enjoy.
But that's the real problem. You can't just average everyone and apply that result to anyone. The "average of everyone" fits exactly NO ONE.
The US Navy figured this out long ago in a famous anecdote in fact. They wanted to fit a cockpit to the "average" pilot, took a shitload of measurements of a lot of airmen, and it ended up nobody fit.
The actual solution was customization and accommodations.
try disabling collecting the history about the videos you've watched in YouTube settings. There are still some recommendations after that but they are less cringe
reply▲My favorite is the new thing where they recommend a "members only" video, from a creator that covers current events, and the video is 2 years old.
reply▲You know what that reminds me very much of? That email client thing that asks you "did you forget to add an attachment?". That's been there for 3 decades (if not longer) before LLMs were a thing, so I'll pass on it and keep waiting for that truly amazing LLM-enabled capability that we couldn't dream of before. Any minute, now.
reply▲> readily discard short or nonsensical input
When "asdfasdf" is actually a package name, and it's in reply to a request for an NPM package, and the question is formulated in a way that makes it hard for LLMs to make that connection, you will get a false positive.
I imagine this will happen more than not.
Using such an expensive technology to prevent someone from making a stupid mistake on a meaningless endeavor seems like a complete waste of time. Users should just be allowed to fail.
reply▲Amen! This is part of the overall societal decline of no failing for anyone. You gotta feel the pain to get the growth.
reply▲if somone from 1960 saw the quadrillions of cpu cycles we are wasting on absolutely nothing every second, they would have an aneurysm
reply▲As someone from 1969, but with an excellent circulatory system, I just roll my eyes and look forward to the sound of bubbles bursting whilst billionaires weep.
reply▲When bubbles burst, is it really the billionaires who are hit the hardest? I'm skeptical.
reply▲Tell you what, let’s make sure this time it is!
Convince them to sink their fortunes in, and then we just make sure it pops.
Expensive now is super cheap 10 years from now though.
reply▲So, like, machine learning. Remember when people used to call it AI/ML? Definitely wasn't as much money being spent on it back then.
reply▲> The end result is I only have to deal with that annoying popup when I really am glad it is there.
Are you sure about that? It will trigger only for what the LLM declares important, not what you care about.
Is anyone delivering local LLMs that can actually be trained on your data? Or just pre made models for the lowest common denominator?
> For example, if you close a youtube browser tab with a comment half written it will pop up an `alert("You will lose your comment if you close this window")`. It does this if the comment is a 2 page essay or "asdfasdf". Ideally the alert would only happen if the comment seemed important but it would readily discard short or nonsensical input. That is really difficult to do in traditional software but is something an LLM could do with low effort.
I agree this would be a great use of LLMs! However, it would have to be really low latency, like on the order of milliseconds. I don't think the tech is there yet, although maybe it will be soon-ish.
It’s because “AI” isn’t a feature. “AI” without context is meaningless.
Google isn’t running ads on TV for Google Docs touting that it uses conflict-free replicated data types, or whatever, because (almost entirely) no one cares. Most people care the same amount about “AI” too.
I want AI to do useful stuff. Like comb through eBay auctions or Cars.com. Find the exact thing I want. Look at things in photos, descriptions, etc
I don't think an NPU has that capability.
Would that be ideal though? Adding enormous complexity to solve a trivial problem which would work I'm sure 99.999% of the time, but not 100% of the time.
Ideally, in my view, is that the browser asks you if you are sure regardless of content.
I use LLMs, but that browser "are you sure" type of integration is adding a massive amount of work to do something that ultimately isn't useful in any real way.
You don't need a LLM for that, a simple Markov Chain can solve that with a much smaller footprint.
reply▲At my current work much of our software stack is based on GOFAI techniques. Except no one calls them AI anymore, they call it a "rules engine". Rules engines, like LLMs, used to be sold standalone and promoted as miracle workers in and of themselves. We called them "expert systems" then; this term has largely faded from use.
This AI summer is really kind of a replay of the last AI summer. In a recent story about expert systems seen here on Hackernews, there was even a description of Gary Kildall from The Computer Chronicles expressing skepticism about AI that parallels modern-day AI skepticism. LLMs and CNNs will, as you describe, settle into certain applications where they'll be profoundly useful, become embedded in other software as techniques rather than an application in and of themselves... and then we won't call them AI. Winter is coming.
Yeah, the problem with the term "AI" is that it's far too general to be useful.
I've seen people argue that the goalposts keep moving with respect to whether or not something is considered AI, but that's because you can argue that a lot of things computers do are artificial intelligence. Once something becomes commonplace and well understood, it's not useful to communicate about it as AI.
I don't think the term AI will "stick" to a given technology until AGI (or something close to it).
No. No-no-no-no-no. I want predictability. I don't want a black box with no tuning handles and no awareness of the context to randomly change the behavior of my environment.
reply▲I’ve seen some thoroughly unhinged suggestions floating around the web for a UI/UX that is wholly generated and continuously adjusted by an LLM and I struggle to imagine a more nightmarish computing experience.
reply▲expedition322 days ago
[-] Honestly some of the recommendations to watch next I get on Netflix are pretty good.
No idea if they are AI Netflix doesn't tell and I don't ask.
AI is just a toxic brand at this point IMO.
Bingo. Nobody uses ChatGPT because it's AI. They use it because it does their homework, or it helps them write emails, or whatever else. The story can't just be "AI PC." It has to be "hey look, it's ChatGPT but you don't have to pay a subscription fee."
reply▲zzo38computer23 hours ago
[-] Hopefully, you could make a browser extension to detect if a HTML form has unsaved changes; it should not require AI and LLM. (This will work better without the document including JavaScripts, but it is possible to work with JavaScripts too.)
reply▲I want a functioning search engine. Keep your goofy opinionated mostly wrong LLM out of my way, please.
reply▲I think they will eventually. It’s always been a very incoherent sales pitch that your expensive PCs are packed full of expensive hardware that’s supposed to do AI things, but your cheap PCs that have none of that are still capable of doing 100% of the AI tasks that customers actually care about: accessing chatGPT.
reply▲Also, what kind of AI tasks is the average person doing? The people thinking about this stuff are detached from reality. For most people a computer is a gateway to talking to friends and family, sharing pictures, browsing social media, and looking up recipes and how-to guides. Maybe they do some tracking of things as well in something like Excel or Google Sheets.
Consumer AI has never really made any sense. It's going to end up in the same category of things as 3D TV's, smart appliances, etc.
I don't remember any other time in the tech industry's history when "what companies and CEOs want to push" was less connected to "what customers want." Nobody transformed their business around 3D TVs like current companies are transforming themselves to deliver "AI-everything".
reply▲I think it does make sense if you're at a certain level of user hardware. If you make local computing infeasible because of the computational or hardware cost it makes it much easier to sell compute as a service. Since about 2014 almost every single change to paid software has been to make it a recurring fee rather than a single payment, and now they can do that with hardware as well. To the financially illiterate paying a $15 a month subscription to two LLMs from their phone they have a $40 monthly payment on for two years seems like a better deal than paying $1,200 for a desktop computer with free software that they'll use a tenth as much as the phone. This is why Nvidia is offering GForce Now the same way in one hundred hour increments, as they can get $20 a month that goes directly to them, with the chance of getting up to an additional $42 maximum if the person buys additional extensions of equal amount (another one hundred hours). That ends up with $744 a year directly to Nvidia without any board partners getting a cut, while a mid grade GPU with better performance and no network latency would cost that much and last the user five entire years. Most people won't realize that long before they reach the end of the useful lifetime of the service they'll have paid three to four times as much as if they had just bought the hardware outright.
With more of the compute being pushed off of local hardware they can cheapen out on said hardware with smaller batteries, fewer ports and features, and weaker CPUs. This lessens the pressure they feel from consumers who were taught by corporations in the 20th century that improvements will always come year over year. They can sell less complex hardware and make up for it with software.
For the hardware companies it's all rent seeking from the top down. And the push to put "AI" into everything is a blitz offensive to make this impossible to escape. They just need to normalize non-local computing and have it succeed this time, unlike when they tried it with the "cloud" craze a few years ago. But the companies didn't learn the intended lesson last time when users straight up said that they don't like others gatekeeping the devices they're holding right in their hands. Instead the companies learned they have to deny all other options so users are forced to acquiesce to the gatekeeping.
The customers are CEOs dreaming of a human-free work force.
reply▲shermantanktop23 hours ago
[-] Suggested amendment: the customers are CEOs dreaming of Wall Street seeing them as a CEO who will deliver a human-free work force. The press release is the product. The reality of payrolls are incidental to what they really want: stock price go up.
It's all optics, it's all grift, it's all gambling.
Just off the top of my head of some "consumer" areas that I personally encounter...
I don't want AI involved in my laundry machines. The only possible exception I could see would be some sort of emergency-off system, but I don't think that even needs to be "AI". But I don't want AI determining when my laundry is adequately washed or dried; I know what I'm doing, and I neither need nor want help from AI.
I don't want AI involved in my cooking. Admittedly, I have asked ChatGPT for some cooking information (sometimes easier than finding it on slop-and-ad-ridden Google), but I don't want AI in the oven or in the refrigerator or in the stove.
I don't want AI controlling my thermostat. I don't want AI controlling my water heater. I don't want AI controlling my garage door. I don't want AI balancing my checkbook.
I am totally fine with involving computers and technology in these things, but I don't want it to be "AI". I have way less trust in nondeterministic neural network systems than I do in basic well-tested sensors, microcontrollers, and tiny low-level C programs.
A lot of consumer tech needs have been met for decades. The problem is that companies aren't able to extract rent from all that value.
reply▲PunchyHamster2 days ago
[-] I do think it makes some sense in limited capacity.
Have some half decent model integrated with OS's builtin image editing app so average user can do basic fixing of their vacation photos by some prompts
Have some local model with access to files automatically tag your photos, maybe even ask some questions and add tags based on that and then use that for search ("give me photo of that person from last year's vacation"
Similarly with chat records
But once you start throwing it in cloud... people get anxious about their data getting lost, or might not exactly see the value in subscription
You and I live in different bubbles. ChatGPT is the go-to for my non-techie friends to ask for advice on basically everything. Women asking it for relationship advice and medical questions, to guys with business ideas and lawsuit stuff.
reply▲Consumer local AI? Maybe.
On the other hand everyone non-technical I know under 40 uses LLMs and my 74 year old dad just started using ChatGPT.
You could use a search engine and hope someone answered a close enough question (and wade through the SEO slop), or just get an AI to actually help you.
“Do my homework assignment for me.”
reply▲Dell are less beholden to shareholder pressure than others, Michael Dell owns 50% of the company since it went public again.
reply▲Meanwhile we got Copilot in Notepad.
reply▲I think part of the issue is that it's hard to be "exciting" in a lot of spaces, like desktop computers.
People have more or less converged on what they want on a desktop computers in the last ~30 years. I'm not saying that there isn't room for improvement, but I am saying that I think we're largely at the state of "boring", and improvements are generally going to be more incremental. The problem is that "slightly better than last year" really isn't a super sexy thing to tell your shareholders. Since the US economy has basically become a giant ponzi scheme based more on vibes than actual solid business, everything sort of depends on everything being super sexy and revolutionary and disruptive at all times.
As such, there are going to be many attempts from companies to "revolutionize" the boring thing that they're selling. This isn't inherently "bad", we do need to inject entropy into things or we wouldn't make progress, but a lazy and/or uninspired executive can try and "revolutionize" their product by hopping on the next tech bandwagon.
We saw this nine years ago with "Long Blockchain Ice Tea" [1], and probably way farther back all the way to antiquity.
[1] https://en.wikipedia.org/wiki/Long_Blockchain_Corp.
Companies don’t really exist to make products for consumers, they live to create stock value for investors. And the stock market loves AI
reply▲The stock market as always been about whatever is the fad in the short term, and whatever produces value in the long term. Today AI is the fad, but investors who care about fundamentals have always cared about pleasing customers because that is where the real value has always come from. (though be careful - not all customers are worth having, some wannabe customers should not be pleased)
reply▲The will of the stock market doesn't influence Dell, they're a privately held corporation. They're no longer listed on any public stock market.
reply▲As someone pointed out, Dell is 50% owned by Michael Dell. So it's less influenced by this paradigm.
reply▲Treating consumers as customers, good.
reply▲PunchyHamster2 days ago
[-] There is place for it but it is insanely overrated. AI overlords are trying to sell incremental (if in places pretty big) improvement in tools as revolution.
reply▲I did use whisper last night to get the captions out of a video file. The standard whisper tool from OpenAI uses CPU. It took more than 20 minutes to fully process a video file that was a little more than an hour long. During that time my 20-Core CPU was pegged at 100% utilization and the fan got very loud. I then downloaded an Intel version that used the NPU. CPUs stayed close to 0% and fans remained quiet. Total task was completed in about 6 minutes.
NPUs can be useful for some cases. The AI PC crap is ill thought out however.
I suggest trying whisper-cpp if you haven't. It's probably
the fastest CPU only version.
But yeah, NPUs likely will be faster.
Depending on the part, it's likely the iGPU will be even faster. The new panther lake has iGPUs with either 80% or 250% the performance of the NPU when at the higher end. But on lower end models, it's lower but still within the same performance class
reply▲satvikpendem4 hours ago
[-] faster-whisper can be faster in many cases, even on CPU.
reply▲Looking at that page, it doesn't seem particularly faster than whisper-cpp, except when using batches - but I'm not clear on what that means.
Does it have a command line utility I can embed into my scripts?
satvikpendem3 hours ago
[-] Batching is essentially running multiple instances at once, ie bundling 8 segments and running them simultaneously on the processing unit, but which obviously takes more RAM to do. Notice, however, that if you drop the precision to int8 from fp16, you use basically the same amount of RAM as whisper.cpp yet it completes in a fraction of the time using batching [0].
Yes, if you check their community integrations section on faster-whisper [1], you can see a lot of different CLIs, GUIs, and libraries. I recommend WhisperX [2], it's the most complete CLI so far and has features like diarization which whisper.cpp does not have in a production-ready capacity.
[0] https://github.com/SYSTRAN/faster-whisper#benchmark
[1] https://github.com/SYSTRAN/faster-whisper#community-integrat...
[2] https://github.com/m-bain/whisperX
If you mean OpenVINO, it uses CPU+GPU+NPU - not just the NPU. On something like a 265K the NPU would only be providing 13 of the 36 total TOPS. Overall, I wish they would just put a few more general compute units in the GPU and have 30 TOPS or something but more overall performance in general.
reply▲They nailed it. Consumers don't care about AI, they care about functionality they can use, and care less if it uses AI or not. It's on the OS and apps to figure out the AI part. This is why even though people think Apple is far behind in AI, they are doing it at their own pace. The immediate hardware sales for them did not get impacted by lack of flashy AI announcements. They will slowly get there but they have time. The current froth is all about AI infrastructure not consumer devices.
reply▲The only thing Apple is behind on in the AI race is LLMs.
They've been vastly ahead of everyone else with things like text OCR, image element recognition / extraction, microphone noise suppression, etc.
iPhones have had these features 2-5 years before Android did.
Apple’s AI powered image editor (like removing something from the background) is near unusable. Samsung’s is near magic, Google’s seems great. So there’s a big gap here.
reply▲That is rather funny because I think Google's and Samsung's AI image actions are completely garbage, butchering things to the point where I'd rather do it manually on my desktop or use prompt editing (which to Google's credit Gemini is fantastic at). Whereas Apple's is flawless in discerning everything within a scene or allowing me to extract single items from within a picture. For example say, a backpack in the background.
reply▲That is unrelated to and unmentioned in the post you are responding to.
reply▲Well if I ever used an slop-image-generator, that’d be an issue, but as I don’t, it’s a bit of a non-event!
reply▲TTS is absolutely horrible on iOS. I have nearly driven into a wall when trying to use it whilst driving and it goofs up what I've said terribly. For the love of all things holy, will someone at Apple finally fix text to speech? It feels like they last touched it in 2016. My phone can run offline LLMs and generate images but it can't understand my words.
reply▲> I have nearly driven into a wall when trying to use it whilst driving and it goofs up what I've said terribly.
People should not be using their phones while driving anyways. My iPhone disables all notifications, except for Find My notifications, while driving. Bluetooth speaker calls are an exception.
It sounds like you mean STT not TTS there?
reply▲You're right, in my rage I typod, its really frustrating, even friends will text me and their text makes no sense, and 2 minutes later "STUPID VOICE TO TEXT" I have a few friends who drive trucks, so they need to be able to use their voice to communicate.
reply▲Better speech transcription is cool, but that feels kinda contrived. Phone calls exist, so do voice messages sent via texting apps, and professional drivers can also just wait a bit to send messages if they really must be text; they're on the job, but if it's really that urgent they can pull over.
reply▲They can also use paper maps instead of GPS.
reply▲I have to say that OpenAI's Whisper model is excellent. If you could leverage that somehow I think it would really improve. I run it locally myself on an old PC with 3060 card. This way I can run whisper large which is still speedy on a GPU especially with faster-whisper. Added bonus is the language autodetection which is great because I speak 3 languages regularly.
I think there's even better models now but Whisper still works fine for me. And there's a big ecosystem around it.
I wonder what the wattage difference is between the iPhone STT and Whisper? How many seconds would the iPhone battery last?
reply▲Kind of a big "only" though. Siri is still shit and it's been 15 years since initial release.
reply▲When I'm driving and tell Siri, "Call <family member name>", sometimes instead of calling, it says, "To who?", and I can't get it to call no matter what I do.
reply▲Amazing how its been 15 years and it still can't discern 15 from 50 when you talk to it.
reply▲All of the reporting about Apple being behind on AI is driving me insane and I hope that what Dell is doing is finally going to be the reversal of this pattern.
The only thing that Apple is really behind on is shoving the word (word?) "AI" in your face at every moment when ML has been silently running in many parts of their platforms well before ChatGPT.
Sure we can argue about Siri all day long and some of that is warranted but even the more advanced voice assistants are still largely used for the basics.
I am just hoping that this bubble pops or the marketing turns around before Apple feels "forced" to do a copilot or recall like disaster.
LLM tech isn't going away and it shouldn't, it has its valid use cases. But we will be much better when it finally goes back into the background like ML always was.
Right! Also I don’t think Siri is that important to the overall user experience on the ecosystem. Sure it’s one of the most visible use cases but how many people really care about that? I don’t want to talk out loud to do tasks usually, it’s helpful in some specific scenarios but not the primary use case. The text counterpart of understanding user context on the phone is more important even in the context of llms, and that what plays into the success of their stack going forward
reply▲are you really asking why someone would like a much better siri?
- truck drivers that are driving for hours.
- commuters driving to work
- ANYONE with a homepod at home that likes to do things hands free (cooking, dishes, etc).
- ANYONE with airpods in their ears that is not in an awkward social setting (bicycle, walking alone on the sidewalk, on a trail, etc)
every one of these interaction modes benefits from a smart siri.
That’s just the tip of the iceberg. Why can’t I have a siri that can intelligently do multi step actions for me? “siri please add milk and eggs to my Target order. Also let my wife know that i’ll pick up the order on my way home from work. Lastly, we’re hosting some friends for dinner this weekend. I’m thinking Italian. Can you suggest 5 recipes i might like? [siri sends me the recipes ASYNC after a web search]”
All of this is TECHNICALLY possible. There’s no reason apple couldn’t build out, or work with, various retailers to create useful MCP-like integrations into siri. Just omit dangerous or destructive actions and require the user to manually confirm or perform those actions. Having an LLM add/remove items in my cart is not dangerous. Importantly, siri should be able to do some tasks for me in the background. Like on my mac…i’m able to launch Cursor and have it work in agent mode to implement some small feature in my project, while i do something else on my computer. Why must i stare at my phone while siri “thinks” and replies with something stupid lol. Similarly, why can’t my phone draft a reply to an email ASYNC and let me review it later at my leisure? Everything about siri is so synchronous. It sucks.
It’s just soooo sooo bad when you consider how good it could be. I think we’re just conditioned to expect it to suck. It doesn’t need to.
> siri please add milk and eggs to my Target order.
Woah woah woah, surely you’re not suggesting that you, a user, should have some agency over how you interact with a store?
No, no, you’re not getting off that easy. They’ll want you to use Terry, the Target-AI, through the target app.
I doubt that anyone is actually suggesting that Siri should not be better, but to me I think the issues with it are very much overblown when it does what I actually ask it to do the vast majority of the time since the reality is most of the time what I actually want to ask it to do are basic things.
I have a several homepods, and it does what I ask it to do. This includes being the hub of all of my home automation.
Yes there are areas it can improve but I think the important question is how much use would those things actually get vs making a cool announcement, a fun party trick, and then never used again.
We have also seen the failures that have been done by trying to treat LLM as a magic box that can just do things for you so while these things are "Technically" possible they are far from being reliable.
I've never used Siri. Never even tried it. It's disabled on my phone as much as I've been able to work out how to do.
reply▲We have a home pod, we use it a lot for simple things like timers when cooking or playing a particular kind of music. They are simple and dumb, but they have become part of our lives. It's just a hands free way to doing simple things we might do on the phone.
We are looking forward to being able to ask Siri to pipe some speech through to an AI
> did not get impacted by lack of flashy AI announcements
To be fair, they did announce flashy AI features. They just didn't deliver them after people bought the products.
I've been reading about possible class action lawsuits and even the government intervening for false advertisement.
Even customers who care about AI (or perhaps should...) have other concerns. With the RAM shortage coming up many customers may choose to do without AI features to save money even though they want it at a lower price.
reply▲Nailed it? Maybe close. They still have a keyboard button dedicated to Copoilot. That thing can’t be reconfigured easily.
reply▲Required for Windows certification nowadays iirc
reply▲Can PowerToys remap it?
reply▲Yes, on my Thinkpad I could remap it with Powertoys. It looks like the sibling comments have had issues though.
For me, the Copilot key outputs the chord "Win (Left) + Shift (Left) + F23". I remapped it to "Ctrl (Right)" and it's functioning as it should.
I have one laptop with a Copilot key in my business. (I didn't even realize that when I bought it.) It takes the place of a modifier key, I think the menu key. Except it outputs a specific keypress (Ctrl+Shift+F23). So it can't be mapped to anything useful like a modifier key. But you can reassign the meaning of Ctrl+Shift+F23.
reply▲SturgeonsLaw21 hours ago
[-] Yep. I installed Claude as a PWA and used Powertoys to remap it to a command that launches it
reply▲You can sort of remap it on windows, but it's somewhat limited in my experience. It shows up as a keyboard chord rather than a simple button press. I think it's LWin+LShift+F23. I ended up simply disabling it entirely on my gaming laptop. I've been meaning to see if it's easier to make it useful on KDE Plasma desktop but haven't yet (though I did remap the HP Omen button to pull down Yakuake instead).
reply▲As someone who spent a year writing an SDK specifically for AI PCs, it always felt like a solution in search of a problem. Like watching dancers in bunny suits sell CPUs, if the consumer doesn't know the pain point you're fixing, they won't buy your product.
reply▲Tbh it's been the same in Windows PCs since forever. Like MMX in the Pentium 1 days - was marketed as basically essential for anything "multimedia" but provided somewhat between no and minimal speedup (v little software was compiled for it).
It's quite similar with Apple's neural engine, which afiak is used very little for LLMs, even for coreML. I know I don't think I ever saw it being used in asitop. And I'm sure whatever was using it (facial recognition?) could have easily ran on GPU with no real efficiency loss.
I have to disagree with you about MMX. It's possible a lot of software didn't target it explicitly but on Windows MMX was very widely used as it was integrated into DirectX, ffmpeg, GDI, the initial MP3 libraries (l3codeca which was used by Winamp and other popular MP3 players) and the popular DIVX video codec.
reply▲Similar to AI PC's right now, very few consumers cared in late 90s. Majority weren't power users creating/editing videos/audio/graphics. Majority of consumers were just consuming and they never had a need to seek out MMX for that, their main consumption bottleneck was likely bandwidth. If they used MMX indirectly in Winamp or DirectX, they probably had no clue.
Today, typical consumers aren't even using a ton of AI or enough to even make them think to buy specialized hardware for it. Maybe that changes but it's the current state.
MMX had a chicken/egg problem; it did take awhile to "take off" so early adopters really didn't see much from it, but by the time it was commonplace it was doing some work.
reply▲ffmpeg didn't come out for 4 years after the MMX brand was introduced!
Of course MMX was widely used later but at the time it was complete marketing.
Apple's neural engine is used a lot by the non-LLM ML tasks all over the system like facial recognition in photos and the like. The point of it isn't to be some beefy AI co-processor but to be a low-power accelerator for background ML workloads.
The same workloads could use the GPU but it's more general purpose and thus uses more power for the same task. The same reason macOS uses hardware acceleration for video codecs and even JPEG, the work could be done on the CPU but cost more in terms of power. Using hardware acceleration helps with the 10+ hour lifetime on the battery.
Yes of course but it's basically a waste of silicon (which is very valuable) imo - you save a handful of watts to do very few tasks. I would be surprised if in the length of my MacBook the NPU has been utilised more than 1% of the time the system is being used.
You still need a GPU regardless if you can do JPEG and h264 decode on the card - for games, animations, etc etc.
Do you use Apple's Photos app? Ever see those generated "memories," or search for photos by facial recognition? Where do you think that processing is being done?
Your macbook's NPU is probably active every moment that your computer is on, and you just didn't know about it.
How often is the device either generating memories or I'm searching for photos? I don't use Apple Photos fwiw, but even if I did I doubt I'd be in that app for 1% of my total computer time, and of that time only a fraction of the time would be spent doing stuff on the ANE. I don't think searching for photos requires that btw, if they are already indexed it's just a vector search.
You can use asitop to see how often it's actually being used.
I'm not saying it's not ever used, I'm saying it's used so infrequently that any (tiny) efficiency gains do not trade off vs running it on the GPU.
Continuously in the background. There's basically a nonstop demand for ML things being queued up to run on this energy-efficient processor, and you see the results as they come in. That indexing operation is slow, and run continuously!
reply▲You also have Safari running OCR on every image and video on every webpage you load to let you select and copy text
reply▲Using VisionOCR stuff on MacOS spins my M4 ANE up from 0 to 1W according to poweranalyzer
reply▲The silicon is sitting idle in the case of most laptop NPUs. In my experience, embedded NPUs are very efficient, so there's theoretically real gains to be made if the cores were actually used.
reply▲Yes but you could use the space on die for GPU cores.
reply▲heavyset_go9 hours ago
[-] At least with the embedded platforms I'm familiar with, dedicated silicon to NPU is both faster and more power efficient than offloading to GPU cores.
If you're going to be doing ML at the edge, NPUs still seem like the most efficient use of die space to me.
It's even worse and sadder. Consumers already paid a premium for that, because the monopolists in place made it unavoidable. And now, years later, engineers (who usually are your best advocates and evangelists when it comes to bringing new technologies to the material world) are desperate to find any reason at all for those things to exist and not be a complete waste of money and resources.
reply▲I spent a few months working on different edge compute NPUs (ARM mostly) with CNN models and it was really painful. A lot of impressive hardware, but I was always running into software fallbacks for models, custom half-baked NN formats, random caveats, and bad quantization.
In the end it was faster, cheaper, and more reliable to buy a fat server running our models and pay the bandwidth tax.
Fundamentally when you think about it, what people know today as AI are things like ChatGPT and
all of those products run on cloud infrastructure mainly via the browser or an app. So it makes perfect sense that customers just get confused when you say "This is an AI PC". Like, what a weird thing to say - my smartphone can do ChatGPT why would I buy a PC to do that. It's just a totally confusing selling point. So you ask the question why is it an AI PC and then you have to talk about NPUs, which apart from anything else are confusing (Neural what?) but bring you back to this conversation:
What is an NPU? Oh it's a special bit of hardware to do AI. Oh ok, does it run ChatGPT? Well no, that still happens in the cloud. Ok, so why would I buy this?
Consumers are not idiots. We know all this AI PC crap is it's mostly a useless gimmick.
One day it will be very cool to run something like ChatGPT, Claude, or Gemini locally in our phones but we're still very, very far away from that.
It’s today’s 3D TVs. It’s something investors got all hyped up about that everybody “has to have“.
There is useful functionality there. Apple has had it for years, so have others. But at the time they weren’t calling it “AI“ because that wasn’t the cool word.
I also think most people associate AI with ChatGPT or other conversational things. And I’m not entirely sure I want that on my computer.
But some of the things Apple and others have done that aren’t conversational are very useful. Pervasive OCR on Windows and Mac is fantastic, for example. You could brand that as AI. But you don’t really need to no one cares if you do or not.
>
Pervasive OCR on Windows and Mac is fantastic, for example.I agree. Definitely useful features but still a far cry from LLMs which is what the average consumer identifies as AI.
Not that far away, you can run a useful model on flagship phones today, something around GPT 3.5's level.
So we're probably only a few years out from today's SOTA models on our phones.
> you can run a useful model on flagship phones today
How?
I think the moral of the story is just don't buy any electronics until you absolutely have to now: your laptop, your desktop, your car, your phone, your tv's. Go third party for maintenance when you can. Install Linux when you can. Only buy things that can be maintained and enjoy what you have.
reply▲I got a new Subaru and the touchscreen is making me insane. I will avoid electronics in cars as much as possible going forward.
It literally has a warning that displays every time you start the car: "Watching this screen and making selections while driving can lead to serious accidents". Then you have to press agree before you can use the A/C or stereo.
Like oh attempting to turn the air conditioner on in your car can lead to serious accidents? Maybe you should rethink your dashboard instead of pasting a warning absolving you of its negative effects?
mixmastamyk3 hours ago
[-] Unless you return it and explain why, then you've supported them with your wallet, the most powerful method of support.
reply▲Usually that legalese goes away after ~30 seconds or when you put it in drive, you rarely have to actually hit “OK”. But I haven’t been in a recent Subaru!
reply▲yeyeyeyeyeyeyee23 hours ago
[-] It's a liability transfer. Thank corp legal.
reply▲How about a recall and replacement for that defective dashboard with one that doesn't cause distractions and accidents since it has buttons that can be felt even by someone wearing winter gloves?
reply▲Finally companies understand that consumers do not want AI products, but just better, stronger, and cheaper products.
Unfortunately investors are not ready to hear that yet...
If the AI-based product is suitable for purpose (whatever "for purpose" may mean), then it doesn't need to be marketed first and foremost as "AI". This strikes me as pandering more to investors than consumers, and even signaling that you don't value the consumers you sell to, or that you regard the company's stock as more of the product than the actual product.
I can see a trend of companies continuing to use AI, but instead portraying it to consumers as "advanced search", "nondeterministic analysis", "context-aware completion", etc - the things you'd actually find useful that AI does very well.
PunchyHamster2 days ago
[-] It's basically being used as "see, we keep up with the times" label, as there is plenty of propaganda that basically goes "move entirely to using AI for everything or you're obsolete"
reply▲The problem is that there are virtually no off-the-shelf local AI applications. So they're trying to sell us expensive hardware with no software that takes advantage of it.
reply▲Yes it's a surprising marketing angle. What are they expecting people to run on these machines? Do they expect your average joe to pop into the terminal and boot up ollama?
Anyone technical enough to jump into local AI usage can probably see through the hardware fluff, and will just get whatever laptop has the right amount of VRAM.
They are just hoping to catch the trend chasers out, selling them hardware they won't use, confusing it as a requirement for using ChatGPT in the browser.
To be fair Ollama does have a GUI.
reply▲I agree with you, and I don't want anything related to the current AI craze in my life, at all.
But when I come on HN and see people posting about AI IDEs and vibe coding and everything, I'm led to believe that there are developers that like this sort of thing.
I cannot explain this.
I see using AI for coding as a little different. I'm producing something that is designed for a machine to consume and react to. Code is the means by which I express my aims to the machine. With AI there's an extra layer of machine that transforms my written aims into a language any machine can understand. I'm still ambivalent about it, I'm proud of my code. I like to know it inside out. Surrendering all that feels alien to me. But it's also undeniable that AI has sped up a bunch of the boring grunt work I have to do in projects. You can write, say, an OpenAPI spec, some tests and tell the AI to do the rest. It's very, very far from perfect but it remains very useful.
But the fact remains that I'm producing something for a machine to consume. When I see people using AI to e.g. write e-mails for them that's where I object: that's communication intended for humans. When you fob that off onto a machine something important is lost.
> I like to know it inside out. Surrendering all that feels alien to me.
It's okay, you'll just forget you were ever able to know your code :)
I've already forgotten most assembly languages I ever used. I look forward to forgetting C++.
reply▲Last part is very common, but what's wrong with assembly languages?
But I wasn't talking about forgetting one language or another, i was talking about forgetting to program completely.
CamperBob221 hours ago
[-] Nothing at all wrong with assembly languages. I just don't need them anymore.
reply▲Partly it's these people all trying to make money selling AI tools to each other, and partly there's a lot of people who want to take shortcuts to learning and productivity without thinking or caring about long term consequences, and AI offers that.
reply▲The "AI" gold rush pays a lot. So they're trying to present themselves as "AI" experts so they can demand those "AI" gold rush salaries.
reply▲I cannot explain this.That usually means you're missing something, not that everyone else is.
reply▲Sometimes, but I didn't get sucked into the crypto/blockchain/NFT hype and feel like that was the right call in hindsight.
reply▲HN had many phases, crypto, js frameworks, the cloud...
The guy coding in C++ still has a great job, he didnt miss anything, is all fucking FOMO.
If you develop software you can’t be as productive without an LLM as a competitor or coworker can be with one.
reply▲SturgeonsLaw21 hours ago
[-] If you have the right soft skills, productivity is decoupled from career advancement
reply▲I am the most productive in my team, by far, 2 promotions in 1 year.
I never use LLMs
Even as a principal software developer and someone who is skeptical and exhausted with the AI hype, AI IDEs can be useful. The rule I give to my coworkers is: use it where you know what to write but want to save time doing it. Unit tests are great for this. Quick demos and test benches are great. Boilerplate and glue are great for this. There are lots of places where trivial, mind-numbing work can be done quickly and effortlessly with an AI. These are cases where it's actually making life better for the developer, not replacing their expertise.
I've also had luck with it helping with debugging. It has the knowledge of the entire Internet and it can quickly add tracing and run debugging. It has helped me find some nasty interactions that I had no idea were a thing.
AI certainly has some advantages in certain use cases, that's why we have been using AI/ML for decades. The latest wave of models bring even more possibilities. But of course, it also brings a lot of potential for abuse and a lot of hype. I, too, all quite sick of it all and can't wait for the bubble to burst so we can get back to building effective tools instead of making wild claims for investors.
I think you've captured how I feel about it too. If I try to go beyond the scopes you've described, with Cursor in my case and a variety of models, I often end up wasting time unless it's a purely exploratory request.
"This package has been removed, grep for string X and update every reference in the entire codebase" is a great conservative task; easy to review the results, and I basically know what it should be doing and definitely don't want to do it.
"Here's an ambiguous error, what could be the cause?" sometimes comes up with nonsense, but sometimes actually works.
> I'm led to believe that there are developers that like this sort of thing.
this is their aim, along with rabbiting on about "inevitability"
once you drop out of the SF/tech-oligarch bubble the advocacy drops off
Well, yes, Dell, everyone knows that, but it is _most_ improper to actually _say_ it. What would the basilisk think?!
reply▲Yes, everybody should buy an AI PC. Buy two! For all we know, that's exactly what we need for AGI... why would you be against that?
reply▲Why would the basilisk care about people spending money on what is clearly a dead end?
reply▲Protip, if you are considering a dell xps laptop, consider the dell precision laptop workstation instead which is the business version of the consumer level xps.
It also looks like names are being changed, and the business laptops are going with a dell pro (essential/premium/plus/max) naming convention.
I have the precision 5690 (the 16inch model) with a ultra 7 processor and 4k touchscreen (2025 model). It is very heavy, but its very powerful. My main gripe is that the battery life is very bad, and it has a 165 watt charger, which wont work on most planes. So if you fly a lot for work, this laptop will die on you unless you bring a lower wattage charger. It also doesn't sleep properly. I often find it in my bag hours after closing it and the fans are going at full blast. It should have a 4th usb port (like the smaller version!). Otherwise I have no complaints (other than about windows 11!).
reply▲After using several Precisions at work, I now firmly believe that Dell does not know how to cool their workstations properly. They are all heavy, pretty bad at energy efficiency and run extremely hot (I use my work machine laid belly up in summer since fans are always on). I’d take a ThinkPad or Mac any day over any Dell.
reply▲Power hungry intel chips and graphics cards are inconvenient in laptops when it comes to battery life and cooling. It is especially noticeable if you spend any time using an M-series macbook pro, where performance is the same or better, but you get 16 hours of battery life. I prefer to use thinkpads, but apple just has a big technological advantage here that stands out in the UX department. I really hope advances are made quickly by competitors to get similar UX in a more affordable package.
reply▲While I appreciate the build quality and ruggedness of the thinkpads, I’d take the bigger trackpad and better screen of the XPS/precision any day. Or, maybe my employer screwed me by giving a shitty thinkpad SKU (it has a 1080p TN panel ffs)..
reply▲I just want a solid laptop that can be used with the lid closed. I want to set it up and never open the lid again. I'll guess I'll keep dreaming.
reply▲Yeah they should make a laptop where you can choose what display you want to use, and which keyboard and mouse for that matter. It could be made cheaper by ditching the screen and keyboard, and heck I wouldn’t even mind if it were a bit bigger or heavier since it’ll just sit on or under my desk. That sort of laptop would be amazing.
reply▲Why would "consumers" as a whole care about an AI specific pc?
Consumers consciously choosing to play games - or serious CAD/image/video editing - usually note they will want a better GPU.
Consumers consciously choosing to use AI/llm? That's a subscription to the main players.
I personally would like to run local llm. But this is far from a mainstream view and what counts as an AI PC now isn't going to cut it.
Dell is cooked this year for reasons entirely outside their control. DRAM and storage/drive shortages are causing costs of those to go to the moon. And Dell's 'inventory' light supply chain and narrow margins puts them in a perfect storm of trouble.
reply▲I can't wait for all the data center fire-sales when the whole "AI" boom goes bust. Ebay is going to be flooded with tech.
reply▲aleph_minus_one21 hours ago
[-] >
I can't wait for all the data center fire-sales when the whole "AI" boom goes bust. Ebay is going to be flooded with tech.
I think a lot of the hardware of these "AI" servers will rather get re-purposes for more "ordinary" cloud applications. So I don't think your scenario will happen.
Yep, hyperscalers go on and on about the "fungible" datacenter capacity in their earning calls as a hedge for a sudden decrease in demand. I could see a scenario where there would be an abundance of GPU capacity, but I’m sure we’d find uses for those too. For instance, there are classic data retrieval workloads that can be accelerated using GPUs.
reply▲Anything but admitting that AI king is naked, here on HN...
reply▲What? No, this is a pretty relevant comment that is being directly caused by AI.
Consumer PCs and hardware are going to be expensive in 2026 and AI is primarily to blame. You can find examples of CEOs talking about buying up hardware for AI without having a datacenter to run it in. This run on hardware will ultimately drive hardware prices up everywhere.
The knock on effect is that hardware manufacturers are likely going to spend less money doing R&D for consumer level hardware. Why make a CPU for a laptop when you can spend the same research dollars making a 700 core beast for AI workloads in a datacenter? And you can get a nice premium for that product because every AI company is fighting to get any hardware right now.
> Why make a CPU for a laptop when you can spend the same research dollars
You might be right, but I suspect not. While the hardware company are willing to do without laptop sales, data centers need the power efficiency as well.
Facebook has (well had - this was ~10 years ago when I heard it) a team of engineers making their core code faster because in some places a 0.1% speed improvement across all their servers results in saving hundreds of thousands of dollars per month (sources won't give real numbers but reading between the lines this seems about right) on the power bill. Hardware that can do more with less power thus pays for itself very fast in the data center.
Also cooling chips internally is often a limit of speed, so if you can make your chip just a little more efficient it can do more. Many CPUs will disable parts of the CPU not in use just to save that heat, if you can use more of the CPU that translates to more work done and in turn makes you better than the competition.
Of course the work must be done, so data centers will sometimes have to settle for whatever they can get. Still they are always looking for faster chips that use less power because that will show up on the bottom line very fast.
flyinghamster2 days ago
[-] See also, Crucial exiting the marketplace. That one hit me out of left field, since they've been my go-to for RAM for decades. Though I also see that as a little bit of what has been the story of American businesses: "It's too much trouble to make consumer products. Let's just make components or sell raw materials, or be middlemen instead. No one will notice."
reply▲So it was RAM a couple months ago and now storage/drives are going to the moon also?
reply▲It was RAM a couple months ago, and it continues to be RAM. Major RAM manufacturers like SK Hynix are dismantling NAND production to increase RAM manufacturing, which is leading to sharp price increases for solid-state storage.
reply▲> What we've learned over the course of this year, especially from a consumer perspective, is they're not buying based on AI .. In fact I think AI probably confuses them more than it helps them understand a specific outcome.Do consumers understand that OEM device price increases are due to AI-induced memory price spike over 100%?
reply▲On the same note, whats going on with Dell's marketing lately?
Dell, Dell Pro, Dell Premium, Dell _Pro_ Premium Dell Max, Dell _Pro_ max...
They went and added capacitive keys on the XPS? Why would you do this...
A lot of decisions that do not make sense to me.
I thought they actually dumbed down the model names. Basically the more adjactives the laptop has, the higher the model is. Now the machines can have pronounciable names and just add generation number every year or so.
Sure, the original numbering system did make sense, but you had to Google what the system meant. Now, it's kind of intuitive, even though the it's just a different permutation of the same words?
seabrookmx18 hours ago
[-] The new XPS's that they just teased at CES bring back the real function keys and have a newly designed aluminum unibody.
I've shied away from Dell for a bit because I had two XPS 15's that had swelling batteries. But the new machines look pretty sweet!
It's a lot easier for people to spend more money when they are confused about their choices.
reply▲Something I learned on HN years ago was the principle that often something that is riding to the top of the hyper curve is usually not a good product, but a good feature in another product.
At CES this year, one of the things that was noted was that "AI" was not being pushed so much as the product, but "things with AI" or "things powered by AI".
This change in messaging seems to be aligning with other macro movements around AI in the public zeitgeist (as AI continues to later phases of the hyper curve) that the companies' who've gone all-in on AI are struggling to adapt to.
The end-state is to be seen, but it's clear that the present technology around AI has utility, but doesn't seem to have enough utility to lift off the hype curve on an continuously upward slope.
Dell is figuring this out, Microsoft is seeing it in their own metrics, Apple and AWS has more or less dipped toes in the pool...I'd wager that we'll see some wild things in the next few years as these big bets unravel into more prosaic approaches that are more realistically aligned with the utility AI is actually providing.
throwaway203715 hours ago
[-] I'm not a game programmer, but is there a use case for NPUs in gaming? One idea: If you had some kind of open world game, like a modern role playing game, where the NPCs could have non-deterministic conversations (1990s-style: "talk with the villagers") that could be pretty cool. Are NPUs are a good fit for this use case?
Does anyone know: How do these vendors (like Dell) think normie retail buyers would use their NPUs?
They still ship their laptops with the Copilot key. Once that is removed then their statement will follow their actions.
reply▲I'd be surprised if Microsoft would sell them Windows licenses or would work with them on drivers if they don't put the Copilot key on the keyboard.
reply▲What is with Microsoft and demanding a key for every new thing they come up with?
reply▲When had Microsoft done this before? For Cortana maybe? I can't recall them ever mandating dedicated physical keys for anything other than the Windows key, but that was over 30 years ago and I assume that's not what you're talking about.
reply▲Ah, my memory was off - it was PC
manufacturers who kept adding more and weird keys (media keys, for example).
All I remember is having all sorts of fun trying to get those keys to work at all in Linux; they often were insanely setup and dependent on windows drivers (some would send a combination keystroke, some wouldn't work unless polled, etc).
They’ve just realised that AI won’t be in the PC, but on a server. Where Dell are heavily selling into - “AI datacenter” counted for about 40% of there infrastructure revenue
reply▲NPU is space that would've probably been better put into something like a low power programmable DSP core, which they more or less are depending on which one you are looking at but with some preconceived ideas on how to feed the DSP its data and get the hardware working. You don't get to simply write programs on them usually from what I've seen.
reply▲Everyone just wants a laptop with the latest NVIDIA graphics card, but also good cooling and a slim design. That's all. People don't care what AI features are built in; that's for Windows and applications.
reply▲Consumers will prioritize products with the latest hardware, good performance, and low price.
reply▲In today's economic environment, cost-effectiveness is a primary consideration for consumers.
reply▲"We're very focused on delivering upon the AI capabilities of a device—in fact everything that we're announcing has an NPU in it—but what we've learned over the course of this year, especially from a consumer perspective, is they're not buying based on AI," Terwilliger says bluntly. "In fact I think AI probably confuses them more than it helps them understand a specific outcome."
--------------
What we're seeing here is that "AI" lacks appeal as a marketing buzzword. This probably shouldn't be surprising. It's a term that's been in the public consciousness for a very long time thanks to fiction, but more frequently with negative connotations. To most, AI is Skynet, not the thing that helps you write a cover letter.
If a buzzword carries no weight, then drop it. People don't care if a computer has a NPU for AI any more than they care if a microwave has a low-loss waveguide. They just care that it will do the things they want it to do. For typical users, AI is just another algorithm under the hood and out of mind.
What Dell is doing is focusing on what their computers can do for people rather than the latest "under the hood" thing that lets them do it. This is probably going to work out well for them.
> People don't care if a computer has a NPU
I actually do care, on a narrow point. I have no use for an NPU and if I see that a machine includes one, I immediately think that machine is overpriced for my needs.
Alas NPUs are in essentially all modern CPUs by Intel and AMD. It’s not a separate bit of silicon, it’s on the same package as the CPU
reply▲True. But if a company is specifically calling out that their machine has an NPU, I assume they're also adding an surcharge for it above what they would charge if they didn't mention it. I'm not claiming that this is a rational stance, only that I take "NPU" as a signal for "overpriced".
reply▲Ahh I hear you that’s a fair observation.
reply▲We do care. We REALLY don't want AI on by default on our PCs.
reply▲Isn't the only AI PC a Mac Studio?
reply▲According to ancient Apple ads, a "Mac" is not a "PC".
reply▲Macs fundamentally can't be personal computers since they're entirely controlled by apple. Any computer running nonfree software can't be a personal one
reply▲lol so the IBM PC isn't a PC?
reply▲There are free BIOSes.
reply▲kube-system23 hours ago
[-] IBM didn't ship any on their PC
reply▲So? You can replace the ROM chip (or flash it, if it's an EEPROM). The whole point of free software is that you don't have to limit yourself to what the manufacturer says you can do.
reply▲kube-system22 hours ago
[-] I was responding to:
> Any computer running nonfree software can't be a personal one
This should have been obvious to anyone paying any attention whatsoever, long before any one of these computers launched as a product. But we can't make decisions on product or marketing based on reality or market fit. No, we have to make decisions on the investor buzzword faith market.
Hence the large percentage of Youtube ads I saw being "with a Dell AI PC, powered by Intel..." here are some lies.
Unfortunately, their common sense has been rewarded by the stock tanking 15% in the past month including 4% just today alone. Dell shows why companies don't dare talk poorly of AI, or even talk about AI in a negative way at all. It doesn't matter that it's correct, investors hate this and that's what a ton of companies are mainly focusing on.
reply▲Should have stayed private. Then they wouldn’t have to care what investors think.
reply▲The whole point of going private is to make the private equity partners a boatload of money by going public again in the future.
reply▲To be fair, Dell has bigger, more fundamental threats out on the horizon right now than consumers not wanting AI.
Making consumers want things is fixable in any number of ways.
Tariffs?..
Supply chain issues in a fracturing global order?..
.. not so much. Only a couple ways to fix those things, and they all involve nontrivial investments.
Even longer term threats are starting to look more plausible these days.
Lot of unpredictability out there at the moment.
I have a "Copilot" button on my new ThinkPad. I have yet to understand what it does that necessitates a dedicated button.
On Linux it does nothing, on Windows it tells me I need an Office 365 plan to use it.
Like... What the hell... They literally placed a paywalled Windows only physical button on my laptop.
What next, an always-on screen for ads next to the trackpad?
It's equivalent to Win + Shift + F23 so you can map it to some useful action if you have a suitable utility at hand.
reply▲Good news: Office 365 has been renamed to Microsoft 365 Copilot.
I'm serious. They dropped the Office branding and their office suite is now called Copilot.
This is good news because it means the Copilot button opens Copilot, which is exactly what you'd expect it to do.
Most consumers aren't running LLMs locally. Most people's on-device AI is likely whatever Windows 11 is doing, and Windows 11 AI functionality is going over like a lead balloon. The only open-weight models that can come close to major frontier models require hundreds of gigabytes of high bandwidth RAM/VRAM. Still, your average PC buyer isn't interested in running their own local LLM. The AMD AI Max and Apple M chips are good for that audience. Consumer dedicated GPUs just don't have enough VRAM to load most modern open-weight LLMs.
I remember when LLMs were taking off, and open-weight were nipping at the heels of frontier models, people would say there's no moat. The new moat is high bandwidth RAM as we can see from the recent RAM pricing madness.
aleph_minus_one21 hours ago
[-] > your average PC buyer isn't interested in running their own local LLM.
This does not fit my observation. It's rather that running one's local LLM is currently far too complicated for the average PC user.
There is one feature that I do care about.
Local speech recognition is genuinely useful and much more private than server based options.
NPUs are just kind of weird and difficult to develop for and integration is usually done poorly.
Some useful applications do exist. Particularly grammar checkers and I think windows recall could be useful. But we don't currently have these designed well such that it makes sense.
A while ago I tried to figure out which APIs use the NPU and it was confusing to say the least.
They have something called the Windows Copilot Runtime but that seems to be a blanket label and from their announcement I couldn't really figure out how the NPU ties into it. It seems like the NPU is used if it's there but isn't necessary for most things.
I wonder if Dell will ever understand why consumers don't care.
reply▲I'll never forget walking through a tech store and seeing a HP printer that advertised itself as being "AI-powered". I don't know how you advertise a printer to make it exciting to customers but this is just ridiculous. I'm glad that tech companies are finally finding out people won't magically buy their product if they call it AI-powered.
reply▲I already have experience with intermitent wipers, they are impossible to use reliably, a newer car I have made the intermitent wipers fully automatic, and impossible to dissable.Now they have figured out how to make intermitent wipers talk, and want to put them in everything.
I forsee a future where humanity has total power and fine controll over reality, where finaly after hundreds of years, there is weather controll good enough to make it rain exactly the right amount for intermitent wipers to work properly, but we are not there yet.
reply▲People dont want feature x (AI). They want problem(s) solved.
reply▲I'm kind of excited about the revival of XPS. The new hardware sounds pretty compelling. I have been longing for a macbook-quality device that I can run Linux on... so eagerly awaiting this.
reply▲I owned a couple XPS 13 laptops in a row and liked them a lot, until I got one with a touch bar. I returned it after a couple weeks and swapped over the to X1 Carbon.
The return back to physical buttons makes the XPS look pretty appealing again.
This is exactly what I was hoping to see. I also returned one I ordered with the feedback that I needed physical function keys and the touchbar just wasn't cutting it for me.
reply▲Sweet, TIL!
I love my 2020 XPS.
The keyboard keys on mine do not rattle, but I have seen newer XPS keyboard keys that do rattle. I hope they fixed that.
People don't want AI PC, cause they don't want to spend 5000 bucks for something that's half as good as the free version of ChatGPT.
But we've been there before. Computers are going to get faster for cheaper, and LLMs are going to be more optimized, cause right now, they do a ton of useless calculations for sure.
There's a market, just not right now.
kittikitti11 hours ago
[-] Consumers could be using AI upwards of 10 hours a day and still say they don't care about it.
reply▲Happy Dell takes user feedback to heart
reply▲I saw the latest xps laptops and I’m really intrigued… finally a high end laptop without an nvidia gpu!
reply▲It seems many products (PCs, TVs, cars, kitchen appliances, etc.) have transitioned from "solve for the customer" to "solve for ourselves (product manufacturers) and tell the customer it's for them, even though it's 99% value to us and 1% value to them".
reply▲The typical consumer doesn't care about any checkbox feature. They just care if they can play the games they care about and word/email/netflix.
That being said, netflix would be an impossible app without gfx acceleration APIs that are enabled by specific CPU and/or GPU instruction sets. The typical consumer doesn't care about those CPU/GPU instruction sets. At least they don't care to know about them. However they would care if they didn't exist and Netflix took 1 second per frame to render.
Similar to AI - they don't care about AI until some killer app that they DO care about needs local AI.
There is no such killer app. But they're coming. However as we turn the corner into 2026 it's becoming extremely clear that local AI is never going to be enough for the coming wave of AI requirements. AI is going to require 10-15 simultaneous LLM calls or GenAI requests. These are things that won't do well on local AI ever.
Even i3 cpu is perfectly fine software decoding 2160p H264, the only consequence is about 2x higher power draw compared to NVidia decoder.
reply▲idontwantthis5 hours ago
[-] I just don’t know what an AI PC is. Does that mean it does shit I don’t tell it to do?
reply▲Seems savvy of Dell. With empty AI hype now the default, saying the quiet part out loud is a way to stand out. Unfortunately, it doesn't mean Dell will stop taking MSFT's marketing money to pre-sell the Right-Ctrl key on my keyboard as the "CoPilot" key.
I wouldn't hate this so much if it was just a labeling thing. Unfortunately, MSFT changed how that key works at a low level so it cannot be cleanly remapped back to right-CTRL. This is because, unlike the CTRL, ALT, Shift and Windows keys, the now-CoPilot key no longer behaves like a modifier key. Now when you press the CoPilot key down it generates both key down and key up events - even when you keep it pressed down. You can work around this somewhat with clever key remapping in tools like AutoHotKey but it is literally impossible to fully restore that key back so it will behave like a true modifier key such as right-CTRL in all contexts. There are a limited number of true modifier keys built into a laptop. Stealing one of them to upsell a monetized service is shitty but intentionally preventing anyone from being able to restore it goes beyond shitty to just maliciously evil.
More technical detail: The CoPilot key is really sending: Shift+Alt+Win+Ctrl+F23 which Windows now uses as the shortcut to run the CoPilot application. When you remap the CoPilot key to right-Ctrl only the F23 is being remapped to right-Ctrl. Due to the way Windows works and because MSFT is now sending F23 DOWN and then F23 UP when the CoPilot key has only been pressed Down but not yet released, those other modifiers remain pressed down when our remapped key is sent. I don't know if this was intentional on MSFT's part to break full remapping or if it's a bug. Either way, it's certainly non-standard and completely unnecessary. It would still work for calling the CoPilot app to wait for the CoPilot key to be released to send the F23 KEY UP event. That's the standard method and would allow full remapping of the key.
But instead, when you press CoPilot after remapping it to Right-Ctrl... the keys actually being sent are: Shift+Alt+Win+Right-Ctrl (there are also some other keypresses in there that are masked). If your use case doesn't care that Shift, Alt and Win are also pressed with Right-Ctrl then it'll seem fine - but it isn't. Your CoPilot key remapped to Right-Ctrl no longer works like it did before or like Left-Ctrl still works (sending no other modifiers). Unfortunately, a lot of shortcuts (including several common Windows desktop shortcuts) involve Ctrl in combination with other modifiers. Those shortcuts still work with Left-Ctrl but not CoPilot remapped to Right-Ctrl. And there's no way to fix it with remapping (whether AutoHotKey, PowerToys, Registry Key, etc). It might be possible to fix it with a service running below the level of Windows with full admin control which intercepts the generated keys before Windows ever sees them - but as far as I know, no one has succeeded in creating that.
> "One thing you'll notice is the message we delivered around our products was not AI-first," Dell head of product, Kevin Terwilliger says with a smile. "So, a bit of a shift from a year ago where we were all about the AI PC."
> "We're very focused on delivering upon the AI capabilities of a device—in fact everything that we're announcing has an NPU in it—but what we've learned over the course of this year, especially from a consumer perspective, is they're not buying based on AI," Terwilliger says bluntly. "In fact I think AI probably confuses them more than it helps them understand a specific outcome."
He's talking about marketing. They're still gonna shove it into anything and everything they can. They just aren't gonna tell you about it.
WTF is an "AI PC"? Most of "AI" happens on the internet, in big datacenters, your PC has nothing to do with that. It will more likely confuse users who don't understand why they need a special PC when any PC can access chatgpt.com.
Now, for some who actually want to do AI locally, they are not going to look for "AI PCs". They are going to look for specific hardware, lots of RAM, big GPUs, etc... And it is not a very common use case anyways.
I have an "AI laptop", and even I, who run a local model from time to time and bought that PC with my own money don't know what it means, probably some matrix multiplication hardware that I have not idea how to take advantage of. It was a good deal for the specs it had, that's the only thing I cared for, the "AI" part was just noise.
At least a "gaming PC" means something. I expect high power, a good GPU, a CPU with good single-core performance, usually 16 to 32 GB of RAM, high refresh rate monitor, RGB lighting. But "AI PC", no idea.
AI PC in MS parlance is a computer with 40+ TOPS NPU built-in. Yes, they are intended for local AI applications.
reply▲> It's not that Dell doesn't care about AI or AI PCs anymore, it's just that over the past year or so it's come to realise that the consumer doesn't.
This seems like a cop out for saving cost by putting Intel GPUs in laptops instead of Nvidia.
How is saving costs a cop out? That's a genuine goal of most businesses.
reply▲Discrete GPU + laptop means 2 hours of battery life. The average customer isn't buying those.
reply