I wish I could paste my profile into a new device without needing to sit through the quiz again.

Thats a great point, and a good articl. I think the example they bring up as "good" and what we are leaning into here, is the idea of transparency and agency of being able to see and modify your profile.

The tension we have been finding is that we dont want to require people to "know how to prompt" to get value out of having a profile, hence our ongoing thinking around how to bootstrap good personal profiles from various data sources.

As Koomen notes, a good profile feels like it could be the best weapon against "AI slop" in a case I want something sharp and specific. But getting to that requires knowing how to prompt most of the time.

Nice, great to see where it nails it, and has the right feel! Using your own comments directly, or in the first instance allowing users to paste in sections of their commented on or upvoted history is a great next step. Not sure yet whether the flow should be in place of or in addition to doing the survey.

Hey so cool to hear! the skim/skip/dive mapped to how we use things, but thats sample size of 3 haha.

The idea of having some kind of thumbs up/down on what you see after getting recs, that gets added to your preferences, or being able to do another round of preferences (rather than just re-doing them like we have now) is for sure on our next steps if we continue with. Were not quite sure what the feedback loops will be yet (we did look at adding whole webhistory for example but that felt like a bit much and pretty invasive).

For the miscats, on a meta level what we are generally interested in is whether they come from compression of the preferences into your user profile (essentially if more or better data is the path to better context for such a specific usecase, or whether there is more bang for buck optimizing the various prompts. Keen to hear if its obvious from looking at your profile what was the case.

If we get serious with this evals are a must next step. We are only 2 days in at the moment :)

This is great to hear in such detial. One of the first cabs of the rank to improve this would be greater user control over what preferences to include, and/or smarter selection of the pool to select. This kind of focused preference is super important, especially as i use hackernews, and clearly for you and i suspect others as well.

In fact I would pose that I have a couple of disparate interests or "profiles" that i would like to have greater control over/support in generating, that are non overlapping sets of topics and types of content. The ability to have greater agency in creating them and managing them is something we are keen to explore.

The article comments one is a toughie, as LLM use skyrockets when you scrape and consume content from the links. It would be awesome to include it, but would likely need to be paid, just from a cost perspective.

Really appreciate the detail here, this makes it easier to turn your examples into a test/eval/feature case.

this is an angle we honestly didn't think about (we are pretty much long time lurkers) but accessing existing HN content is a great idea! I didn't even know there was a page of upvoted submissions :) It doesn't look like thats available via the API, but a copy paste of some text should work just aswel, all we pass through is titles and urls to the LLM anyway to generate the profile so its much the same.

More generally a next feature we want for ourselves is a way to add just some generic text and "update" the profile with that, rather than generate it fresh exclusively off of the 30 examples. This circles back to us using this as a focus point to think about what data is enough to generate a good user profile, and what good is.

thats grand, thank you! we really want to look at both using comments as a source for preference, and applying your preferences to comments to perhaps flag/sort/filter once you are on a page. Will have to think about the user flow, and the way in which preferences differ for comments vs posts

Yes us to! I spend heaps of time playing with the link between preferences and the profile.

We aren't really sure yet how best to surface _why_ the model predicts what it does. You can hover over the skim label and there is a bit of reasoning text, which might shed some light on why for now. We will think more about how to make these relationships more clear in the process of tightening them up and generally improving them.

Once the relationships are a bit more clear theres probably the 80/20 rule of work to tighten up those predictions.

Are you referring to what's in the textarea on the edit profile text dialog? Seems to be a simple concatenation of the titles according to how I tagged them. If it's indeed this what you're referring to, what did you find interesting about this?

how ever you like! i tend to do it off the title, but you could click into it if you like.

we came to this by looking at how a "user profile" in plain english could be both used and generated by LLMs, but once we were looking at this we did discuss traditional recsys. Two things against it for this usecase: bootstrapping preferences with a low number of data points, and no "unified" storage of all users preferences or pre-existing dataset is difficult with trad ML or statistical methods. Also having your preferences or "model" if you will in plain english gives a sense of agency, transparency and individuality to your recomendataions that are at least difficult, if not impossible to communicate with other types of models.

Id love to have those assumptions challenged though, if there are examples you could point me towards.

hahaha thats a great test case, gona remember that for evals. Did it notice anything? or did it just get confused. we found if you have random preferences it often just says to skim half and skim half, which i suspect it might do in your case.

It _in theory_ should try and pick up a content style (funny stuff??) even if the tech is seemingly random, but i wouldn't be surprised if it just failed.

I genuinely find that interesting to hear. what about the 30 examples felt different to say the frontpage? (assuming it did)

On a meta level i was suuuuper conscious of writing every word of this post/comments myself, as my prior is that HN's community is very intollerant of and highly sensitive to low effort content, whether via AI or not. This is despite using AI tools for lots of other parts of work (drafting, coding, summarising, brainstorming etc).

Do you think HN has become more accepting of AI slop, the slop is becoming harder to detect, or isnt as discerning as i assume?

yeah we see that a bit as well, i promise its not hardcoded in haha.

On the LLM side of things we are using Gemini 2.5 flash, mostly for speed, and found it to be reasonably good quality at a vibe level compared to something heavier like claude 4, probably because we've worked hard to keep the task very simple and explicit. But in saying that there are a bunch of comments on quality that really highlight that if we want to get serious about that we should put in some user feedback loops and evals.

Its all in JS/TS, using vercel ai for the LLM calls, storage is local, but in order to really dig into quality we might start saving things, but to do that well we'd have to add auth/users etc. and we wanted to keep it light for a demo. We have been recently exploring langfuse for tracing, and are really liking that, and will probably look at using them for first pass evals when we get to it for this project.

We also talked quite a bit about non-LLM recsys and aside from time to set up and do well, something I really like is the sense of transparency and agency. you can see your profile, and edit it if you like to see the change in your results. I almost think wed lean further into that rather than folding in some trad DS or recsys stuff even if that might make the results better. Just musings at this point though.

yeah seeing so many people using this its clear we should add some way for people to indicate when things felt off vs good, so that we can start tweaking the system, maybe with some evals.

We played around with the idea of a "fun" or "random" category, but ultimately didn't include it in this little first demo, as we found it super hard to have it not be just literally random (although that might not be a bad thing as you say)

On the topic of different moods and headspaces, thats one of the things more broadly we are really thinking about outside of this demo, and hadn't really considered for here but should. What different data can we use (in this case maybe just a different survey for different "profiles"), and how can a user manage those different profiles and front pages will be questions to answer.

Id be really interested to know if anyone has done topic grouped or themed frontpages for hackernews, as this would map well to that concept. ill have a look.

thanks for checking it out and taking the time to give some feedback!

When weve been testing things, we often find that if there wasnt a great match between the options when picking preferences, and whats currently on the front page, that the context it generates will result in a lot of skips (understandably, but not great UX). Right now can try regenerating your context (and going through the process again), or manually editing it to get to different results.

Theres also some work for us to better select the options when picking preferences, or ensuring we always surface some deep dives.

Applying the same process to more pages, or bubling up content from multiple pages, or new is a great idea. cool to hear thats where you would look.

hackernews is such a rich source of "non trivial" content, and we are really seeing this when using it for a project compared with just consuming it as an individual.

We are focusing right now on how comments could be used to build up a better user context, and your comment has made me think about how we can feed comments in (instead of just titles and urls) for your selected preferences to make a better profile, without needing to scrape anything (expensive and slow).

But i think that would only work because of their quality and relevance generally, which would for sure make it an interesting knowledge source for pure LLM search. Feels like something someone should build or maybe already has! keen to see if anyone knows of projects like that.