ScreenCoder: An intelligent UI-to-code generation system
62 points
26 days ago
| 5 comments
| github.com
| HN
dv_dt
26 days ago
[-]
Lol, I have been exploring in the opposite direction. Given a set of basic CRUD endpoints for various data structures, including links to other structures. Generate a basic UI for them, with guidance on added ui to generate based on input and data oriented interaction specs.

This is cool though, I think there are going to be an explosion of different higher level development patterns that will be useful in a lot of different situations that AI will enable. It will be interesting to see which ones get wider adoption, and which stay useful but in smaller focused niches.

reply
javcasas
26 days ago
[-]
reply
j45
26 days ago
[-]
Domain driven design or data driven design has it's benefits for sure.

A scenario that comes up is the combination of understanding the problem space, use of tools and what the developer knows/needs to learn can shine their path in different direction to get to the outcome.

reply
_fat_santa
26 days ago
[-]
At my org we have been using the Figma MCP server to generate code from Figma designs. Spent about a day writing rules to keep the AI on guardrails but it's been very smooth sailing since then and any generated code needs minimal changes.
reply
alooPotato
26 days ago
[-]
Ohh nice can you say more? Do you somehow map your Figma components to React components?
reply
_fat_santa
24 days ago
[-]
I can share that we use this MCP server: https://github.com/GLips/Figma-Context-MCP.

As far as rules, it's very dependent on the codebase you are working in. My recommendation for rolling something like this out is take about a day or so and try to build UI components with it, and then add to your CLAUDE.md/.clinerules/etc as the AI screws up, and there will be a ton of screw ups. Funnily enough the first rule I had to write was to tell Cline to use the MCP plugin because it kept trying to run a fresh copy and could never connect.

reply
vvram
26 days ago
[-]
Nice! What kind of rules do you recommend? Are any of the rules in OSS?
reply
wiradikusuma
26 days ago
[-]
I guess it's good for "Dreamweaver 2.0" in the sense that it produces plain HTML. But my understanding is most serious web applications use a framework of some sort, such as React, Vue, or Svelte, and each is opinionated.
reply
jagged-chisel
26 days ago
[-]
> … most serious web applications use a framework of some sort …

So many things wrong with this sentiment.

Define “serious.”

Do people think it’s just not possible to make web apps without frameworks?

Take the output from this tool and adapt it to whatever framework you want … if you feel that’s necessary.

reply
pzo
26 days ago
[-]
It's so much harder to make very accurate image to plain HTML conversion than plain HTML to react et at. LLM are very good with translating between different web frameworks.
reply
rane
26 days ago
[-]
There's this one new cool technology that could be just the thing for converting the HTML to use web framework of preference.
reply
kocial
26 days ago
[-]
Oh! What about CSS frameworks like Tailwind, Bootstrap, etc.?

How on earth was it able to generate and clone Images just from UI? Like did it get the source of the URL as well, where it could link those images or what did it actually do there?

reply
burkaman
26 days ago
[-]
I think it's just cropping the provided screenshot: https://github.com/leigest519/ScreenCoder/blob/main/image_re...
reply
vivzkestrel
26 days ago
[-]
Would be nice if it generated HTMX
reply
stronglikedan
26 days ago
[-]
It's just a PR away, so get crackin'!
reply