WorldTUI: I trained a 360M model to pretend to be a terminal
文章洞見
原文內容目前無法安全取得，因此未產生文章摘要。
討論洞見
尚無可用的討論摘要。
代表性留言
Thank for posting it, happy to answer any questions!
simedw · HN #50191705（在新分頁開啟）
LS, ls, els, ELS, lsl, and so forth each seem to perform an ls. Once I do Xls, none of the others work anymore.
IncRnd · HN #50197758（在新分頁開啟）
Finally they’ll stop scrolling HTML or Java sample code on screen.
qubex · HN #50199455（在新分頁開啟）
The mistake correction capabilities are fascinating, have you experimented around that more, like breaking it up in 2 models? one that generates one that repairs?Wondering if would make any difference in results, which could be impactful given your observation that "Self-correction was one of the more effective training changes"
nico · HN #50199240（在新分頁開啟）
Would this let you actually create a file on your filesystem (as best as the model can), or is it just like a window into its own little world?
ambicapter · HN #50200229（在新分頁開啟）
Very very cool! Impressed that it can be done with only fine tuningOn a very similar theme, I recently built a “lossy git”, which stores your code by training a neural network, then it generates the code back to you on checkout* web demo: https://nicobrenner.github.io/lossy-git/demo.html* cli version: https://nicobrenner.github.io/lossy-git/The web demo runs entirely on the browser, both the version control and the neural network training/inference. You can commit and checkout both images or code - you can also paste/“upload” your own images, no server requests (everything clears from the page on reload)
nico · HN #50192162（在新分頁開啟）
That's very cool project. I read this blog post a few days ago about one way to look at memory encoding in models https://alexwang.ai/posts/associative-memory/ that you might find relevant
simedw · HN #50192306（在新分頁開啟）
That's hilarious. I bet if I committed my code into that repo, it would check out with fewer bugs.
InsideOutSanta · HN #50193460（在新分頁開啟）
This is a micro-implementation of what research calls a "Neural Computer", in which the entire UI itself is generative and driven by user actions.Neural Computers provide the illusion you are interacting with an underlying operating system/application, but are actually a recurring (user input) -> (predicted pixel output) loop.In theory, it would be impossible to tell a perfect "Neural" computer apart from a "real" one. Which is kind of a wild thought.https://arxiv.org/abs/2604.06425
gavinray · HN #50199895（在新分頁開啟）
Great way to get people to type in their passwords.
YeGoblynQueenne · HN #50201097（在新分頁開啟）
>In theory, it would be impossible to tell a perfect "Neural" computer apart from a "real" one. Which is kind of a wild thought.This isn't suprising. After all, you can use 2 neurons to model a NAND gate, and you can build up a CPU from NAND gates only.https://nandgame.com/You don't even need transformers to model a CPU using neural nets. The only problem becomes training them, which becomes very tricky from end to end, especially from shell command to output because the gradients are often going to reverse quite a bit due to binary nature, and the recurrence of memory outputs back into the input (we still don't know how to accurately train RNNS)With LLMS, uou have to do something like have the LLM forget the KV caches after every invocation and instead rely on injected context of a state, which it then also outputs at the end of the terminal command, and then train it on that. That way given enough parameters which translates to resolution, and some command preprocessing (like splitting a complex chained shell commands in a single line), you can guarantee that for every possible input you get accurate output and state change. So instead of an efficient RNN, you basically end getting a much wider lookup table of sorts.
ActorNightly · HN #50200146（在新分頁開啟）
>I used Qwen2.5-Coder-0.5B as the starting point. I needed a small model to get close to realtime performance. Being a coding model, it had also already seen plenty of CLI usage.Interesting. I wanted to ask, how that model compares with the fine-tuned version, but I just saw the author provided benchmarks. Interesting that fine-tuning improved the memory so significantly, wouldn't have expected that.Though, on second thought, I had an experience where I was able to, as a human, fine-tune my memory beyond what I previously thought was possible!Reminds me of one of my favorite jokes.Excuse me, can you tell me how to get to Carnegie Hall?—Practice, man... Practice...
andai · HN #50191790（在新分頁開啟）
I haven’t run extensive tests yet, but larger models also get better at memory.
simedw · HN #50191957（在新分頁開啟）
> … I had an experience where I was able to, as a human, fine-tune my memory beyond what I previously thought was possible!Please elaborate.
treetalker · HN #50192092（在新分頁開啟）
One of the first very cool demos I've seen of ChatGPT was in a post where the author prompted AI "pretend to be a terminal". Then went on to execute commands like `cd .fbi`, `ls files`, `cat files/confidential.txt`. Very fun demo. Looking forward to exploring similar idea with this terminal
ponyous · HN #50192980（在新分頁開啟）
The craziest thing with that is you could "install" lynx browser and "browse" the alternate universe web within this terminal. At the time -- the whole chat bot and LLM thing was very new-- it seemed crazy and impossible that that kind of behavior was possible from "just some next token prediction"! Like, if you "created" a file on this terminal and it was saved , or downloaded a webpage etc. It sucked when they put in some guardrails and the terminal was no more
hydrolox · HN #50198970（在新分頁開啟）
I think this was right at the beginning, in the GPT-3.5 era still.I always wanted to put it behind an open SSH server as a "fake" honeypot of sorts, see what attackers and script kiddies do with it. But I guess by now someone's tried that already..
Doohickey-d · HN #50194180（在新分頁開啟）
> Is it useful? Not really.So interestingly enough Anthropic has been using models of this kind for their alignment evaluations: https://alignment.anthropic.com/2026/reward-seeker/#:~:text=...
saagarjha · HN #50193863（在新分頁開啟）
The useful thing I actually want to do with this: Zero-latency ssh.1. Train a smaller (<10M-50M) model. Lower accuracy is fine; I want latency <10ms.2. Then create a "reactive" version of `ssh` that gives you the perception of zero latency over shitty networks and high latency ssh connections across the world. Basically, the local screen should respond instantly to my keystrokes (including how arrow, tab, etc. esc keys respond in various TUI apps), and then correct itself when the server actually responds.
qurren · HN #50197302（在新分頁開啟）
This could be useful for honeypots (cybersecurity) although it also might have its own attack vector if the honeypot isn’t setup carefully. Sure, you can already virtualize some shells but I’m sure those have limits that this might not have.Cool project, thanks for sharing!
dwoosley · HN #50201655（在新分頁開啟）
This reminds me of this post from ~3 years ago: https://news.ycombinator.com/item?id=33847479"Building A Virtual Machine inside ChatGPT"- https://www.engraved.blog/building-a-virtual-machine-inside/It's mentioned in the "Related Work" section too.
dpflan · HN #50194694（在新分頁開啟）
I've built something very similar but with off the shelf models: https://github.com/beleon/vibeSHEDIT (comparison)Very impressive, and a lot more work than it looks from the outside (the DAgger loop especially). I built something similar in June, vibeSH, a shell that hallucinates the whole computer, but from the other direction: no training, just a frontier model and a long prompt, so maybe the comparison is useful.The granularity is different though. You model keypresses and predict which rows change, I batch keystrokes and have the model repaint the whole screen each round trip. Non interactive cli programs work very good, but interactive cli programs like vim, less and top do work, but it feels like SSH over a 1996 modem, you type a burst, wait, and the screen catches up. Yours is going to be a lot more responsive at that.The big model buys consistency, for as long as the session fits in the context window. Files come back the way you wrote them, python output is usually right instead of guessed, nested paths stay where they are. The price is an API call per round trip and a cost that grows with the session (and it's not 208 MB and doesnt run offline).Mine offers an @ai prefix that talks to the director instead of the machine, so mid-session you can say "@ai make this a haunted Solaris machine from 1996" and the filesystem, the motd and the error messages all change to match, and the directives stick.Your hardest problem and mine are the same one I think. For you it's a 360M model losing track of state, for me it's the context window filling up, after which the box gets amnesia and old files have quietly rewritten themselves. I decided early not to fight that completely and made it canon (the box has bad memory), your DAgger setup is an actual fix for it, which I don't have.
rowbin · HN #50194980（在新分頁開啟）
And the write up: https://blog.leonbecker.de/i-built-a-shell-that-hallucinates...
rowbin · HN #50195033（在新分頁開啟）
I think this project is a very good way to explore the question:>What's the difference between a model that predicts intelligent output, and intelligence itself?It doesn't directly attack the question, but gives a neatly constrained thought experiment which you can see running.
OtherShrezzing · HN #50192906（在新分頁開啟）
i agree absolutely with you. and this kind of reinforce my idea that, like this predicted terminal could not be considered a real terminal, predicted intelligence could not be considered real intelligence
metchio · HN #50192946（在新分頁開啟）
Isn't that just the turing test? I suppose you have you answer now that models can easily pass it.
luipugs · HN #50193072（在新分頁開啟）
It would be interesting to expose this on the internet. I wonder what the scrapers etc… would do with it.
__mp · HN #50191576（在新分頁開啟）
It would be cool to see the results of training on programs that emit kitty graphics commands!
robpruzan · HN #50202003（在新分頁開啟）
Very cool. Fun project. I feel like there’s a wide field to be explored once you understand fine tuning good existing models
oilkillsbirds · HN #50191530（在新分頁開啟）
The next step is training a model to pretend to be the internet.(You could run lynx in that terminal to train/test it without images)
amelius · HN #50193370（在新分頁開啟）
I remember a couple of these from a few years ago. Long gone now of course but a fun demo of AI hallucinating the internet endlessly, generated pages from URLs that had more URLs in them to navigate to.
randfur · HN #50193667（在新分頁開啟）
There was a whole fake gui OS with similar concept right
obilgic · HN #50191486（在新分頁開啟）
VibeOS - Fully Hallucinated Operating Systemhttps://www.youtube.com/watch?v=z3pV6FHvcgM
andai · HN #50191754（在新分頁開啟）
athropic had an experiment that did exactly this. we need more experimentation in this space.
prodigycorp · HN #50191599（在新分頁開啟）
I guess, with enough latency and/or memory, everything can be a filesystem
h1fra · HN #50193994（在新分頁開啟）
Yeah. And with enough energy to burn, anything can be emulated.
em3rgent0rdr · HN #50196506（在新分頁開啟）
I was searching the blog but couldn't find details on the fine-tuning process. What was the hardware and software used? Any learnings there?
ChanderG · HN #50193798（在新分頁開啟）
> I had a blast building this. Is it useful? Not really.Maybe instead of simulating a terminal you could use it to predict what you'd type next?So simulate _you_, not the terminal.
amelius · HN #50193398（在新分頁開啟）
This is super cool! If you publish the weights I would love to bake it into Rune to run a screensaver hallucinating a terminal.
ernestrc · HN #50193946（在新分頁開啟）
It's very tricky to make it hallucinate fake commands. I've been trying `dogsay` (instead of `cowsay`) but no luck. Any tips?
suhdonghwi · HN #50193591（在新分頁開啟）
This would be great for a honeypot, where can observe attackers, use keystrokes for training data to identify attacks.
thinkski · HN #50198960（在新分頁開啟）
I haven't tried yet with this model, but Cowrie already provides support for an LLM backend. [1][1] https://docs.cowrie.org/en/latest/LLM.html
wmchen · HN #50200621（在新分頁開啟）
this is quite similar to amazon-science Cyber-Zero ?https://github.com/amazon-science/Cyber-Zerothe paper is quite interesting - https://arxiv.org/html/2508.00910v2
sandGorgon · HN #50193198（在新分頁開啟）
The 'grid' thinking used here is quite interesting.I'd love to see a video of this playing game of life.
6thbit · HN #50198659（在新分頁開啟）
Questions for folks familiar with fine-tuning: what are the best resources to learn fine tuning?
wy1981 · HN #50193659（在新分頁開啟）
Reminds me of this project from Microsoft. A Quake II “simulation” model where the whole application was just a model.https://www.microsoft.com/en-us/research/articles/whamm-real...
psandor · HN #50191555（在新分頁開啟）
These are called world models, which is why the project is called WorldTUI. For another example, take a look at Google's recent Genie models. https://deepmind.google/blog/genie-2-a-large-scale-foundatio...
soulofmischief · HN #50191688（在新分頁開啟）
I wonder how much a small model could do if it had a tool calling API for a filesystem, an embedded busybox, and a single output token for 'emit what busybox would do here'. Then presumably the model parameters could learn everything that diverges from what busybox can handle.
Lerc · HN #50192077（在新分頁開啟）
I think the toolcalling should be system prompts then the agent is the userspace
alex7o · HN #50192347（在新分頁開啟）
Very cool to do this as a Terminal!I did it for web apps last year, basically a web server that feeds all requests into an LLM.https://www.ohad.com/blog/voidware/
ohadpr · HN #50192466（在新分頁開啟）
I remember doing something like this in the ancient era of GPT 3 or 4. I just told it that it was a terminal responding to commands and it was surprisingly consistent.
air7 · HN #50193295（在新分頁開啟）
This is more useful than you’d think. Consider using something like this to drive a high-interaction honeypot.
Meleagris · HN #50194670（在新分頁開啟）
That's a very cool idea, maybe a bit expensive to scale.
simedw · HN #50194723（在新分頁開啟）
Interesting idea, but I'm not sure why anyone would prefer that over a typical non-generative honypot.
root_axis · HN #50197466（在新分頁開啟）
Wait, how did it `sleep 3`? I thought LLMs didn't have a concept of time
yjftsjthsd-h · HN #50191670（在新分頁開啟）
It’s still rendering frames constantly, no? So it just figures the next X frames are all doing nothing?
peder · HN #50191733（在新分頁開啟）
from the post:Time is an input too. If nothing is pressed, the model gets a WAIT token with a duration instead of a key, which is how snake keeps moving on its own.
sprobertson · HN #50191737（在新分頁開啟）
I regretfully suspect this actually is the future of UI. You just need smart local models running fast enough to keep the framerate high, but that’s inevitable in the long term, isn’t it?
thorum · HN #50191704（在新分頁開啟）
We had decades to work on making software more performant, but instead most users' space-age 2nm chips spend most of their time running Javascript because it was faster to make software that way.
skohan · HN #50191801（在新分頁開啟）
Why would you do that instead of having the model do the task you're using the UI for? It could generate UI widgets on-the-fly as needed to take richer inputs.
gr_norm · HN #50191787（在新分頁開啟）
Wow! This this reminds me of when we tried doing this with gpt 3 and.. it worked!The tmux thing had me confused for a moment until i realized it was emulating tmux. Super fun. Is the model window width aware?
prodigycorp · HN #50191594（在新分頁開啟）
The first part of the context is the window width:``` BOS SCREEN_START 32 96 ```I trained it on three different resolutions, but it should kind of work with in between ones too.
simedw · HN #50193666（在新分頁開啟）
good old days https://news.ycombinator.com/item?id=33847479
serje1480 · HN #50191981（在新分頁開啟）
Holy cow that’s amazing. I’d bet future apps will be built this way.
aurareturn · HN #50194251（在新分頁開啟）
Quite possibly. Because the models are "eating" the harness --they don't need it to tell them to do multiple steps with planning, implement, review, or to spawn sub tasks, etc. anymore.I also assume that in some way, in order to produce correct code without having to do a lot of but fixes, the larger models have gained implicit to execution engines or at keast predictors in their weights.There are also several projects that have trained VMs into layers. Or embedded them in various ways.There are two major potential advantages if you can really make it work well. One of them is potentially huge speed improvements by iterating inside the accelerator without deciding and running external programs. The other is being self-contained, eliminating the need for most of the harness complexity.Another reason I think this is likely to become a real thing is that it seems with hardware improvements with flash and within a few years fully in-memory compute, we are going to get much larger, faster and smarter models. So they will be able to more easily absorb the complexity of emulating systems with more fidelity, as well as operate more quickly.You can also imagine that as models get larger and faster, it is a natural next step in training. The top models now automatically run the software the generate and test it thoroughly. They do not need to be told to do this. By feeding enough of those parts of the sessions back into a large enough model to train on, the system gains the ability to predict the outcome without having to run the code. So it's the core part of a training pipeline already built into recent models.There also might be some convergence with continuous learning. Because altering weights based on some token context seems like it could be similar to keeping a program resident. And maybe the continuous learning part provides the "compilation" step.
ilaksh · HN #50195080（在新分頁開啟）
I fry my brain everyday trying to keep pace, but honestly, stop it guys. Give the guy a break.This is the most magical time in my adult life. The 70s, 80s, and 90s, look like a painting compared to the current pace of things. I am literally tired at the end of each day. Dog tired.Great job OP. I mean. This is nuts.
_bobm · HN #50194735（在新分頁開啟）
wonder if the PHP on this hallucinated terminal is patched already.
4ggr0 · HN #50193160（在新分頁開啟）
Perhaps it has a hallucinated PHP that never had those bugs.
claykkari · HN #50193609（在新分頁開啟）
I tried it and is not working remotely as close as the demo gif. It keeps telling me that "test1" folder that I want to create already exists (not true), then letting me do "mkdier test2" (sic) but then the folder is not there when ls. I mean all its expected, this has no state, follow no rules and is making up things from the current context, but is not as "magical" as I thought when watching the demo.
donperignon · HN #50192212（在新分頁開啟）
Why did it return -1 to 10 - 4
Muhammad523 · HN #50193910（在新分頁開啟）
So, uhh, what's your sudo password? Can't get this thing to install software...
mastax · HN #50197463（在新分頁開啟）