So glad I have no idea what happen yesterday and Im unaffected 
3 Likes
Crash was apparently worse than the covid crash and worse than the FTX crash.
How is it that you missed the near mass suicide that has quickly been forgotten already 
2 Likes
No idea how I miss these things 
If it does not involve running a node on it I’m not interested
5 Likes
This is something I’d like to explore more as to your thoughts on this. I find it hard to imagine I run an APP that runs for a short while be it 1 minutes or a word processor type APP for 2 or 3 hours. Just does not seem very productive at all.
But this is a question for another topic and maybe one day when you get around to it, might make a topic on it explaining what you are thinking about this and how you’d make it worthwhile to run nodes within apps.
2 Likes
It’s also a non solution while ETH gas is required, so not really relevant.
A simpler and short version for now. I will get back on the horse soon.
So you have a personal AI, it is helping you with your whole life. Always on! or a humanoid robot, again always on. These should be local to you AI models (titans, llm etc.) with infinite memory. We want that memory to be ONLY local to you. However the app is always running. So it needs funding to store data etc. but we don’t want the hassle of paying and wallets and all that stuff. For normal folk we want all that to be advanced usage IMO.
So these always on apps (the new computer interface in many senses) can run nodes. That actually makes nodes more effective as the app is looking across the network all the time, whereas a node does not, so we fake that in nodes to refresh the routing table, with apps, that is done and done well, with no fake searches etc…
So then we have memory data banks, so for titans for example there is core memory (short term), persistent (long term rules) and long term (more) mutable memory. Some of this can be shared and some not, but it all has to be local to you and protected from any central authority.
That’s the basics of it. I will hopefully show a POC of this when we publish the API, which I am desperate for.
I am hopeful that will not be the case for too long, but I am working around that where I can at the moment.
16 Likes
I wonder whether melding these together is necessary either, tbh. It is easy enough to have both the client and node app pointing to the same wallet anyway.
4 Likes
Yes, a series of small apps that are used infrequently and for short periods should all be able to share the same wallet invisibly I would hope.
7 Likes
We should also look at EIP-2771 (Trusted Forwarders): and meta transaction type patterns to see if there are ways for clients to pay without fees in clever ways, i.e. include a few extra ANT for the forwarder to handle this. I am sure there are a ton of approaches to that, but this one could be interesting.
7 Likes
Maybe a little off topic, and perhaps you’ve already seen this. The guy that managed to run deepseek-r1 on a bunch of macs has an open source project which allows LLMs to be split across multiple computers / devices. It’s a slight tangent to the local AI idea, but makes it feasible to distribute LLMs across nodes. It currently partitions the LLM across machines, and sends only the last calculation (end of 1 partition → start of new partition a few hundred kb) between machines. The guys ideals seem pretty compatible with autonomi.
5 Likes
How is that infinite memory working? i only checked some videos about local LLM’s it seems they recommend like a Nvidia RTX 4090 with 24GB Vram to get the best experience when running local LLM.
There are several attempts at this.
- transformers squared
- Titans
I also think small models that are fine tuned with old memories (so no forgetting) can be useful. However titans etc. do have forgetting algorithms in them. So it’s a lot of work in that space, but this all leads to open ended AI which is where it’s exciting and dangerous too.
What is amazing is each move forward give us as people many more powers of collaboration and research to build on. So it’s kinda almost self perpetuating with human in the loop right now. It’s about to explode though
7 Likes
Seem very interesting, it will be exciting to follow the progress coming years. I have thoughts about getting a rtx 5090 when ANT becomes a success, it would be nice if you can get away with something cheaper and not throw away $2400 at Nvidia with their monopoly 60% margin. 
And I thought I was a cool kid with my 3090!
1 Like
Hopefully AMD Strix Halo’s top chip will give ok performance with larger models with up to 128gb RAM (96gb for GPU) for a lot less money than an RTX5090 or Mac with 96/128gb ram… but we’ll see.
Also, with Titans, I wonder whether there won’t be the same disadvantage if you have a limited memory compared to current models? Or would that be unaffected?
1 Like
Will a APU have enough compute compared to a dedicated GPU, is RAM that much of a bottleneck?
It depends. GPU memory (shared between CPU & GPU on Strix Halo and Apple M platforms) is a bottleneck if you want to run larger models, hence why AMD suggest Strix Halo will get more than 2x the performance of a 5090 on a 70b parameter model. (edit: actually, 4090 was the comparison AMD made, not 5090)
But, of course if a model fits in 32gb, the 5090 is a far more powerful GPU and would hugely outperform the APU.
My laptop has rainbow light shows all over the keyboard 
That’s what you get for buying a gaming laptop for coding and playing with llms.
Gf was impressed at least.
EDIT: it is though shit hot for compiling Rust, which is a massive win.
7 Likes
I get the feeling that the Strix Halo is being compared to the laptop 4090, that one is not in the same league as a desktop 4090. A laptop 4090 is like a desktop 4080, but I hope the Strix Halo will be good and strong, I need to look into that.
Cooler than me, I only have a 3070 with 8GB Vram, which Nvidia claimed was enough, but it turned out not to be the case. At least I bought it for $550 on the first weeks after the release as I understood that the covid toilet paper zombies soon would buy them all and cause shortages.