I’ve just pushed a new version which:
- (sn_httpd) Tidied up the code since disabling streaming, which should improve download times
- (sn_httpd) Uses latest autonomi libs
- (docker) Silences noisy logs (sn_networking mostly)
I’ve just pushed a new version which:
Submitted a feature request: Feature(API): Allow data streaming for gets · Issue #2421 · maidsafe/safe_network · GitHub
Begone! I had some wrestling with streams in Rust, and I don’t wish this on anyone I’d like it to be more like an implementations of Read
and Write
.
Check out the Dev Forum
I’ve made a number of improvements and released a new version (v0.3.5) of sn_httpd.
sn_httpd binaries can be found here: Release v0.3.5 · traktion/sn_httpd · GitHub
docker images can be found here: https://hub.docker.com/repository/docker/traktion/sn_httpd/general
Docker is probably the easiest to get going!
Changes include:
Images and MP3s, etc, seem to load in a couple of seconds or so from my local docker instance now, which is pretty cool.
I hope you like it and more changes are to come! I want to have a play with directory listings and file uploads again too.
Some screenshots!
I’ve pushed a new release (v0.3.6) with the following release notes:
Release binaries: Release v0.3.6 · traktion/sn_httpd · GitHub
Docker latest updated too: https://hub.docker.com/r/traktion/sn_httpd
I updated the test file to access some of the test files folks have uploaded too (assuming sn_httpd is running on localhost:8080): http://localhost:8080/c8430f24e51ad9225c48a7e9c4cb8fcd9fec19066c178fe1458847b85e670597/streaming.html
Obligatory screenshots:
(Yes, I know… too many tabs! I was in a rush! )
Fyi, I managed to get Foxyproxy working on android with Firefox. It lets me whitelist a specific *.autonomi URL pattern to send to sn_httpd.
As android proxy settings are usually system wide and exclude only, this is great news.
I can now dedicate firefox to autonomi only or let it do clearnet and autonomi. Then chrome (or whatever) can still be used for regular clearnet browsing.
Why does it matter? It is cool to use DNS style resolution instead or xor addresses. It is gives us an easy way to do this on android.
Also, as android can run sn_httpd in termux, out of the box (just turn hotspot on to get a dns resolver for seed servers), it gives us more options. Both the browser and server can run on the device, avoiding external hosting, etc.
In fact, we can also resolve everything in a path only way, e.g. http://autonomi/traktion/streaming.html
There could be security issues/benefits, but interesting option.
fantastic work @Traktion iv got a docker set up just having a play with it now
Yes, fantastic work on that. I’ve been moving to a localhost server approach myself as a custom browser (awe style) is out of reach. So very interested in how you are handling this.
My new URL scheme seems clunky but was designed to support versioning in a regular browser. I’d like to have feedback on it sometime in case there’s a better way.
Great to see this @Traktion, very cool indeed.
BTW I have just published the first version [cough] of an API for handling, er… versioned data (see dweb / trove::TroveHistory on crates.io). Even though it’s going to need a rewrite for Transactions it should be fine pretty much as is, though I’ll make the terminology more generic (ie remove refs to Registers) so you might find it useful.
I’ve been meaning to take a look at AWE addressing approach. It definitely feels like an area to align on!
So far, sn_https just looks first 2 levels of hierarchy, with the first being the archive and the second being the file within it. It used to be a bit more conveluted, but with the archives available in the API, it makes life a lot easier.
So, http://localhost:8080/archive/filename is the same as http://autonomy/archive/filename or http://archive.autonomi/filename, etc.
Edit: fwiw, the middle one is neat… narrowing hierarchy is cool. If we don’t have to worry about CORS, certs, etc, it would be nice. I’ve not dedicated thought to those implications yet though.
Sub directories in an archive should slot into the middle easily enough, but not tested that yet!
The archive itself can either be XOR or a ‘name’ stored in a register. The name lets us point to the latest archive xor address.
Under the hood, there is a public root name list (register) for pointing a another private app list (register). The latter can be appended (updated) to the latest archive one update.
So, the idea is, first come first serve for public root names. Then the names can be privately managed to point where they want to.
It’s pretty rudimentary, but works… sort of. I rely on CLI to update the registers atm and the history was funky last time I messed with it. I’m assuming transactions can solve that though.
Performance wise, sn_httpd now loads all names into a hashmap on startup. That may not scale, but ok for now. In theory, the list could be stored on a flat file (or archived there) and maybe the whole thing could be spun off into a resolver app… maybe even an existing DNS resolver could be bent to fit?
I’ll take a look at the crate too. Will be interesting to see! I need to do the crates thing at some point too… not enough time in the day!
Thanks for that. I won’t get into questions just yet. Ideally I’ll write an explainer for key design areas and you could comment on that, compare to your approach etc. I find it helps to write things up, because I gain a better understanding of what I’ve done!
Any feedback or questions you have for me will be welcome if you do look at dweb or awe.
awe now uses dweb but I’ve not pushed that change yet so you will see dweb within awe until I do. I chose not to use Archive but stick with my own type for now. I don’t think they have had time to think things through so my types are not much more complex but much more capable and shouldn’t need to change for a while, hopefully never. Famous last words. I use a single type to cater for both a file system and a website style application (so providing for extending metadata for files, web redirects etc).
On CORS, I’ve not looked either because I wasn’t needing to. Moving to the browser I will, but am hoping it might help plug leaks from Autonomi sites to the conventional web. I really don’t know if that is likely though!
Publishing to crates.io
is fairly painless BTW, good instructions and helpful error messages make it straightforward.
Changes have now been tested and pushed to awe and dweb, including updates for the upcoming renamed Autonomi modules.
That means others can’t easily build them, but the code can be explored.
Next I’ll be moving awe functionality to dweb-cli at which point awe will be archived.
Versioning has been broken by the removal of the merkle_reg from Registers (in the upcoming Autonomi release) so I’m waiting for Transactions to become useable in order to restore that.
I’d like to try this. I’m on a chromebook in Linux Developer mode and installed docker.io. Did docker run --rm -it -p 8080:8080 traktion/sn_http, and it seemed to be working until this line:
[2024-12-24T01:08:13Z ERROR ant_bootstrap::contacts] Failed to get bootstrap addrs from URL http://139.59.198.251/bootstrap_cache.json: reqwest::Error { kind: Request, url: “http://139.59.198.251/bootstrap_cache.json”, source: TimedOut }
Can you walk me through the steps I need to take to get there?
@Helen glad to see you are trying it out!
I was literally pushing new docker images last night, as the latest network was incompatible.
From looking at those log entries, you just need to wait a while until it logs that it has ‘Started listener’.
The autonomi libs seem to take a bit longer to bootstrap now, but it should get there.
Trying it now… lots of “Failed to dial” errors… just let it run?
Yup! It seems a bit noisy on the error logging in the new library. After about a minute, it should be ready.
Okay! Still waiting for it. It’s been hanging here for a while:
[2024-12-24T19:51:49Z INFO actix_web::middleware::logger] 172.17.0.1 “GET /autonomi/4b5efdacd5d87da24f72fe6ddb214ab9df7aa875b198a20ccdbdc5dc6d7a0d6a HTTP/1.1” 404 39 “-” “Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36” 2.093666
Looks like it is listening if you’re getting those logs. You should get some messages about downloading chunks.
EDIT: Try this URL if that doesn’t work:
http://localhost:8080/7a8554af57f3664c1f7896cae2394119d8ead2d49e736b060d8a66fc6a2befa1/BegBlag.mp3
Yeah, it’s working! So cool. Now, how can I develop a web page that can be viewed by others?
If you want to start with a static site, you can upload a directory files as a ‘public archive’.
You may have figured it out from the logs, but the directory in the link above is the archive. If you go to the directory instead, you will see a listing (of only 1 file in this case):
http://localhost:8080/7a8554af57f3664c1f7896cae2394119d8ead2d49e736b060d8a66fc6a2befa1/
If you upload an archive, you will see a bunch of files in the archive and you can link any of them.
I was waiting for a stable network again to do a bit more dev to allow ‘index.html’ to auto-load when available. I’m also having issues uploading archives at the moment, as it’s taking out my router, which is adding to the dev challenge!
Anyway, if your uploads work, you should be able to see how it comes together.
EDIT: To add, you can upload a directory as a public archive using:
ant file upload -p -x mydirectory
It seems to not be allowing uploads due to not recognizing my wallet. I tried setting SECRET_KEY as well as importing it… any insight?