:IF: Colony - Search Autonomi Simply

Bug report :beetle:

3 issues I’m tracking currently:

  • AppImage doesn’t execute on Arch linux. This seems to be a Tauri issue, not something specific to Colony. I’m unsure how to fix this one:
Could not create default EGL display: EGL _BAD_PARAMETER. Aborting...

If any other developers with Tauri apps have figured this out, I’d love to hear how you fixed it.

  • Clicking ā€˜Upload All Pods’ with updates too quickly can cause network synchronization issues. Basically what happens is, the scratchpads are pushed out onto the network and are working their way through, then another one is sent. Sometimes the network will see this as a fork, or 2 versions of the truth. @Anselme has made a patch to the autonomi crate to enable me to arbitrate the issue on the app side and I have the code to do this working, but I need an official release in order for me to update Colony itself. So for now, wait like 5 minutes between clicking ā€˜Upload All Pods’ just to be safe.

  • If you add a local pod as a pod reference in one of your other local pods, you won’t be able to see it in the GUI. This was a bug in colonylib. I’ve fixed it locally and it works, but this fix is stacked on top of the forked scratch pad error handling stuff I mentioned above. So for now, just avoid referencing your own pods.

1 Like

I suggest you open a Tauri issue to find out. They’ve been very helpful with me in the past. The Discord can be good too.

2 Likes

hi @zettawatt, I have uploaded my first file waves.mp4 but how do I actually find it in Search? Or - how do other people find that file? Can you see it in the list of files? (because I do not)

edit: I think I found it. I had to go File Management > Your Pods > Edit > Transfer File to the Pod > Edit > Choose video metadata, etc > Then upload my pods. (very complicated for a new, noob user like me to find and figure out)

Now I can see Waves.mp4 in the search… Don’t expect nothing extraordinary, it’s just a downloaded clip from Pixabay :joy: Which by the way may be a great resource for new uploads, what do you think?

2 Likes

I was just coming on here to help walk you through it, but I was too slow :smile:. Excellent. This was the one part of the UX we couldn’t figure out how best present to the user. Uploading is easy, just push a button, but writing the metadata will always require a human touch, especially for things that are new. Like I did in the ia_downloader program, I plan to add the capability to scrape databases and call an LLM to help populate eventually. If you have any suggestions, they will be greatly appreciated.

At a high level, there are always these 3 steps:

  • upload a file
  • attach metadata to that file in your pod
  • share your pod address with others so they can find it

So for everyone else to be able to find it, all you need to do is share your pod address here, I’ll add it as a reference to the Genesis Pod, and everyone who installs Colony from here on out will be able to search for any file that has metadata in that pod in the future. Or if you want to bootstrap someone else onto the network, simply give them your pod address and they’ll tie into everything that is connected in this way. As the connections get formed between people, the topology will turn into a web instead of a spoke and wheel, and the Genesis Pod will no longer be that important, just one more among many.

After you post that address, you’ll be the first one that has shared their pod, ever :tada: !! @southside and @aatonnomicc , you guys better catch up :laughing:!

4 Likes

Quickly before they catch up… this is my ā€œwhat is podā€ Pod address a67d5401e2720ba381092df3ac868d92533f2cbcf2bc2555e75957e5ce346ea8fa3c4792c9e9c9a5df2ae769972ea258 :joy:

Edit: Now I can reply calmly… :grin: what I would suggest is to attach metadata & create Pod association at the time of upload, for example having a prompt before hitting the upload button, to fill in all necessary information for the file to be found (public file) or to skip this dialog (private file).

All in 1 step when uploading, not doing it afterwards and figuring out how to do it. Is that technically possible?

2 Likes

now that I shared my pod address, people can basically see all files uploaded into this pod? Does it work like a kind of folder on the network? Sorry, I still have many basic questions and need to understand all of this new terminology. I actually never heard the term pod yet (maybe because I’m not English native), but is there some other possible word that could explain its meaning? Is folder the right word?

3 Likes

It works :tada:!! I was able to find your ā€˜waves.mp4’ video file!

Yes, this is possible. I’ll need to cache the metadata information and plug it into the pod on successful upload. The ā€˜Upload All Pods’ operation will need to be a separate operation since it locks up the app while it is operating, but I could put an indicator somewhere that tells the user they need to upload their pods for people to see the things they’ve uploaded.

No worries, I just didn’t explain it well. I agree that folder is probably a better word because we’re kind of building a file system on the network. If you’re used to Unix land the mapping is like this:
pod → folder : store all of your files’ information in this container
pod reference → symbolic link: link to other folders (pods) on the network.

Using file system terminology, you added your waves.mp4 file to your folder (pod). Then you sent me the path (autonomi address) to your folder (pod). Then I added that path as a symbolic link (pod reference) in my folder (pod).

Now when someone looks at my folder (pod) and does a find -L (search) they can find every file that matches their query through all folders (pods) and symbolic links (pod references).

I’m not hung up on the names, if it makes sense to change them, I can do that. I more or less put those in because I didn’t know what to call them when I started coding this thing up :rofl:

6 Likes

Now I get it much more, thank you for the explanation. In my brain, the term folder works much better, and the term folder path makes much more sense now.

I guess there will be a need of constant education for newcomers, so they can understand these new things. Maybe a small question mark icon attached to these words like pod, pod reference, etc. with a small pop-up explanation would help a lot. Like in-app education.

If you explain things in terms which I already know, I have much better understanding how to use the app and what can be done with it.

3 Likes

I think it is better, but also wonder if ā€˜index’ fits even better, as a ā€˜pod’ as I understand it is a list of references to files with info about them?

2 Likes

I think directory and folder are accurate and familiar terms for such a structure. Both are indexes containing metadata and references to content, but structured in a way that isn’t apparent from the term index.

An index is fairly accurate too, but less familiar wrt this kind of content.

I think @zettawatt chose Pod because he is alluding to Solid Pods, having decided to follow that approach and use RDF. RDF defines structures called Containers, which are similar to directories and folders (hierarchical), but less familiar.

4 Likes

I took a lot of inspiration from the Solid project, which like @happybeing said, was the original use of the term ā€˜pod’. I’ll look into what it will take to change everything from ā€˜pod’ to ā€˜folder’ and ā€˜pod reference’ to ā€˜link’.

5 Likes

Colony v1.0.0 released

  • Updated to latest autonomi crate
  • Updated dweb binary to v0.10.5
  • Random fixes and exception handling
  • Leveraging new ScratchPad::Fork() error handling

Binaries can be found here: Release Colony v1.0.0 Ā· zettawatt/colony Ā· GitHub

7 Likes

Colony v1.0.1 released

  • Enhancements for flakey network issues. Basically handles the case where if scratchpads/pointers can’t be found on the network, the sync operation will continue and rebuild the local cache to the best of its ability with the information it finds.

Latest binaries can be found here: Release Colony v1.0.1 Ā· zettawatt/colony Ā· GitHub

6 Likes

Files uploaded via colony app - are they immutable (and theoretically permanent) only?

1 Like

Yes, the uploaded files are all immutable and use the standard API so they can be downloaded by any tool. The metadata however is mutable. I plan to support private data in the future, but I haven’t implemented that functionality yet.

7 Likes

Colony v1.1.0 released

  • Spent the day working on UI enhancements, cleanup, and bug fixes. Overall the UI looks much cleaner than what it did before.
  • On the Downloads page, clicking on a downloaded file will open it with the default system application.
  • Added wallet balance info on the Wallets page
  • Saved the context on the Search page. When you navigate off of that page and back, it will redraw the old context.

Latest binaries can be found here: Release Colony v1.1.0 Ā· zettawatt/colony Ā· GitHub

14 Likes

Autonomi web pages are incredibly slow via colony/dweb, like 3 or 4 minutes to load the atlas page. Is this normal now? Does the network have some problems? I don’t remmeber it being this slow. My internet is fine.

1 Like

I think it is still recovering from the issues with the last version. It’s getting better, but still pretty slow.

I do think it encourages creative solutions to improve performance though.

1 Like

That’s better than it was a few days ago. Many didn’t load at all. The fact that they’re loading now is a good sign that the network is recovering. Pre-update, pages would load in about 30 seconds on my poor wifi connection at my house.

3 Likes

When we sort the long term issue with Pointers it to be much quicker.

Until then I’m afraid the versioned web will be slow. This doesn’t seem to be a priority I’m afraid.

3 Likes