[No code has been written. I’m revealing my plan early with the hope of maximizing Verifi’s usefulness through collective input. Also because my dev machine is currently occupied with the Node Rewards Program. When the Node Rewards Program ends in December, this plan will have been refined, I’ll begin on the code, and should have an early build in a git repo ready for cloning after a week.]
Background
For a data+ network to succeed, it must be trustworthy. There are several aspects to trustworthiness, including security, integrity, and performance. Verifi is focused on integrity.
Verifi defines integrity as: uploaded files aren’t lost or corrupt. Although that can currently be accomplished using the autonomi
CLI app, it’s a manual process that must be repeated for each upload. Verifi automates the process, enabling on-going verification of any number of uploads.
Goals
Automatic verification that uploaded files aren’t lost or corrupt benefits the network in several ways:
- Users, whether individuals or corporations, have peace-of-mind that the network has integrity, and that should an error occur at any time, they will know and can re-upload. (and hopefully report the error to developers)
- Developers have additional tooling for validating design and code.
- Testers have additional tooling for verifying and load-testing the network.
Organization
Verifi is written in open-source Rust and structured as both a CLI app and library. Although most usage of Verifi is through its CLI app, separating the core functionality into a library makes it more useful to other projects in the network’s ecosystem.
Concepts
- Notification - An alert informing a user of a verification error.
- Upload item - An original file, its md5 hash, its upload address, and optionally a comment.
- Upload list - One or more upload items.
Features
- Command-line arguments:
- Either an upload list or path(s) to files containing an upload list.
- Optional repetition delay, (
continuous
, an amount of time, oronce
) defaulting to running once and exiting.
- Output notifications to the command-line.
Stages
Stage 1
- Integrate with the network by directly invoking the
autonomi
CLI app.
Stage 2
- Add an API network integration method.
- Add optional concurrent verification.
Stage 3
- Add additional notification methods, such as playing a sound or sending an email.
- Add OpenTelemetry support for diagnostics integration.
- Engage FYEO (@TammyFYEO, @Brian_FYEO) in a security audit of the codebase.
Notes
Verifi is hopefully a contribution to:
Moreover, they’re likely validating it too with regular downloads. - @Traktion
But I suspect that the up/downloading, especially the downloading is nowhere near the amounts we would see in the actual network. - @Toivo
What we are looking for is the chance to help where we can and taking part in structured testing directed by Autonomi is exactly what we should be doing…another team download and verify them. Tens of thousands of times. - @Southside
imo the Emerging Autonomi Community needs its own Test work to do development on… - @rreive
large uploads fail a lot of the time with being unable to get quotes and also the failed verification check. but I am guessing… - @aatonnomicc
Also tagging @happybeing, @neo, I hope you don’t mind.
Feedback
How does Verifi, and this plan, suck? How could they be better?