Tut, tut, I see you aren’t following the github closely enough and shall notify the re-education department.
3 Likes
Guilty as charged. I will try to do better.
To be fair though it is a full time job to stay on top of it!
5 Likes
info!("New peer added to routing table: {peer:?}, now we have #{} connected peers", self.connected_peers);
@roland
maidsafe:main ← RolandSherwin:enable_connected_peers_log_vdash
opened 06:03PM - 01 Nov 23 UTC
- used by vdash
## Description
### Summary generated by Reviewpad on 01 Nov… 23 18:03 UTC
This pull request adds logging of the connected peers when a new peer is added to the routing table in the `chore(log)` module. The patch includes changes to the `event.rs` file, where the count of connected peers is cached and logged along with the new peer information.
11 Likes
roland
November 7, 2023, 4:35am
5356
We’ve also added a log marker for it, PeersInRoutingTable(usize). You can maybe grep for this instead, as this should be less prone to accidental deletion.
6 Likes
I recon it must be about time for a testnet
9 Likes
An interesting issue, defines a new binary ‘node-manager’ and commands which will be used to manage nodes as services, as an alternative to running then manually.
opened 08:46PM - 08 Nov 23 UTC
- [ ] Provide an `sn_rpc_client` crate
- [ ] Node manager: `start` control comm… and
- [ ] Node manager: `stop` control command
- [ ] Node manager: `status` command
- [ ] Node manager: `uninstall` command
- [ ] Node manager: release process
- [ ] Node manager: `upgrade` command
- [ ] Node manager: `run` command
- [ ] Node manager: `kill` command
- [ ] Remove node control from the RPC protocol
- [ ] Refactor churn tests to use the node manager for node control
- [ ] Use signals in `safenode` for graceful shutdown
- [ ] Provide `node-manager` command in `safeup`
- [ ] Testnet Deploy: refactor to deploy nodes with node manager
- [ ] Testnet Deploy: trigger upgrades for existing testnets
- [ ] Testnet Deploy: GHA workflow for running testnet upgrades
---
### Provide an `sn_rpc_client` crate
We would like the node manager to use the RPC client for its `status` command, and possibly other things. We can avoid repeating the code that's defined in the `safenode_rpc_client` binary by extracting out a new crate which defines a client abstraction that wraps the Protobuf generated code and returns data structures that are more convenient to work with.
This is a prerequisite for a `status` command in the node manager.
---
### Node manager: `start` control command
The node manager should implement a command that starts all installed node services, with additional capability to start individual services, perhaps based on peer ID.
This command would benefit from having the `sn_rpc_client` crate available because we can use it to determine the peer ID of the launched service, which can be stored as part of the node registry that will be maintained by the node manager. This registry will allow control of individual services based on the peer ID.
The end to end tests should exercise this functionality.
This is a prerequisite for an `upgrade` command.
---
### Node manager: `stop` control command
The node manager should provide a command that stops all installed node services, with additional capability to stop individual services, perhaps based on peer ID.
The end to end tests should exercise this functionality.
This is a prerequisite for an `upgrade` command.
---
### Node manager: `status` command
The node manager should provide a command that shows the status of all services, with the RPC client providing the ability to obtain more information than just 'running/stopped'.
---
### Node manager: `uninstall` command
The node manager should provide a command that uninstalls all node services, with additional capability to uninstall individual services, perhaps based on peer ID.
---
### Node manager: release process
Provide a release process for the node manager for releasing the binaries and publishing the crate. We need this for installation of the node manager via `safeup`.
---
### Node manager: `upgrade` command
The node manager should provide a command that upgrades all installed/running nodes to the latest version, with additional capability to upgrade indvidual services, perhaps based on peer ID.
This process probably looks something like this:
* Obtain the latest version of `safenode`
* Download the `safenode` binary at the latest version
* For each installed/running service:
- Gracefully stop the running service
- Overwrite the `safenode` binary
- Start the service with the new binary
If we want to have reliable upgrades, we'll probably need to have graceful shutdown implemented first.
---
### Node manager: `run` command
The node manager should provide a command that runs `safenode` processes. This would run the nodes without installing them as services, to facilitate easily running local networks for testing or other purposes. The idea would be for this to supplant the `testnet` binary for running local networks.
---
### Node manager: `kill` command
The node manager should provide a command that kills `safenode` processes. This does not apply to nodes that are installed as services. The idea would be for this to supplant the `testnet` binary for running local networks.
---
### Remove node control from the RPC protocol and `safenode`
The node manager is taking the approach of using the OS service infrastructure to run `safenode`. If we encourage the use of the node manager, the RPC protocol should not deal with node control. If the OS is managing the service, using RPC for node control could lead to undefined behaviour, especially on Windows.
It would be preferable for RPC to be informational, or deal within the network domain, and let the OS deal with node control.
Once node control is removed from RPC, it would also need to be removed from `safenode`.
---
### Refactor churn tests to use the node manager for node control
The churn tests currently use the RCP client to restart a node. If we removed node control from `safenode`, the tests would need to be refactored to use the node manager.
This is probably dependent on the node manager having `run` and `kill` commands for running local networks.
---
### Use signals in `safenode` for graceful shutdown
With node control removed, we would have the opportunity to implement graceful shutdown by responding to operating system signals.
---
### Provide `node-manager` command in `safeup`
We will need a mechanism for installing the node manager, particularly as it will be necessary to distribute it with additional dependencies, like WinSW on Windows. We can implement a command in `safeup` for this.
---
### Testnet Deploy: refactor to deploy nodes with node manager
When we have working `install/start/stop` commands for the node manager, we can upgrade `testnet-deploy` to use this mechanism for installing the node on cloud VMs.
---
### Testnet Deploy: trigger upgrades for existing testnets
When we have a working `upgrade` command for the node manager, we can use `testnet-deploy` to trigger an upgrade process for any existing testnet that was deployed using `testnet-deploy`. This should serve as a deploy process for any nodes that have been provided by Maidsafe as part of an alpha/beta network.
---
### Testnet Deploy: GHA workflow for running testnet upgrades
Provide a GHA workflow for launching testnet upgrade processes.
8 Likes
Toivo
November 16, 2023, 8:16am
5361
I was just about to post that! Maybe they reduce the messaging overhead as nodes commit to a price and need to store? No constant re-checking with every failed chunk?
4 Likes
joshuef
November 16, 2023, 3:17pm
5362
They do that, but they’re not back in the prev sense. See the updoot for more deets
7 Likes
Has anyone tried this?
Looks like a awesome bit of kit @chriso
12 Likes
chriso
December 5, 2023, 11:04am
5364
Hey Josh, thanks. It’s still very much under active development and subject to change, literally on a daily basis. The short-term aim is to use it for upgrading nodes, which would enable us to push out changes but keep our testnets running.
Of course if you decide to use it, any feedback would be welcome. I might recommend just running it on VMs for the moment. It installs services on the machines where it runs, and how it does so can be subject to change, so it could potentially leave your machine in a state where you would need to manually clean things up.
14 Likes
That is a game changer.
Thanks for the tip on using a VM I’ll definitely be kicking the wheels.
16 Likes
Ledger hardware support being worked on:
maidsafe:main ← bochaco:feat-cli-ledger-integration
opened 07:42PM - 07 Dec 23 UTC
## Description
### Summary generated by Reviewpad on 07 Dec 23 19:42 UTC
Thi… s pull request contains several changes across multiple files.
In the `sn_cli/src/subcommands/wallet.rs` file, the changes include importing the `ledger_get_addr` function, modifying the `WalletCmds` enum to include a new variant `Address` with a boolean field `ledger`, adding a command-line argument to the `Address` variant, and updating the `wallet_cmds_without_client` function to handle the new `Address` variant.
In the `main.rs` file, the diff modifies the code related to the wallet commands, changing the pattern matching syntax and adding a pattern match for `WalletCmds::Address` in an if statement.
The `Cargo.lock` file has been updated with various changes, including additions and updates to package dependencies such as "coins-ledger", "memoffset", "hidapi-rusb", "libusb1-sys", "pkg-config", "rusb", "serde_json", and "vcpkg".
A new module called `ledger` has been added to the `subcommands/mod.rs` file.
A new file `ledger.rs` has been added in the `sn_cli/src/subcommands` directory, which includes license information, constant definitions, and the implementation of functions related to interacting with a Ledger hardware wallet.
In the `Cargo.toml` file, changes include adding the `coins-ledger` dependency, modifying the `reqwest` dependency, and adding the `serde_json` dependency.
Please review these changes and ensure they are appropriate for the project. Let me know if you need further assistance with this code review.
14 Likes
You can tell things are hotting up when the forum is buzzing and people are too busy to post here for twelve days!
Meanwhile, here’s a boring testing PR being used to sneak out the following task at the bottom of a to-do list :
client/node API documentation and testing
opened 09:00AM - 19 Dec 23 UTC
enhancement
A list of desirable testing:
- [ ] Launch a WAN network for nightly reviews.
…
- [ ] automate large uploads and other smoketesting for testnets
- [ ] Normal client tests against wan
- [ ] Tests to check node dispersion and price setting is as expected?
- [ ] CLI output tests
- [ ] Compatibility tests (#1065 )
- [ ] client/node API documentation and testing
- [ ]
12 Likes
Will be next update huge?
1 Like
Testnets seem remarkably stable these days so we might see more stress testing, such as with churn, but there’s already a new issue that I think needs attention.
At the moment pricing seems to cause great disparities in node earnings, which will be bad if it causes lots of churn while people restart low earning nodes to boost earnings.
So I think we’re going to need quite a few tests to figure out how to price storage in a way that manages supply without causing unwanted inefficiencies such as churn caused by node ID fishing.
10 Likes
I wonder if the issue with big uploads is likely to be addressed. So far I don’t think anyone has been able to upload that 36gb LLM if I remember correctly?
What are the main issues that the team are aiming to sort in the next tests?
Are they;
Pricing model / tokenomics to promote more even earning between nodes
Big upload issues
Are there any other big issues?
Certainly seems like the tests are doing great and seems like not far to go, though the last few issues can be the toughest! Looking forward to the next tests whatever they may bring
9 Likes
Correct. I have it on a cloud snapshot, ready to try again on the next testnet. My max upload from home is ~4Mb/min so I need to run this from the cloud to see what the real bottleneck is.
6 Likes
I will try again on the next test as well. I have fairly fast upload speeds on my connection, but I expect that isn’t the bottleneck with uploads to current testnets.
5 Likes