- Feb 03, 2025
-
-
STEVAN Antoine authored
this is an attempt at making benchmarks easier to work with - `benchmarks run` will read benchmarks from NUON data and run them - `benchmarks plot` will plot benchmark results - the output format will be something - a directory whose name is the hash of the CPU spec and the Komodo commit hash - contains `cpu.json` with the CPU info - contains `komodo.txt` with the Komodo commit hash - contains NDJSON result files >
**Note** > > results will typically be uploaded to https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results > **Note** > > this MR goes alongside the [`komodo-benchmark-results@restart`](https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results/-/compare/main...restart) branch ## changelog - bump Nushell to 0.101.0 - parallel `$in` => remove useless `let input = $in` when possible, e.g. still required when using the function's `$in` in a `for` loop - `group-by` changed => `group-by x --to-table` will now produce a table with columns `x` and `items` instead of `group` and `items` as in 0.95.0 - add link to results repo - the Nushell benchmarks lib - rename `--force` to `--no-confirm (-y)` - add `--append` - reject columns that GPLT will complain about, e.g. `$.points.k` for the FRI plots - add `--save` to the FRI plot - move the "field" and "curve group" benchmarks from `benchmarks/src/bin/operations/` to `benchmarks/src/bin/` - remove `benchmarks/params/fri.nu` because it's been uniformized with the other methods - rewrite the README - add main function to `benchmarks/` that runs the benchmarks from a NUON record specification - simplify the output of FRI run ## TODO - [x] fix "_atomic operations_" (done in 4f69a1d6) - [x] check that _plotting_ still works ## images ### Field     ### Linear algebra    ### FEC      ### ZK   ### FRI        
-
- Nov 26, 2024
-
-
STEVAN Antoine authored
this replaces the "_nothing to do_" messages from Nushell commands with more explicite errors.
-
- Aug 01, 2024
-
-
STEVAN Antoine authored
## changelog - `src/main.rs` has been moved to a new crate: `bins/saclin` which stands for **S**emi-**A**VID **CLI** in **N**ushell - dependencies of `komodo` have been fixed - Nushell and Rust tests have been split in the Makefile: by default, only Rust tests will run locally and Nushell tests and examples can be run manually if desired. The CI will still run everything. - the README has been updated - test images have been moved to `assets/` - the majority of the old `./nu-utils/` module have been moved to internals of `./benchmarks/` and imports have been fixed - `cargo.nu` has been moved to `./bins/` and a new `./bins/README.md` mentions it - `./bins/saclin/` has been created and should be a self-contained Rust crate + Nushell module
-
- Jul 12, 2024
-
-
STEVAN Antoine authored
this MR is two-fold - it restructures the two main Nushell modules so that they are easier to read and use - it improves the "run" and "plot" modules for the benchmarks ## changelog - `.nushell/` is now renamed to `nu-utils/` - `benchmarks/` is now a valid Nushell module which exports a bunch of modules - `benchmarks linalg`: measure and plot linear algebra operations - `benchmarks setup`: measure and plot trusted setup building - `benchmarks commit`: measure and plot crafting commitments - `benchmarks recoding`: measure and plot the recoding of shards - `benchmarks fec`: measure and plot FEC operations, such as encoding and recoding, and allow combining these results with the pure recoding ones - the submodules of `benchmarks` typically have a `run` and a `plot` command, whith the exception of `benchmarks fec` which has a `run` module and multiple "plot" commands in `benchmarks fec plot` - the "run" commands will create a random temp file by default and ask for confirmation otherwise if the output file already exists, unless `--force` is used - snippetds in `benchmarks/README.md` have been updated
-
STEVAN Antoine authored
## results     
-
- Jul 05, 2024
-
-
STEVAN Antoine authored
- show the log of the degree for high values - don't show the "time" Y label because the units are in the times values - don't rotate the X tick labels ## results  
-
- May 29, 2024
-
-
STEVAN Antoine authored
this MR turns `./.nushell/` into a directory module by - adding `mod.nu` - exporting all the modules all uses of `.nushell/` have been fixed to not mention `.nu` internal modules anymore. >
**Note** > the `.nushell venv` module has been removed because, when the `$venv.VENV` activation script is not there, Nushell can't parse the whole `.nushell` module, which is very annoying to have to rely of the state of the external filesystem to be able to simply parse a module...
-
- May 28, 2024
-
-
STEVAN Antoine authored
## new structure for the repository - benchmarks are in `./benchmarks/` and can be run with either `cargo run --package benchmarks --bin <bench>` or the commands in `./benchmarks/README.md` ``` ├── Cargo.toml ├── README.md └── src └── bin ├── commit.rs ├── fec.rs ├── linalg.rs ├── operations │ ├── curve_group.rs │ └── field.rs ├── recoding.rs ├── setup.rs └── setup_size.rs ``` - examples are now in `./bins/` as standalone binaries and can be run either with `cargo run --package <pkg>` or with the help of the `cargo bin` command from `.nushell/cargo.nu` ``` ├── curves │ ├── Cargo.toml │ ├── README.md │ └── src │ └── main.rs ├── inbreeding │ ├── build.nu │ ├── Cargo.toml │ ├──...
-
- May 23, 2024
-
-
STEVAN Antoine authored
up until now, elliptic curves have been hardcoded in the benchmarks, forcing to run them on all supported curves... this MR makes it possible to use only a subset of curves. >
**Note** > when running the same commands from !104, minus the "inbreeding" ones which are not affected by this MR, the time goes from 12min 33sec to 4min 28sec ## TODO - [x] setup - [x] commit - [x] recoding - [x] fec - [ ] linalg - [ ] setup size - [ ] field operations - [ ] group operations > **Note** > because all the unticked bullet points above are far from critical to the paper and do require to measure all curves, i won't change these for now -
STEVAN Antoine authored
this MR moves run and plot commands from `examples/benches/README.md` to - `scripts/setup/`: `run.nu` and `plot.nu` - `scripts/commit/`: `run.nu` and `plot.nu` - `scripts/recoding/`: `run.nu` and `plot.nu` - `scripts/fec/`: `run.nu` and `plot.nu` - `scripts/inbreeding/`: `build.nu`, `run.nu` and `plot.nu` to generate all the figures at once ```bash use scripts/setup/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/setup.ndjson use ./scripts/setup/plot.nu; plot data/setup.ndjson --save ~/setup.pdf use scripts/commit/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/commit.ndjson use ./scripts/commit/plot.nu; plot data/commit.ndjson --save ~/commit.pdf use scripts/recoding/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/recoding.ndjson use ./scripts/recoding/plot.nu; plot data/recoding.ndjson --save ~/recoding.pdf use scripts/fec/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/fec.ndjson use ./scripts/fec/plot.nu; plot encoding data/fec.ndjson --save ~/encoding.pdf use ./scripts/fec/plot.nu; plot decoding data/fec.ndjson --save ~/decoding.pdf use ./scripts/fec/plot.nu; plot e2e data/fec.ndjson --save ~/e2e.pdf use ./scripts/fec/plot.nu; plot combined data/fec.ndjson --recoding data/recoding.ndjson --save ~/comparison.pdf use ./scripts/fec/plot.nu; plot ratio data/fec.ndjson --recoding data/recoding.ndjson --save ~/ratio.pdf ./scripts/inbreeding/build.nu ./scripts/inbreeding/run.nu --output data/inbreeding.nuon ./scripts/inbreeding/plot.nu data/inbreeding.nuon --save ~/inbreeding.pdf ``` >
**Note** > this took around 27min 18sec in total on my machine with 14min 45sec for the inbreeding section only and 12min 33sec for the rest
-