Skip to content
Snippets Groups Projects
  1. Feb 03, 2025
    • STEVAN Antoine's avatar
      rework benchmarks: script and output format (dragoon/komodo!193) · ab0d9a8b
      STEVAN Antoine authored
      this is an attempt at making benchmarks easier to work with
      - `benchmarks run` will read benchmarks from NUON data and run them
      - `benchmarks plot` will plot benchmark results
      - the output format will be something 
        - a directory whose name is the hash of the CPU spec and the Komodo commit hash
        - contains `cpu.json` with the CPU info
        - contains `komodo.txt` with the Komodo commit hash
        - contains NDJSON result files
      
      > :bulb: **Note**
      >
      > results will typically be uploaded to https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results
      
      > :bulb: **Note**
      >
      > this MR goes alongside the [`komodo-benchmark-results@restart`](https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results/-/compare/main...restart) branch
      
      ## changelog
      - bump Nushell to 0.101.0
        - parallel `$in` => remove useless `let input = $in` when possible, e.g. still required when using the function's `$in` in a `for` loop
        - `group-by` changed => `group-by x --to-table` will now produce a table with columns `x` and `items` instead of `group` and `items` as in 0.95.0
      - add link to results repo
      - the Nushell benchmarks lib
        - rename `--force` to `--no-confirm (-y)`
        - add `--append` 
        - reject columns that GPLT will complain about, e.g. `$.points.k` for the FRI plots
        - add `--save` to the FRI plot
      - move the "field" and "curve group" benchmarks from `benchmarks/src/bin/operations/` to `benchmarks/src/bin/`
      - remove `benchmarks/params/fri.nu` because it's been uniformized with the other methods
      - rewrite the README
      - add main function to `benchmarks/` that runs the benchmarks from a NUON record specification
      - simplify the output of FRI run
      
      ## TODO
      - [x] fix "_atomic operations_" (done in 4f69a1d6)
      - [x] check that _plotting_ still works
      
      ## images
      
      ### Field
      ![complex_curve_group_operations](/uploads/57b36926cce041cf405a9b44f190b8b8/complex_curve_group_operations.png)
      ![complex_field_operations](/uploads/0747c85dbaff8980561aa9d922fcd5e7/complex_field_operations.png)
      ![simple_curve_group_operations](/uploads/974cf70fed68f8d8ac898d54be3f27be/simple_curve_group_operations.png)
      ![simple_field_operations](/uploads/a3d4d0dcdeb35d4c434eaa38fb51e7b5/simple_field_operations.png)
      
      ### Linear algebra
      ![linalg-inverse](/uploads/bc290ffa39459ce0f9bbd393b50b7b98/linalg-inverse.png)
      ![linalg-mul](/uploads/96d8c2a63ed48d6a0d3508b4a948153b/linalg-mul.png)
      ![linalg-transpose](/uploads/128e35eca91497d8aadb0130c05aeee3/linalg-transpose.png)
      
      ### FEC
      ![encoding](/uploads/405c4d3ef9ec5135ebdd7ce2e6c96bfe/encoding.png)
      ![decoding](/uploads/d793234d44e9fc6f34f0c2a9372863cd/decoding.png)
      ![recoding](/uploads/413021de997c86d45b1287fcfe7804c7/recoding.png)
      ![combined](/uploads/6d1c3ae6d3bf5547434ca29ae80b5536/combined.png)
      ![ratio](/uploads/6cebd7a0bcef57d1b1256bc3941c1b0a/ratio.png)
      
      ### ZK
      ![setup](/uploads/1feb169452aa3274dc924edf772c9a5b/setup.png)
      ![commit](/uploads/0a2775c2116ca7d3fa7b956b934d1565/commit.png)
      
      ### FRI
      ![commits_single](/uploads/4602725e551a025d42815183a61d11b2/commits_single.png)
      ![commits_single_normalized](/uploads/2ffebea940af1cbbe9bd64499343e0e9/commits_single_normalized.png)
      ![end_to_end](/uploads/46917abd2f5976dfa6d3039cc0ab2c0e/end_to_end.png)
      ![evaluating](/uploads/cf4dc496cd5144615bf5f9b06d27dccd/evaluating.png)
      ![proofs](/uploads/b0828bfa62c2226c8d63c9ecd387049f/proofs.png)
      ![proofs_normalized](/uploads/8fbf0d713884e2e147c62313a001f379/proofs_normalized.png)
      ![proving](/uploads/161539dc412330be1878cdda82c3d966/proving.png)
      ![verifying_single](/uploads/2df7e777481d7789478386f8e83e0783/verifying_single.png)
      ab0d9a8b
  2. Nov 26, 2024
  3. Aug 01, 2024
    • STEVAN Antoine's avatar
      refactor repo architecture (!161) · 3608e95a
      STEVAN Antoine authored
      ## changelog
      - `src/main.rs` has been moved to a new crate: `bins/saclin` which stands for **S**emi-**A**VID **CLI** in **N**ushell
      - dependencies of `komodo` have been fixed
      - Nushell and Rust tests have been split in the Makefile: by default, only Rust tests will run locally and Nushell tests and examples can be run manually if desired. The CI will still run everything.
      - the README has been updated
      - test images have been moved to `assets/`
      - the majority of the old `./nu-utils/` module have been moved to internals of `./benchmarks/` and imports have been fixed
      - `cargo.nu` has been moved to `./bins/` and a new `./bins/README.md` mentions it
      - `./bins/saclin/` has been created and should be a self-contained Rust crate + Nushell module
      3608e95a
  4. Jul 12, 2024
    • STEVAN Antoine's avatar
      refactor the Nushell modules (dragoon/komodo!158) · dad69f2c
      STEVAN Antoine authored
      this MR is two-fold
      - it restructures the two main Nushell modules so that they are easier to read and use
      - it improves the "run" and "plot" modules for the benchmarks
      
      ## changelog
      - `.nushell/` is now renamed to `nu-utils/`
      - `benchmarks/` is now a valid Nushell module which exports a bunch of modules
        - `benchmarks linalg`: measure and plot linear algebra operations
        - `benchmarks setup`: measure and plot trusted setup building
        - `benchmarks commit`: measure and plot crafting commitments
        - `benchmarks recoding`: measure and plot the recoding of shards
        - `benchmarks fec`: measure and plot FEC operations, such as encoding and recoding, and allow combining these results with the pure recoding ones
      - the submodules of `benchmarks` typically have a `run` and a `plot` command, whith the exception of `benchmarks fec` which has a `run` module and multiple "plot" commands in `benchmarks fec plot`
      - the "run" commands will create a random temp file by default and ask for confirmation otherwise if the output file already exists, unless `--force` is used
      - snippetds in `benchmarks/README.md` have been updated
      dad69f2c