Skip to content
Snippets Groups Projects
  1. Feb 03, 2025
    • STEVAN Antoine's avatar
      rework benchmarks: script and output format (dragoon/komodo!193) · ab0d9a8b
      STEVAN Antoine authored
      this is an attempt at making benchmarks easier to work with
      - `benchmarks run` will read benchmarks from NUON data and run them
      - `benchmarks plot` will plot benchmark results
      - the output format will be something 
        - a directory whose name is the hash of the CPU spec and the Komodo commit hash
        - contains `cpu.json` with the CPU info
        - contains `komodo.txt` with the Komodo commit hash
        - contains NDJSON result files
      
      > :bulb: **Note**
      >
      > results will typically be uploaded to https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results
      
      > :bulb: **Note**
      >
      > this MR goes alongside the [`komodo-benchmark-results@restart`](https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results/-/compare/main...restart) branch
      
      ## changelog
      - bump Nushell to 0.101.0
        - parallel `$in` => remove useless `let input = $in` when possible, e.g. still required when using the function's `$in` in a `for` loop
        - `group-by` changed => `group-by x --to-table` will now produce a table with columns `x` and `items` instead of `group` and `items` as in 0.95.0
      - add link to results repo
      - the Nushell benchmarks lib
        - rename `--force` to `--no-confirm (-y)`
        - add `--append` 
        - reject columns that GPLT will complain about, e.g. `$.points.k` for the FRI plots
        - add `--save` to the FRI plot
      - move the "field" and "curve group" benchmarks from `benchmarks/src/bin/operations/` to `benchmarks/src/bin/`
      - remove `benchmarks/params/fri.nu` because it's been uniformized with the other methods
      - rewrite the README
      - add main function to `benchmarks/` that runs the benchmarks from a NUON record specification
      - simplify the output of FRI run
      
      ## TODO
      - [x] fix "_atomic operations_" (done in 4f69a1d6)
      - [x] check that _plotting_ still works
      
      ## images
      
      ### Field
      ![complex_curve_group_operations](/uploads/57b36926cce041cf405a9b44f190b8b8/complex_curve_group_operations.png)
      ![complex_field_operations](/uploads/0747c85dbaff8980561aa9d922fcd5e7/complex_field_operations.png)
      ![simple_curve_group_operations](/uploads/974cf70fed68f8d8ac898d54be3f27be/simple_curve_group_operations.png)
      ![simple_field_operations](/uploads/a3d4d0dcdeb35d4c434eaa38fb51e7b5/simple_field_operations.png)
      
      ### Linear algebra
      ![linalg-inverse](/uploads/bc290ffa39459ce0f9bbd393b50b7b98/linalg-inverse.png)
      ![linalg-mul](/uploads/96d8c2a63ed48d6a0d3508b4a948153b/linalg-mul.png)
      ![linalg-transpose](/uploads/128e35eca91497d8aadb0130c05aeee3/linalg-transpose.png)
      
      ### FEC
      ![encoding](/uploads/405c4d3ef9ec5135ebdd7ce2e6c96bfe/encoding.png)
      ![decoding](/uploads/d793234d44e9fc6f34f0c2a9372863cd/decoding.png)
      ![recoding](/uploads/413021de997c86d45b1287fcfe7804c7/recoding.png)
      ![combined](/uploads/6d1c3ae6d3bf5547434ca29ae80b5536/combined.png)
      ![ratio](/uploads/6cebd7a0bcef57d1b1256bc3941c1b0a/ratio.png)
      
      ### ZK
      ![setup](/uploads/1feb169452aa3274dc924edf772c9a5b/setup.png)
      ![commit](/uploads/0a2775c2116ca7d3fa7b956b934d1565/commit.png)
      
      ### FRI
      ![commits_single](/uploads/4602725e551a025d42815183a61d11b2/commits_single.png)
      ![commits_single_normalized](/uploads/2ffebea940af1cbbe9bd64499343e0e9/commits_single_normalized.png)
      ![end_to_end](/uploads/46917abd2f5976dfa6d3039cc0ab2c0e/end_to_end.png)
      ![evaluating](/uploads/cf4dc496cd5144615bf5f9b06d27dccd/evaluating.png)
      ![proofs](/uploads/b0828bfa62c2226c8d63c9ecd387049f/proofs.png)
      ![proofs_normalized](/uploads/8fbf0d713884e2e147c62313a001f379/proofs_normalized.png)
      ![proving](/uploads/161539dc412330be1878cdda82c3d966/proving.png)
      ![verifying_single](/uploads/2df7e777481d7789478386f8e83e0783/verifying_single.png)
      ab0d9a8b
  2. Nov 26, 2024
  3. Aug 01, 2024
    • STEVAN Antoine's avatar
      refactor repo architecture (dragoon/komodo!161) · 3608e95a
      STEVAN Antoine authored
      ## changelog
      - `src/main.rs` has been moved to a new crate: `bins/saclin` which stands for **S**emi-**A**VID **CLI** in **N**ushell
      - dependencies of `komodo` have been fixed
      - Nushell and Rust tests have been split in the Makefile: by default, only Rust tests will run locally and Nushell tests and examples can be run manually if desired. The CI will still run everything.
      - the README has been updated
      - test images have been moved to `assets/`
      - the majority of the old `./nu-utils/` module have been moved to internals of `./benchmarks/` and imports have been fixed
      - `cargo.nu` has been moved to `./bins/` and a new `./bins/README.md` mentions it
      - `./bins/saclin/` has been created and should be a self-contained Rust crate + Nushell module
      3608e95a
  4. Jul 12, 2024
    • STEVAN Antoine's avatar
      refactor the Nushell modules (dragoon/komodo!158) · dad69f2c
      STEVAN Antoine authored
      this MR is two-fold
      - it restructures the two main Nushell modules so that they are easier to read and use
      - it improves the "run" and "plot" modules for the benchmarks
      
      ## changelog
      - `.nushell/` is now renamed to `nu-utils/`
      - `benchmarks/` is now a valid Nushell module which exports a bunch of modules
        - `benchmarks linalg`: measure and plot linear algebra operations
        - `benchmarks setup`: measure and plot trusted setup building
        - `benchmarks commit`: measure and plot crafting commitments
        - `benchmarks recoding`: measure and plot the recoding of shards
        - `benchmarks fec`: measure and plot FEC operations, such as encoding and recoding, and allow combining these results with the pure recoding ones
      - the submodules of `benchmarks` typically have a `run` and a `plot` command, whith the exception of `benchmarks fec` which has a `run` module and multiple "plot" commands in `benchmarks fec plot`
      - the "run" commands will create a random temp file by default and ask for confirmation otherwise if the output file already exists, unless `--force` is used
      - snippetds in `benchmarks/README.md` have been updated
      dad69f2c
    • STEVAN Antoine's avatar
      fix colors and markers for elliptic curves in plots (dragoon/komodo!157) · 8853e01b
      STEVAN Antoine authored
      ## results
      ![setup](/uploads/1f3a383689d3c8fd5a3f1331deabdec0/setup.png)
      ![commit](/uploads/bcf3bfdd991725abb97cbe4f8890840b/commit.png)
      ![inverse](/uploads/35316c887c6b2347baa86c4db119f762/inverse.png)
      ![mul](/uploads/262c27e99d7f2b5c8fb45761607a8717/mul.png)
      ![transpose](/uploads/2378460cb687d0a9320a24a840ed2eef/transpose.png)
      8853e01b
  5. Jul 05, 2024
    • STEVAN Antoine's avatar
      improve setup and commit plots (dragoon/komodo!152) · 4bd3943c
      STEVAN Antoine authored
      - show the log of the degree for high values
      - don't show the "time" Y label because the units are in the times
        values
      - don't rotate the X tick labels
      
      ## results
      ![setup](/uploads/75dfd2033801a88e446e0ce8cae24167/setup.png)
      ![commit](/uploads/cca5bdfabb8f4242d21bf68079a5566a/commit.png)
      4bd3943c
  6. May 29, 2024
    • STEVAN Antoine's avatar
      make `.nushell/` a proper module (dragoon/komodo!123) · 45540a7c
      STEVAN Antoine authored
      this MR turns `./.nushell/` into a directory module by
      - adding `mod.nu`
      - exporting all the modules
      
      all uses of `.nushell/` have been fixed to not mention `.nu` internal modules anymore.
      
      > :bulb: **Note**  
      > the `.nushell venv` module has been removed because, when the `$venv.VENV` activation script is not there, Nushell can't parse the whole `.nushell` module, which is very annoying to have to rely of the state of the external filesystem to be able to simply parse a module...
      45540a7c
  7. May 28, 2024
    • STEVAN Antoine's avatar
      split `examples/` into `benchmarks/` and `bins/` (dragoon/komodo!117) · bb626120
      STEVAN Antoine authored
      ## new structure for the repository
      - benchmarks are in `./benchmarks/` and can be run with either `cargo run --package benchmarks --bin <bench>` or the commands in `./benchmarks/README.md`
      ```
      ├── Cargo.toml
      ├── README.md
      └── src
          └── bin
              ├── commit.rs
              ├── fec.rs
              ├── linalg.rs
              ├── operations
              │   ├── curve_group.rs
              │   └── field.rs
              ├── recoding.rs
              ├── setup.rs
              └── setup_size.rs
      ```
      
      - examples are now in `./bins/` as standalone binaries and can be run either with `cargo run --package <pkg>` or with the help of the `cargo bin` command from `.nushell/cargo.nu`
      ```
      ├── curves
      │   ├── Cargo.toml
      │   ├── README.md
      │   └── src
      │       └── main.rs
      ├── inbreeding
      │   ├── build.nu
      │   ├── Cargo.toml
      │   ├──...
      bb626120
  8. May 23, 2024
    • STEVAN Antoine's avatar
      dont' hardcode curves in the benchmarks (dragoon/komodo!99) · d7e84161
      STEVAN Antoine authored
      up until now, elliptic curves have been hardcoded in the benchmarks, forcing to run them on all supported curves...
      
      this MR makes it possible to use only a subset of curves.
      
      > :bulb: **Note**  
      > when running the same commands from !104, minus the "inbreeding" ones which are not affected by this MR, the time goes from 12min 33sec to 4min 28sec :tada: 
      
      
      ## TODO
      - [x] setup
      - [x] commit
      - [x] recoding
      - [x] fec
      - [ ] linalg
      - [ ] setup size
      - [ ] field operations
      - [ ] group operations
      
      > :bulb: **Note**  
      > because all the unticked bullet points above are far from critical to the paper and do require to measure all curves, i won't change these for now
      d7e84161
    • STEVAN Antoine's avatar
      Refactor plot commands (dragoon/komodo!104) · 0f43be24
      STEVAN Antoine authored
      this MR moves run and plot commands from `examples/benches/README.md` to
      - `scripts/setup/`: `run.nu` and `plot.nu`
      - `scripts/commit/`: `run.nu` and `plot.nu`
      - `scripts/recoding/`: `run.nu` and `plot.nu`
      - `scripts/fec/`: `run.nu` and `plot.nu`
      - `scripts/inbreeding/`: `build.nu`, `run.nu` and `plot.nu`
      
      to generate all the figures at once
      ```bash
      use scripts/setup/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/setup.ndjson
      use ./scripts/setup/plot.nu; plot data/setup.ndjson --save ~/setup.pdf
      
      use scripts/commit/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/commit.ndjson
      use ./scripts/commit/plot.nu; plot data/commit.ndjson --save ~/commit.pdf
      
      use scripts/recoding/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/recoding.ndjson
      use ./scripts/recoding/plot.nu; plot data/recoding.ndjson --save ~/recoding.pdf
      
      use scripts/fec/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/fec.ndjson
      use ./scripts/fec/plot.nu; plot encoding data/fec.ndjson --save ~/encoding.pdf
      use ./scripts/fec/plot.nu; plot decoding data/fec.ndjson --save ~/decoding.pdf
      use ./scripts/fec/plot.nu; plot e2e data/fec.ndjson --save ~/e2e.pdf
      
      use ./scripts/fec/plot.nu; plot combined data/fec.ndjson --recoding data/recoding.ndjson --save ~/comparison.pdf
      use ./scripts/fec/plot.nu; plot ratio data/fec.ndjson --recoding data/recoding.ndjson --save ~/ratio.pdf
      
      ./scripts/inbreeding/build.nu
      ./scripts/inbreeding/run.nu --output data/inbreeding.nuon
      ./scripts/inbreeding/plot.nu data/inbreeding.nuon --save ~/inbreeding.pdf
      ```
      
      > :bulb: **Note**  
      > this took around 27min 18sec in total on my machine with 14min 45sec for the inbreeding section only and 12min 33sec for the rest
      0f43be24