Skip to content
Snippets Groups Projects
  1. Apr 10, 2025
  2. Apr 08, 2025
  3. Apr 03, 2025
    • STEVAN Antoine's avatar
      use `gitlab.isae-supaero.fr:a.stevan/nob.rs` to build (!202) · 4962757c
      STEVAN Antoine authored
      that's an attempt at using Rust to build itself.
      
      this is using [`gitlab.isae-supaero.fr:a.stevan/nob.rs@e4b03cdd`](a.stevan/nob.rs@e4b03cdd).
      
      > :bulb: **Note**
      >
      > to be honest, this is not a 100% replacement of the `Makefile`...
      >
      > `make.rs` does a lot more and provides a full CLI with easy-to-use options, e.g. instead of `make fmt` and `make fmt-check`, we now have `./make.rs fmt` and `./make.rs fmt --check`
      >
      > (see the API below)
      
      ## the API
      ```
      Usage: make [OPTIONS] [COMMAND]
      
      Commands:
        fmt      Formats the code
        check    Checks the code
        clippy   Runs Clippy
        test     Runs the tests
        version  Shows the version of all the tools used,
        doc      Builds the documentation
        help     Print this message or the help of the given subcommand(s)
      
      Options:
        -h, --help      Print help
        -V, --version   Print version
      ```
      ```
      Usage: make fmt [OPTIONS]
      
      Options:
        -c, --check  Only checks instead of really formatting
      ```
      ```
      Usage: make check
      ```
      ```
      Usage: make clippy
      ```
      ```
      Usage: make test [OPTIONS]
      
      Options:
        -v, --verbose   Be extra verbose with the output of the tests
        -e, --examples  Run the examples instead of regular tests
      ```
      ```
      Usage: make version
      ```
      ```
      Usage: make doc [OPTIONS]
      
      Options:
        -o, --open      Open the documentation in the browser
        -p, --private   Document private items
        -f, --features  Document all features
      ```
      
      ## running the pipeline in the GitHub mirror
      ```bash
      const GH_API_OPTIONS = [
          -H "Accept: application/vnd.github+json"
          -H "X-GitHub-Api-Version: 2022-11-28"
      ]
      let res = gh api ...$GH_API_OPTIONS /repos/dragoon-rs/komodo/actions/runs | from json
      ```
      ```bash
      let runs = $res.workflow_runs
          | where head_branch == "use-nob-to-build"
          | select id head_sha status conclusion run_started_at
          | into datetime run_started_at
          | sort-by run_started_at
      ```
      ```bash
      $runs
          | update id { $"[`($in)`]\(https://github.com/($GITHUB_MIRROR)/actions/runs/($in)\)" }
          | update run_started_at { format date "%Y-%m-%dT%H:%M:%S" }
          | to md --pretty
      ```
      | id                                                                             | head_sha                                 | status    | conclusion | run_started_at      |
      | ------------------------------------------------------------------------------ | ---------------------------------------- | --------- | ---------- | ------------------- |
      | [`14237650542`](https://github.com/dragoon-rs/komodo/actions/runs/14237650542) | d67f1cfd | completed | success    | 2025-04-03T07:44:14 |
      | [`14237741570`](https://github.com/dragoon-rs/komodo/actions/runs/14237741570) | 9ef598a1 | completed | success    | 2025-04-03T07:49:40 |
      | [`14238086977`](https://github.com/dragoon-rs/komodo/actions/runs/14238086977) | 0a79edf3 | completed | success    | 2025-04-03T08:09:13 |
      | [`14238175174`](https://github.com/dragoon-rs/komodo/actions/runs/14238175174) | a84b2b12 | completed | success    | 2025-04-03T08:13:52 |
      | [`14239395984`](https://github.com/dragoon-rs/komodo/actions/runs/14239395984) | 8594c9bf | completed | success    | 2025-04-03T09:16:00 |
      4962757c
  4. Apr 02, 2025
  5. Feb 03, 2025
    • STEVAN Antoine's avatar
      rework benchmarks: script and output format (!193) · ab0d9a8b
      STEVAN Antoine authored
      this is an attempt at making benchmarks easier to work with
      - `benchmarks run` will read benchmarks from NUON data and run them
      - `benchmarks plot` will plot benchmark results
      - the output format will be something 
        - a directory whose name is the hash of the CPU spec and the Komodo commit hash
        - contains `cpu.json` with the CPU info
        - contains `komodo.txt` with the Komodo commit hash
        - contains NDJSON result files
      
      > :bulb: **Note**
      >
      > results will typically be uploaded to https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results
      
      > :bulb: **Note**
      >
      > this MR goes alongside the [`komodo-benchmark-results@restart`](https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results/-/compare/main...restart) branch
      
      ## changelog
      - bump Nushell to 0.101.0
        - parallel `$in` => remove useless `let input = $in` when possible, e.g. still required when using the function's `$in` in a `for` loop
        - `group-by` changed => `group-by x --to-table` will now produce a table with columns `x` and `items` instead of `group` and `items` as in 0.95.0
      - add link to results repo
      - the Nushell benchmarks lib
        - rename `--force` to `--no-confirm (-y)`
        - add `--append` 
        - reject columns that GPLT will complain about, e.g. `$.points.k` for the FRI plots
        - add `--save` to the FRI plot
      - move the "field" and "curve group" benchmarks from `benchmarks/src/bin/operations/` to `benchmarks/src/bin/`
      - remove `benchmarks/params/fri.nu` because it's been uniformized with the other methods
      - rewrite the README
      - add main function to `benchmarks/` that runs the benchmarks from a NUON record specification
      - simplify the output of FRI run
      
      ## TODO
      - [x] fix "_atomic operations_" (done in 4f69a1d6)
      - [x] check that _plotting_ still works
      
      ## images
      
      ### Field
      ![complex_curve_group_operations](/uploads/57b36926cce041cf405a9b44f190b8b8/complex_curve_group_operations.png)
      ![complex_field_operations](/uploads/0747c85dbaff8980561aa9d922fcd5e7/complex_field_operations.png)
      ![simple_curve_group_operations](/uploads/974cf70fed68f8d8ac898d54be3f27be/simple_curve_group_operations.png)
      ![simple_field_operations](/uploads/a3d4d0dcdeb35d4c434eaa38fb51e7b5/simple_field_operations.png)
      
      ### Linear algebra
      ![linalg-inverse](/uploads/bc290ffa39459ce0f9bbd393b50b7b98/linalg-inverse.png)
      ![linalg-mul](/uploads/96d8c2a63ed48d6a0d3508b4a948153b/linalg-mul.png)
      ![linalg-transpose](/uploads/128e35eca91497d8aadb0130c05aeee3/linalg-transpose.png)
      
      ### FEC
      ![encoding](/uploads/405c4d3ef9ec5135ebdd7ce2e6c96bfe/encoding.png)
      ![decoding](/uploads/d793234d44e9fc6f34f0c2a9372863cd/decoding.png)
      ![recoding](/uploads/413021de997c86d45b1287fcfe7804c7/recoding.png)
      ![combined](/uploads/6d1c3ae6d3bf5547434ca29ae80b5536/combined.png)
      ![ratio](/uploads/6cebd7a0bcef57d1b1256bc3941c1b0a/ratio.png)
      
      ### ZK
      ![setup](/uploads/1feb169452aa3274dc924edf772c9a5b/setup.png)
      ![commit](/uploads/0a2775c2116ca7d3fa7b956b934d1565/commit.png)
      
      ### FRI
      ![commits_single](/uploads/4602725e551a025d42815183a61d11b2/commits_single.png)
      ![commits_single_normalized](/uploads/2ffebea940af1cbbe9bd64499343e0e9/commits_single_normalized.png)
      ![end_to_end](/uploads/46917abd2f5976dfa6d3039cc0ab2c0e/end_to_end.png)
      ![evaluating](/uploads/cf4dc496cd5144615bf5f9b06d27dccd/evaluating.png)
      ![proofs](/uploads/b0828bfa62c2226c8d63c9ecd387049f/proofs.png)
      ![proofs_normalized](/uploads/8fbf0d713884e2e147c62313a001f379/proofs_normalized.png)
      ![proving](/uploads/161539dc412330be1878cdda82c3d966/proving.png)
      ![verifying_single](/uploads/2df7e777481d7789478386f8e83e0783/verifying_single.png)
      ab0d9a8b
  6. Nov 13, 2024
  7. Nov 05, 2024
  8. Sep 23, 2024
  9. Aug 01, 2024
    • STEVAN Antoine's avatar
      refactor repo architecture (!161) · 3608e95a
      STEVAN Antoine authored
      ## changelog
      - `src/main.rs` has been moved to a new crate: `bins/saclin` which stands for **S**emi-**A**VID **CLI** in **N**ushell
      - dependencies of `komodo` have been fixed
      - Nushell and Rust tests have been split in the Makefile: by default, only Rust tests will run locally and Nushell tests and examples can be run manually if desired. The CI will still run everything.
      - the README has been updated
      - test images have been moved to `assets/`
      - the majority of the old `./nu-utils/` module have been moved to internals of `./benchmarks/` and imports have been fixed
      - `cargo.nu` has been moved to `./bins/` and a new `./bins/README.md` mentions it
      - `./bins/saclin/` has been created and should be a self-contained Rust crate + Nushell module
      3608e95a
  10. Jul 12, 2024
    • STEVAN Antoine's avatar
      refactor the Nushell modules (!158) · dad69f2c
      STEVAN Antoine authored
      this MR is two-fold
      - it restructures the two main Nushell modules so that they are easier to read and use
      - it improves the "run" and "plot" modules for the benchmarks
      
      ## changelog
      - `.nushell/` is now renamed to `nu-utils/`
      - `benchmarks/` is now a valid Nushell module which exports a bunch of modules
        - `benchmarks linalg`: measure and plot linear algebra operations
        - `benchmarks setup`: measure and plot trusted setup building
        - `benchmarks commit`: measure and plot crafting commitments
        - `benchmarks recoding`: measure and plot the recoding of shards
        - `benchmarks fec`: measure and plot FEC operations, such as encoding and recoding, and allow combining these results with the pure recoding ones
      - the submodules of `benchmarks` typically have a `run` and a `plot` command, whith the exception of `benchmarks fec` which has a `run` module and multiple "plot" commands in `benchmarks fec plot`
      - the "run" commands will create a random temp file by default and ask for confirmation otherwise if the output file already exists, unless `--force` is used
      - snippetds in `benchmarks/README.md` have been updated
      dad69f2c
  11. Jul 05, 2024
  12. May 29, 2024
  13. May 28, 2024
    • STEVAN Antoine's avatar
      split `examples/` into `benchmarks/` and `bins/` (dragoon/komodo!117) · bb626120
      STEVAN Antoine authored
      ## new structure for the repository
      - benchmarks are in `./benchmarks/` and can be run with either `cargo run --package benchmarks --bin <bench>` or the commands in `./benchmarks/README.md`
      ```
      ├── Cargo.toml
      ├── README.md
      └── src
          └── bin
              ├── commit.rs
              ├── fec.rs
              ├── linalg.rs
              ├── operations
              │   ├── curve_group.rs
              │   └── field.rs
              ├── recoding.rs
              ├── setup.rs
              └── setup_size.rs
      ```
      
      - examples are now in `./bins/` as standalone binaries and can be run either with `cargo run --package <pkg>` or with the help of the `cargo bin` command from `.nushell/cargo.nu`
      ```
      ├── curves
      │   ├── Cargo.toml
      │   ├── README.md
      │   └── src
      │       └── main.rs
      ├── inbreeding
      │   ├── build.nu
      │   ├── Cargo.toml
      │   ├── consts.nu
      │   ├── mod.nu
      │   ├── plot.nu
      │   ├── README.md
      │   ├── run.nu
      │   └── src
      │       ├── environment.rs
      │       ├── main.rs
      │       └── strategy.rs
      ├── rank
      │   ├── Cargo.toml
      │   └── src
      │       └── main.rs
      └── rng
          ├── Cargo.toml
          └── src
              └── main.rs
      ```
      
      - Nushell modules are now located in `./.nushell/`
      
      ## changelog
      apart from the changes to the general structure of the repo:
      - `binary.nu` -> `.nushell/binary.nu`
      - new `cargo bin` command from `.nushell/cargo.nu`
      - `error throw` is now defined in `.nushell/error.nu`
      - main TOML has been greatly simplified because the dependencies of "examples" have been moved to the associated crates
      - the rest is basically the same but in the new structure
      bb626120
  14. Apr 26, 2024
    • STEVAN Antoine's avatar
      migrate recoding benchmark to PLNK (dragoon/komodo!89) · 22f7943d
      STEVAN Antoine authored
      this MR
      - moves the last "recoding" benchmark to `examples/benches/`
      - moves the README, which is now all alone, to `examples/benches/`
      - adds a mention to `examples/benches/README.md` in `README.md`
      - some minor improvements to the bench README
      
      ## TODO
      - [x] find a way to plot the "recoding" results (thanks to !90)
      22f7943d
  15. Jan 12, 2024
    • STEVAN Antoine's avatar
      test all shard permutations in the CLI test (!14) · 94b544af
      STEVAN Antoine authored
      related to
      - !13
      
      ## description
      in !13, i noticed that decoding can be pretty sensitive to the order of the shards...
      this is why i thought testing all possible permutations of the shards could be beneficial :thinking:
      
      see the [run attached to this MR](https://gitlab.isae-supaero.fr/dragoon/komodo/-/jobs/11281#L1120) to see all the combinations of shards used in the tests, e.g. `[0, 2, 3]` means that shards `0`, `2` and `3` have been used and shards `1` and `4` have been "_lost_" 
      
      ## changelog
      this MR
      - adds an inline `math` module to `tests/cli.nu` which defines
        - `choose` which computes all the sets of $k$ choose $n$ in $[|0, ..., n - 1|]$
        - `perm` which computes all the permutations on $[|0, ..., n - 1|]$
        - see the inline `_test_choose` and `_test_perm` commands for concrete examples of their behaviours
      - compute all the permutation of $k'$ choose $n$ shards, where $k'$ ranges from $k$ to $n$
      - run the reconstruction test on all these cases
      - moves the CLI example from `README.md` to `examples/cli.nu`
      
      > **Note**  
      > also removes the newline added to the stdout of `komodo` commands, so that the output is prettier, without spurious empty lines
      94b544af
  16. Dec 06, 2023
  17. Dec 01, 2023
  18. Nov 30, 2023
Loading