Skip to content
Snippets Groups Projects
  1. May 31, 2024
    • STEVAN Antoine's avatar
      add information about experiment to `inbreeding load` (dragoon/komodo!134) · 142e673e
      STEVAN Antoine authored
      this can be used by `plot` without passing extra arguments, i.e. the pipeline becomes
      ```bash
      use bins/inbreeding
      
      let experiment = "..."
      ```
      ```bash
      inbreeding load $experiment | inbreeding plot
      ```
      instead of having to know the value of `k` and do
      ```bash
      inbreeding load $experiment | inbreeding plot --options { k: $k }
      ```
      142e673e
  2. May 29, 2024
  3. May 28, 2024
    • STEVAN Antoine's avatar
      include the standard deviation of "inbreeding" measurements (dragoon/komodo!119) · a29b511d
      STEVAN Antoine authored
      this adds `$.diversity.e` to the output of `inbreeding load` and input of `inbreeding plot` to show errors bars in the final plot.
      
      these can be discarded by running something similar to
      ```bash
      inbreeding load $experiment
          | reject diversity.e
          | inbreeding plot --options { k: $OPTS.k }
      ```
      a29b511d
    • STEVAN Antoine's avatar
      split "inbreeding" load and plot (dragoon/komodo!118) · 31d58f4c
      STEVAN Antoine authored
      plotting "inbreeding" results is now done in two steps
      - `load` the data from raw experiment files with `inbreeding load`
      - `plot` the results by piping this data into `inbreeding plot`
      31d58f4c
    • STEVAN Antoine's avatar
      split `examples/` into `benchmarks/` and `bins/` (dragoon/komodo!117) · bb626120
      STEVAN Antoine authored
      ## new structure for the repository
      - benchmarks are in `./benchmarks/` and can be run with either `cargo run --package benchmarks --bin <bench>` or the commands in `./benchmarks/README.md`
      ```
      ├── Cargo.toml
      ├── README.md
      └── src
          └── bin
              ├── commit.rs
              ├── fec.rs
              ├── linalg.rs
              ├── operations
              │   ├── curve_group.rs
              │   └── field.rs
              ├── recoding.rs
              ├── setup.rs
              └── setup_size.rs
      ```
      
      - examples are now in `./bins/` as standalone binaries and can be run either with `cargo run --package <pkg>` or with the help of the `cargo bin` command from `.nushell/cargo.nu`
      ```
      ├── curves
      │   ├── Cargo.toml
      │   ├── README.md
      │   └── src
      │       └── main.rs
      ├── inbreeding
      │   ├── build.nu
      │   ├── Cargo.toml
      │   ├── consts.nu
      │   ├── mod.nu
      │   ├── plot.nu
      │   ├── README.md
      │   ├── run.nu
      │   └── src
      │       ├── environment.rs
      │       ├── main.rs
      │       └── strategy.rs
      ├── rank
      │   ├── Cargo.toml
      │   └── src
      │       └── main.rs
      └── rng
          ├── Cargo.toml
          └── src
              └── main.rs
      ```
      
      - Nushell modules are now located in `./.nushell/`
      
      ## changelog
      apart from the changes to the general structure of the repo:
      - `binary.nu` -> `.nushell/binary.nu`
      - new `cargo bin` command from `.nushell/cargo.nu`
      - `error throw` is now defined in `.nushell/error.nu`
      - main TOML has been greatly simplified because the dependencies of "examples" have been moved to the associated crates
      - the rest is basically the same but in the new structure
      bb626120
  4. May 27, 2024
    • STEVAN Antoine's avatar
      mix colors for hybrid recoding strategies (dragoon/komodo!112) · 9d6e6e7a
      STEVAN Antoine authored
      - define `scripts/color.nu` to manipulate RGB colors, especially mix two colors together
      - compute the color of _hybrid recoding strategies_ as a weighted sum of the two _simple recoding strategies_ involved, e.g. if the strategy is "10% of the time recode 2 shards and 90% of the time recode 3", then the color of that curve will be 10% the color of the simple strategy recoding 2 shards and 90% the color of the other simple strategy recoding 3 shards
      - make the _hybrid_ curves transparent and dashed
      
      ## example
      ![foo](/uploads/71afca07a87fc38e3fa005cebfab4e50/foo.png)
      9d6e6e7a
    • STEVAN Antoine's avatar
      add timestamp to measurements and delay start (dragoon/komodo!111) · 1f16e7e2
      STEVAN Antoine authored
      - add a timestamp to all the measurements of the _diversity_ from `inbreeding/mod.rs`
      - allow to delay the measurement starts with `--measurement-schedule-start`, to help completing already existing measurements
      
      > :exclamation: **Important**  
      > existing measurement files will have to change shape from
      > ```
      > table<strategy: string, diversity: list<float>>
      > ```
      > to
      > ```
      > table<strategy: string, diversity: table<t: int, diversity: float>>
      > ```
      1f16e7e2
  5. May 24, 2024
    • STEVAN Antoine's avatar
      make "inbreeding" scripts a module (dragoon/komodo!109) · d594d917
      STEVAN Antoine authored
      just a small QoL improvement
      d594d917
    • STEVAN Antoine's avatar
      measure "inbreeding" for multiple recoding scenarii (dragoon/komodo!108) · 11cd2cdc
      STEVAN Antoine authored
      this MR is two-fold
      - refactor `run.nu` and `plot.nu` from `scripts/inbreeding/` into Nushell modules with `--options` as argument instead of `options.nu` (a7cebb95, 6b72191f and 5f1c4963)
      - introduce another level of depth to the measurements (a0e52e95)
      
      > :bulb: **Note**  
      > in the table below
      > - $s$ is the number of recoding scenarii averages together
      > - $m$ is the number of measurements per point
      > - two iterations of the same experiment are shown side by side for comparison
      
      s   |    m | . | .
      :--:|:----:|:-------------------------:|:-------------------------:
      1   | 10   | ![inbreeding_1_10.1](/uploads/c593393edb3513c9d77b0fe134c27fd7/inbreeding_1_10.1.png) | ![inbreeding_1_10.2](/uploads/97c85b36833112de51a2b756ade53479/inbreeding_1_10.2.png)
      1   | 100  | ![inbreeding_1_100.1](/uploads/af4da1d7cf76ef43fb39c2a3a529b7cd/inbreeding_1_100.1.png) | ![inbreeding_1_100.2](/uploads/e187298709d524437dea503be6ac555f/inbreeding_1_100.2.png)
      1   | 1000 | ![inbreeding_1_1000.1](/uploads/394821777baeff9fec589440ba4c554c/inbreeding_1_1000.1.png) | ![inbreeding_1_1000.2](/uploads/1d592b791075f204f8a0ebdd739403dd/inbreeding_1_1000.2.png)
      10  | 100  | ![inbreeding_10_100.1](/uploads/3c822e7669e9f0b4d97919e5a3bd4bca/inbreeding_10_100.1.png) | ![inbreeding_10_100.2](/uploads/aa6b54ec64f82ca386dae4e262dcd0b6/inbreeding_10_100.2.png)
      100 | 10   | ![inbreeding_100_10.1](/uploads/7ef383143d981717b0ad01dee0359eb0/inbreeding_100_10.1.png) | ![inbreeding_100_10.2](/uploads/ce71a32a7f6e9ba3c4dae563aeefd856/inbreeding_100_10.2.png)
      100 | 100  | ![inbreeding_100_100.1](/uploads/e2038273051f959d8be69fef9ba7a493/inbreeding_100_100.1.png) | ![inbreeding_100_100.2](/uploads/0ef30735597e1a6812484d1cac4d34ca/inbreeding_100_100.2.png)
      
      we can see that
      - the smaller the $s$, the more different the two figures are on each line -> this is likely due to the fact that, if only one recoding scenario is used, then repeating the same experiment will result in very different results and measurements. Running the same experiment $s$ times and averaging helps reducing the variance along this axis
      - the smaller the $m$, the more noisy the measures of each points -> this is simply because, when $m$ is small, the variance of the empirical means measured for each point is higher
      
      ## final results
      ![inbreeding](/uploads/e561f4e4acad8eedbb3ccf1a4666c302/inbreeding.png)
      ![inbreeding_100_100_1](/uploads/37ab0eacb5159137595579dfbb20250c/inbreeding_100_100_1.png)
      11cd2cdc
  6. May 23, 2024
    • STEVAN Antoine's avatar
      Refactor plot commands (dragoon/komodo!104) · 0f43be24
      STEVAN Antoine authored
      this MR moves run and plot commands from `examples/benches/README.md` to
      - `scripts/setup/`: `run.nu` and `plot.nu`
      - `scripts/commit/`: `run.nu` and `plot.nu`
      - `scripts/recoding/`: `run.nu` and `plot.nu`
      - `scripts/fec/`: `run.nu` and `plot.nu`
      - `scripts/inbreeding/`: `build.nu`, `run.nu` and `plot.nu`
      
      to generate all the figures at once
      ```bash
      use scripts/setup/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/setup.ndjson
      use ./scripts/setup/plot.nu; plot data/setup.ndjson --save ~/setup.pdf
      
      use scripts/commit/run.nu; seq 0 13 | each { 2 ** $in } | run --output data/commit.ndjson
      use ./scripts/commit/plot.nu; plot data/commit.ndjson --save ~/commit.pdf
      
      use scripts/recoding/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/recoding.ndjson
      use ./scripts/recoding/plot.nu; plot data/recoding.ndjson --save ~/recoding.pdf
      
      use scripts/fec/run.nu; seq 0 18 | each { 512 * 2 ** $in } | run --ks [2, 4, 8, 16] --output data/fec.ndjson
      use ./scripts/fec/plot.nu; plot encoding data/fec.ndjson --save ~/encoding.pdf
      use ./scripts/fec/plot.nu; plot decoding data/fec.ndjson --save ~/decoding.pdf
      use ./scripts/fec/plot.nu; plot e2e data/fec.ndjson --save ~/e2e.pdf
      
      use ./scripts/fec/plot.nu; plot combined data/fec.ndjson --recoding data/recoding.ndjson --save ~/comparison.pdf
      use ./scripts/fec/plot.nu; plot ratio data/fec.ndjson --recoding data/recoding.ndjson --save ~/ratio.pdf
      
      ./scripts/inbreeding/build.nu
      ./scripts/inbreeding/run.nu --output data/inbreeding.nuon
      ./scripts/inbreeding/plot.nu data/inbreeding.nuon --save ~/inbreeding.pdf
      ```
      
      > :bulb: **Note**  
      > this took around 27min 18sec in total on my machine with 14min 45sec for the inbreeding section only and 12min 33sec for the rest
      0f43be24
Loading