Skip to content

Adding dynamic progress based starting of plots. #1091

Open
NPutting wants to merge 5 commits intoswar:developmentfrom
NPutting:main
Open

Adding dynamic progress based starting of plots. #1091
NPutting wants to merge 5 commits intoswar:developmentfrom
NPutting:main

Conversation

@NPutting
Copy link
Copy Markdown

@NPutting NPutting commented Jun 3, 2021

Adding dynamic progress percent based starting. This waits until the average percent is greater than
(running + 1)/2 * 1/max_concurrent such as for a max concurrent of 8 it would use these values:

current running average %
0 0.00
1 12.50
2 18.75
3 25.00
4 31.25
5 37.50
6 43.75
7 50.00
8 56.25
9 62.50

such as if there were 7 already running, the average would have to be 50% or greater such as,
12.5%, 25%, 37.5%, 50%, 62.5%, 75%, and 87.5 % as they average out to 50%

This should help in two cases:
First, when starting up the manager and plots are going faster than the average time fixed stagger minutes waste time.
Second, the user shouldn't have to try and guess/calculate the correct stagger minutes just their max parallel plots.

This change is completely passive, defaulting to false both in new and old configs without the option. All other config options such as max concurrent, max concurrent with early start, stagger minutes, and max for phase 1 still apply as well. I've documented the option in the config.yaml.default, however, I haven't added anything to the primary README.md yet as I wasn't sure if this would be a default option in the future.

I've been trying this out for about a day so far and plots seem to stagger well. I used a 10-minute minimum stagger so it doesn't delay extra and the average spacing is close to the 48-50 minutes I was using before once steady state is reached.

Info on not met percentages is also included in the INFO level logs similar to other config options and looks like this:
2021-06-02 21:39:23 [INFO]: Minimum percent not met, 51.97 vs 56.25

@NPutting NPutting changed the base branch from main to development June 3, 2021 02:53
@NPutting
Copy link
Copy Markdown
Author

NPutting commented Jun 3, 2021

Changed the PR to the development branch as that seems to be what other PRs were doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants