Adding dynamic progress based starting of plots. #1091
Open
NPutting wants to merge 5 commits intoswar:developmentfrom
NPutting:main
Open
Adding dynamic progress based starting of plots. #1091NPutting wants to merge 5 commits intoswar:developmentfrom NPutting:main
NPutting wants to merge 5 commits intoswar:developmentfrom
NPutting:main
Conversation
Adding progress percent based starting
Author
|
Changed the PR to the development branch as that seems to be what other PRs were doing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding dynamic progress percent based starting. This waits until the average percent is greater than
(running + 1)/2 * 1/max_concurrent such as for a max concurrent of 8 it would use these values:
such as if there were 7 already running, the average would have to be 50% or greater such as,
12.5%, 25%, 37.5%, 50%, 62.5%, 75%, and 87.5 %as they average out to 50%This should help in two cases:
First, when starting up the manager and plots are going faster than the average time fixed stagger minutes waste time.
Second, the user shouldn't have to try and guess/calculate the correct stagger minutes just their max parallel plots.
This change is completely passive, defaulting to false both in new and old configs without the option. All other config options such as max concurrent, max concurrent with early start, stagger minutes, and max for phase 1 still apply as well. I've documented the option in the config.yaml.default, however, I haven't added anything to the primary README.md yet as I wasn't sure if this would be a default option in the future.
I've been trying this out for about a day so far and plots seem to stagger well. I used a 10-minute minimum stagger so it doesn't delay extra and the average spacing is close to the 48-50 minutes I was using before once steady state is reached.
Info on not met percentages is also included in the INFO level logs similar to other config options and looks like this:
2021-06-02 21:39:23 [INFO]: Minimum percent not met, 51.97 vs 56.25