Skip to content

Added namespace progress report#2168

Merged
mdellweg merged 1 commit into
pulp:mainfrom
fao89:namespacepb
Apr 16, 2025
Merged

Added namespace progress report#2168
mdellweg merged 1 commit into
pulp:mainfrom
fao89:namespacepb

Conversation

@fao89
Copy link
Copy Markdown
Member

@fao89 fao89 commented Apr 8, 2025

No description provided.

@fao89 fao89 force-pushed the namespacepb branch 2 times, most recently from 845d131 to 8b1afc8 Compare April 8, 2025 12:13
rochacbruno
rochacbruno previously approved these changes Apr 8, 2025
Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you seen #2139? I'm not sure this takes into account this fix.

Edit See my second review

Copy link
Copy Markdown
Contributor

@gerrod3 gerrod3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: my earlier review. I do think this takes care of namespace optimization fix that I made in #2139, so I am willing to close that in favor of this. Can you add a bugfix changelog to this PR as well?

Comment thread pulp_ansible/app/tasks/collections.py Outdated

tasks = []
msg = _("Parsing Namespace Metadata")
async with ProgressReport(message=msg, code="sync.parsing.namespace", total=0) as pr:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async with ProgressReport(message=msg, code="sync.parsing.namespace", total=0) as pr:
async with ProgressReport(message=msg, code="sync.parsing.namespace", total=len(self.namespace_shas)) as pr:

for namespace, namespace_sha in self.namespace_shas.items():
tasks.append(loop.create_task(self._add_namespace(namespace, namespace_sha)))
await asyncio.gather(*tasks)
pr.total = pr.done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pr.total = pr.done

Since we know the total before hand we don't have to set it to done. In fact the amount we are going to add vs the amount that did get added can differ, so maybe let it be reflected in the done vs total amounts.

Copy link
Copy Markdown
Member Author

@fao89 fao89 Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept this because I only increment when the download succeeds,
so this would adjust the total of downloads

Comment thread pulp_ansible/app/tasks/collections.py Outdated
Comment on lines 720 to 723
await self.parsing_namespace_progress_bar.aincrement()
return True

return False
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need the

Suggested change
await self.parsing_namespace_progress_bar.aincrement()
return True
return False
await self.parsing_namespace_progress_bar.aincrement()
log.info("Failed to find namespace...")

The return True/False is no longer needed, so maybe we add some log statements for when we fail to add the namespace metadata.

if pr.message == "Parsing CollectionVersion Metadata":
assert pr.total == pr.done
if pr.message == "Parsing Namespace Metadata":
assert pr.total == pr.done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assert pr.total == pr.done
assert pr.total == pr.done == 1

I feel that we should know the total amount for this test sync since it is only one collection.

fixes: pulp#2138
Signed-off-by: Fabricio Aguiar <fabricio.aguiar@gmail.com>
@fao89 fao89 requested a review from gerrod3 April 14, 2025 14:15
@mdellweg mdellweg merged commit bf4079b into pulp:main Apr 16, 2025
12 checks passed
@patchback
Copy link
Copy Markdown

patchback Bot commented Apr 16, 2025

Backport to 0.21: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply bf4079b on top of patchback/backports/0.21/bf4079b75aa35790229b51850041726626978a42/pr-2168

Backporting merged PR #2168 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/pulp/pulp_ansible.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/0.21/bf4079b75aa35790229b51850041726626978a42/pr-2168 upstream/0.21
  4. Now, cherry-pick PR Added namespace progress report #2168 contents into that branch:
    $ git cherry-pick -x bf4079b75aa35790229b51850041726626978a42
    If it'll yell at you with something like fatal: Commit bf4079b75aa35790229b51850041726626978a42 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x bf4079b75aa35790229b51850041726626978a42
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Added namespace progress report #2168 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/0.21/bf4079b75aa35790229b51850041726626978a42/pr-2168
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link
Copy Markdown

patchback Bot commented Apr 16, 2025

Backport to 0.20: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply bf4079b on top of patchback/backports/0.20/bf4079b75aa35790229b51850041726626978a42/pr-2168

Backporting merged PR #2168 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/pulp/pulp_ansible.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/0.20/bf4079b75aa35790229b51850041726626978a42/pr-2168 upstream/0.20
  4. Now, cherry-pick PR Added namespace progress report #2168 contents into that branch:
    $ git cherry-pick -x bf4079b75aa35790229b51850041726626978a42
    If it'll yell at you with something like fatal: Commit bf4079b75aa35790229b51850041726626978a42 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x bf4079b75aa35790229b51850041726626978a42
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Added namespace progress report #2168 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/0.20/bf4079b75aa35790229b51850041726626978a42/pr-2168
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link
Copy Markdown

patchback Bot commented Apr 16, 2025

Backport to 0.24: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply bf4079b on top of patchback/backports/0.24/bf4079b75aa35790229b51850041726626978a42/pr-2168

Backporting merged PR #2168 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/pulp/pulp_ansible.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/0.24/bf4079b75aa35790229b51850041726626978a42/pr-2168 upstream/0.24
  4. Now, cherry-pick PR Added namespace progress report #2168 contents into that branch:
    $ git cherry-pick -x bf4079b75aa35790229b51850041726626978a42
    If it'll yell at you with something like fatal: Commit bf4079b75aa35790229b51850041726626978a42 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x bf4079b75aa35790229b51850041726626978a42
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR Added namespace progress report #2168 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/0.24/bf4079b75aa35790229b51850041726626978a42/pr-2168
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants