progress bar optional manual control

[AnkitAhlawat7742]

Fix: #7939

Summary

The default behavior for the progress bar has been adjusted based on dataset size. It remains enabled for small datasets and is automatically disabled for larger datasets to improve performance. Users retain full control and can explicitly enable or disable the progress bar using the progress_bar parameter.

Changes

Modified the logic to disable the progress bar when the number of files exceeds 16.

[AnkitAhlawat7742]

Hi @lhoestq ,
Can you please trigger the CI CD pipeline ?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[AnkitAhlawat7742]

@lhoestq I have applied the required Ruff formatting changes in the load.py file.
Could you please trigger the CI/CD pipeline again?

[AnkitAhlawat7742]

HI @lhoestq ,
Just a gentle reminder to please trigger the CI/CD pipeline for this PR..!!

[AnkitAhlawat7742]

HI @julien-c ,
can you please trigger the the CI/CD pipeline for this PR.

[AnkitAhlawat7742]

Hi @albertvillanova ,
Could you please trigger the CI/CD pipeline?, i can see it is pending from long time
Thanks

[lhoestq]

I don't think we want to add more arguments to this method.
Progress bars can already be disabled like this

from datasets.utils import disable_progress_bars

disable_progress_bars()

[AnkitAhlawat7742]

Hey @lhoestq ,
My intention was to provide more fine-grained control at the method level, particularly for multiprocessing scenarios where multiple progress bars can clutter the logs. This would give users the flexibility to explicitly enable or disable the progress bar as needed,which aligns with the concern raised in the issue #7939