Skip to content

Made ORSTools ready for qgis_process#325

Open
jannefleischer wants to merge 80 commits intoGIScience:mainfrom
jannefleischer:qgis_process_ready
Open

Made ORSTools ready for qgis_process#325
jannefleischer wants to merge 80 commits intoGIScience:mainfrom
jannefleischer:qgis_process_ready

Conversation

@jannefleischer
Copy link
Copy Markdown
Contributor

@jannefleischer jannefleischer commented May 22, 2025

Because I need to run our plugin OS-WALK-EU (which depends on isocrone calculations from ORStools) from within a qgis_process setup headless I needed some upstream changes:

  1. moved the initiation of the processing toolbox to initProcessing, so qgis_process can skip everything with a GUI involved
  2. Added some recasting of parameters to int, because qgis_process interprets numbers as strings...
  3. Added an configuration-processing-algorithm (so my headless machine can have a custom provider). Added a test to that, but never done tests before (just a urban planner doing some programming...)

Now I can run something like this from the command line:

qgis_process plugins enable "ORStools" && qgis_process run "ORS Tools:set_provider_config_to_ors_via_algorithm" -- distance_units=meters area_units=m2 ellipsoid=EPSG:7019 ors_provider_name=ors.onmyserver.de ors_provider_api_key= ors_provider_url=https://ors.onmyserver.de ors_provider_timeout=240 ors_provider_overwrite=true

followed by:

echo '{                                                                                                            
        "distance_units": "meters", 
        "area_units": "m2",
        "ellipsoid": "EPSG:7019",
        "inputs": {
                "INPUT_PROVIDER": 1,
                "INPUT_PROFILE": 0,
                "INPUT_POINT_LAYER": "/home/fleischer/srv-fs1/u/02_Code_Working_Trees/os-walk-eu/SampleData/SampleGrid_centroids.shp",
                "INPUT_FIELD": "GRD_ID",
                "INPUT_METRIC": 1,
                "INPUT_RANGES": "250,500,750,1000",
                "LOCATION_TYPE": 0,
                "INPUT_SMOOTHING": 0,
                "INPUT_AVOID_FEATURES": [],
                "INPUT_AVOID_COUNTRIES": null,
                "INPUT_AVOID_BORDERS": null,
                "OUTPUT": "/home/fleischer/srv-fs1/u/02_Code_Working_Trees/os-walk-eu/SampleData/SampleGrid_centroid_isochrones_orstools.shp"
    }
}' | qgis_process run "ORS Tools:isochrones_from_layer" -

Due to None/null-Values and [] as parameters, only the json-piping is working.

Copy link
Copy Markdown
Collaborator

@koebi koebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey,
thanks a lot for the contribution!

I had a quick look at everything but the new processing algorithm - will have to look at that one in detail :)

Comment thread ORStools/ORStoolsPlugin.py Outdated
Comment thread ORStools/ORStoolsPlugin.py Outdated
Comment thread ORStools/ORStoolsPlugin.py Outdated
Comment thread ORStools/proc/isochrones_layer_proc.py Outdated
@jannefleischer
Copy link
Copy Markdown
Contributor Author

Thanks for the feedback, removed old stuff.

jannefleischer and others added 23 commits May 26, 2025 11:09
Added sink commitment following qgis-pull-request #59241
Added sink commitment following qgis-pull-request #59241
This reverts commit 89ad098.
@merydian
Copy link
Copy Markdown
Collaborator

Hi, this would be a really cool feature to have I think. I would however be -1 on introducing three new processing algorithms for this sole reason. @koebi proposed to hide the new processing algorithms from the toolbox. Could you have a look at how that would work?

Otherwise I'd also be open for a video call to discuss details if you would like.

@jannefleischer
Copy link
Copy Markdown
Contributor Author

jannefleischer commented Jul 1, 2025

Well, I understand that adding three (rather trivial) algorithms would somewhat bloat the plugin, though if you really want to run qgis headless you need a way to configure the configuration (e.g. add a custom service if you want to use one).

I run this from a jupyterhub child container with obviously no way of accessing a UI. So from my perspective the addition of a configuration via processing toolbox is a must. Deletion and getting ID isn't. But would be easier to use, I think.

Though currently it isn't beautiful that there are (somewhat) double structures to modify config. Maybe modify gui to use the processing tool internally, too. To safe code lines.

Edit: A, now I understand what you are saying. I'll check how to useHideFromToolbox-flag...

@jannefleischer
Copy link
Copy Markdown
Contributor Author

jannefleischer commented Jul 1, 2025

So, I have hidden the algorithms now. Though should be documented somewhere that they exist?

@jannefleischer
Copy link
Copy Markdown
Contributor Author

@koebi Anything to be done before merged?

@jannefleischer jannefleischer requested a review from koebi August 28, 2025 09:51
@koebi
Copy link
Copy Markdown
Collaborator

koebi commented Aug 28, 2025

Hey,
thanks a lot for your continuous work :)

So, I have hidden the algorithms now. Though should be documented somewhere that they exist?
Yep :) We are currently re-working our plugin documentation, #308 is there to keep track of some ToDos.

Do you have any idea why the tests do not run completely?

@jannefleischer
Copy link
Copy Markdown
Contributor Author

I have absolutely no clue. And if I initiate them locally I almost instantly hit my apikey-limit. So sadly I am no help here :/ .

- Added `pre-commit` to requirements for code quality checks.
- Refactored `pytest_sessionstart` in `conftest.py` to ensure API key is set correctly.
- Enhanced `TestCommon` to include retry logic for OverQueryLimit in client requests.
- Updated `TestGui` to improve mouse event handling and added cleanup in `tearDown`.
- Implemented new tests in `TestProc` for directions layer optimization.
- Refactored field creation tests in `test_wrapper.py` to use unified `create_qgs_field` function.
- Fixed imports in `qgis_interface.py` to use the correct PyQt5 module from QGIS.
@jannefleischer
Copy link
Copy Markdown
Contributor Author

@koebi I have included changes from main into this. THough haven't tested my work at all with the new changes. Could you maybe have the test run on this?

@jannefleischer
Copy link
Copy Markdown
Contributor Author

When I run the tests in my fork, everything runs fine. https://github.com/jannefleischer/orstools-qgis-plugin/actions/runs/22182555888

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants