Fix build on macOS#11
Conversation
librt stuff is in system standard library on macos
|
Dear @controversial, thank you for your PR. I have seen your fork previously, and the changes that you made for macOS. Thank you for providing these changes as a PR now! EDIT: I am sorry, I mixed up your fork with this one, also containing adaptions for macOS: https://github.com/Empty2k12/map-matching-2 Unfortunately, as you said, I have no macOS system available and therefore, it is very difficult for me to support a macOS build. I could use GitHub Actions for building on macOS, yes. However, I had various bugs in the past that were OS dependent and could only be resolved by live-debugging on the respective system. Without a macOS system, I cannot live-debug the application, and therefore, I cannot practically support the builds that GitHub Actions would create. Sometimes, the bugs came from underlying libraries or OS differences of definitions, which needed workarounds in my code then for the respective OSs. As such, it is very difficult for me to support macOS, simply because I cannot resolve any bugs that happen during runtime needing live-debugging for tracking them down. One example for this is the disabled rpmalloc library that you say crashed on macOS. Officially, rpmalloc supports macOS, so I am not sure what happens here and I would need to live-debug the issue. Maybe the crash can be circumvented, but I have no way of checking this. I would love to be able to support all three major OS, but I only can support Windows and Linux with the tech that I currently have. GitHub actions would make it look like I can support macOS, but I cannot, in practice, help with any issues that arise down the line. Another reason for why I currently have not used GitHub actions is that for Linux, I use a custom Docker build-chain that currently needs around 40-50 minutes to build itself. I suspect this will lead to a time-out with GitHub actions. But I don't actually know. However, it takes a long time to build when everything is deleted after a build has been run. Maybe, I can circumvent a part of this issue by supplying these containers over DockerHub for the GitHub Actions to pull them, but this would also mean that I have to support the build environment in addition to the application itself and update these regularly, since other people can pull them, too. Furthermore, the build process itself also needs around 10-15 minutes at least (in practice, probably even more, since free GitHub actions are resource limited) and at least 16 GB of main memory, although I suspect this depends on the available number of threads. I don't know if GitHub actions works with these circumstances. I have not found the time to try it out, yet. Do you have any suggestions for resolving these issues? Maybe the first thing I should try is the GitHub actions build system. But even then, I am unsure about whether I can support macOS builds in practice. Even "unofficial" support may be an issue when people use the builds and have issues that I cannot reproduce. I am not happy about such a situation. I think, there is container support on macOS: https://github.com/apple/container I am open for suggestions, but for the moment, I will leave this PR open, until I have an idea how this could be resolved sustainably. Thank you for your suggestion! It is very appreciated! |
|
Yeah I agree it would be difficult to maintain real support for macOS in an ongoing way without a machine to develop on. My suggestion around for GitHub actions is mostly that you could have an action that simply runs build and executes
You can use actions/cache to keep a warm Since the changes in this PR are mostly all changes to the build process — rather than adjustments to application logic — it seems unlikely to me that future changes would cause major regressions here that would require extensive debugging on macOS? Maybe it would be possible to allow the project to be built on macOS but declare that macOS support is “unofficial” |
|
Dear @controversial, thanks for your quick reply! Ooh, actions/cache sounds interesting. I looked a little bit into it, but it seems that cache entries older than 7 days are automatically removed and there is a size limit of 10 GB. Unfortunately, a debug build on Windows already has a folder size of 9 GB, and on Linux GCC its 7 GB. The release build in Windows has 4.5 GB and on Linux 4 GB. And that is without the Docker build containers, which each have around 3 GB for GCC and Clang each. I suspect, this becomes tricky. Could work for a release-build only, but not for all paths. Concerning the unit tests, unfortunately, the coverage is not complete. The OS-dependent bugs that I encountered in the past were mostly not reflected by the unit tests. For example, I had a bug only on Windows that only appeared with importing road networks with memory mapping enabled when the intermediate files became larger than 4 GB. The error did not appear when memory mapping was disabled or the intermediate files were disabled and the output was below 4 GB. Only when a single file became larger than 4 GB during the import, the error happend - but not during importing, instead, later when matching on these files, because they were corrupt. Such errors are very difficult to address with unit tests. The error came from an underlying library, not my application. It doesn't make much sense to write unit tests that create such large test data for the purpose of catching rare bugs from underlying libraries. Such tests would consume a load of time during deployment. But as such, it slipped through. I could "unofficially" support macOS, but as soon as such an error arises, I have no possibility to resolve such an error without live-debugging on the actual OS. Especially because it was so complex, as the corrupted file made the matching crash, not the preparation. And with such an error in existence, I would probably have no other possibility as to remove the macOS build, to prevent further issues for other people using my software and experiencing unresolvable crashes. I completely understand your intention and if I had something like a Java or Python application that is "OS-independent by design" (as long as I don't write OS-specific code), I would feel much safer with "unofficial" support, since I knew the application "technically" works out of the box. But with this C++ application, hmmm... I am amazed enough that it seems to build so easily on macOS. But I have to think about it. I have to think about it and maybe have it dependent on GitHub actions support, which is, as said, quite difficult with the resources this project requires for compilation. Or I add it without GitHub actions support, but have no way of verifying whether it builds or not. I will have to think about this. But I am more willing to find and decide on a sustainable solution now that you sent your PR. |
2 participants
These are the changes I had to make to get the project to build on macOS.
FRAMEWORK DESTINATIONin theinstall(TARGETS)command, which CMake requires on macOS when usingRUNTIME_DEPENDENCIESsrc/library/src/io/memory_mapped/utils.cpptemplatekeyword which is enforced by AppleClang where gcc is more lenientlibrton macOS since it’s built-in and fails to linkAlso I saw you mentioned in #9 that you don’t have a way to test on macOS; I wanted to suggest that you could set up Github Actions to run build & test on all 3 platforms (windows, macos, linux); i think it would be free. you might even be able to link the artifacts from GHA runs to the Releases to distribute prebuilt binaries as well.