Merging of different CLIP model variants?

[maxlund]

Hi, I'm wondering if there is any documentation of what classes of models you can merge. For example, do you support other CLIP models from e.g. open_clip?

[tanganke]

Yes, I've recently added fusion_bench.modelpool.OpenCLIPVisionModelPool to support loading OpenCLIP models. The recommended version of the OpenCLIP library is 2.0.2 (as specified in the task_vectors repository's environment.yml. Higher versions should also work, though the code includes compatibility handling for version differences.

This implementation is still in testing phase. If you encounter any issues while using it, please report them so we can improve the functionality.