Add modules proposal

[goaaats]

This pull request proposes a scheme to introduce libraries into the Dalamud ecosystem, to alleviate pressure on developers during patch days and to improve the code sharing experience.

I would love honest input from first and third-party developers. Don't expect this to be implemented as is soon, as it is quite broad in scope and I don't have a lot of time to work on features myself. Maybe an acceptable outcome of this proposal is that we extract bits and bobs that would be easier to implement but still provide a lot of value.

Please read the entire proposal before commenting. Thanks!

Read a rendered version here: https://github.com/goaaats/DIPs/blob/modules/text/0000-modules.md

[NotNite]

This is really exciting! I'm glad that we're solving plugin libraries and splitting up Dalamud via the same mechanism. Some initial thoughts:

• Optional dependencies are pretty important to get right IMO. We have a lot of plugins with some base functionality that optionally use IPC to enhance it (some that come to mind: Craftimizer and Macro Mate, Wotsit and Teleporter, etc) and I'm sure that someone might use the module system in this way. I'm unsure how optional dependencies would be presented to the user, though.
  • Optional dependencies might also benefit bringing up Dalamud on patch days. As an example, let's say /xldata depends on a bunch of services in module plugins for its separate tabs; if a single module that's only used in one tab fails to load, and the other modules still work, the tab can just be disabled instead of the entire window failing to load. Dalamud at present will usually completely die in this state (I'm reminded of when fly text broke the entire boot sequence one patch), and having it in a half-loaded state is better than not loaded at all imo.
• If plugin repositories can also publish module plugins, how will duplicate module IDs from multiple repositories be handled? With plugins, users have the option to choose which plugin from which repository they click Install on, but that's explicitly a non-goal here. In any case, I presume we'll disallow overriding module plugins on the main repository, in the same way we do for plugins already.

[Minmoose]

I echo much of what NotNite has said;

This is very excellent and I do very much support this, would make a module that could be used for posing hooks from Brio and an IPC for Brio.

There are a few problems for me.

Optional dependencies: Brio has a hard rule to never depend on another plugin or service outside of Dalamud or CS for any core functionality I don't see a way to consume a third-party module like a Penumbra or C+ IPC module without it becoming a hard load requirement for Brio and wouldn't be able to load without those modules being there and available. It would be hard to justify using it without it being optional

And then, where do the modules come from and how would we be able to tell a user where to get the appropriate repository, there were discussions on discord about making sure there isn't a malicious module pulled in by a malicious custom repo and in turn a malicious module getting used by a 3pp inadvertently, so there needs to be some way to fully qualify that the module being consumed is the real module.

Some examples discussed on Discord at the time of writing:

• Fully qualifying the module and repo name, e.g. https:/TPR/CustomRepo.json#internalName
• A list of preferred repositories that the module comes from as a first choice if available
• Load the module from the repo the plugin came from as a first choice
• On the consumer side, bind it to a nuget package somehow, so it's verifiable the right code is being pulled in
• Create a super simple central repository for all module's that other plugins can reference to verify trust

While I wouldn't really mind any of them, and while the central repository one does feel nice, I do recognize it's not really something Dalamud would do, so my opinion is it should be some combination of being backed by nuget, loading the module from the same repository as a first try and/or fully qualifying the module, I feel that creates the most trust in the code being pulled in, without Dalamud as a platform having to manage anything for 3pp

[NotNite]

I personally dislike the concept of a central registry, because it defeats half of the purpose of custom repositories (independent developer control). I also think it's important to allow modules to come from a repo they aren't being requested from, as a module author may host it on their own repo (e.g. Kami may make a separate KTK repo that users add) or it might be served on one of many possible URLs (a hypothetical Penumbra module would be published on Sea of Stars and the original Penumbra repository simultaneously).

I think I prefer having the resolver prioritize modules on the same repository, then falling back to modules from specifically allowed repositories, such that a malicious repository can't publish a malicious module that gets automatically installed. This is different from the threat model of CRPs, because modules are installed automatically by the resolver without any user input, while users pick which variant of a duplicate plugin to install.

Note that I'm not sure how to best handle versioning or updates with this model. A malicious plugin installing a malicious module is probably out of scope. I also haven't put much thought into how signing/NuGet could be incorporated, but it would probably be a more elegant solution than specifying repository URLs.

[Zinbhe]

One issue I came across while writing a toy resolver was testing releases. A few of the questions that will have to be answered are:

• Can modules have testing versions? If so:
  • Do users opt into testing modules?
  • Does Testing interfere with non-Testing for the "only one version of a module may be loaded" rule?
• If a Testing plugin requests a shared module version that is incompatible with non-Testing plugins, do the non-Testing plugins get disabled?

They aren't impossible to answer, but they do inform the resolver design, and matter for both user and developer experience so should be considered carefully.